To help you understand the incremental impact of campaigns, we’ve released a campaign holdout feature that allows you to exclude a group of users from being messaged by a campaign. Now with campaign holdout reporting you’ll be able to run and analyze tests comparing outcomes for the group of users who receive messages from a campaign (the test group) against the group of users who did not (the control group) and assess performance lift from campaign exposure directly within the Blueshift platform.
We support two types of holdout (control) groups:
- Global Control Group: A percentage of users will be held out from ALL campaigns and syndications. This allows you to analyze the overall impact of your marketing strategy.
- Campaign Control Group: A group of users will be held out from a specific campaign.
Holdout reports are easily accessible from the campaign index screen, the campaign detail screen, and the campaign journey screen through the ‘Holdout Report’ menu option. In the holdout report screen you’ll be able to see the conversion lift from running a campaign for a selected goal metric and time period as well as the statistical significance level of the results.