By default, campaign experiments assigns a user to a test or control group every time they enter a search (this is known as "search-based splits"). This means that if someone performs three searches, two of those searches could be subject to test treatments and the third could be subject to the control treatment. With our new cookie-based splits, you now have the option to assign searchers to dedicated test and control groups. This means that if someone is mapped to a test group and performs three searches, all three will be subject to test treatments.
What type of split option you decide to use is at your discretion. It is important to understand the options and align with how you want your options to be tested.
Cookie-based might be a better option in the following cases:
- Testing specific audiences
- You want to set up a landing page test that runs similar to other conversion rate optimization testing tools
Search-based splits might be a better option in the following cases:
- You want to keep consistency with previous search-based tests
- Low volume campaigns
Let’s say you want to test the impact of Accelerated Mobile Pages (AMP), pages which often load in under a second. If you set this experiment up using search-based splits, a given user might experience both AMP and non-AMP experiences as their inclusion in test or control is re-assessed for each search they do. Enabling cookie-based splits allows you to keep user experiences consistent, as the user is permanently assigned to your experiment's test or control group.
Learn more about experiments here.
