About certainty of lift

Certainty of lift, available only for Brand Lift, is an important metric to understand the reliability of your lift results. It represents the likelihood that the measured lift is generated by your campaigns and not due to chance.

How it works

Certainty of lift is calculated as 1 - p-value and can sometimes be referred to as the 'statistical significance' or the 'confidence' of lift results. The p-value shows how likely your lift results would be if the ads were actually ineffective. A high certainty, corresponding to a low p-value, indicates that results are unlikely to have happened purely by chance. High certainty is a strong indication that your ads generated lift.

Understand Brand Lift levels of certainty

Google tries to collect enough responses to detect lift with the highest certainty (90%), but lower certainty results can still be helpful in making advertising decisions. In your Google Ads account, you can find lift results and their associated certainty for all studies with a measurement certainty is above 70%. Results below 70% aren’t reported because they wouldn’t be statistically strong enough to be useful and trustworthy.

Note: Certainty of lift isn’t available for all accounts. If you don’t have it in your account, you can only find results with high certainty (>90%). Certainty of lift is only available for Brand Lift. Search and Conversion Lift results are only shared with the highest certainty of lift (>90%).

How to interpret and use results with certainty of lift

The following table gives general guidance on how to interpret lift results with different certainty of lift. Note that the certainty of lift is rounded down by increments of 5% for simplicity. You can use the table below for guidance, but it is recommended that you interpret the results based on your business needs and risk tolerance.

Certainty of lift	Interpretation
≥90%	High confidence – These results are the most reliable since they’re unlikely to be due to chance. You can use high confidence results to make decisions related to your ads as there’s strong evidence that your ads created lift.
85%	Medium confidence – These results have a slight chance that they could have been affected by noise. It’s recommended that you use them as directional insights or for low-risk decisions.
80%
75%	Low confidence – These results could be due to chance and may not accurately reflect your ads’ performance. It’s recommended that you use low confidence results for directional insights or very low risk decisions. You can gather more survey responses (via remeasurement) to increase certainty before making significant decisions.
70%
'No lift'	Unreliable results – Results with a certainty below 70% don’t provide enough evidence of lift and are reported as 'no lift' in Google Ads.

It’s important to note that Brand Lift aims to detect lift at the highest certainty of 90%. Showing lower certainty results doesn’t degrade the quality of results, but allows you to get data points that would otherwise be unavailable.

Understand low certainty

Low certainty results don’t always mean that ads are ineffective. These results can help you to get insights that should be further verified. A low certainty of lift means that the measurement resolution was unable to detect the lift with high confidence. This can happen either because not enough survey responses were collected or because the lift is low.

Low survey counts: Low survey counts (below 4100 responses) leads to low measurement power, which means a lower certainty of lift is more likely. In particular, you will have lower survey count per segment when segmenting your results, for example, by age, gender or campaigns. You can use remeasurement to increase survey responses
Low absolute lift: When lift is below 2%, it’s hard to detect with high certainty. However, a low absolute lift isn’t necessarily an indicator of poor performance. Campaigns with low absolute lift can still outperform on cost per lifted user (CPLU).

Compare segment-level lifts with different certainty of lift

It’s likely that different segments, for example different age groups, will have a different certainty of lift. As explained above, you shouldn’t conclude that the segment with highest certainty of lift is the best performing segment.

First, note that it’s recommended that you use absolute lift or CPLU to compare segments (do not use lifted users). Second, note that lift performance between different segments is often similar (high overlap of confidence intervals) which makes it hard to clearly identify the best segments. However, if you want to optimise your campaign towards the best performing segments, it’s recommended that you select the segments with the highest absolute lift (or lowest CPLU) to make the best decision on average. Note however, that the lower the certainty of lift, the higher the chance that the measured performance could be due to noise. If you have multiple segments with similar performance, it can be wise to select the ones with highest certainty to minimise risk. If you’re unsure how to identify the best performing segments, please ask your account representative.

Understand confidence intervals and levels

When referring to lift of an ad, people usually refer to the lift 'point estimate' which is the most likely lift generated by the ad. However, in Google Ads, you can also find a confidence interval for all brand lift metrics which is an estimated range in which your result could fall. This range is defined by an upper and lower bound which are the highest and lowest values where your lift is likely to actually be. Lift results use 80% two-sided confidence intervals, which means that there is an 80% chance that the true lift is between the lower bound and upper bound. This also means that you have a 90% chance that the lift is greater than the lower bound.

Example: You may view that your relative lift is 35%, which is the point estimate. However, you can also see that the confidence interval goes from 30% to 40%, which means that there is an 80% chance that the true lift is between 30% (the lower bound) and 40% (the upper bound). Another way to look at this is that there is a 90% chance that lift is greater than the 30% (the lower bound).

Note that when the certainty of lift is smaller than 90% then the lower bound of the confidence interval will be smaller than 0 because Google can’t guarantee with more than 90% certainty that the lift was positive.

Frequently asked questions

Can I choose my own minimum certainty of lift

No. Results are always shown if their certainty is above 70%. If you want a higher limit (like 80%) discard any results that don’t meet your limit. It’s not possible to set a limit lower than 70%.

How can I increase my certainty of lift?

Certainty of lift depends on the accuracy of the measurement. To increase measurement accuracy you can:

Use remeasurement to increase survey collection.
Measure ad recall or awareness to determine the highest chance of getting lift with high certainty.
Talk to your Account manager to select ad campaigns with high lift.

How can I learn more about how certainty of lift is calculated?

There’s always some natural randomness in surveys that can lead to fluctuations in data. This is often referred to as 'random measurement noise'. This random noise may lead to brand lift measuring positive lift despite the ads not creating lift in reality. The p-value quantifies how likely a measured lift can be due to noise if the ads did not generate lift. If you have a very low p-value, it’s very unlikely that the lift measured was a result of random noise, and it’s certain that the ad campaigns caused lift.

The certainty of lift is calculated as 1 - p-value and expressed as a percentage. The higher this number is (and therefore the lower the p-value), the more certain that the ads caused lift.

Example 1: Ad shows 5% absolute lift, p-value = 0.01: This means that there is a 1% chance of seeing 5% lift due to random measurement noise. This leads to a high certainty (99%) that an ad caused lift.

Example 2: Ad shows 5% absolute lift, p-value = 0.35: This means there's a 35% chance of seeing 5% lift due to random measurement noise. This leads to a low certainty (65%) and is not reliable enough evidence that the ad caused lift.

Why would I experience low certainty of lift in segments?

When slicing data by segment, for example, by campaigns, each segment only has a subset of all the surveys. Because the individual segments have fewer surveys than the overall study, it’s expected to be harder to detect lift with high certainty. Note that if one segment has more reach than another, for example, if one campaign has more budget than another one, it will collect more surveys and is likely to have a higher certainty of lift, even though the lift for that segment could be smaller. Detecting lift is most challenging on segments with the smallest reach.

Where can I find the certainty for each in segments?

In the lift report there is an expandable table below the charts. You can review the table to find all your lift metrics, including the certainty of lift.

Where do I find confidence intervals?

Confidence intervals are included on lift graphs and can be found by hovering over lift results in tables. The graphs allow you to inspect how much uncertainty there is in the measurement. Additionally, when comparing segments this allows you to quickly inspect if the confidence intervals of two segments overlap. The higher the overlap the less certain you can be that one segment is better than another. On all graphs, the confidence intervals are clipped at 0 but the exact values can be found by hovering over lift results in tables.

Was this helpful?

How can we improve it?