Short answer: Incrementality testing measures the sales your ads actually caused, by comparing a group that saw your ads with a similar group that did not. The gap between them is the incremental effect. Smaller brands spending roughly €10k to €50k a month usually do need it, but a simple geo holdout or pause test is enough. A full measurement programme is not.
Key takeaways
- Incrementality means the sales that would not have happened without the ads. Attributed conversions are the sales the platform gives itself credit for. The two are rarely the same.
- Every incrementality method needs a comparison group that did not see the ads: a random holdout audience, a set of regions, or a period with ads switched off.
- Platform lift studies from Meta and Google are useful when your account qualifies. A geo holdout test is the most practical option you can run yourself on your own sales data.
- A fair geo test runs three to six weeks, uses regions with a stable sales history and enough orders, and keeps everything else unchanged.
- Incremental ROAS, not reported ROAS, tells you whether the next euro of spend pays back. Below roughly €10k a month or with one channel only, simpler checks usually do the job.
Most brands we speak to have the same problem in different words. Meta says it drove €90,000 last month, Google says it drove €70,000, the email tool claims another €30,000, and the shop only took €140,000 in total. Every platform is counting partly the same customers, and every platform has a reason to count generously.
Incrementality testing gets you out of that loop. Instead of asking which channel touched the sale, it asks: if we had not spent this money, how much revenue would we have lost? Below are the main methods, a simple test for a modest budget, and when it is not worth the effort yet.
What is incrementality in marketing?
Incrementality is the extra outcome your marketing caused: the sales, sign-ups or leads that would not have happened if the activity had not run. Everything else, the customers who would have bought anyway, is not incremental, even if they clicked an ad on the way to the checkout.
Think of a customer who already loves your brand, types your name into Google, clicks your branded search ad and buys. Google Ads records a conversion. But if the ad had not been there, most such customers would have clicked the organic result just below it and bought anyway. The ad was present at the sale. It did not cause it.
Now think of someone who had never heard of you, sees an Instagram video ad and buys a week later via direct traffic. The ad caused that sale, and the reporting may barely show it.
So attribution and incrementality can be wrong in both directions. Attribution over-credits ads that sit close to the purchase, such as branded search and retargeting, and often under-credits ads that create new demand, such as prospecting on Meta, YouTube or TikTok.
Attribution is still useful for optimising within a channel. It just cannot tell you what would have happened without the spend. Incrementality testing creates that comparison on purpose.
Why don't platform-reported conversions show incrementality?
Because a conversion report only shows who bought after seeing or clicking an ad. It cannot show who would have bought without it. And the people ad platforms are best at finding are often the people already most likely to buy.
Platforms are built to show ads to the people most likely to convert: past customers, yesterday's visitors, people searching your brand name. The reported return looks excellent, but many of them were going to buy regardless.
Three things make the gap worse:
- Overlapping windows. Meta's default attribution setting credits a sale up to seven days after a click or one day after a view. Google Ads, your email tool and your affiliate network each have their own windows. The same order can be claimed by all of them.
- View-through credit. A sale a day after someone scrolled past an ad can count.
- Modelled conversions. Where tracking is limited, platforms fill gaps with their own estimates.
Platforms are not lying. They are answering a different question. For more on reading each channel's figures, our guides on whether Facebook ads are worth it and whether Google Ads are worth it cover break-even maths per channel.
In our experience, these two campaign types show the highest reported ROAS and the lowest share of incremental sales. They are cheap to test with a pause and the result often frees up budget straight away.
How do you measure incrementality?
Every method works the same way underneath: you compare people or places that could see your ads with similar people or places that could not, and the difference in sales is the incremental effect. The methods differ in how that comparison group is created, how much they cost, and how much you have to trust someone else's maths.
Platform conversion lift studies
Meta and Google both offer conversion lift studies. The platform randomly splits your target audience into a test group, which can see your ads, and a holdout group, which cannot. After the study, it compares conversions in both groups and reports the lift.
Random assignment is the gold standard, because the two groups are alike in every way except the ads. The catch is access. Meta offers Conversion Lift through the Experiments area of Ads Manager for some accounts, and through its account teams for others. Google Ads offers Conversion Lift studies, based on users or on geography, for accounts that meet its eligibility criteria. Minimum spend and conversion requirements apply on both platforms, they vary by account, market and study type, and they change over time, so check directly with the platform or your account representative before planning around one.
Two limits: a lift study measures one platform in isolation, and the platform still runs the maths. Check the result against your own sales data.
Geo holdout and matched-market tests
A geo test switches ads off, or changes spend, in some regions while keeping them running in comparable regions. You then compare sales in the two sets of regions against how they normally relate to each other. Because you measure sales in your own shop or CRM, the result covers everything: online orders, people who saw an ad and searched later, and sales that no pixel tracked.
A matched-market test is the more careful version: you choose regions whose sales have moved together closely in the past. Meta's open-source GeoLift tool helps with this if your analyst works in R, but at modest budgets a spreadsheet and careful design go a long way.
On/off or time-based tests
The simplest test of all: switch a campaign off for a period and see what happens to total sales, then switch it back on. It costs nothing to set up and anyone can do it.
The weakness is that time is a poor control group. Sales move week to week with weather, paydays, competitors' sales and newsletters. A pause test detects large effects, such as branded search adding almost nothing, and struggles with moderate ones.
PSA and ghost ads
These are two ways of building a clean holdout inside an ad platform. In a PSA test, the holdout group is shown an unrelated public service announcement instead of your ad, so both groups are reached in the same way. The downside is that you pay for those PSA impressions.
Ghost ads avoid that cost. The platform logs which holdout users would have been shown your ad, shows them nothing from you, and compares like with like. You will rarely build either yourself, but they explain why a proper lift study beats simply comparing people who saw ads with people who did not.
Marketing mix modelling
Marketing mix modelling, or MMM, uses statistics on two or more years of weekly sales and spend data to estimate how much each channel contributes, while accounting for seasonality, pricing and promotions. Meta's Robyn and Google's Meridian are both free, open-source MMM tools.
MMM suits brands with several channels, offline sales or TV, and a long history. For a brand spending €20k a month on two channels it is usually overkill, and where it is used, incrementality tests are the best way to calibrate it.
Marketing analytics services that test what your ads really add
We design and read incrementality tests on your own sales data, from a two-week branded search pause to a full geo holdout, and turn the result into a budget decision. If a test would be too small to tell you anything, we say so before you spend.
See our marketing analytics servicesHow do you run a simple geo holdout test?
Pick regions that normally sell in a stable ratio to each other, switch the ads off in a minority of them for three to six weeks, and compare actual sales in those regions with what the usual ratio predicted. At €10k to €50k a month this is usually the most informative test you can run without a platform's help.
Step by step
- Choose one question. For example: "How much revenue does our Meta prospecting actually add?" Test one channel or campaign type at a time, or you will not know what caused the result.
- Export sales by region. Pull at least eight to twelve weeks of daily or weekly orders and revenue by shipping region from Shopify or your CRM. In Germany that might be federal states or postcode areas, in the UK counties or larger regions.
- Pick holdout and control regions. Choose holdout regions that together make up roughly 15 to 30 percent of sales, and whose sales have tracked the control regions closely in the past. Avoid splitting cities from their commuter belts, such as Berlin and Brandenburg, because people see ads in one place and order to another.
- Set the baseline. Calculate the ratio of holdout sales to control sales for each week of the pre-period, and note how much it normally moves. That week-to-week movement is your noise.
- Exclude the holdout regions from targeting in the channel you are testing. Leave every other channel, price and promotion exactly as it is in all regions.
- Run three to six weeks. Three is a minimum for most online shops, longer if your products have a long consideration period or you sell fewer orders.
- Compare and calculate. Use the baseline ratio to predict what the holdout regions would have sold with ads, compare it with what they actually sold, and divide the gap by the spend you held back.
A worked example in simple numbers
The numbers below are an example, not a benchmark. Picture an online shop in Germany spending about €30,000 a month on Meta ads and asking whether that spend really pays back.
Baseline
Over the previous ten weeks, the chosen holdout regions sold €25 for every €100 sold in the control regions. Week to week, that ratio moved between €24 and €26.
During the test
Meta is switched off in the holdout regions for four weeks. That holds back about €6,000 of spend. The control regions sell €200,000, and the holdout regions sell €42,000.
The result
Expected holdout sales were €50,000 (25 percent of €200,000). Actual sales were €42,000. The ads were causing about €8,000 of sales in those regions, or 16 percent of their revenue.
The drop is well outside the normal range. If the ratio had landed at €24.50, inside the €24 to €26 band, you could not tell the effect apart from noise, and the honest conclusion would be "inconclusive", not "Meta does nothing".
How do you calculate incremental ROAS from a test?
Divide the incremental revenue by the spend that produced it. In the example, €8,000 of incremental revenue from €6,000 of held-back spend gives an incremental ROAS of about 1.33. That is the figure to compare with your break-even, not the ROAS shown in Ads Manager.
Now compare it with what the platform reported. Suppose Ads Manager showed a ROAS of 3.0 for the same campaigns over the same weeks. Dividing 1.33 by 3.0 gives about 0.44. In other words, roughly 44 percent of the revenue Meta was claiming was genuinely caused by Meta. You can use that factor, sometimes called an incrementality multiplier, to adjust the platform's reported numbers until you test again.
The last step is turning revenue into profit. If your contribution margin after product cost, shipping, fees and returns is 55 percent, each euro of spend returns about €0.73 of margin on the first order (1.33 multiplied by 0.55). On first orders alone, that spend loses money.
"Meta adds real sales, but less than reported. At an incremental ROAS of 1.33 it only pays back if new customers buy again. Our repeat data says a typical new customer brings another €40 of margin within a year, so we keep prospecting, cut retargeting, and retest in six months."
One subtlety: the result applies to the spend level you tested. To decide whether to scale, a test that increases spend in some regions answers better than one that removes it.
What can go wrong with an incrementality test?
Most failed tests fail for the same handful of reasons: they are too short, too small, run at the wrong time, or something else changed while they were running. All of these are avoidable with a bit of planning.
Do smaller brands need incrementality testing?
If you spend more than about €10k a month across two or more paid channels, yes, in a simple form. One or two well-designed tests a year will usually tell you more about where your money should go than any attribution report. Below that, or with very few orders, simpler checks are usually enough.
When it is not worth it yet
- Spend is low. Below roughly €10k a month, or for a single campaign spending a few hundred euros, the effect you are trying to measure is usually smaller than the normal noise in your sales.
- Orders are few. A shop with a handful of orders a day, or a B2B company closing a few deals a month, rarely has the volume for a clean regional split.
- There is only one channel. If all your paid spend is on one platform, your blended cost per new customer, total spend divided by new customers, already tells you most of what you need.
- Tracking basics are broken. If you cannot reliably export orders by region and date, fix that first.
- The result would not change anything. If you would keep the budget the same whatever the test said, do not run it.
For these brands, the practical substitute is watching blended numbers over time: total marketing spend against new customers and total revenue in your own data, plus the occasional pause test on branded search or retargeting. Our performance marketing services are set up that way: blended targets first, platform numbers second, and tests when the budget justifies them.
Check whether you have enough data for a geo test
- Export the last twelve weeks of orders from your shop with shipping region and date.
- Group them into two sets of regions, roughly 20 percent and 80 percent of sales, and calculate the weekly ratio between them.
- Look at how much that ratio moves from week to week. If it moves by more than a few percent, you will need a bigger holdout, a longer test or a clearer question.
This takes under an hour and tells you whether a test is feasible before you change anything in an ad account.
If you would rather test a specific platform, the same logic applies to Meta ads services and Google Ads management services: any agency or freelancer running your accounts should be able to propose a test, explain what it can and cannot show, and accept a result that makes their channel look smaller. For brands that want testing built into a wider plan across channels, a growth marketing agency should treat it as a routine part of deciding where the next euro goes.
Frequently asked questions
Not quite. A standard A/B test compares two versions of an ad or page, and both groups see something from you. An incrementality test compares seeing your ads with not seeing them at all. That is why it measures whether the spend adds sales, not which ad is better.
One that is above your break-even after margin and, if you have real repeat data, after the value of later orders. A shop with a 50 percent margin needs an incremental ROAS of at least 2.0 to break even on first orders. There is no universal good number, only your own maths.
For most smaller brands, one or two tests a year on the biggest channel or the most doubtful campaign type is enough. Retest after big changes in spend, offer or creative strategy. Testing continuously only makes sense at much larger budgets.
Email is easy: hold back a random slice of your list from a campaign or flow and compare revenue per subscriber. SEO is harder because you cannot switch it off by region, so most brands judge it on organic traffic and revenue trends over months rather than a holdout test.
Ask how they would design a test for your budget, how they will check it has enough volume, which sales data they will judge it on, and what they did last time a test made their own channel look worse. A credible answer uses your shop or CRM data and admits when a test would be inconclusive.
A little, in the holdout regions, and only to the extent the ads were genuinely working. You also save the spend you held back. If the test shows almost no drop, you have found budget you were wasting, which is the most valuable result of all.
What to do next
You do not need a data science team to find out what your ads really add. You need one clear question, your own sales data and a few weeks of patience.
- Add up what each ad platform claimed last month and compare it with total revenue in your shop. If the claims add up to more, testing is worth your time.
- Start with a two to four week pause on branded search or retargeting, then plan a geo holdout on your biggest channel using the steps above.
- If you want a second opinion on whether your data can support a test, ask for a free written audit. We send it within 48 hours, including "do not test yet" if that is what we find.