Running your own geo holdout test: measure what your ads really contribute without expensive tooling

Switch your ads off in one region for a few weeks and compare what happens to revenue. That is how you measure incrementality with tools you already have, and this is the playbook.

A geo holdout test is the simplest way to measure what your ads truly contribute: you switch your campaigns off in one region for a few weeks and compare revenue there with the rest of your market. If revenue in that region drops hard, your ads were incremental. If it largely holds up, the platform was claiming revenue that would have come anyway. You do not need an expensive measurement platform for this, just region targeting, your store data and discipline.

What exactly is a geo holdout test?

Attribution in Ads Manager tells you which revenue the platform credits to itself. Incrementality is a different question: how much of that revenue would not have existed without the ads? The most honest answer comes from actually pulling the plug somewhere and watching what happens. A geo holdout does that in a controlled way: one region becomes the measurement group without ads, while the rest of your market keeps running as the comparison.

The principle is the same as the holdout studies run by large measurement platforms, just executed with tools every B2C advertiser already owns. You trade statistical refinement for simplicity, and for most decisions that simplicity is plenty: you want to know whether your biggest channel actually drives revenue, not publish a scientific paper.

How do you set up the test in five steps?

1. Pick a test region large enough to produce reliable numbers, but small enough that you can afford the revenue dip. A province or a cluster of postal areas usually works better than a single city.
2. Establish the baseline: how did revenue in that region compare to the rest of your market over recent months? That ratio is your measuring stick, not the absolute revenue.
3. Exclude the region from targeting across all campaigns on the channel you are testing. Half-excluding is not a test: any active campaign type still reaching the region pollutes your measurement.
4. Let the test run for two to four weeks without touching it, and change nothing major about your campaigns, prices or promotions during that window.
5. Afterwards, compare the revenue ratio during the test with the baseline. The deviation in the excluded region is your indication of what the ads really added there.

Schedule the test deliberately in a quiet, representative period. Not right before Black Friday, not in the middle of your peak season, and not in a week where you were already making big changes to your account. The more boring the period, the cleaner the measurement. Communicate internally what is running, too, so nobody launches a discount code halfway through or switches the excluded region back on because revenue there is slipping. That slip is not the test failing, it is exactly the signal you were looking for.

Which pitfalls ruin the measurement?

The setup is simple, the discipline is the real work. These are the mistakes we see most often:

• Choosing a region so small that normal weekly fluctuations are bigger than the effect you are trying to measure.
• Testing too briefly: in the first days after switching off, the region still benefits from earlier ad exposure, so the later weeks tell the real story.
• Running a promotion, product launch or PR moment during the test window, at which point you no longer know what you are measuring.
• Picking a region that is structurally different, like the area around your physical store or one with a strong local seasonal pattern.
• Forgetting that other channels keep running: that is fine, as long as they stay identical inside and outside the test region.

You only know what your ads are worth once you have pulled the plug somewhere.

What do you do with the outcome?

If revenue in the holdout region clearly drops, congratulations: your ads drive real revenue and you can scale with more confidence. If revenue largely holds, that is no reason to switch everything off, but it is a reason to rethink your mix. Often it turns out retargeting and branded traffic are claiming revenue that comes from your brand itself, and that your budget is worth more in prospecting, where it brings in net-new customers.

Treat the outcome as a decision tool, not a final verdict. One test measures one channel, in one period, at one budget level. Repeat the test a few times a year or after major changes to your mix, and put the results next to your platform numbers and your post-purchase survey. The more independent signals point the same way, the sturdier your decisions become.

So document the whole setup in one place: which region you excluded, in which period, which campaigns were off and what the baseline was. That feels like overkill for such a simple test, but six months from now you will want to put this outcome next to a fresh measurement. Knowing exactly what you did back then is the difference between spotting a trend and starting from scratch.

Conclusion

A geo holdout test is not an academic exercise, it is a practical audit of your biggest cost line: does the story your dashboards tell actually hold up? Pick a region, fix your baseline, pull the plug and look honestly at what happens. Translating those outcomes into sharper budget choices, more prospecting where it counts and a media buying approach that steers on real revenue instead of claimed revenue, that is exactly the work we do for brands every day. Want to know what your numbers really say? Book a call and we will gladly take a look with you.