Most CFOs have asked their CMO the same question at some point: "What would happen if we cut this marketing spend?" It's a fair question. CPG marketing budgets now consume 19.5% of gross sales, and that percentage is climbing. Yet when pressed for proof of causation rather than correlation, many marketing teams struggle to answer.
“That 4x return on ad spend might look healthy in a dashboard, but if 60% of those conversions would have occurred organically, your true return is closer to 1.6x. You're essentially paying to reach customers who were already walking through the door.”
The evidence is concrete. When eBay ran a large scale experiment pausing their brand search ads, sales remained largely unchanged. They'd been spending millions to capture demand that already existed. The ads weren't creating growth; they were just claiming credit for it.
The Incrementality Gap
According to a 2024 survey by the Association of National Advertisers, 71% of advertisers now rank incrementality as their most important retail media KPI. This represents a fundamental shift in how marketing effectiveness is measured. Yet 36% of CPG brand marketers admit they struggle to prove investment incrementality, and a third say they're only measuring it at a basic level.
- CPG brands spent $52.3 billion on retail media in 2024 alone
- 85% now advertise across four or more retail media networks
- Amazon and Walmart dominate with 84.2% market share
- Neither platform makes incrementality testing straightforward
What Incrementality Testing Actually Reveals
Incrementality testing answers the question traditional analytics can't: what would have happened if you hadn't spent this money? It's the difference between correlation and causation, between claiming credit and proving it.
The methodology varies. Geo lift tests divide regions into treatment and control groups, measuring sales differences when advertising is increased or paused in specific areas. Holdout testing goes dark on ad spend for a subset of customers or geographies while maintaining normal spending elsewhere. Synthetic control methods create weighted combinations of untreated regions to simulate what would have happened without the intervention.
Recent incrementality experiments show iROAS (incremental return on ad spend) ranging from 253% to 1,609% across advertisers. Generally, anything above 2.0 indicates strong incremental performance. Below 1.0 means you're largely capturing demand that would have converted organically anyway.
Consider a real example: a beauty brand running campaigns across Amazon, Ulta, and Sephora used geo based holdout testing and discovered Amazon achieved 2.8x iROAS while other platforms underperformed. Without incrementality measurement, they would have continued splitting budgets based on last click attribution rather than actual incremental lift.
The Measurement Challenge
- 44% cite concerns about accuracy or reliability
- 43% struggle applying tests across different ad types and retailers
- 41% point to limited tools or technologies
- 37% cite lack of standardisation
Geo holdout tests carry substantial opportunity costs. Holding out 50% of the US from seeing ads could mean millions in lost revenue daily, so most tests use smaller holdout percentages matched carefully to treated regions. The tests also can't control for weather, local economic conditions, or seasonality, which introduces noise into the results.
Synthetic control methods offer more flexibility when clean holdouts aren't possible, using algorithmic approaches to construct comparable control groups from existing data. Meta's GeoLift library and other tools have made these techniques more accessible, though they still require statistical sophistication to implement correctly.
Making It Operational
Only 26% of in house marketers currently conduct incrementality testing themselves. Most rely on retail media platforms or third party measurement providers. Between Q1 and Q3 2024, the number of retail media networks offering marketing mix modelling access rose 50%, showing platforms are responding to advertiser demands.
The most sophisticated approach combines multiple methods. Use geo tests to validate marketing mix models. Run holdout experiments on high spend channels. Deploy synthetic controls when randomised testing isn't feasible. Track iROAS alongside traditional ROAS to understand the full picture.
“For CFOs evaluating marketing effectiveness, incrementality testing provides what correlation based metrics cannot: evidence of causation.”
It identifies which investments drive profitable growth and which destroy margin by paying for sales that would have happened anyway. In an environment where every percentage point of margin matters, that distinction is worth substantially more than a vanity ROAS metric.
The Executive Takeaway
If your marketing team can't answer "what would happen if we cut this spend?" with data rather than opinion, you're flying blind. Incrementality testing might reveal uncomfortable truths about wasted spend, but it also surfaces opportunities to reallocate budgets toward channels that actually drive new growth. The 71% of advertisers prioritising incrementality aren't chasing a trend. They're demanding proof that marketing spend creates value rather than just claiming credit for it.