Picture a review with six new video ads on the screen. One changes the opening. Another changes the offer. The music, presenter, product demonstration and call to action move at the same time. A week later, the dashboard says version two won.
What should the next brief repeat? Usually, no one can say.
That is not only a media-testing problem. It is a production problem. The ads were made to launch, but not made to leave behind an interpretable learning.
For global brands working with Chinese production and supply-chain teams, this distinction matters more as platforms broaden their measurement products. On October 5, 2026, Uber Advertising announced a U.S. restaurant product that combines its existing Sponsored Listings test-and-control experimentation with de-identified, aggregated bank-card transaction findings from Affinity Solutions. The stated aim is to measure effects across Uber Eats and physical restaurant locations. This proves that Uber launched a specific capability; it does not prove that every platform, category or campaign has solved cross-channel incrementality. It does show the question platforms are trying to answer: not merely who received credit for a sale, but whether advertising created business that would not otherwise have happened. Uber’s announcement explains the scope and data inputs.
Two reports, two different questions
Attribution asks: which ad, click or channel receives credit for a conversion that occurred?
Incrementality asks: which outcomes would not have occurred without the advertising intervention?
Attribution is useful for routine reporting, optimization and event feedback. Incrementality needs a credible counterfactual: a holdout, randomized control, matched market, geo experiment or another defensible estimate of what would have happened without the treatment.
The IAB and IAB Europe Guidelines for Incremental Measurement in Commerce Media compare experiments, model-based counterfactuals, econometric models and hybrid proxies across causal strength and scope. Their practical value is not an instruction to run an RCT for every campaign. It is a reminder to match rigor to the decision. A directional test may be sufficient for choosing between two hooks. A major market-entry or budget-allocation decision deserves a stronger counterfactual and tighter controls.
A creative brief should also describe the experiment
Google’s experiment guidance recommends a clear hypothesis tied to a business goal, one variable per experiment, metrics chosen in advance, stable baseline campaigns and a record of what was learned. Its Display & Video 360 guidance explicitly says to keep all arms identical except for the tested variable. Those practices do not automatically turn every A/B test into incrementality research. They do remove some of the most common production-side contamination.
ONCE proposes treating every testable asset family as a Measurement-Ready Creative Unit with six fields:
| Field | What the buyer should specify |
|---|---|
| Decision hypothesis | The real decision that changes if the result is positive, negative or inconclusive |
| One creative variable | A single identifiable change: hook, product action, proof, offer or call to action |
| Frozen base and treatment IDs | Control and variant names mapped to market, language, placement and live dates |
| Counterfactual and measurement owner | Who selects the platform experiment, holdout, matched market or directional comparison—and who owns interpretation |
| Observation window and intervention log | Start and end dates, plus changes in budget, audience, landing page, price or inventory |
| Learning record | What the result supports, what it does not support and what the next brief should retain or retest |
The point is not more paperwork for editors. It is a shared source of version truth across production, media and analytics. When AI can generate dozens of variants quickly, variable discipline becomes scarcer than variation. Our earlier article on the six controls brands need in automated creative systems covers permissions, approvals, change logs and exit rights; the additional requirement here is to make each permitted change measurable.
Four ways a convincing result becomes unusable
First, multiple creative variables changed together. If the hook, music, presenter and price all move, a winning treatment does not tell the next producer what to repeat.
Second, the baseline drifted during the test. A change in budget, targeting or landing page means the teams are no longer comparing the question they started with.
Third, the control was contaminated elsewhere. A held-out market may still see the same promotion through creators, email, retail media or an offline activation. The measured difference cannot be assigned cleanly to one treatment.
Fourth, the volume cannot support the intended decision. An inconclusive result, wide uncertainty or too few conversions may justify “we do not know yet,” not a ceremonial winner selected from the highest point estimate.
These invalidation conditions should be agreed before launch. Otherwise, the review process can turn random movement into a creative lesson.
Price the learning, not only the number of exports
Production estimates often divide cost by the number of videos delivered. But ten unrelated variations may produce only one ranking that nobody can explain. A more useful comparison for an always-on team is:
Cost per interpretable creative learning = (strategy + production + test media + data integration + analysis + version governance) ÷ completed tests that support an action
A completed test should have a credible baseline or control, a stable treatment, a documented observation window and a result that can change a defined decision. Use the project’s real costs; do not borrow an industry-average denominator.
This changes the production company’s economics as well. Selling more edits alone invites price compression. Owning hypothesis design, modular capture, version governance and learning records turns the deliverable from extra files into decision infrastructure. That does not make a production team a causal-inference specialist. Statistical design and final interpretation must remain with qualified measurement owners.
Choose the lightest method that can support the decision
For a short campaign with low spend, low conversion volume and limited downside, use directional creative comparisons. Control one variable, keep the record and avoid causal claims.
For an established campaign with enough volume, use a platform experiment or qualified measurement partner. Freeze the comparison window and connect the result to one creative or budget decision. Google’s Conversion Lift documentation is explicit about comparing treatment and control to estimate incremental conversions rather than attributed conversions.
For always-on, multi-market programs where allocation decisions are material, include geo holdouts, matched markets or another fit-for-purpose incrementality method in the annual measurement design. Cross-channel contamination, seasonality, pricing and inventory changes become part of the production and media calendar, not an afterthought.
OpenAI’s current advertising documentation illustrates why the distinction matters. Its conversion measurement guide covers click attribution, Pixel and Conversions API event flows, deduplication and modeled measurement. Separately, OpenAI describes geo-based incrementality work as an early-stage partnership effort in its October 2026 measurement announcement. Both capabilities can coexist. They should not be described as the same thing.
Before ordering the next batch, replace “How many videos do we need?” with three questions:
- Which decision is this batch meant to change?
- Which single creative variable may move in each comparison?
- Where will the result be recorded for the next brief?
Without those answers, more creative will often produce more reporting, not more learning. The durable asset is not the file that won once. It is a creative learning that another team can reuse, challenge and test again.
