An Advertising Film Strategy Is Not a Scorecard: What Evidence Should Make You Change It?

Advertising Film Strategy Research 10/10

An advertising film proposal can look complete in a meeting room. The objective is covered, the insight is elegant, the mood board is coherent, the production list is thorough, and the final slide contains a set of measurable outcomes. The harder question is often left unanswered: if the proposal is wrong, where would the team see it?

The direct answer is this: a reliable advertising film strategy does not compress marketing, psychology, creativity, production and data into one composite score. It turns each proposed route into a set of judgments that can fail. The team should state what it expects to change, why this expression could change it, what evidence the camera must provide, what the real media environment lets people encounter, and which observation would make the team choose another route.

This article presents ONCE's seven-layer falsifiable planning chain. It reorganizes evidence and production judgments from the first nine articles in this series. It is not a universal academic theory validated across categories, and it does not imply that ONCE has conducted randomized sales experiments or attribution studies for clients. Its purpose is more practical: before the budget is locked, it lets brand, marketing, creative and production teams compare two proposals that genuinely answer the same problem.

Research scope: why having data can still mean answering the wrong question

As of 6 October 2026, this article focuses on four bodies of material: comparisons between observational advertising attribution and randomized experiments; statistical power and uncertainty in large advertising experiments; algorithmic delivery as a confound in platform creative tests; and current official documentation for video experiments, brand lift and geographic incrementality. It also revisits the completed studies in this series on problem diagnosis, creative briefs, product demonstrations, source credibility, risk framing and campaign content systems.

Together, these sources warn that a large dataset does not guarantee that the right problem has been identified. In 2019, Gordon, Zettelmeyer, Bhargava and Chapsky compared randomized results with several observational models across 15 US Facebook advertising experiments. The data comprised roughly 500 million user-experiment observations and 1.6 billion impressions. The formal abstract reports that observational methods often failed to recover the randomized estimates even after conditioning on extensive demographic and behavioral variables. The finding supports the narrower claim that attribution records do not automatically provide a counterfactual—not that all observational data is useless. The publisher record and abstract describe the design and limits.

Lewis and Rao's 2015 analysis of 25 large field experiments shows another problem: individual sales are volatile relative to the cost of an ad exposure, so return estimates can remain imprecise even in very large studies. The study reports a median confidence interval on ROI wider than 100 percentage points, and notes that an informative experiment may require more than ten million person-weeks in some settings. The practical implication is not that smaller brands should stop measuring. It is that failing to detect a difference is not the same as proving no difference. With a limited budget, the question should become narrower rather than the noise being presented as certainty. The formal abstract and bibliographic record set out that evidence.

A 2025 article by Braun and Schwartz in the Journal of Marketing adds a third complication. In some online advertising A/B tests, random assignment to an experiment arm does not mean that each ad is ultimately exposed to the same mix of people. Delivery algorithms may optimize different creatives toward different users, confounding creative response with who received the exposure. This result should not be generalized to every platform or experimental setup, but it is enough to require a distinction between assignment, delivery and actual exposure. We reviewed the formal abstract and accessible article record; the full text was restricted.

Measurement therefore cannot sit at the end of a plan as a decorative acceptance criterion. It must begin with the first sentence that defines the problem.

The seven-layer falsifiable planning chain

Layer 1: marketing bottleneck — which judgment must change?

Falling sales are a business symptom, not an advertising-film task. The problem may be that the target audience cannot retrieve the brand, does not understand the product, cannot connect a feature to its own situation, does not trust the claim, perceives excessive risk, or already believes the message but faces price, distribution, supply, service or process friction.

The first line of the plan should not say “increase sales” or “build a premium image.” It should define an observable communication gap. For example: “Target procurement teams know this class of equipment but cannot judge how it reduces uncertainty around line-change downtime.” If interviews, search behavior, sales notes and presales questions show that buyers already understand and believe this point, the film should not keep spending budget on the same explanation. The first article in this series explains in more detail what an advertising film can and cannot change.

Layer 2: audience situation — what is the person doing when they watch?

The same person processes information differently in a commuting feed, at a launch event, in a sales presentation, in an internal procurement chat and on a search landing page. The plan should specify whether the viewer has motivation, time and sound; what prior attitude exists; who else participates in the decision; and what information this viewer must carry to the next colleague.

A falsifiable prediction can be concrete. If a proposal assumes that technical buyers will engage in effortful processing, an early test should at least show that they can restate the important conditions and ask questions about the evidence. If they remember only the emotion and setting but cannot articulate the claim, the information structure has not completed its intended task. “They liked it” is not an adequate defense.

Layer 3: proposition and evidence — what gives the camera the right to make the claim?

A proposition must connect three things: the audience judgment to be changed, an observable fact and the conditions under which the promise holds. Stronger claims carry a heavier burden of proof. Visual exaggeration, an actor's reaction and editing rhythm cannot substitute for real product conditions.

Article four described the creative brief as a promise-evidence contract. The integrated framework adds one more field: the disconfirmation condition. If a prototype cannot reliably reproduce the claim, the comparison is unfair, the expert lacks relevant knowledge, or the essential fact exists only in voice-over, the team should narrow the claim, obtain better evidence or select another proposition. Postproduction should not be asked to conceal the gap.

Layer 4: expression structure — format carries a mechanism, not an industry label

Brand stories, problem-solution films, demonstrations, comparisons, testimonials and corporate content are not items on an aesthetic menu. They organize different causal relationships. A story can place a product reason inside a human situation. A demonstration can make an invisible process observable. A testimonial can introduce source experience. A comparison can expose the conditions under which a difference holds.

Two proposals can be compared only after the communication task is fixed. If Route A uses a character story to answer why the change matters, while Route B uses a demonstration to answer whether the product can deliver it, the routes are not merely competing on creativity. They are competing on which bottleneck deserves priority. Each route needs its own prediction before the team decides whether to combine them. The more polished treatment should not win by default.

Layer 5: psychological mechanism — persuasion must reconnect to a product reason

Emotion, narrative transportation, processing fluency, mental simulation, source credibility and gain or loss framing explain intermediate processes. None of them automatically produces a purchase. Every mechanism needs a “then what?” clause.

“Make the viewer anxious” is not a complete mechanism. Anxiety can serve a decision only when it corresponds to a real risk, the film offers a credible and feasible response, and the brand is correctly identified as part of that response. Likewise, helping someone imagine use is not the same as establishing performance. In a high-risk decision, mental simulation still needs transparent demonstration conditions or evidence from an appropriate source.

Layer 6: production and media constraints — if the relationship cannot survive the film and exposure, it is not strategy

The strategy must pass through prototype readiness, locations, performers, approved claims, shooting days, horizontal and vertical composition, sound conditions, captions, review and version delivery. A causal relationship that works only in a 60-second horizontal film with sound cannot be assumed to survive in a six-second silent vertical placement.

Media conditions also change the estimand. Paid distribution can control part of the sequence, while organic recommendation cannot guarantee that the hero film comes first. If an algorithm delivers different assets to different people, the observed result cannot be attributed to creative alone. ONCE's brand-film production solution plans the hero, shorter versions, aspect ratios and follow-up assets together. The value is not simply more deliverables; it is ensuring that each version preserves attribution cues and completes its own communication task.

Layer 7: outcome measurement — write the estimand before choosing the metric

“Did it work?” can refer to at least five different questions:

• Process measures: Was the film seen, attended to, understood, correctly attributed and remembered with its evidence?
• Brand measures: Did ad recall, awareness, association or consideration change?
• Behavioral measures: Did search, site activity, trial, inquiry or purchase change?
• Attribution output: Which observed touchpoint received credit for behavior that occurred?
• Incremental effect: Would the outcome have been different without the advertising intervention?

These cannot impersonate one another. Completion rate can diagnose an opening or pacing, but it cannot establish incremental purchasing. Last-click attribution records a path but does not automatically construct the counterfactual. Brand-lift research estimates a specified brand outcome rather than sales return.

Google's current video-experiment documentation asks advertisers to start with a hypothesis, select a success metric and keep campaign settings aligned while changing creative. Its current Brand Lift documentation describes survey comparisons between exposed viewers and eligible people who were not exposed, for outcomes such as recall, awareness and consideration. These tools demonstrate that an experiment can be organized around a clear question. Availability, budget, delivery mechanics and sample size still determine which question it can answer.

Replace the total score with a three-column evidence ledger

To keep the seven-layer chain from becoming a more elaborate scorecard, record only three fields for every load-bearing relationship:

Direct evidence, adjacent evidence and assumption are not quality scores. Direct evidence is close to the current market, audience, product and outcome. Adjacent evidence requires explicit transfer conditions. An assumption must be tested in this project. A team can begin production with assumptions; it cannot present them as facts.

How to make two proposals genuinely comparable

Suppose an intelligent device for enterprise buyers needs an advertising film. Presales records suggest that buyers know the functions but remain uncertain about whether deployment will interrupt existing operations. The team proposes two routes:

• Route A follows a project manager through a tense working day and promises that change need not disrupt order.
• Route B demonstrates the crucial stages before, during and after deployment, including how the system rolls back when an exception occurs.

Do not begin by asking which route feels more premium. Return both to the same estimand: can the target buying group judge deployment risk and controllability more accurately?

Route A assumes that a human situation will increase value relevance and make the risk legible to nontechnical decision makers. It fails if viewers remember the pressure but cannot explain how the product reduces risk. Route B offers diagnostic evidence. It fails if the steps are clear but the audience still cannot see why the organizational change is worth making. With sufficient budget, the answer may be A to establish the meaning of the problem and B to provide proof. With constrained resources, the diagnosed bottleneck should determine priority.

That is the heart of the framework. It does not award the story eight points and the demonstration seven. It asks them to offer different mechanisms and disconfirmable predictions for the same problem.

Minimum validation: different budgets support different strengths of counterfactual

Before production: test comprehension and proof, not sales causality

Scripts, storyboards, animatics and rough cuts can be shown to people who approximate the decision situation. Use unprompted restatement, brand attribution, objection logging and evidence recognition to identify severe misunderstanding, missing proof and misattribution. This stage can eliminate a broken route. It cannot forecast commercial sales.

An online-first 2024 study on video-ad attention manipulated high and low attention in a laboratory and compared eye tracking, heart rate, skin conductance, EEG and self-report. Its accessible methods and results report that some commonly used attention measures did not consistently discriminate the manipulated conditions, while heart rate performed better in that task. The transfer lesson is limited but useful: even a process measure needs a precise construct and validation. Attention is not one interchangeable number.

During distribution: randomized holdouts answer a local causal question

When a platform or owned channel can randomly assign eligible units to treatment and control, and the arms remain comparable except for the creative intervention, the team can estimate an average treatment effect for that audience, delivery system and period. Actual exposure, noncompliance, cross-group contamination, power, conversion delay and concurrent promotions still matter. Randomization gives a local answer under a defined setup, not the permanent value of the creative.

When individual randomization is impossible: geo experiments need enough regions and a stable environment

Google's 2026 Meridian GeoX technical material and current official documentation position geo experiments as a publisher-agnostic option for incrementality measurement and support holdback, go-dark and heavy-up designs. The official implementation guidance also asks for comparable treatment and control geographies, the avoidance of parallel regional tests, limited changes during the study, and sufficient historical data. Contamination, budget and the minimum detectable effect all affect feasibility. Geo experimentation is not a small-sample shortcut. Too few regions, different trends, cross-region movement or concurrent promotions can make the result hard to interpret.

When only natural variation exists: a quasi-experiment must disclose its identification assumptions

Budget changes, policy shifts, channel launches or scheduling differences sometimes create natural comparisons. Difference-in-differences, synthetic control or interrupted time-series approaches may be useful. The team still needs to explain why the comparison represents the counterfactual, whether prior trends were comparable, and whether concurrent events or spillovers exist. Without these conditions, the analysis remains observational. Model complexity does not turn correlation into randomization.

Failure is not “the film did not work”; it tells you which layer to update

When a result fails to support the prediction, locate the failure before replacing the whole strategy:

• It was not seen: examine media opportunity, the opening and visibility.
• It was seen but the brand was not identified: examine attribution cues and the content system.
• The brand was identified but the message was not understood: examine information order, expression structure and demonstration.
• The message was understood but not believed: examine claim strength, evidence conditions and the source.
• It was believed but no action followed: examine risk, price, distribution, service and the next-step friction.
• Behavior rose but incrementality remains unknown: preserve the attribution result and build a more credible counterfactual.

The same result can have competing explanations. Low inquiry volume may mean the film failed to change a judgment. It may also mean the media missed the buying group, the landing page failed, supply was constrained or sales follow-up stopped. The framework is not there to give the team a fast excuse. It specifies the most important layer to change next.

Public-work observation: a framework can explain a choice, not manufacture a result

ONCE's public KTC Smart Display case page follows a movable display across living spaces rather than organizing the visible description around a feature list. From the text and imagery available on the page, the product's movement, the human situation and the change of space can be read as a filmable spine connecting proposition and execution.

The complete film and audio did not play in this research environment, and no public randomized experiment, brand-lift result or sales data was available. The case therefore cannot validate the framework or establish an effect on memory or sales. It shows how one planning relationship can enter production; effectiveness would still require evidence for the relevant audience, delivery and outcome.

Conclusion: the best proposal is not the hardest to challenge

Advertising film planning should not promise to eliminate uncertainty. It should put uncertainty in the right place: whether the bottleneck was diagnosed correctly, whether the audience is in the assumed situation, whether the claim has sufficient proof, whether the expression carries the mechanism, whether production and media preserve the important relationship, and whether the measurement answers the original question.

At approval, ask every route to complete the same five sentences: Which judgment should change? Why could this expression change it? What must the film show? Which process and outcome will we examine first? What evidence would make us choose another route? A proposal that answers those questions clearly is ready to enter budget and production.

If you are preparing a brand film, product film or campaign system, bring the business symptom, audience questions, publishable evidence, media conditions and available data into the first conversation. ONCE can turn them into comparable, filmable and testable routes before deciding the scale of production.

References reviewed through 6 October 2026

• Gordon, B. R., Zettelmeyer, F., Bhargava, N. and Chapsky, D. (2019). A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook. Marketing Science, 38(2), 193–225. DOI: 10.1287/mksc.2018.1135. Formal abstract, scope and bibliographic record reviewed.
• Lewis, R. A. and Rao, J. M. (2015). The Unfavorable Economics of Measuring the Returns to Advertising. The Quarterly Journal of Economics, 130(4), 1941–1973. DOI: 10.1093/qje/qjv023. Formal abstract and bibliographic record reviewed.
• Braun, M. and Schwartz, E. M. (2025). Where A/B Testing Goes Wrong: How Divergent Delivery Affects What Online Experiments Cannot (and Can) Tell You About How Customers Respond to Advertising. Journal of Marketing, 89(2), 71–95. DOI: 10.1177/00222429241275886. Formal abstract, stated scope and accessible article information reviewed; full text restricted.
• Hartnett, N. et al. (2024, online first). How to accurately measure attention to video advertising. International Journal of Advertising. DOI: 10.1080/02650487.2024.2435164. Accessible method and result descriptions reviewed.
• Google (2026). Meridian GeoX: An Open-Source Framework for Precision and Efficiency in Geo Experiments, together with current Google Ads documentation for geo experiments, video experiments and Brand Lift. The research page did not establish peer-review status; platform documentation supports current features and implementation conditions only.