Every number on this page is made up. It is real model output, but the brand, sales and spend are synthetic data we generated, so we also know the true answer and can show it.
A made-up CPG brand: 12 regions, 3 years of weekly store sales, 16 paid media channels (about $67M a year), plus price, promotion weeks, distribution and competitor activity. Fitted with Google's open-source Meridian engine on our own machine.
Sales back per $1 of spend, middle estimate and 90% range, after feeding the model three lift tests (Linear TV, Connected TV, non-brand search). Bars run from $0 to $5. Sorted by spend.
| Channel | Spend / yr | Back per $1 (90% range) |
|---|---|---|
| Linear TV lift test | $10.8M | $2.01 ($1.59 to $2.62) · true $1.80 |
| Search: non-brand lift test | $8.3M | $2.60 ($2.07 to $3.30) · true $2.40 |
| Paid social: performance | $8.0M | $0.74 ($0.21 to $2.09) · true $1.60 |
| Shopping / Performance Max | $6.2M | $1.47 ($0.44 to $4.52) · true $2.00 |
| Retail media: on-site | $5.3M | $1.71 ($0.78 to $3.33) · true $2.10 |
| Connected TV lift test | $4.6M | $2.17 ($1.71 to $2.71) · true $2.20 |
| Paid social: reach | $3.6M | $0.79 ($0.23 to $2.19) · true $1.40 |
| Search: branded | $3.5M | $0.91 ($0.20 to $3.01) · true $3.00 |
| Display & native | $3.1M | $0.80 ($0.22 to $3.02) · true $0.90 |
| Retail media: off-site | $2.5M | $0.93 ($0.23 to $3.39) · true $1.60 |
| Affiliate & partnerships | $2.5M | $0.94 ($0.29 to $3.27) · true $1.70 |
| Online video | $2.4M | $1.15 ($0.30 to $3.67) · true $1.50 |
| Direct mail & catalogs | $2.3M | $1.62 ($0.34 to $6.51) · true $1.00 |
| Out-of-home | $1.7M | $1.10 ($0.31 to $4.22) · true $1.10 |
| Audio | $1.3M | $1.46 ($0.33 to $4.39) · true $1.30 |
| Influencer / creator | $1.0M | $1.00 ($0.24 to $3.64) · true $1.20 |
Sales are not profit. This made-up brand keeps 40 cents of gross margin on each $1 of sales, so a channel needs about $2.50 back per $1 to pay for itself (made-up margin; a pilot uses yours). 1 of 16 channels clears that on the middle estimate (Search: non-brand); 0 clear it across their whole 90% range. True: 1 of 16 (Search: branded). Below break-even does not mean cut: the channel may still be worth its cost for reasons this model does not see (new buyers, retailer commitments), and its range may be too wide to say. It means the case for it has to be made on more than this number.
The same data and model with no lift tests. The biggest channels come out at about half their true return and the ranges are wide: the model is mostly repeating its starting assumptions. This is why an MMMflow pilot asks for your past tests on day one.
| Channel | Spend / yr | Back per $1 (90% range) |
|---|---|---|
| Linear TV | $10.8M | $0.98 ($0.36 to $2.34) · true $1.80 |
| Search: non-brand | $8.3M | $1.19 ($0.42 to $2.36) · true $2.40 |
| Paid social: performance | $8.0M | $0.62 ($0.21 to $1.94) · true $1.60 |
| Shopping / Performance Max | $6.2M | $1.13 ($0.32 to $2.95) · true $2.00 |
| Retail media: on-site | $5.3M | $1.76 ($0.81 to $3.34) · true $2.10 |
| Connected TV | $4.6M | $0.92 ($0.26 to $2.62) · true $2.20 |
| Paid social: reach | $3.6M | $0.78 ($0.23 to $2.28) · true $1.40 |
| Search: branded | $3.5M | $0.86 ($0.24 to $2.57) · true $3.00 |
| Display & native | $3.1M | $0.74 ($0.18 to $2.37) · true $0.90 |
| Retail media: off-site | $2.5M | $0.84 ($0.20 to $2.58) · true $1.60 |
| Affiliate & partnerships | $2.5M | $0.87 ($0.20 to $2.83) · true $1.70 |
| Online video | $2.4M | $1.15 ($0.30 to $3.23) · true $1.50 |
| Direct mail & catalogs | $2.3M | $1.63 ($0.32 to $6.56) · true $1.00 |
| Out-of-home | $1.7M | $1.05 ($0.26 to $4.15) · true $1.10 |
| Audio | $1.3M | $1.23 ($0.27 to $3.72) · true $1.30 |
| Influencer / creator | $1.0M | $0.94 ($0.25 to $3.41) · true $1.20 |
Upper funnel creates mid-funnel demand: people see the TV ad, then search the brand name. A second made-up brand built that way: Linear TV, Connected TV, Online video send part of their effect through branded searches, and paid branded search mostly catches searches that would have clicked the free listing anyway.
Read naively, it looks strong: $0.78 back per $1 ($0.23 to $2.09). With one brand-search holdout test it drops to $0.46 ($0.35 to $0.57). True: $0.40.
Read naively, Linear TV returns $0.43: the searches it causes get credited to search. With a geo lift test it is $1.89 ($1.52 to $2.40). True: $1.80.
| Channel | Naive read | Full funnel, with lift tests | Extra brand searches per $1k | True (return · searches) |
|---|---|---|---|---|
| Linear TV lift test | $0.43 | $1.89 ($1.52 to $2.40) | 196 (± 51) | $1.80 · 180 |
| Connected TV lift test | $0.70 | $2.12 ($1.68 to $2.66) | 248 (± 214) | $2.20 · 220 |
| Online video | $1.20 | $1.09 ($0.28 to $3.88) | not measurable yet | $1.50 · 150 |
Back per $1. A geo lift test reads the whole effect, searches included, so the full-funnel number is the one to budget on. The brand searches column shows how much of that runs through search. Online video had no lift test and its spend is too small to measure its search effect: test it next.
In this made-up brand about 12% of region-weeks are promotion weeks (15% off). Price and the promotion flag go into the model as things the brand controls, next to media, so a promotion week's extra sales are not handed to whatever media ran that week.
How it's worked out: expected sales as fitted, minus expected sales for the same weeks with the promotion off and price back to normal. True: $31.3M a year, inside the model's range.
Same data, fitted again with price and promotion left out of the model. In this made-up brand, promotion weeks fall at random, not alongside media flights, so leaving them out did not shift much credit to media (average return per $1 across all media, spend-weighted: $1.42 left out, $1.44 in; true $1.83). It made the model less sure: the typical channel's 90% range is $3.18 wide with promotions left out and $2.35 with them in. Real brands often run promotions and media together (launches, holidays). When we timed this brand's promotions with its TV flights and fitted again, TV's lift test kept its estimate in place either way; untested channels moved in both directions. With no TV lift test at all, the model did not hand TV the promotion credit either, but it got TV's size wrong whether promotions were in or out ($1.13 and $0.89 back per $1; true $1.80), because it fell back on its starting assumption. Promotions in the model plus lift tests is what keeps the credit where it belongs. The six biggest channels:
| Channel | Promotions left out | Promotions in the model | True |
|---|---|---|---|
| Linear TV | $2.00 ($1.59 to $2.55) | $1.97 ($1.52 to $2.49) | $1.80 |
| Search: non-brand | $2.62 ($2.04 to $3.30) | $2.55 ($2.02 to $3.19) | $2.40 |
| Paid social: performance | $0.72 ($0.22 to $2.05) | $0.63 ($0.19 to $1.61) | $1.60 |
| Shopping / Performance Max | $0.77 ($0.23 to $2.59) | $1.13 ($0.34 to $2.83) | $2.00 |
| Retail media: on-site | $1.18 ($0.23 to $3.65) | $1.71 ($0.78 to $3.48) | $2.10 |
| Connected TV | $2.16 ($1.67 to $2.71) | $2.12 ($1.69 to $2.74) | $2.20 |
A second made-up version of the brand runs four kinds of promotion weeks (15% off alone, or with a feature ad, an in-store display, or both) and has a trade spend file: the brand pays for the 15% off on every unit sold that week, plus a fee for each feature or display. Price, the promotion flag, feature and display go into the model as separate things the brand controls. All promotions: $21.8M a year of trade spend, $36.1M a year of sales from promotion weeks, $1.66 back per $1 (true $1.65).
| Promotion | Events (3 yrs) | Trade spend / yr | Sales from it / yr | Sales per $1 of trade (90% range) | Profit per $1 of trade, rough | True (sales · profit) |
|---|---|---|---|---|---|---|
| Price cut only | 89 | $7.5M | $11.2M | $1.49 ($1.45 to $1.53) | -$0.40 (-$0.42 to -$0.39) | $1.47 · -$0.41 |
| Price cut + feature | 55 | $6.0M | $10.5M | $1.75 ($1.72 to $1.78) | -$0.30 (-$0.31 to -$0.29) | $1.75 · -$0.30 |
| Price cut + display | 46 | $4.2M | $7.0M | $1.65 ($1.61 to $1.69) | -$0.34 (-$0.36 to -$0.32) | $1.65 · -$0.34 |
| Price cut + feature + display | 39 | $4.1M | $7.5M | $1.82 ($1.79 to $1.85) | -$0.27 (-$0.28 to -$0.26) | $1.82 · -$0.27 |
| Adding a feature ad (its fee only) | – | $2.0M | $7.5M | $3.74 ($3.58 to $3.90) | $0.50 ($0.43 to $0.56) | $3.75 · $0.50 |
| Adding an in-store display (its fee only) | – | $1.3M | $4.0M | $3.09 ($2.89 to $3.34) | $0.24 ($0.15 to $0.34) | $3.33 · $0.33 |
An event is one run of promotion weeks in one region. A type with fewer than about 12 events (8 for all three together) shows "not enough events yet" instead of a number: single events vary too much for fewer to settle its average (see the table at the bottom of this tab).
Profit per $1 uses a made-up 40% gross margin: 40 cents of each extra sales dollar, minus the $1 of trade. It is rough on purpose (it leaves out the margin lost on discounted units and what the retailer keeps); a pilot uses your own margins and your trade terms. At this margin every promotion as run here loses money (the four rows with a price cut): a price cut needs $2.50 of sales per $1 of trade to break even, and the best here brings back $1.82. Brands still run promotions for reasons this sum leaves out (retailer commitments, trial, keeping shelf space), but this is the number the revenue manager should see first. Counted on their own, the feature and display fees do pay back. The price cut alone returns least here because the brand pays the discount on units it would have sold anyway; the feature and display fees return more per dollar because they are small fees on top. These made-up promotions lift sales by the same share every time and don't overlap with media, which is why the ranges are so narrow: expect wider ranges on real promotions.
Real promotions are messier, so a third made-up version makes them harder: discounts of 10% to 25%, lifts that differ from event to event and region to region, about half of promotion weeks inside TV flights, and a dip the week after each promotion because shoppers stocked up. Fitted the same way as above, sales per $1 of trade looks fine, but it counts only the promotion week. Adding one more input, "the week after a promotion", lets the model take the dip off:
| Promotion | Events (3 yrs) | Promotion week only (90% range) | Net of the week-after dip (90% range) | Profit per $1, net, rough | True |
|---|---|---|---|---|---|
| Price cut only | 97 | $1.48 ($1.40 to $1.54) | $0.98 ($0.91 to $1.06) | -$0.61 | $1.02 net · $1.46 before the dip |
| Price cut + feature | 54 | $1.61 ($1.55 to $1.66) | $1.27 ($1.21 to $1.33) | -$0.49 | $1.27 net · $1.52 before the dip |
| Price cut + display | 42 | $1.63 ($1.57 to $1.70) | $1.31 ($1.24 to $1.38) | -$0.48 | $1.25 net · $1.53 before the dip |
| Price cut + feature + display | 36 | $1.72 ($1.67 to $1.78) | $1.47 ($1.41 to $1.53) | -$0.41 | $1.49 net · $1.72 before the dip |
Net of the dip, the price cut alone brings back about $0.98 per $1 of trade, not $1.48. 4 of 4 net ranges contain the true value; without the week-after input, 2 of 4 promotion-week ranges missed the true value (they were too narrow). One thing the harder data does not support: splitting a promotion into "what the feature ad added" and "what the display added". Those two came back at $3.02 and $4.26 per $1 of fee (true $3.73 and $3.57, both outside the model's ranges), so a readout reports the promotion types as run, net of the dip, and treats any feature-versus-display split as a question for a test.
The type averages above are tight because they pool every event. Revenue managers plan one event at a time, and on the harder made-up data single events spread widely around their type's average (net of the week-after dip, per $1 of trade):
| Promotion | Type average | Middle 80% of single events | Events under $1 back | Events to pool for a usable number |
|---|---|---|---|---|
| Price cut only | $1.02 | $0.74 to $1.49 | 47% | 12 |
| Price cut + feature | $1.27 | $0.82 to $2.03 | 26% | 12 |
| Price cut + display | $1.24 | $0.75 to $2.01 | 29% | 12 |
| Price cut + feature + display | $1.49 | $0.99 to $2.16 | 14% | 8 |
"Usable" here means the pooled number lands within 20% of the type's true average at least 8 times in 10. A single event read from weekly regional sales (this week against the other regions, minus the same gap the week after) is off by about $0.23 per $1 for a price cut alone (median). So a readout shows each type's number with how many events it rests on, and calls a type "not enough events yet" below about 12. At the made-up 40% margin, only 3% or fewer of events in any type cover their trade spend.
The model can't split a feature ad from a display on this data, so a pilot can add a small in-market test: at one discount (15% off), run three kinds of events side by side: price cut only, price cut plus feature, price cut plus display. Each fee's sales per $1 is the extra over the price-cut-only events, divided by the fee. Simulated on fresh made-up events, with the single-event read error measured above: with 16 events of each kind (48 in all), the feature fee reads $4.05 per $1 (90% of tests between $2.22 and $6.12) against a true $4.07. The test reads a fee above the $2.50 break-even 9 times in 10 with about 16 events per kind for the feature fee and 48 for the smaller display fee. It answers "does this fee pay", not "which one pays more": in this made-up world the two are too close to rank with any test a brand would run.
Channels without a lift test, ranked by how many dollars of uncertainty they carry (range width × spend). A geo test on the top one or two teaches the model the most per test.
| Channel | Spend / yr | 90% range per $1 | Dollars of uncertainty |
|---|---|---|---|
| Shopping / Performance Max | $6.2M | $0.44 to $4.52 | $25.2M |
| Paid social: performance | $8.0M | $0.21 to $2.09 | $15.0M |
| Direct mail & catalogs | $2.3M | $0.34 to $6.51 | $14.4M |
| Retail media: on-site | $5.3M | $0.78 to $3.33 | $13.6M |
| Search: branded | $3.5M | $0.20 to $3.01 | $9.9M |
Weeks 1 to 3 of a pilot: before any model runs, every file is checked and the totals are tied out to your own sales and spend. This is the same check run on the made-up brand's files.
| Check | Result |
|---|---|
| No personal data in any file (emails, phone numbers, customer IDs) | passed |
| Every file has the columns the model needs | passed |
| Weeks are ISO dates and start on a Monday | passed |
| Every channel maps to one of the 23 channel groups | passed |
| No blank spend (a channel that didn't run is 0) | passed |
| No missing weeks or region x channel rows | passed |
| Spend and delivery move together (cost per 1,000 impressions) | passed |
| Sales and media cover the same weeks | passed |
| Sales are summable (revenue, units), not rates | passed |
| Enough history (at least 2 years weekly for a regional model) | passed |
| Price, promotion and distribution are included | passed |
| Gaps in outside factors are filled on purpose, never with zeros | passed |
Same total budget, about $67M a year, and no channel moves more than 30% from today. Worked out with Meridian's budget optimizer on the lift-tested model, two ways. Turn on "Show the true answer" to see what each plan would really have done.
Move about $7.9M a year. The model expects $3.9M more sales a year ($462k to $7.0M, 90% range). It pays for this by cutting channels it has no test for, such as Search: branded (model $0.91 per $1, true $3.00) and Retail media: off-site (model $0.93 per $1, true $1.60). True result: -$455k. The model was wrong about the channels it cut; the best possible under the same rules was $1.8M.
Only the three lift-tested channels may move. The model expects $44k (-$405k to $496k): no real gain. They are already about balanced. True: -$87k (best possible $51k).
On this 12-region brand the best go-dark test reads these channels only to within about $0.80 to $2.80 per $1 (one standard error), and the tested-channels plan may now move six channels. The three new tests: Shopping / Performance Max read $4.33 (true $2.00); Paid social: performance read $0.77 (true $1.60); Direct mail & catalogs read $1.92 (true $1.00). The model now expects $3.3M ($661k to $7.8M). True result: -$769k (best possible $672k). Noisy tests taken at face value made the plan worse, not better (one simulated draw of test noise). The fix is test size, not more tests: the finest regions the brand has, matched region sets, 12 to 16 weeks, and a test's own range must clear the move before money moves.
Each test's read and its 90% range. A channel may gain budget only if its whole range sits above the average read of the six tested channels ($2.17), and may lose budget only if its whole range sits below it. Everything else stays where it is.
| Tested channel | Test read per $1 | 90% range | Rule | True |
|---|---|---|---|---|
| Linear TV | $1.84 | $0.75 to $2.92 | stays | $1.80 |
| Connected TV | $1.78 | -$0.63 to $4.20 | stays | $2.20 |
| Search: non-brand | $2.37 | $0.96 to $3.77 | stays | $2.40 |
| Shopping / Performance Max | $4.33 | $2.41 to $6.25 | may gain | $2.01 |
| Paid social: performance | $0.77 | -$0.61 to $2.14 | may lose | $1.60 |
| Direct mail & catalogs | $1.92 | -$2.65 to $6.50 | stays | $1.00 |
Here the rule lets Shopping / Performance Max gain and paid social performance lose. Both moves are wrong: these were unlucky reads (Shopping read high, paid social read low), though the rule kept the damage small. One set of reads proves little, so we repeated the whole test-then-move decision over 4,000 draws of test noise on the six tested channels (about $40M a year). Best possible under the same limits: $665k a year.
| Test size | Trust the reads: average result / yr | Loses money | With the rule: average / yr | Loses money | Moves anything |
|---|---|---|---|---|---|
| 12 regions | -$529k | 78% | $1k | 5% | 12% |
| 12 regions, matched | -$357k | 69% | $10k | 5% | 15% |
| 50 regions | $73k | 40% | $37k | 6% | 24% |
| 50 regions, matched | $122k | 35% | $45k | 6% | 26% |
| Assumed 15% error | $413k | 4% | $240k | 2% | 73% |
Fast stand-in, not a full Meridian fit for each draw: it knows each channel's true curve shape, so a real model does no better. Taken at face value, 12-region tests lose money most of the time; the rule almost never loses but rarely finds a move. Bigger, better-matched tests are what turn the rule into gains.
The table above uses a fast stand-in. Here the same decision runs through full Meridian fits and Meridian's own budget optimizer, on 6 sets of test reads sized like 50 matched regions. "Random" reads are drawn at random; "Picked" reads were chosen because the rule lets something move on them, so the rule gets exercised.
| Test reads | What the rule allows | Trust the model: expects / yr | With the rule: expects / yr | Stand-in, with the rule (true) |
|---|---|---|---|---|
| Random #1 | nothing moves (no test clearly on both sides of the average) | $2.3M / true $222k | $0k / true $0k | $0k |
| Random #2 | nothing moves (no test clearly on both sides of the average) | $827k / true $602k | $0k / true $0k | $0k |
| Random #3 | nothing moves (no test clearly on both sides of the average) | $2.8M / true $8k | $0k / true $0k | $0k |
| Picked #4 | Search: non-brand gains; Paid social: performance loses | $1.7M / true -$471k | $1.3M / true $295k | $290k |
| Picked #5 | Linear TV loses; Paid social: performance loses; Direct mail & catalogs gains | $1.9M / true -$129k | $1.2M / true -$213k | -$213k |
| Picked #6 | Search: non-brand gains; Shopping / Performance Max gains; Paid social: performance loses | $1.7M / true -$527k | $1.3M / true $149k | $170k |
Result: with the rule, full Meridian and the stand-in end up in the same place on every set of reads (true result within about $20k a year: $0, $0, $0, +$295k vs +$290k, -$213k vs -$213k, +$149k vs +$170k). The rule's one loss comes from a test that read high by chance (direct mail); about 1 in 20 rule-backed moves should go wrong this way. What full Meridian does NOT get right is its own forecast: it expects $0.8M to $2.8M a year from every trust-the-model plan, while the most any of them really made was $0.6M, three of them lost money, and its 90% range missed the true result on 3 of 6. So the readout shows each test's range and what the rule allows, never the model's expected gain as a promise. Six fits with short chains; a check, not a general number.
What we would tell this brand: don't move the big amounts yet. The money is in the untested channels, and the model can't yet say which way. Test the top ones on "Where to test next", then rerun the plan.
| Channel | Spend / yr now | Trust-the-model plan | True effect / yr |
|---|---|---|---|
| Direct mail & catalogs | $2.3M | +$0.7M (+30%) | $303k |
| Linear TV lift test | $10.8M | +$3.2M (+30%) | $2.8M |
| Search: non-brand lift test | $8.3M | +$2.5M (+30%) | $2.4M |
| Connected TV lift test | $4.6M | +$1.2M (+26%) | $1.2M |
| Audio | $1.3M | +$0.2M (+16%) | $119k |
| Online video | $2.4M | +$0.1M (+3%) | $47k |
| Out-of-home | $1.7M | -$0.0M (-2%) | -$19k |
| Shopping / Performance Max | $6.2M | -$0.3M (-5%) | -$317k |
| Retail media: on-site | $5.3M | -$0.3M (-6%) | -$331k |
| Influencer / creator | $1.0M | -$0.2M (-24%) | -$153k |
| Affiliate & partnerships | $2.5M | -$0.7M (-30%) | -$714k |
| Display & native | $3.1M | -$0.9M (-30%) | -$490k |
| Paid social: reach | $3.6M | -$1.1M (-30%) | -$806k |
| Paid social: performance | $8.0M | -$2.4M (-30%) | -$2.0M |
| Retail media: off-site | $2.5M | -$0.8M (-30%) | -$723k |
| Search: branded | $3.5M | -$1.1M (-30%) | -$1.7M |
Illustration only, on synthetic data; not a client result and not a claim about any real brand or channel. Short sampling runs (2 chains). Meridian is Google's open-source marketing mix model; MMMflow is not affiliated with Google.