Sample readout · made-up brand

What an MMMflow readout looks like

Every number on this page is made up. It is real model output, but the brand, sales and spend are synthetic data we generated, so we also know the true answer and can show it.

A made-up CPG brand: 12 regions, 3 years of weekly store sales, 16 paid media channels (about $67M a year), plus price, promotion weeks, distribution and competitor activity. Fitted with Google's open-source Meridian engine on our own machine.

16 of 16channels whose true return sits inside the model's range (with 3 lift tests)
14 of 16the same, without lift tests
0.70 → 0.37typical miss per $1, before and after lift tests

Sales back per $1 of spend, middle estimate and 90% range, after feeding the model three lift tests (Linear TV, Connected TV, non-brand search). Bars run from $0 to $5. Sorted by spend.

ChannelSpend / yrBack per $1 (90% range)
Linear TV lift test$10.8M$2.01 ($1.59 to $2.62) · true $1.80
Search: non-brand lift test$8.3M$2.60 ($2.07 to $3.30) · true $2.40
Paid social: performance$8.0M$0.74 ($0.21 to $2.09) · true $1.60
Shopping / Performance Max$6.2M$1.47 ($0.44 to $4.52) · true $2.00
Retail media: on-site$5.3M$1.71 ($0.78 to $3.33) · true $2.10
Connected TV lift test$4.6M$2.17 ($1.71 to $2.71) · true $2.20
Paid social: reach$3.6M$0.79 ($0.23 to $2.19) · true $1.40
Search: branded$3.5M$0.91 ($0.20 to $3.01) · true $3.00
Display & native$3.1M$0.80 ($0.22 to $3.02) · true $0.90
Retail media: off-site$2.5M$0.93 ($0.23 to $3.39) · true $1.60
Affiliate & partnerships$2.5M$0.94 ($0.29 to $3.27) · true $1.70
Online video$2.4M$1.15 ($0.30 to $3.67) · true $1.50
Direct mail & catalogs$2.3M$1.62 ($0.34 to $6.51) · true $1.00
Out-of-home$1.7M$1.10 ($0.31 to $4.22) · true $1.10
Audio$1.3M$1.46 ($0.33 to $4.39) · true $1.30
Influencer / creator$1.0M$1.00 ($0.24 to $3.64) · true $1.20

Sales are not profit. This made-up brand keeps 40 cents of gross margin on each $1 of sales, so a channel needs about $2.50 back per $1 to pay for itself (made-up margin; a pilot uses yours). 1 of 16 channels clears that on the middle estimate (Search: non-brand); 0 clear it across their whole 90% range. True: 1 of 16 (Search: branded). Below break-even does not mean cut: the channel may still be worth its cost for reasons this model does not see (new buyers, retailer commitments), and its range may be too wide to say. It means the case for it has to be made on more than this number.

The same data and model with no lift tests. The biggest channels come out at about half their true return and the ranges are wide: the model is mostly repeating its starting assumptions. This is why an MMMflow pilot asks for your past tests on day one.

ChannelSpend / yrBack per $1 (90% range)
Linear TV$10.8M$0.98 ($0.36 to $2.34) · true $1.80
Search: non-brand$8.3M$1.19 ($0.42 to $2.36) · true $2.40
Paid social: performance$8.0M$0.62 ($0.21 to $1.94) · true $1.60
Shopping / Performance Max$6.2M$1.13 ($0.32 to $2.95) · true $2.00
Retail media: on-site$5.3M$1.76 ($0.81 to $3.34) · true $2.10
Connected TV$4.6M$0.92 ($0.26 to $2.62) · true $2.20
Paid social: reach$3.6M$0.78 ($0.23 to $2.28) · true $1.40
Search: branded$3.5M$0.86 ($0.24 to $2.57) · true $3.00
Display & native$3.1M$0.74 ($0.18 to $2.37) · true $0.90
Retail media: off-site$2.5M$0.84 ($0.20 to $2.58) · true $1.60
Affiliate & partnerships$2.5M$0.87 ($0.20 to $2.83) · true $1.70
Online video$2.4M$1.15 ($0.30 to $3.23) · true $1.50
Direct mail & catalogs$2.3M$1.63 ($0.32 to $6.56) · true $1.00
Out-of-home$1.7M$1.05 ($0.26 to $4.15) · true $1.10
Audio$1.3M$1.23 ($0.27 to $3.72) · true $1.30
Influencer / creator$1.0M$0.94 ($0.25 to $3.41) · true $1.20

Upper funnel creates mid-funnel demand: people see the TV ad, then search the brand name. A second made-up brand built that way: Linear TV, Connected TV, Online video send part of their effect through branded searches, and paid branded search mostly catches searches that would have clicked the free listing anyway.

Paid branded search

Read naively, it looks strong: $0.78 back per $1 ($0.23 to $2.09). With one brand-search holdout test it drops to $0.46 ($0.35 to $0.57). True: $0.40.

TV gets its search halo back

Read naively, Linear TV returns $0.43: the searches it causes get credited to search. With a geo lift test it is $1.89 ($1.52 to $2.40). True: $1.80.

ChannelNaive readFull funnel, with lift testsExtra brand searches per $1kTrue (return · searches)
Linear TV lift test$0.43$1.89 ($1.52 to $2.40)196 (± 51)$1.80 · 180
Connected TV lift test$0.70$2.12 ($1.68 to $2.66)248 (± 214)$2.20 · 220
Online video$1.20$1.09 ($0.28 to $3.88)not measurable yet$1.50 · 150

Back per $1. A geo lift test reads the whole effect, searches included, so the full-funnel number is the one to budget on. The brand searches column shows how much of that runs through search. Online video had no lift test and its spend is too small to measure its search effect: test it next.

In this made-up brand about 12% of region-weeks are promotion weeks (15% off). Price and the promotion flag go into the model as things the brand controls, next to media, so a promotion week's extra sales are not handed to whatever media ran that week.

$31.2M a yearsales the model credits to promotion weeks (90% range $30.8M to $31.5M)

How it's worked out: expected sales as fitted, minus expected sales for the same weeks with the promotion off and price back to normal. True: $31.3M a year, inside the model's range.

What if promotions are left out?

Same data, fitted again with price and promotion left out of the model. In this made-up brand, promotion weeks fall at random, not alongside media flights, so leaving them out did not shift much credit to media (average return per $1 across all media, spend-weighted: $1.42 left out, $1.44 in; true $1.83). It made the model less sure: the typical channel's 90% range is $3.18 wide with promotions left out and $2.35 with them in. Real brands often run promotions and media together (launches, holidays). When we timed this brand's promotions with its TV flights and fitted again, TV's lift test kept its estimate in place either way; untested channels moved in both directions. With no TV lift test at all, the model did not hand TV the promotion credit either, but it got TV's size wrong whether promotions were in or out ($1.13 and $0.89 back per $1; true $1.80), because it fell back on its starting assumption. Promotions in the model plus lift tests is what keeps the credit where it belongs. The six biggest channels:

ChannelPromotions left outPromotions in the modelTrue
Linear TV$2.00 ($1.59 to $2.55)$1.97 ($1.52 to $2.49)$1.80
Search: non-brand$2.62 ($2.04 to $3.30)$2.55 ($2.02 to $3.19)$2.40
Paid social: performance$0.72 ($0.22 to $2.05)$0.63 ($0.19 to $1.61)$1.60
Shopping / Performance Max$0.77 ($0.23 to $2.59)$1.13 ($0.34 to $2.83)$2.00
Retail media: on-site$1.18 ($0.23 to $3.65)$1.71 ($0.78 to $3.48)$2.10
Connected TV$2.16 ($1.67 to $2.71)$2.12 ($1.69 to $2.74)$2.20

By promotion type, per $1 of trade spend

A second made-up version of the brand runs four kinds of promotion weeks (15% off alone, or with a feature ad, an in-store display, or both) and has a trade spend file: the brand pays for the 15% off on every unit sold that week, plus a fee for each feature or display. Price, the promotion flag, feature and display go into the model as separate things the brand controls. All promotions: $21.8M a year of trade spend, $36.1M a year of sales from promotion weeks, $1.66 back per $1 (true $1.65).

PromotionEvents (3 yrs)Trade spend / yrSales from it / yrSales per $1 of trade (90% range)Profit per $1 of trade, roughTrue (sales · profit)
Price cut only89$7.5M$11.2M$1.49 ($1.45 to $1.53)-$0.40 (-$0.42 to -$0.39)$1.47 · -$0.41
Price cut + feature55$6.0M$10.5M$1.75 ($1.72 to $1.78)-$0.30 (-$0.31 to -$0.29)$1.75 · -$0.30
Price cut + display46$4.2M$7.0M$1.65 ($1.61 to $1.69)-$0.34 (-$0.36 to -$0.32)$1.65 · -$0.34
Price cut + feature + display39$4.1M$7.5M$1.82 ($1.79 to $1.85)-$0.27 (-$0.28 to -$0.26)$1.82 · -$0.27
Adding a feature ad (its fee only)–$2.0M$7.5M$3.74 ($3.58 to $3.90)$0.50 ($0.43 to $0.56)$3.75 · $0.50
Adding an in-store display (its fee only)–$1.3M$4.0M$3.09 ($2.89 to $3.34)$0.24 ($0.15 to $0.34)$3.33 · $0.33

An event is one run of promotion weeks in one region. A type with fewer than about 12 events (8 for all three together) shows "not enough events yet" instead of a number: single events vary too much for fewer to settle its average (see the table at the bottom of this tab).

Profit per $1 uses a made-up 40% gross margin: 40 cents of each extra sales dollar, minus the $1 of trade. It is rough on purpose (it leaves out the margin lost on discounted units and what the retailer keeps); a pilot uses your own margins and your trade terms. At this margin every promotion as run here loses money (the four rows with a price cut): a price cut needs $2.50 of sales per $1 of trade to break even, and the best here brings back $1.82. Brands still run promotions for reasons this sum leaves out (retailer commitments, trial, keeping shelf space), but this is the number the revenue manager should see first. Counted on their own, the feature and display fees do pay back. The price cut alone returns least here because the brand pays the discount on units it would have sold anyway; the feature and display fees return more per dollar because they are small fees on top. These made-up promotions lift sales by the same share every time and don't overlap with media, which is why the ranges are so narrow: expect wider ranges on real promotions.

Harder promotions: the week after counts too

Real promotions are messier, so a third made-up version makes them harder: discounts of 10% to 25%, lifts that differ from event to event and region to region, about half of promotion weeks inside TV flights, and a dip the week after each promotion because shoppers stocked up. Fitted the same way as above, sales per $1 of trade looks fine, but it counts only the promotion week. Adding one more input, "the week after a promotion", lets the model take the dip off:

PromotionEvents (3 yrs)Promotion week only (90% range)Net of the week-after dip (90% range)Profit per $1, net, roughTrue
Price cut only97$1.48 ($1.40 to $1.54)$0.98 ($0.91 to $1.06)-$0.61$1.02 net · $1.46 before the dip
Price cut + feature54$1.61 ($1.55 to $1.66)$1.27 ($1.21 to $1.33)-$0.49$1.27 net · $1.52 before the dip
Price cut + display42$1.63 ($1.57 to $1.70)$1.31 ($1.24 to $1.38)-$0.48$1.25 net · $1.53 before the dip
Price cut + feature + display36$1.72 ($1.67 to $1.78)$1.47 ($1.41 to $1.53)-$0.41$1.49 net · $1.72 before the dip

Net of the dip, the price cut alone brings back about $0.98 per $1 of trade, not $1.48. 4 of 4 net ranges contain the true value; without the week-after input, 2 of 4 promotion-week ranges missed the true value (they were too narrow). One thing the harder data does not support: splitting a promotion into "what the feature ad added" and "what the display added". Those two came back at $3.02 and $4.26 per $1 of fee (true $3.73 and $3.57, both outside the model's ranges), so a readout reports the promotion types as run, net of the dip, and treats any feature-versus-display split as a question for a test.

One event is not the average

The type averages above are tight because they pool every event. Revenue managers plan one event at a time, and on the harder made-up data single events spread widely around their type's average (net of the week-after dip, per $1 of trade):

PromotionType averageMiddle 80% of single eventsEvents under $1 backEvents to pool for a usable number
Price cut only$1.02$0.74 to $1.4947%12
Price cut + feature$1.27$0.82 to $2.0326%12
Price cut + display$1.24$0.75 to $2.0129%12
Price cut + feature + display$1.49$0.99 to $2.1614%8

"Usable" here means the pooled number lands within 20% of the type's true average at least 8 times in 10. A single event read from weekly regional sales (this week against the other regions, minus the same gap the week after) is off by about $0.23 per $1 for a price cut alone (median). So a readout shows each type's number with how many events it rests on, and calls a type "not enough events yet" below about 12. At the made-up 40% margin, only 3% or fewer of events in any type cover their trade spend.

A test that settles whether a feature or display fee pays

The model can't split a feature ad from a display on this data, so a pilot can add a small in-market test: at one discount (15% off), run three kinds of events side by side: price cut only, price cut plus feature, price cut plus display. Each fee's sales per $1 is the extra over the price-cut-only events, divided by the fee. Simulated on fresh made-up events, with the single-event read error measured above: with 16 events of each kind (48 in all), the feature fee reads $4.05 per $1 (90% of tests between $2.22 and $6.12) against a true $4.07. The test reads a fee above the $2.50 break-even 9 times in 10 with about 16 events per kind for the feature fee and 48 for the smaller display fee. It answers "does this fee pay", not "which one pays more": in this made-up world the two are too close to rank with any test a brand would run.

Channels without a lift test, ranked by how many dollars of uncertainty they carry (range width × spend). A geo test on the top one or two teaches the model the most per test.

ChannelSpend / yr90% range per $1Dollars of uncertainty
Shopping / Performance Max$6.2M$0.44 to $4.52$25.2M
Paid social: performance$8.0M$0.21 to $2.09$15.0M
Direct mail & catalogs$2.3M$0.34 to $6.51$14.4M
Retail media: on-site$5.3M$0.78 to $3.33$13.6M
Search: branded$3.5M$0.20 to $3.01$9.9M

Weeks 1 to 3 of a pilot: before any model runs, every file is checked and the totals are tied out to your own sales and spend. This is the same check run on the made-up brand's files.

156 weeks2023-01-02 to 2025-12-22
12 regions17 channel groups
11.2%media spend as a share of sales
Ready0 items to fix first, 0 to mention
CheckResult
No personal data in any file (emails, phone numbers, customer IDs)passed
Every file has the columns the model needspassed
Weeks are ISO dates and start on a Mondaypassed
Every channel maps to one of the 23 channel groupspassed
No blank spend (a channel that didn't run is 0)passed
No missing weeks or region x channel rowspassed
Spend and delivery move together (cost per 1,000 impressions)passed
Sales and media cover the same weekspassed
Sales are summable (revenue, units), not ratespassed
Enough history (at least 2 years weekly for a regional model)passed
Price, promotion and distribution are includedpassed
Gaps in outside factors are filled on purpose, never with zerospassed
Also noted:
  • national-only controls in a regional model (shock_flag): Meridian may reject them with one knot per week
Tying out to your own totals needs your finance team's numbers, so it can't be shown on made-up data; in a pilot the files must match them within 1% before modeling starts. The lift tests on the other tabs were simulated separately.

Same total budget, about $67M a year, and no channel moves more than 30% from today. Worked out with Meridian's budget optimizer on the lift-tested model, two ways. Turn on "Show the true answer" to see what each plan would really have done.

Trust the model everywhere

Move about $7.9M a year. The model expects $3.9M more sales a year ($462k to $7.0M, 90% range). It pays for this by cutting channels it has no test for, such as Search: branded (model $0.91 per $1, true $3.00) and Retail media: off-site (model $0.93 per $1, true $1.60). True result: -$455k. The model was wrong about the channels it cut; the best possible under the same rules was $1.8M.

Move only what a test backs

Only the three lift-tested channels may move. The model expects $44k (-$405k to $496k): no real gain. They are already about balanced. True: -$87k (best possible $51k).

Then run three more tests the same size, and move what they back

On this 12-region brand the best go-dark test reads these channels only to within about $0.80 to $2.80 per $1 (one standard error), and the tested-channels plan may now move six channels. The three new tests: Shopping / Performance Max read $4.33 (true $2.00); Paid social: performance read $0.77 (true $1.60); Direct mail & catalogs read $1.92 (true $1.00). The model now expects $3.3M ($661k to $7.8M). True result: -$769k (best possible $672k). Noisy tests taken at face value made the plan worse, not better (one simulated draw of test noise). The fix is test size, not more tests: the finest regions the brand has, matched region sets, 12 to 16 weeks, and a test's own range must clear the move before money moves.

The rule: a channel moves only when its own test backs the move

Each test's read and its 90% range. A channel may gain budget only if its whole range sits above the average read of the six tested channels ($2.17), and may lose budget only if its whole range sits below it. Everything else stays where it is.

Tested channelTest read per $190% rangeRuleTrue
Linear TV$1.84$0.75 to $2.92stays$1.80
Connected TV$1.78-$0.63 to $4.20stays$2.20
Search: non-brand$2.37$0.96 to $3.77stays$2.40
Shopping / Performance Max$4.33$2.41 to $6.25may gain$2.01
Paid social: performance$0.77-$0.61 to $2.14may lose$1.60
Direct mail & catalogs$1.92-$2.65 to $6.50stays$1.00

Here the rule lets Shopping / Performance Max gain and paid social performance lose. Both moves are wrong: these were unlucky reads (Shopping read high, paid social read low), though the rule kept the damage small. One set of reads proves little, so we repeated the whole test-then-move decision over 4,000 draws of test noise on the six tested channels (about $40M a year). Best possible under the same limits: $665k a year.

Test sizeTrust the reads: average result / yrLoses moneyWith the rule: average / yrLoses moneyMoves anything
12 regions-$529k78%$1k5%12%
12 regions, matched-$357k69%$10k5%15%
50 regions$73k40%$37k6%24%
50 regions, matched$122k35%$45k6%26%
Assumed 15% error$413k4%$240k2%73%

Fast stand-in, not a full Meridian fit for each draw: it knows each channel's true curve shape, so a real model does no better. Taken at face value, 12-region tests lose money most of the time; the rule almost never loses but rarely finds a move. Bigger, better-matched tests are what turn the rule into gains.

Checked with the full engine

The table above uses a fast stand-in. Here the same decision runs through full Meridian fits and Meridian's own budget optimizer, on 6 sets of test reads sized like 50 matched regions. "Random" reads are drawn at random; "Picked" reads were chosen because the rule lets something move on them, so the rule gets exercised.

Test readsWhat the rule allowsTrust the model: expects / yrWith the rule: expects / yrStand-in, with the rule (true)
Random #1nothing moves (no test clearly on both sides of the average)$2.3M / true $222k$0k / true $0k$0k
Random #2nothing moves (no test clearly on both sides of the average)$827k / true $602k$0k / true $0k$0k
Random #3nothing moves (no test clearly on both sides of the average)$2.8M / true $8k$0k / true $0k$0k
Picked #4Search: non-brand gains; Paid social: performance loses$1.7M / true -$471k$1.3M / true $295k$290k
Picked #5Linear TV loses; Paid social: performance loses; Direct mail & catalogs gains$1.9M / true -$129k$1.2M / true -$213k-$213k
Picked #6Search: non-brand gains; Shopping / Performance Max gains; Paid social: performance loses$1.7M / true -$527k$1.3M / true $149k$170k

Result: with the rule, full Meridian and the stand-in end up in the same place on every set of reads (true result within about $20k a year: $0, $0, $0, +$295k vs +$290k, -$213k vs -$213k, +$149k vs +$170k). The rule's one loss comes from a test that read high by chance (direct mail); about 1 in 20 rule-backed moves should go wrong this way. What full Meridian does NOT get right is its own forecast: it expects $0.8M to $2.8M a year from every trust-the-model plan, while the most any of them really made was $0.6M, three of them lost money, and its 90% range missed the true result on 3 of 6. So the readout shows each test's range and what the rule allows, never the model's expected gain as a promise. Six fits with short chains; a check, not a general number.

What we would tell this brand: don't move the big amounts yet. The money is in the untested channels, and the model can't yet say which way. Test the top ones on "Where to test next", then rerun the plan.

ChannelSpend / yr nowTrust-the-model planTrue effect / yr
Direct mail & catalogs$2.3M+$0.7M (+30%)$303k
Linear TV lift test$10.8M+$3.2M (+30%)$2.8M
Search: non-brand lift test$8.3M+$2.5M (+30%)$2.4M
Connected TV lift test$4.6M+$1.2M (+26%)$1.2M
Audio$1.3M+$0.2M (+16%)$119k
Online video$2.4M+$0.1M (+3%)$47k
Out-of-home$1.7M-$0.0M (-2%)-$19k
Shopping / Performance Max$6.2M-$0.3M (-5%)-$317k
Retail media: on-site$5.3M-$0.3M (-6%)-$331k
Influencer / creator$1.0M-$0.2M (-24%)-$153k
Affiliate & partnerships$2.5M-$0.7M (-30%)-$714k
Display & native$3.1M-$0.9M (-30%)-$490k
Paid social: reach$3.6M-$1.1M (-30%)-$806k
Paid social: performance$8.0M-$2.4M (-30%)-$2.0M
Retail media: off-site$2.5M-$0.8M (-30%)-$723k
Search: branded$3.5M-$1.1M (-30%)-$1.7M

Assumptions and limits

  • History: 3 years of weekly data in 12 regions. Meridian's guidance is at least 2 years weekly for a regional model.
  • Starting assumptions: Meridian's default return-per-dollar prior for every channel, except the three with lift tests, whose prior is centred on the test's read (spread 0.15 on the log scale).
  • How settled the model is: the worst channel's R-hat is 1.06 with lift tests and 1.08 without (1.00 is fully settled; the usual bar is under 1.05). That is nearly settled: these are short sampling runs (2 chains, 250 kept draws each). A pilot runs longer chains until every channel is under 1.05, and the readout says so before any number is used.
  • What the model can't see: anything not in the files (a competitor's price cut, a retailer's shelf reset), and how much of branded search TV created, unless a brand-search holdout test is run.

Illustration only, on synthetic data; not a client result and not a claim about any real brand or channel. Short sampling runs (2 chains). Meridian is Google's open-source marketing mix model; MMMflow is not affiliated with Google.

Talk to us about a pilot