Unequal markets need an explicit baseline in a geographic marketing test

A treatment region and a control region rarely have identical revenue levels. Subtracting their raw totals can mostly measure their difference in size. Even comparing their changes requires a decision about how the control should predict the treatment market in the absence of the advertising change.

Google Research's geo-experiment work uses geographic regions as experimental units and emphasizes the design of treatment and control conditions. The financial calculation should follow that design. A convenient control region selected after the results are visible can create a persuasive comparison without a credible counterfactual.

The examples below are deterministic illustrations of alternative comparison assumptions. A single pair of before-and-after totals does not establish that either assumption is correct or provide enough information for a reliable uncertainty estimate. Use the arithmetic to inspect the model, not to substitute for experimental design.

Suppose an invented treatment market has $500,000 of revenue in the baseline period and a control market has $300,000. During the test, treatment revenue rises to $550,000 and control revenue rises to $360,000. Looking only at the treatment market gives a $50,000 increase, or 10% growth.

The control grew 20%. If the approved comparison assumes the treatment would have grown by the same proportion without the intervention, expected treatment revenue is $500,000 multiplied by 1.20, or $600,000. Actual treatment revenue is $50,000 below that counterfactual.

Equivalently, the baseline scale factor is $500,000 divided by $300,000, or five-thirds. Multiplying the control's $360,000 test-period revenue by that factor gives $600,000. The calculation is transparent, but the stable proportional relationship is an assumption that needs evidence from the pretest history.

An additive difference-in-differences calculation subtracts the control's dollar change from the treatment's dollar change. Here treatment rises $50,000 and control rises $60,000, so the estimated difference is negative $10,000. That differs from the negative $50,000 proportional calculation.

The additive version assumes a comparable absolute change would have occurred; the proportional version assumes a comparable percentage change. Neither is automatically appropriate merely because it produces a more favorable result. The market relationship and the experiment's statistical design should determine the comparison.

A richer model may use multiple regions, pretest covariates or other supported structure. Whatever method is used, finance should be able to explain the revenue counterfactual and reproduce the resulting increment. A lift percentage without its baseline model is incomplete evidence for a budget decision.

Review whether the candidate treatment and control markets moved together before the intervention, including seasonal peaks, promotions and unusual local events. One baseline total can conceal very different weekly patterns. Keep the pretest window and any exclusions fixed before inspecting the treatment result.

Where the design permits it, test the proposed prediction rule on earlier held-out periods. Examine the size and direction of its errors and whether a few large regions dominate the fit. This evaluates the relationship used for the counterfactual; it does not by itself prove that every future shock will affect the markets similarly.

Do not rescale the control using post-treatment outcomes that the campaign could have affected. That can absorb part of the effect the experiment is trying to measure. Preserve the original weights or model version and document any necessary correction as a new analytical decision.

Keep changes in the commercial population visible. A new store or acquired customer portfolio can raise one region's revenue without changing demand at existing locations. Decide whether the experiment's outcome covers the entire business footprint or a stable comparable population. Apply that definition to both conditions and explain any opening, closure or acquisition that affects the comparison.

An advertising intervention can coincide with a regional stock shortage, store closure, price change or competitor event. Record those changes and determine whether they undermine the planned comparison. A negative lift estimate may reflect a valid advertising result, a separate shock or inadequate measurement; the totals alone cannot decide.

Exposure and purchasing can cross geographic boundaries. A customer may see an advertisement in one region and buy in another, or a national campaign may reach both conditions. The design needs an appropriate treatment of that spillover. Simply labeling regions treatment and control does not ensure their experiences remain separate.

Use outcome data with consistent geography and timing. Shipping address, billing address and location at exposure can assign the same purchase differently. Choose the definition appropriate to the design and retain the mapping rules so a data change does not appear as commercial lift.

Compare the estimated revenue difference with the incremental spending between conditions, then apply the relevant contribution economics. Keep the revenue estimate, cost assumptions and fixed campaign costs separate. A precise counterfactual revenue model still does not establish profit if the resulting sales have different fulfillment or return costs.

The geographical unit matters for uncertainty. Thousands of customer purchases inside a few regions do not automatically provide thousands of independent treatment assignments. Ask the analyst to use an inference method consistent with the region-level design and any matching or repeated test periods.

Present the estimate with its uncertainty and operating limits. A result from a particular market mix and spend change may not support the same effect at a much larger national budget. Show what the test directly identifies and what additional assumptions a wider rollout would require.

Save the region assignment, pretest observations, model or scaling rule, test-period outcomes, spending differences and known disruptions. Reconcile the revenue population to the finance source used for the contribution calculation. Missing regions or late data should remain visible in the release record.

The final explanation should identify the assumed untreated outcome, the observed difference and the evidence supporting that comparison. Management can then assess a geographical marketing test through the change it was designed to measure, rather than a raw growth figure that confuses market size with campaign effect.