Geo Holdout Experiments: A Practical Design for Link Campaigns

TL;DR. A geo holdout experiment splits your regions into treatment and control, runs the campaign in the treatment regions only, and reads the difference against a matched control to isolate causal lift. It is the closest thing marketing has to a clinical trial. But link building carries a structural catch that paid-media guides never mention: a backlink lifts your whole domain, not a region, so the authority effect leaks into your control group and cancels out. That means a geo holdout on a link campaign measures the geo-specific increment — regional coverage, local relevance, local demand — and is blind to the global authority lift. Read a flat result as “no regional edge,” never as “links do not work.” Run the Geo-Testability Decision Tree first; if your campaign passes, use the 7-step blueprint and the matched-market thresholds below; if it fails, drop to the within-site or time-based fallbacks.

The question a geo holdout is built to answer

Every link builder eventually faces the same interrogation from a client, a CFO, or a board: the rankings moved — which link did that? The honest answer is usually some version of “we cannot fully separate it,” because over the reporting window you earned four placements, published three pages, watched a competitor shed a chunk of their profile, and sat through a core update. That uncertainty is a theme we have written about at length in reporting link building results without overpromising. A geo holdout is the one design that can convert that shrug into a number.

The logic is borrowed wholesale from advertising measurement, where attribution has quietly collapsed under cookie loss and walled gardens. Instead of tracking users, you divide geography. You change something in the treatment regions, hold the control regions steady, and the gap between what the treatment regions did and what a matched control predicts they would have done is your incremental lift. It needs no cookies, no pixels, and no user-level tracking — which is exactly why it survived the privacy era intact. Google formalised the approach in 2011, when Vaver and Koehler introduced the geo experiment and its geo-based regression framework; the industry has been refining it ever since.

The appeal for link builders is obvious. If you could prove a digital PR push caused a measurable traffic lift rather than merely appearing alongside one, you would end the “correlation dressed as a win” problem that undermines so much link reporting. The catch — and it is a real one — is that links behave differently from ads inside a geo design. Most of this guide is about respecting that difference instead of pretending it away.

The concept underneath is a single counterfactual question: what would these regions have done anyway? Incrementality is the share of the outcome that happened because of the intervention rather than alongside it. The arithmetic is deliberately simple — lift is the treatment outcome minus the control outcome, and incrementality is that gap expressed as a fraction of the treatment: (treatment − control) ÷ treatment. If treated regions generate a 1.5% conversion rate and matched control regions sit at 0.5%, then two-thirds of the treated conversions were genuinely caused and one-third would have occurred regardless. Everything a normal dashboard shows you — last-click credit, sessions, referral counts — is a measure of association; only a holdout measures causation.

That is why a geo holdout sits above the two methods most teams already lean on. Attribution reports which touchpoints preceded a conversion, but proximity is not cause; it happily credits demand that was always going to convert. Marketing-mix modelling reads correlations across historical spend, which is useful for strategy but too coarse to isolate one campaign. A holdout is the only one of the three built on a controlled comparison rather than an observed one — which is precisely what makes it worth the design effort, and precisely why its structural limits for links matter so much.

Start here: the Geo-Testability Decision Tree

Before you design anything, you need to know whether your specific campaign can be geo-tested at all. Most cannot, and finding that out after a six-week test is an expensive way to learn it. Run every campaign through these five gates in order. The first gate it fails tells you which design to use — or whether to walk away.

Gates one to four are standard geo-experiment hygiene. Gate five is the one unique to links, and it is the one that quietly wrecks most attempts. The next section explains why.

The catch nobody mentions: domain authority contaminates your control

Here is the mechanism that separates a link geo-test from an ad geo-test, and it is worth slowing down for because almost every write-up on the subject gets it wrong.

When you run a paid campaign in treatment regions only, the ads reach only those regions. The control regions are genuinely untouched, so the difference between them is clean. When you earn a batch of links, those links raise your domain’s authority — and authority is not a regional quantity. As the measurement firm Recast puts it in its analysis of the structural limits of geo-testing, you cannot delist your site in California and leave it live in Nevada. Google does not hold a separate PageRank for the North of England and another for the South. The link you build in service of a “treatment region” lifts your rankings everywhere at once, including in every control region you were counting on to stay flat.

The contamination, stated plainly. In an ad geo-test, treatment gets the intervention and control does not. In a link geo-test, treatment gets the intervention and so does control — because the authority the links create is global. Your “control” is partly treated. That deflates the measured gap toward zero and means a geo holdout on links can only ever isolate the geo-differential: the extra lift in treated regions beyond the domain-wide lift that both groups received.

This changes what the test is for. A geo holdout on a link campaign does not measure “the value of the links.” It measures whether the links produced additional lift in the regions they were designed to serve — the part attributable to local relevance, regionally concentrated press coverage, or demand generated in a specific market. Those are real and worth measuring. But the largest single effect of most link campaigns — the global authority bump — is invisible to the design because it shows up equally in both arms.

The practical consequence is a reading rule you must internalise before you run anything: a flat geo result never means “links do not work.” It means “these links did not create a regional edge over and above the authority they gave the whole domain.” Miss that distinction and you will kill effective campaigns on the strength of a test that was structurally incapable of seeing their main effect. If you want the counterpart argument for how a single link’s authority actually moves a page, we model it in predicting ranking impact from a single backlink.

A quick illustration fixes the idea. Suppose your links create a true +10% domain-wide authority lift and an additional +6% regional edge where the coverage landed. The treated regions rise by roughly 16% in total; the control regions rise by 10% from the same authority. Your geo holdout compares 16% against 10% and reports a lift of about 6% — accurate, but only the regional slice. The 10% that both arms share is real value your links produced, and it is completely invisible to this design. Someone who did not understand the mechanism would look at “only 6%” and conclude the campaign underdelivered, when in fact the links did most of their work in a place the test cannot look. That is the trap, and naming it is the whole point of running gate five before you commit.

Which link activities are actually geo-testable

The contamination problem is not a reason to abandon geo holdouts — it is a filter that tells you which campaigns suit them. The rule of thumb: a link activity is geo-testable to the degree its mechanism is regional, not global. The more of the effect that runs through local coverage, local relevance, or local demand rather than raw domain authority, the more a geo holdout can see.

Notice the pattern: the campaigns that pass are the ones already discussed as regional and local plays in our overview of link building strategies that actually work. Digital PR with a regional angle is the single best fit, because press coverage is the most geographically concentrated thing link building produces.

Concentrating a campaign geographically is a deliberate act, not an accident of where coverage happens to land. To make a regional digital PR test valid, you build the regional angle into the asset itself: a data story broken down by region so that each outlet has a local hook, pitched to a target list of regional titles and the regional desks of national publications, and timed so the treatment regions get the push while the control regions get nothing. If your PR naturally scatters coverage evenly across the country, it is not a candidate for a geo holdout — the treatment boundary you need simply does not exist. The same logic applies to local link building: a citation and partnership drive concentrated in three cities gives you three treated markets and a clean control in the cities you left alone. The discipline is in the restraint — deliberately not working the control regions for the duration of the test, even when it feels like leaving links on the table.

The 7-step Geo Holdout Blueprint

Once a campaign clears the decision tree, here is the design, start to finish. Treat it as a checklist you can run on Monday morning; the numeric thresholds are drawn from established geo-experiment practice and are your defaults until your own data says otherwise.

Step 1 — Fix the hypothesis and the primary KPI

Write one testable sentence: “Concentrating our Q3 digital PR in [treatment regions] will lift regional organic clicks by at least [X]% versus a matched control.” Your KPI must be geo-visible and business-relevant — regional organic clicks or conversions, not referring domains and not total sessions. Pick the outcome that would actually change a budget decision, and commit to it in writing before launch so you cannot move the goalposts afterwards.

Step 2 — Assemble at least six months of pre-test geo data

You need a clean baseline to match markets and estimate natural variance. Six months is the practical minimum; more is better where seasonality is strong. Pull organic clicks and impressions by region from Search Console (the API exposes region-level breakdowns beyond the UI’s country view) and cross-reference conversions by region in GA4. The tools you already own for this live in our best link building tools guide; GeoLift and Google’s open-source packages, mentioned in Step 7, sit on top of that raw export.

Step 3 — Select and match markets

This is where tests are won or lost. Treatment and control regions must move together before the experiment begins. Require a high historical correlation on the primary KPI in the pre-period — BCG’s matched-market practice sets the bar at roughly 95% pre-test correlation, and it is a good target. Aim for 10–15 comparable markets per side where you can; more units mean more power to detect smaller effects. If your markets are systematically different in population, baseline trend, or seasonality, the result is confounded no matter how cleanly you run everything else.

Market-match scorecard. Score each candidate control region against your treatment set on four axes, 0–2 each (max 8). Keep only regions scoring 6+: (1) pre-period KPI correlation ≥90% = 2; (2) similar baseline organic volume (within ±25%) = 2; (3) matching seasonality shape = 2; (4) geographic isolation from treatment regions (no shared media market, no adjacency) = 2. Regions below 6 add noise, not power.

Step 4 — Run a power analysis and set your MDE

Power is decided before launch, never after. Simulate the confidence-interval width you would get under different test lengths and effect sizes, and design for at least 80% power to detect your minimum detectable effect. In practice a well-powered geo test targets a 2–5% MDE and runs four to six weeks. Shorter or smaller and you risk an underpowered null — seeing “no effect” only because you never gave the design enough signal to detect one. This is the single most common way geo tests mislead: they are read as evidence of absence when they are merely absence of power.

The intuition is worth making concrete, because it governs whether your test is worth running at all. MDE is the smallest true effect your design could reliably catch. It is driven by three things: the volatility of your KPI in the pre-period (noisier regions need a bigger effect to stand out), the number and quality of matched markets (more, better-correlated units tighten the estimate), and the duration of the test (longer windows average out day-to-day noise). If your power analysis says the design can only detect a 9% lift but the regional edge you plausibly expect is 4%, the test is dead on arrival — you would run six weeks only to produce an ambiguous null. The fix is to raise power before launch, not to reinterpret the result after: add markets, extend the window, or accept that this campaign is not measurable at the effect size it can realistically produce.

Step 5 — Assign treatment and control, then defend the boundary

Assign regions to arms and then protect the seam between them. Spillover — the technical name is a SUTVA violation, where treating one unit changes another unit’s outcome — pushes measured lift toward zero. Mitigate it the way ad-measurement teams do: exclude regions adjacent to treatment from the control group, prefer larger and more isolated units, and monitor control-region metrics during the test for unexplained movement. For link campaigns you have an extra, unavoidable spillover source — the authority wash-out from Section 3 — which no buffer can remove. Buffers handle media and audience bleed; only design choice handles authority bleed.

Step 6 — Run with integrity

The fastest way to void a geo test is to change something mid-flight. No budget shifts, no new targeting, no “while we’re at it” additions in either arm once the clock starts. Log every external event — a competitor promotion, a viral moment, a local news story, an unrelated algorithm update — because any of them can masquerade as lift. If an identifiable exogenous shock hits one region, the disciplined move is to exclude that region from the final analysis, decided by a rule you wrote down in advance.

Step 7 — Analyse with the right model for your geo count

Which estimator you use depends on how many geos you have. The three-rung ladder below covers essentially every case a link or PR team will meet. The deeper analysis methods behind these — difference-in-differences, Bayesian forecasting and single-campaign synthetic controls — each get their own treatment elsewhere in this measurement cluster; here you only need to pick the right rung.

The few-geo case matters most for link builders, because regional PR usually runs in one or two markets, not fifteen. That is precisely what Kerman, Wang and Vaver built time-based regression for. If you want a turnkey implementation, Meta’s open-source GeoLift package handles market selection, power analysis and synthetic-control inference end to end, and Google’s GeoexperimentsResearch implements both GBR and TBR. Both are R packages that assume an analyst on the team — budget for that, or keep the design simple enough to read from a matched-control chart.

However you estimate it, report the result as a range, never a point. A modern read expresses lift with a credible or confidence interval — for example, “+4.2% lift, 90% interval +1.8% to +6.5%.” The interval is the decision object, not the headline number: if it clears zero comfortably, you have a real effect; if it straddles zero, you have an inconclusive test, not a negative one; and an overly narrow interval from an overfit model should make you more suspicious, not less. Whether you act depends on where the whole interval sits relative to the effect that would change your budget.

Pre-registration template (fill in before launch). Hypothesis: “[campaign] in [treatment regions] will lift [KPI] by ≥[MDE]% vs matched control.” Primary KPI: [regional non-branded organic clicks]. Markets: treatment […], control […], scorecard ≥6. Pre-period: [6+ months], correlation [≥95%]. Window: [4–6 weeks], fixed dates. MDE / power: [2–5%] at [80%]. Exclusion rule: “drop any region hit by an identified external shock.” Decision: “if the interval clears +[X]%, [scale regionally / roll out nationally]; if it straddles zero, [hold / redesign].” Sign and date it — a test you cannot pre-commit to acting on should not run.

A worked example, with the honest interpretation

Make it concrete. A UK home-services brand commissions a regional data study — “the towns spending the most on home improvements in 2026” — and pitches it hard to press in the North of England, Scotland and Wales, deliberately holding the South and Midlands as an untouched control. The regions were matched on six months of Search Console data and cleared the scorecard at 6+; the test ran six weeks; the KPI was regional non-branded organic clicks to the site’s service pages.

The read. In the pre-period the two arms tracked at ~96% correlation. During the test, a time-based regression predicted the treatment regions would have generated about 41,000 organic clicks with no campaign. They actually generated about 44,050 — a modelled lift of roughly 3,050 clicks, or +7.4%, with a 90% interval spanning approximately +3% to +12%. The interval clears zero, so the regional effect is real, not noise.

What that number is — and is not. The +7.4% is the geo-differential: the extra demand created where the press coverage landed (regional referral traffic, local branded search, local relevance). It is not the total value of the links the study earned. Those links also raised the domain’s authority, which lifted the control regions too — so the domain-wide effect is sitting inside the 41,000-click counterfactual, invisible to this design. Report the 7.4% as “the regional edge from concentrating our PR,” and measure the authority effect separately with a whole-domain before/after synthetic control. Two questions, two designs; conflating them is the classic error.

Translate it for the CFO in the currency they use: 3,050 incremental monthly clicks at a 3% landing-page conversion rate and a £600 customer value is roughly £55,000 in annualised incremental value from the regional edge alone — before counting a penny of the domain-wide authority lift the test could not see. That framing is defensible precisely because it is narrow. It claims only what the design can support, which is what keeps a report credible when the next campaign underperforms.

When geo fails: within-site and time-based fallbacks

Plenty of link campaigns fail the decision tree at gate one or gate five — the mechanism is global authority with no regional angle. Geography cannot help you there, but two other holdout logics can.

The within-site (matched-page) holdout

Instead of splitting regions, split pages. Choose a set of comparable target pages, point your link building at half (treatment pages) and hold the other half untouched (control pages), and compare ranking and traffic trajectories. This sidesteps the authority-contamination problem because you are testing page-level effects within one domain, where links genuinely do favour specific URLs. We have seen this design work cleanly in practice: the split-group internal-equity experiments documented in our analysis of PageRank sculpting used exactly this test-vs-control-page structure to isolate the effect of concentrating equity. The main caveat is contamination in the other direction — internal links and shared authority mean treated pages can lift control pages — so keep the two sets topically and structurally separate.

To run one cleanly, pair pages before you start: match on current ranking band, search volume, page type and existing link profile, so each treated page has a genuine twin in the control set. Aim for at least eight to ten pairs — the same “more units, more power” logic applies here as in the geo version. Point new links only at the treated pages for a fixed window, hold the control pages completely untouched, and compare the two sets’ movement in average position and organic clicks. Because both sets live on one domain under one authority profile, the design controls for the domain-wide factors that muddy a naive before-and-after on a single page. Its blind spot is the mirror image of the geo test’s: it sees page-level effects cleanly but cannot isolate anything that moves the whole domain at once.

The time-based (whole-domain) holdout

For a genuinely national authority play, the counterfactual is not a region or a page — it is time. Model what the whole domain would have done absent the campaign, using its own pre-period trend and, ideally, a synthetic control built from comparable competitor or category signals, then read the divergence after launch. This is the natural home for measuring the domain-wide authority lift that a geo holdout deliberately cannot see. It is less clean than a randomised geo split — external shocks and seasonality are harder to rule out — which is why it pairs best with the disciplined patience our data on how long link building takes argues for: the signal takes months to separate from the noise.

Reading the result without fooling yourself

Geo holdouts fail quietly, in ways that look like clean answers. Watch for these five, which account for most misreadings:

1. Reading the geo-differential as total ROI. The headline error. The number is the regional edge, not the value of the links. Never let it stand in for the whole campaign’s worth.
2. The underpowered null. “No significant lift” from a small, short test is not evidence the campaign failed; it is evidence the test could not detect the effect you were looking for. Power is set before launch, and a null only means something if the design had the power to find a real effect.
3. Spillover deflation. Bleed between adjacent regions — media, audience, or authority — compresses lift toward zero. If treatment and control are not genuinely isolated, expect an underestimate and say so.
4. External shocks. A local competitor promotion, a regional news event, or a store opening in one arm confounds everything. Monitor for them, and exclude affected regions by a pre-declared rule rather than a post-hoc hunch.
5. Mid-test contamination. Any change to either arm after the clock starts — a stray outreach push, a new landing page, a budget tweak — breaks the comparison. Integrity violations are the quiet killer of otherwise sound tests.

The discipline that prevents all five is pre-registration: before launch, write down the hypothesis, the primary KPI, the MDE, the test window, the exclusion rules, and the decision the result will drive. A geo test you were unwilling to act on should not have been run. And a result you can only interpret after seeing it is not a measurement — it is a story. For the broader benchmark data that contextualises any lift you find, our link building statistics for 2026 is the reference to keep open beside your analysis.

Three questions teams always ask

“Can I geo-test if I only work one region?” Yes, but not with a region-vs-region split — you use a synthetic control instead. With one treated market and a pool of untreated candidate regions, GeoLift-style methods weight the candidates into a synthetic clone of your treated region and read the divergence after launch. It is the standard answer to the small-geo reality that dominates link and PR work, and it is exactly why time-based regression exists.

“How long before I can trust the result?” Four to six weeks of treatment is the usual floor, but that is a power statement, not a calendar rule: you run until the design has enough signal to detect your MDE, which you settled before launch. Ending early because the first fortnight looks flat is how underpowered nulls are manufactured. This is a different clock from the ranking-impact timeline — links can take months to move rankings — so measure a fast-moving proxy such as regional organic clicks rather than waiting on the slow one.

“What if the test comes back flat?” First, confirm it was powered to find the effect you cared about; an underpowered flat is uninformative. If it was well-powered and still flat, the honest reading is narrow and useful: the campaign produced no regional edge beyond the domain-wide authority it built. That is a verdict on regional concentration, not on link building — and it might rightly push you toward a national design next time rather than away from links altogether.

The Monday-morning version

You do not need an R package or a data scientist to take the first real step. Open Search Console, export the last six months of organic clicks split by region, and run your next planned campaign through the Geo-Testability Decision Tree. If it is a regional PR or local-link play, you have a genuine candidate for a holdout — match your markets, set an MDE, and measure the regional edge honestly. If it is a national authority play, you have just saved yourself a wasted test and can reach for the within-site or time-based design instead.

The whole point of causal measurement is not to produce bigger numbers — it is to produce true ones, and to know exactly what each number can and cannot bear. A geo holdout, used on the right campaign and read with the authority-contamination caveat firmly in mind, is one of the few tools in link building that can move a claim from “the rankings went up around the time we built links” to “our regional PR caused a measurable, bounded lift we can defend line by line.” In a discipline drowning in correlation, that is worth designing carefully for. Understanding what link building is and how authority actually accrues is the foundation; proving it caused the lift is the frontier this cluster is built to map.