SEO experiments and migrations

An SEO experiment is a change to a set of pages, such as new titles, that you want to measure. A site migration moves pages to new URLs with redirects. Both questions compare traffic before and after a known date.

MeasureScale can test whether the pages’ traffic changed after a change went live. It can measure how much traffic redirected URLs kept. It can also store, change, remove and list experiments on a property.

How to read the answer

• A significant result means the change is larger than normal day-to-day noise. Then look at the size of the uplift and its range.
• Not significant does not mean no effect. If the test is flagged as weak, a small real effect can go undetected. Wait for more data before you decide.
• No estimate usually means a period has fewer than 14 covered days, or runs past the last settled day.
• Other events in the same period can move traffic. Control pages remove only site-wide movement.
• For a migration, the retention figure shows what share of the old traffic the new URLs kept. Start with the redirects at the top of the list.

What it answers

• Did clicks, impressions or sessions on the changed pages rise or fall after the change?
• Is the change larger than the normal day-to-day noise?
• How large a change could the test detect with this much data?
• After a migration, what share of the old URLs’ traffic do the new URLs keep?
• Which redirects lost the most traffic?

Covered days

MeasureScale first reads the property’s own daily totals for the two periods. It uses those totals to tell “no traffic” from “no data”.

• The covered span runs from the first to the last day with data. It never extends past the last settled day. A settled day is a day whose data Google has finished processing.
• A day outside the covered span is unknown. MeasureScale never counts it as 0.
• A covered day with no row for the pages counts as 0. Search Console sends no row for a day without traffic.
• For GA4, a covered day is a day when the property recorded sessions.

A period that is not fully covered gives no estimate. This happens, for example, when a period runs past the last settled day. Data windows and freshness explains settled days.

SEO experiments

Inputs

You name the pages when you ask, or you name a stored experiment.

• The test pages are the pages you changed. Give 1 to 100 full URLs.
• The control pages are optional. They are similar pages you did not change. Give up to 100. A page cannot be both a test page and a control page.
• A stored experiment can be named by its name or its id. It supplies any pages, control pages, source or metric you leave out.
• You must give the day the change went live, also with a stored experiment.

The dates you give always set the periods. MeasureScale does not take the dates from a stored experiment.

The two periods

• The post period starts on the day the change went live. By default, it runs 28 days, or to the last settled day if that comes first. You can give a different end date.
• The pre period has the same number of days. It ends on the day before the change went live.

If the pre period ends before the earliest date your plan allows, MeasureScale rejects the start date.

Matching pages

You can measure Search Console clicks or impressions, or GA4 sessions. Search Console clicks is the default.

• Search Console matches page URLs. MeasureScale lower-cases the scheme and host. It drops the query string and fragment, and removes a trailing slash. The path keeps its case.
• GA4 matches landing page paths. MeasureScale takes the path of each URL you give. It must equal a landing page path exactly, including case.

GA4 needs a GA4 property linked to the Search Console property. Without one, the result says so.

How it works

• MeasureScale compares the test pages’ daily traffic before and after the change went live.
• With control pages, it first removes site-wide movement that the control pages also show.
• It also removes a steady trend that runs through both periods.
• A statistical test then reports whether the remaining change is likely to be real.

The control pages need traffic in the pre period. Without it, MeasureScale rejects the control pages.

The result is significant when the p-value is below 0.05 and both periods have enough data. The test is flagged as weak when the smallest effect it can detect is above 10%. It is also flagged when that effect cannot be computed. A small real effect can then go undetected.

The result shows the uplift as a percent, with a 95% range. It also lists up to 20 of your pages that had no data in either period.

When there is no estimate

MeasureScale gives no estimate in these cases:

• A period has fewer than 14 covered days.
• A period is not fully covered.
• The page rows reached the request cap, so a missing day could read as 0.
• Google returned no data at all for the property in the two periods.

The result then shows the daily averages before and after, and the reason. It is not significant and is flagged as weak. It has no uplift, no range and no p-value.

Site migrations

Inputs

• You give the old and new URL pairs. Give 1 to 500 pairs.
• You must give the day the new URLs went live.
• You can give the whole span. The default is the last 28 settled days.

Give the pairs when you ask. Stored redirect maps are not available. A chain of redirects counts as separate pairs, so map each old URL to its final URL.

The migration date must fall after the start of the span and on or before its end. Otherwise, MeasureScale rejects the request.

• The before period runs from the start of the span to the day before the migration.
• The after period runs from the migration date to the end of the span.

The two periods can differ in length. Daily averages make them comparable.

How it works

MeasureScale normalizes every URL the same way as for experiments. It compares each old URL’s daily traffic before the migration with the daily traffic of the old and new URL after it.

• The after value includes the old URL, because some traffic can stay on it.
• When the old and new URLs normalize to the same URL, MeasureScale counts it once.
• Retention is after as a percent of before. It is empty when the old URL had no traffic before.
• Each pair gets a status from the decay classifier.
• A statistical test compares the daily totals of all mapped URLs, after against before.

Order

Pairs are sorted by the change in daily traffic, largest loss first. Ties are sorted by old URL.

When there is no comparison

MeasureScale gives no comparison in these cases:

• The before or after period is not fully covered.
• The page rows reached the request cap.
• The mapped URLs had no traffic in the before period.

The pairs then show only the daily averages before and after. The result has no retention, no status and no overall test.

Limits

• Before and after, not a controlled trial. Other events in the same period can move traffic. Control pages remove site-wide movement only.
• Independent days. The tests treat each day as independent. Traffic on one day resembles the next, so the p-values can look stronger than they are.
• A fixed control scale. The scaling of the control pages is treated as exact. Its own error does not enter the test.
• Request caps. MeasureScale reads at most 200,000 Search Console page rows for each batch of pages. When it reaches that cap, rows can be missing, and you get no estimate. A GA4 pull stops at 100,000 rows with the same effect.
• GA4 sessions from every channel. GA4 experiment sessions include every channel, not only organic search. GA4 experiment results do not say whether GA4 sampled the data.
• Search Console only for migrations. Migration comparisons read Search Console data only.
• Unmapped URLs. The migration compares only the URLs in the pairs. Other URLs on the site are not included.
• Stored experiments. A property holds at most 500 experiments.