Did That Campaign Really Lift Sales? An Email Marketer's Way to Find Out

Email marketing has a measurement problem, and most teams solve it by accident. The open rate comes from the ESP, the click rate from the same dashboard, and the revenue figure from the store. Three systems, three definitions, and one number that gets reported in the Monday meeting as though it settled the question.

It usually raises a better one: would those sales have happened anyway?

That question decides budgets. An email programme that can show it created demand gets funded next quarter. One that cannot gets cut. And a campaign reaching people who were already going to buy looks identical in the dashboard to a campaign that created demand, at least until you check how many of those buyers had visited the store that week, or were on their third look at the same product page.

What the numbers you already have can and cannot tell you

An email dashboard describes the email well and the business badly. Open rate tells you the subject line and your sending reputation got the message into an inbox and got it read. Click rate tells you the offer was interesting. Neither one tells you whether the person would have bought without the message.

The usual fix is a segment split: send the campaign to one part of the list, hold it back from another, compare the revenue. That is a real experiment and it is worth running, with one caveat that is easy to forget. The two groups have to be equivalent before you start, which means random assignment and a holdout big enough to matter. Splitting "recently engaged" against "everyone else" answers a different question, because those groups were never the same.

The other common comparison is worse: last month against this month. Seasonality, a sale, a competitor's promotion, a payday week and the weather all move the totals, and you cannot subtract any of them afterwards.

Everything downstream of the click is a variable too

A campaign is not one decision. It is a queue of them. Subject line, offer, send time, landing page, price presentation, shipping terms, checkout copy. The click only delivers the visitor to the rest of the queue.

That is where most email teams stop, and where the money is. Two emails can send identical traffic and produce different revenue depending on what the visitor finds after the click, and the things that move that are merchandising decisions on the page rather than copy in the message.

• The price shown, and the price after a promotion ends. A discount that lifts orders while shrinking margin is doing something very different from a price that simply converts better. Only a controlled comparison tells you which one you bought.
• The shipping promise. Free shipping over a threshold is the most common offer in ecommerce email, and the threshold is usually copied off a competitor instead of tested. If your campaign sends traffic to a store where the threshold sits just above the average basket, you are paying for clicks that will not convert.
• The landing page match. An email promising a bundle that lands on a generic collection page has a match problem, and you can fix it without touching the message.

Run the experiment where the traffic lands

The cleanest way to know whether an email worked is to run the test on the store rather than in the ESP.

On Shopify, price is the experiment most stores want first, because a small change applies to every order. Two price points run against two randomly assigned groups of live visitors at the same time, with revenue per visitor reported for each, so a promotional price and a permanent price get judged on the same scale.

Shipping is the second, and it works the same way. One threshold against another, or a flat rate against a threshold, with conversion rate, average order value and revenue per visitor reported for each. Those three numbers together decide whether a free shipping offer is growth or a quiet discount.

For stores that want the page-level questions in one place, Elevate A/B Testing runs price, shipping, theme, content, product image, split URL and checkout experiments on Shopify. Plans start at $49 a month with a 14-day free trial, and the price and shipping tests sit on the $99 Standard plan (Elevate pricing page, read 25 October 2026).

If the store sits on a platform whose rules will not let you hold two versions of a page at once, any campaign test is a guess. That applies to marketplaces and social shopfronts as much as to a self-hosted site. The measurement has to happen somewhere you control the page. CRO testing on Shopify is what gets you there when the store is your own: two versions of a page compared on live traffic, instead of reading tea leaves in a dashboard.

A practical sequence

• Fix deliverability first. No experiment survives a campaign that lands in spam. List hygiene, authentication and verification still come first.
• Define the metric before you send. Revenue per email delivered, or revenue per visitor to the landing page. Pick one, and pick it in advance.
• Hold out a random slice. Two to five percent of the list, randomly selected and held back from the campaign, gives you a baseline that is not contaminated by who happened to engage.
• Test one thing on the store. Run the landing offer as a controlled experiment instead of changing three things at once.
• Let the test finish before you read it. Decide the sample size in advance. A lead after 200 visitors tells you nothing yet.
• Log the decision as well as the outcome. What you rolled out, what you reverted, what is still open. Next quarter's plan depends on it.

Know the odds before you commit the quarter

Analytics-Toolkit ran the numbers on 1,001 A/B tests: 33.5% came back with a statistically significant positive result, and the winners averaged a 15.9% lift. Two tests in three come back inconclusive or flat. Teams that plan for that run a sequence of experiments and bank the wins. Teams that bet a quarter on one idea usually end up explaining a flat quarter.

The short version

• Open and click rates describe the email. The business shows up somewhere else.
• A before-and-after comparison measures the season. Use random assignment and a holdout.
• The offer, price and shipping terms on the landing page are all variables, and all testable.
• Revenue per visitor is the number that settles it. Order count can move the wrong way and still read as a win.
• Expect roughly one clear winner in three tests, and decide your metric before you send.