What Is A/B Testing? A Plain-English Explainer

You've probably changed a button from blue to green because someone had a hunch it would get more clicks. A/B testing is what you do instead of the hunch. So, in the plainest terms: what is A/B testing? It's showing two versions of the same thing to two random halves of your audience, then keeping whichever one performs better. No opinions, no seniority contest, no "well, on my screen it looks fine." You let the numbers settle it. This explainer walks through the idea from zero, no statistics degree required.

So what is A/B testing, in one sentence?

If someone asks you what is A/B testing and you have ten seconds, this is the answer: it's a way to compare two versions of something — a headline, a button, a whole page — by splitting your live visitors between them and measuring which one gets more people to do what you want. Version A is usually what you have now (the "control"). Version B is the one change you want to try (the "variant"). Everything else stays identical, which is the part beginners skip and the part that makes it work.

Think of it as a fair race. If you want to know whether a green button beats a blue one, you can't show the green button to Monday's visitors and the blue one to Friday's — Friday might just be a better day. You show both at the same time, to comparable random groups, so the only thing that differs is the button color. Whatever wins, wins because of the color and nothing else.

How A/B testing works, step by step

The mechanic is simple enough to write on a napkin:

1. Pick one thing to change. One. A headline, a call-to-action, an image. Change two things at once and you'll never know which one moved the needle.
2. Decide what "better" means before you start — more sign-ups, more purchases, more clicks. This is your success metric.
3. Split your traffic randomly. Half see version A, half see version B. Software (Optimizely, VWO, Google's tools, or your own) handles the coin flip.
4. Wait. Let enough people go through both versions that the result isn't just luck.
5. Read the result and ship the winner.

That "wait" step is where most homemade A/B tests fall apart, and it's worth its own section.

An example — made up, but typical

A SaaS pricing page has a button that reads "Start free trial," and it converts 4% of visitors. Someone suggests "Get started free" instead. Instead of arguing, the team runs a test: half of visitors see the old label, half see the new one, for two weeks. At the end, "Get started free" converts 4.9% and "Start free trial" holds at 4.0%. That's a measured lift, so the new copy ships. The decision took data instead of the loudest voice in the room.

Notice what the test did not tell them: why the new label worked, or whether an entirely different page would beat both. It answered one narrow question, and it answered it cleanly. That's the whole deal you're signing up for.

The one rule beginners break

The trap is impatience. You launch a test, check it after two days, see version B "winning," and call it. Then it stops winning in production and you're confused.

Early results are mostly noise. With small numbers, random luck can make either version look like the champion. You have to decide up front how many visitors you need and how long to run, then wait for that number before you look at who won. Calling a test early is the single most common way teams fool themselves — the "win" was a coin landing heads three times in a row. The fix is boring and it works: set the finish line before the race, and don't peek.

What can you A/B test?

Almost anything a visitor sees or clicks: headlines, button text and color, images versus video, form length, page layout, pricing presentation, email subject lines. The best candidates are high-traffic pages where a small percentage lift is worth real money, and changes you can describe in one sentence. If you can't state the change in a sentence, it's too big for a clean test.

What you can't usefully A/B test is anything with too little traffic to ever reach a reliable number, or a "change" so sweeping that a win tells you nothing about what specifically helped.

Where A/B testing stops being enough

A/B testing is brilliant at telling you which version performs better. It is silent on a harder question: can a person even tell what your interface is doing, and whether it's the right thing to do? You can spend a year optimizing a button on a page that confuses everyone who lands on it — and win every test while losing the customer.

That's the difference between optimization and clarity. A/B testing tunes what already exists; it can't tell you that the whole flow is asking the wrong thing. For that you need to watch real people struggle, not just count clicks — which is why testing and qualitative UX work belong together, not in competition.

Take it further

Now that you know what A/B testing is at a plain-English level, the next step is the discipline that makes tests trustworthy — sample size, significance, and the stopping rule this explainer only gestured at. That's all in AB Testing: The Complete Guide, and the wider A/B testing hub collects the strategy and case studies around it.

And when a test can't answer why users behave the way they do, the qualitative counterpart is the UX Clarity framework — the same lens we apply in a Full UX Audit to find what's worth testing before you spend the traffic.

Sources: NN/g — A/B Testing 101 · HBR — The Surprising Power of Online Experiments.

Ready to find out whether you're optimizing the right page — or perfecting the wrong one? Apply for a Full UX Audit →