Introduction to A/B test sample-size planning
A/B test sample-size planning is less about chasing a magic number and more about making your experiment assumptions visible. When you know the current conversion rate, the lift you hope to detect, and the confidence you want before declaring a winner, this calculator gives you a fast way to turn those inputs into a traffic target you can actually compare with your launch plan.
That matters because small lifts can require a surprisingly large audience, while stricter confidence or higher power settings make the requirement even larger. The calculator helps you see those tradeoffs before you spend budget, hold back a release, or keep an experiment running past the point where it is still useful.
The sections below walk through the planning question, how to choose the inputs, what the built-in math is doing, and how to read the result without overreacting to a number that is either too small to trust or too large to be practical.
What A/B experiment question this calculator answers
The specific question behind A/B Test Sample Size Calculator is simple: how many visitors do you need in each arm of a conversion test before the result is worth believing? It answers the practical launch question, “Do I have enough traffic to detect the lift I care about?” before you ship the experiment.
In practice, that means balancing the baseline conversion rate, the minimum detectable effect, the confidence level, and the statistical power target against your traffic reality. If those settings do not fit the amount of traffic you can actually collect, the calculator makes that mismatch obvious early, before a test drags on with no clear endpoint.
This is useful for landing pages, email campaigns, onboarding flows, pricing screens, ad creatives, and any other experiment where the outcome is a conversion rate rather than a continuous measurement.
How to use this A/B sample-size calculator
- Enter Baseline Conversion Rate (Control, %): with the unit shown beside the field.
- Enter Minimum Detectable Effect (% Improvement): with the unit shown beside the field.
- Enter Confidence Level (%): with the unit shown beside the field.
- Enter Statistical Power (%): with the unit shown beside the field.
- Enter Test Type: with the unit shown beside the field.
- Enter Traffic Split: with the unit shown beside the field.
- Press Calculate Sample Size to refresh the visitor counts, total sample, duration estimate, and summary box.
- Review whether the answer is a believable visitor count and whether it moves in the direction you expect before you compare scenarios.
If you are comparing launch plans, keep a short note of the inputs you used so you can repeat the same experiment later or explain why one scenario needs more traffic than another.
Inputs: choosing baseline rate, lift, confidence, and power
The form on this page focuses on the few choices that usually dominate a conversion-rate experiment. The labels are already tied to the calculator's units, so your job is to bring in values that match the same percentage basis rather than mixing decimals, percentage points, and whole numbers in the same run. If your analytics tool reports a conversion as a decimal fraction, translate it into the percentage format shown here before you type it in.
- Baseline Conversion Rate (Control, %): the control rate from a recent reporting window that matches the traffic source you plan to test.
- Minimum Detectable Effect (% Improvement): the smallest relative lift that would actually change your decision. Smaller lifts are harder to detect and require more visitors.
- Confidence Level (%): how cautious you want to be before you call a winner. Higher confidence asks for more traffic.
- Statistical Power (%): how likely the test should be to catch a real lift if it exists. Higher power also increases the sample target.
- Test Type: one-tailed if you only care whether the variant improves the metric, or two-tailed if you also want to catch a drop in performance.
- Traffic Split: the share of traffic you plan to send to each arm. Match the setting to your launch plan so the experiment model and the live test stay in sync.
Any value you see already filled in should be treated as a starting suggestion, not a recommendation. Swap in your own campaign data before you rely on the estimate, and if you are unsure about the right assumption, rerun the calculator with a conservative version and a more ambitious version to see how wide the planning range becomes.
One of the most common mistakes in A/B planning is assuming a benchmark from someone else’s site will behave the same way on your audience. A control page with heavy brand recognition, a checkout page with returning customers, and a cold-traffic landing page can each produce very different baseline rates, so it is worth starting from your own data whenever possible.
Formulas: how the A/B sample-size estimate is built
The calculator's math starts with the baseline conversion rate and the relative lift you want to detect. It turns those into a control rate and a variant rate, then estimates how much separation there is between the two proportions. From there, the confidence and power choices determine how much evidence the calculator demands before it is satisfied.
In practical terms, that is why tiny lifts are expensive to measure. If the expected change is only a small bump above the control rate, the gap between the two groups is narrow and the estimate has to look at far more visitors before random variation stops dominating the picture. When you choose a stronger lift, the sample target falls because the signal is easier to see.
The page also keeps the planning idea simple enough to interpret at a glance. You can think of it as a conversion-rate comparison that grows stricter when you raise confidence or power and becomes easier when the expected improvement is larger. That is why the same calculator can suggest a modest audience for a bold redesign and a much larger one for a subtle copy change.
Traffic split matters in your experiment design even when the main sample estimate is dominated by rate, lift, confidence, and power. A balanced split is usually the easiest plan to reason about, while a lopsided split can be useful for operational reasons but should be chosen deliberately rather than by accident.
Worked example: sizing a landing-page test from your own baseline rate
Use your own analytics instead of placeholder numbers when you test this calculator. Start with the current control conversion rate, pick the smallest lift that would actually change a business decision, and compare the returned sample target with the traffic you can realistically collect during the period you care about. If the target is much larger than your normal flow, that usually means the hypothesis is too subtle for the traffic you have or the experiment window is too short.
When the answer looks too large, do not immediately assume the calculator is being conservative. It may be telling you that you need a more dramatic treatment, a longer running window, or a different decision threshold. When the answer looks comfortably small, double-check that the lift is not overly optimistic and that you have not accidentally used a rate from a different audience.
A useful way to think about the result is to ask three questions: does the lift matter enough to justify the test, does your traffic support the required sample, and will the test still be relevant by the time the sample is reached? If any of those answers is no, the sample-size estimate is doing its job by warning you before you commit time or budget.
Because the calculator works from the numbers you enter, the most honest “example” is the one tied to your own control page, your own campaign, and your own traffic pattern. That makes the output much more useful than a fictional walkthrough with made-up values that would never match your experiment.
How baseline conversion rate changes the sample-size estimate
The most important sensitivity in an A/B sample-size calculator is usually the baseline conversion rate. When the control rate is lower, each visitor contributes less conversion signal, so the calculator needs a larger audience to separate real lift from random fluctuation. When the control rate is higher, the same relative improvement is easier to see because the test collects more successful outcomes per visitor.
If you want to understand that sensitivity without a table of placeholder scenarios, change only one input at a time. Hold the desired lift, confidence, and power steady, then nudge the baseline rate up or down and watch how fast the required sample changes. That habit is more useful than memorizing a generic lower-or-higher rule because it tells you which assumption actually dominates your experiment plan.
The same logic applies to the minimum detectable effect. A tiny lift can turn a manageable test into a traffic-hungry one, while a larger lift may be easy to detect but only worth testing if the product change is meaningful enough to ship. The calculator helps you see that tension instead of guessing at it.
If you are planning a series of tests, this is also a good place to compare experiments by business impact. A low-traffic idea that changes a metric by a lot may be a better candidate than a carefully optimized tweak that barely moves the baseline, because the sample target will be much easier to satisfy in the former case.
How to read the A/B sample-size result
Once the calculator returns a number, treat it as a launch-planning guide rather than a verdict. Ask whether the per-variant visitor count fits a full business cycle, whether the total sample fits your acquisition budget, and whether a slightly stronger or weaker lift would change the plan materially. If the answer to those checks is yes, the estimate is probably good enough for planning.
After the calculation finishes, confirm that the result is a visitor count, that it is plausible for your traffic, and that it moves in the direction you expect when you tighten the settings or lower the expected lift. If you can answer yes to those checks, the output is doing the job of a practical estimate instead of a toy number.
If you want to keep the scenario, use the Copy Results to Clipboard button and paste the text into your notes, ticket, or experiment brief. That gives you a simple record of the settings you used without needing a separate export workflow.
Limitations and assumptions for A/B sample-size planning
No A/B sample-size calculator can capture every real-world detail. This one aims for a practical balance: enough realism to guide a decision, but not so much complexity that the page becomes difficult to use. Keep the following limits in mind when you plan the experiment:
- Input interpretation: read each label literally, because changing the meaning of a field changes the estimate.
- Percentage basis: enter control rate and effect values in the same percentage format that the form uses.
- Two-group comparison: the estimate is intended for a straightforward A/B conversion test rather than a multi-armed or sequential design.
- Rounding: the output is rounded to whole visitors and whole days, so small differences are normal.
- Missing factors: seasonality, audience quality shifts, device mix, delayed conversions, and other operational details are not modeled here.
If you use the output for compliance, safety, medical, legal, or financial decisions, treat it as a starting point and confirm the assumptions with authoritative guidance. The best use of the calculator is to make the experiment logic explicit: you can see which assumptions drive the sample target, change them openly, and explain the plan clearly to a teammate or stakeholder.
