Tools/Sample Size Calculator

A/B Test Sample Size Calculator

Find the exact visitors needed per variant to detect your minimum effect at your chosen significance + power. Two-proportion z-test with pooled variance.

Current CR you want to improve

Smallest relative improvement you want to detect

Confidence level (1 - α)

Probability of detecting a true effect (1 - β)

Plan duration too?

Use the full A/B Test Calculator for sample size + duration in one place.

Plan + duration

Frequently Asked Questions

What sample size do I need for an A/B test?

Sample size depends on baseline conversion rate, minimum detectable effect (MDE), significance level, and statistical power. A page converting at 3% wanting to detect a 10% relative lift (3% to 3.3%) at 95% significance + 80% power needs ~30,000 visitors per variant. Lower baseline CR + smaller MDE = larger sample.

How does this sample-size calculator work?

Uses the standard two-proportion z-test formula: n = (Zα + Zβ)² × (p1(1-p1) + p2(1-p2)) / (p2-p1)². Inputs: p1 (baseline rate), p2 (p1 × (1 + MDE)), Zα (significance), Zβ (power). Output is sample size per variant; total sample = per-variant × number of variants.

What's the difference between MDE and effect size?

MDE (minimum detectable effect) is the smallest improvement you want to be able to confidently distinguish from noise. Effect size is the actual observed improvement. Plan with MDE; analyze with observed effect size. Smaller MDE = larger required sample. Common values: 5-20% relative MDE for established sites, 30-50% for new pages with limited traffic.

What statistical power should I use?

80% is the industry standard (20% false-negative rate). Use 90% for high-stakes tests where missing a real winner is costly. Use 70% only when traffic is severely constrained — accept that 30% of true winners will be missed.

Do I need a different sample for each variant?

Yes. Each variant needs the calculated sample-per-variant. A 2-variant test (A vs B) needs 2× the per-variant count; 3-variant test needs 3×. Multi-variant tests also need a Bonferroni correction to the significance level — divide alpha by the number of comparisons.