A/B Test Significance Calculator
Input visitors and conversions for control and variant. Get the p-value, z-score, confidence level, and winner — using a two-proportion z-test with pooled variance.
Total visitors who saw the control
How many converted in control
Total visitors who saw the variant
How many converted in variant
Want to plan your next test?
Use our sample-size + duration calculator to plan how many visitors you need before launching.
Plan a testFrequently Asked Questions
What is statistical significance in A/B testing?
Statistical significance is the probability that the observed difference between control and variant is real and not due to random chance. A 95% confidence level (p-value < 0.05) is the industry standard. Below 95% means the result could plausibly be noise; above means you can act on it with reasonable certainty.
What p-value should I aim for in A/B testing?
P-value < 0.05 (equivalent to 95% confidence) is the standard threshold for declaring a winner. For high-stakes tests (pricing, checkout flow), use p < 0.01 (99% confidence). Never use p < 0.10 (90%) for permanent decisions — false-positive rate is too high.
How does this significance calculator work?
This calculator uses a two-proportion z-test with pooled variance. Input the visitors and conversions for each variant; the calculator computes the z-score, two-tailed p-value, and confidence level. Formula: z = (rateB - rateA) / sqrt(p_pooled × (1 - p_pooled) × (1/visA + 1/visB)).
Why does my A/B test show 'not significant' despite a clear winner?
Sample size is too small for the observed effect. A 10% lift on 100 visitors per variant rarely reaches significance; the same 10% lift on 5000 visitors per variant usually does. Use the A/B test sample-size calculator first to plan your test, then this significance calculator to evaluate results.
Can I peek at the p-value before the test ends?
No. Stopping a test early when it 'looks significant' inflates false-positive rates dramatically — sometimes from 5% to 30%. Pre-commit your sample size before launching, run the test to completion, then check significance. If you must peek, use sequential testing methods (mSPRT) instead of fixed-horizon z-tests.