A/B Test Calculator
Determine if your test results are statistically significant
Test Data
Conversion Rate Comparison
Detailed Metrics
Methodology: Two-Proportion Z-Test
This calculator uses a two-proportion Z-test to determine statistical significance. This is the standard approach for comparing two independent proportions (conversion rates).
How it works: We calculate a pooled proportion from both groups, then compute the standard error of the difference between the two conversion rates. The Z-score measures how many standard errors the observed difference is from zero (no difference).
Key formulas:
Pooled SE = sqrt(p_pool * (1 - p_pool) * (1/n1 + 1/n2))
Z = (p1 - p2) / SE
CI = (p1 - p2) +/- Z_critical * SE
Two-tailed test: We use a two-tailed test because we're interested in detecting differences in either direction (variant could be better or worse). The p-value represents the probability of observing a difference this extreme if there were truly no difference between groups.
Assumptions: This test assumes independent samples, random assignment, and sufficient sample size (generally n*p > 5 and n*(1-p) > 5 for both groups). Results are approximate for small samples.