Reading and Interpreting Statistical Results
What You’ll Learn
You’ll learn how to read and correctly interpret the statistical outputs from your split tests, including conversion rates, lift percentages, and statistical significance markers. Understanding these metrics is critical because misinterpreting results can lead to implementing changes that actually harm your business performance.
Key Concepts
Statistical results from split testing platforms present data in standardized formats that tell you whether observed differences between variants are meaningful or due to random chance. Most A/B testing tools display conversion rates for each variant alongside calculated metrics that help you understand the magnitude and reliability of differences. Learning to read these outputs correctly ensures you make data-driven decisions rather than relying on gut feelings or incomplete data interpretation.
- Conversion Rate Calculation: Conversion rate is calculated by dividing the number of conversions by total visitors and multiplying by 100. For example, if variant A receives 5,000 visitors with 150 conversions, the conversion rate is (150/5000) × 100 = 3%.
- Lift Percentage: Lift represents the percentage improvement of one variant over the control and is calculated as ((Variant Rate – Control Rate) / Control Rate) × 100. If your control converts at 3% and variant B converts at 3.6%, the lift is ((3.6-3)/3) × 100 = 20% lift.
- Statistical Significance Indicator: Most platforms display a significance level (often as a checkmark, green indicator, or percentage) showing the confidence level that results aren’t due to chance. A result marked as “95% statistically significant” means there’s only a 5% probability the difference occurred randomly.
- Sample Size and Duration: The statistical validity of results depends on having sufficient visitors and running the test long enough to account for day-of-week and seasonal variations. Tests should typically run for at least 2 weeks or until you reach 100+ conversions per variant to ensure reliable results.
Practical Application
Access your current or most recent split test results in your testing platform and locate the conversion rate, lift percentage, and significance level for each variant. Write down these numbers and calculate what the lift percentage means in absolute terms for your business—for instance, if you get 10,000 monthly visitors and see a 15% lift, that’s 150 additional conversions per month.