ab 테스트 계산기

이 도구는 어떻게 사용하나요?

Statistical significance means that the observed difference between your control and variant is unlikely to have occurred by random chance. Typically, a p-value less than 0.05 (at 95% confidence level) is considered statistically significant. This means there is less than a 5% probability that the difference is due to chance alone.

사용량에 제한이 있나요?

도구가 업데이트되나요?

Test Significance analyzes data you have already collected from a running or completed A/B test to determine if the difference between variants is statistically significant. Sample Size Calculator helps you plan a new experiment by estimating how many visitors you need per variant to detect a meaningful difference with your desired confidence level and statistical power.

95% confidence is the standard for most A/B tests and is recommended for most use cases. 90% confidence is acceptable for exploratory tests where you want faster results. 99% confidence is used for high-stakes decisions where you need near certainty. The higher the confidence level, the more visitors you need to reach significance.

Statistical power (typically 80% or 90%) is the probability that your test will detect a true effect when one exists. A test with low power might miss a real improvement, leading to a false negative. Higher power means you are more likely to detect meaningful differences, but it also requires a larger sample size.

Yes, but with caution. Running multiple simultaneous tests on the same page or user flow can lead to interaction effects where the results of one test influence another. To avoid this, ensure your tests target different, non-overlapping user segments, or use a multi-variate testing framework that accounts for interactions between variables.