*Estimate sample upfront* for more confidence

Use a sample size calculator to figure out how many visitors you need based on conversion rate, desired outcome, and risk tolerance. A planned sample size goal will give you a * measure of confidence* in your data and an objective criterion for *stopping the experiment*.

*Example*: Your conversion rate is 20% and you want to see if your variations managed to lift that by at least 10%. The calculator will show that you can expect to detect a statistically significant lift with 6,000 visitors per variation (including control) 80% of the time. This is called "power analysis". If you ran the experiment with 1,000 visitors and saw a statistically significant 60% lift, you would know you're still far from your original plan of 6,000. You might be sceptical of this and decide to run your experiment longer, by which time the lift might drop to 20%.

Be realistic. Analysis can show that an experiment has no chance of success. In that case, adjust your tactics and avoid running a *futile experiment*.

To detect *smaller effects* or have a *higher chance of success*, you'll need more visitors.

If *time is constrained*, you'll need to increase risk of failure or aim for less definitive results.