Maximize Your Wins With The Least Effort

Our Practical Takeaways For Running A/B Tests

1. If You Don't Test, You Surely Won't Gain Anything

This is a given. By choosing not to run any experiments your impact will be 0% at the end of your testing timeframe. Test to win. :)

2. Prebuild Experiments To Save Time

Experiments take time to build. You could prebuild tests when you have an experiment running in order to save time and raise your testing velocity. Don't just wait for a test to stop to and begin figuring out what to do next aftwards. Always have the next test ready.

3. Smaller Tests Take Less Effort With A Smaller Impact Potential

Usually we observe smaller change tests to require less effort, have a smaller impact, and require more time to run.

4. Bigger Tests Take More Effort With A Bigger Impact Potential

Usually we observe bigger change tests to require more effort, have a bigger impact, and require less time to run. Bigger change tests might be more effective in the beginning when there are many "high certainty" changes that could be optimized together. The effectiveness of bigger change tests may decay over time.