Try to find the biggest improvements before time runs out. Prebuild tests quickly to avoid wasting time.
Start by capturing your best ideas, whether they are big or small. Then as you're planning what to test next, consider these guidelines:
|Small Ideas||Big Ideas|
|Scope||Isolated changes to page elements or copy with specific intent||Several combined changes|
|Effort||Low effort (1-3 days)||High effort (up to 2 weeks)|
|Test duration||Long. Best if idea is important and you're willing to wait. Harder to confirm the effect with high certainty.||Short. Best if you want results sooner. Typically higher certainty results.|
|Potential||Typically smaller gains/loses up to 10%, but can add up over time.||Bigger gains of 10% to 30% are possible.|
|Learning||Run isolated, small tests if it's important to pinpoint cause.||Combine multiple ideas to create bigger tests if gains are more important than knowing why a test won or lost. If you combine related changes taht represent a single meaningful unit or component, you can still get a clear cause from such a test.|
If nothing is being tested, pick a quick idea you can built and start within a day. This way you avoid non-testing days.
If you are more confident in your small idea than the big idea, go with the small idea first.
If you have good ideas for a low-traffic page and a high traffic page, start with the high traffic page. Perhaps you have ideas that can run as a template sidewide or on multiple similar pages, pulling greatest site traffic into the test. However, if the low traffic page is more important to your bottom line, test there first.
If you've only tested small ideas and had no results, test a big idea, and vice versa.
As you're combining ideas for a single big test, the risk is that good changes may get canceled by bad changes. To mitigate this, support your ideas with user research, analytics, and patterns based on past tests that worked. Isolate very experimental ideas. Visit our GoodUI Evidence test repository or subscribe to GoodUI Datastories to get the latest patterns and analysis.
Some big test ideas are not combinations of multiple ideas but can be though of as single meaningful units. For example, a social proof widget will include the copy itself, navigation controls, and so on. All those piece are required to create the overall unit. You can learn a great deal from such tests. Big tests than can be turned into reusable patterns centered on a specific idea or intent are ideal
Every time you start a test, start prebuilding the next. Prioritization and preparation should be an ongoing activity. If you find that a big idea is taking too long to design and the current test may be ended soon, build a small idea that can be launched instead.
If you have any questions, let me know on Twitter (@vladmalik)