All Latest 646 A/B Tests

Test #379 on Learnwithhomer.com by Stanley ZuoStanley Zuo Oct 15, 2021 Mobile Pricing X.X% Sales

Stanley Tested Pattern #112: Lower Price Frames On Learnwithhomer.com

Pattern #112: Lower Price Frames - Variant A
Pattern #112: Lower Price Frames - Variant B

In this experiment, the annual plan was standardized and framed in a monthly price context. This was more comparable with the monthly plan price above. Please also note that the annual billing price was also kept and clearly stated. Impact on overall sales and annual plan sales was measured.

Total Sample: 3,437 • Statistical Power at 5% MDE: 22.3%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #378 by Jakub LinowskiJakub Linowski Oct 07, 2021 Desktop Mobile Product X.X% Sales

Jakub Tested Pattern #119: Unselected Or Selected Defaults

Pattern #119: Unselected Or Selected Defaults - Variant A
Pattern #119: Unselected Or Selected Defaults - Variant B

Here we have an experiment with a variation that preselected an option for a club duration. The control required customers to first express the choice for how many months they would like to order a product for. Whereas the variation defaulted to 12 months from the beginning.

Impact on adds-to-cart and sales was measured. The experiment unfortunately had to be stopped early due to another embedded variation that was performing poorly. And so it does not have many transactions.

Total Sample: 2,953 • Statistical Power at 5% MDE: 5.2%

Test #375 on Backstage.com by Stanley ZuoStanley Zuo Sep 17, 2021 Mobile Listing X.X% Sales

Stanley Tested Pattern #32: Condensed List On Backstage.com

Pattern #32: Condensed List - Variant A
Pattern #32: Condensed List - Variant B

Similar to experiment 373, listing descriptions were shortened dynamically using exposable "more" links AND dates/location data was removed. This way, the variation showed shorter listings and therefore more listings per screens. Impact on listing clicks (progression) along with membership starts was measured.

Total Sample: 44,045 • Statistical Power at 5% MDE: 92.5%

Test #374 on Expertinstitute.com by Ardit VeliuArdit Veliu Sep 15, 2021 Desktop Mobile Home & Landing X.X% Leads

Ardit Tested Pattern #9: Multiple Steps On Expertinstitute.com

Pattern #9: Multiple Steps - Variant A
Pattern #9: Multiple Steps - Variant B

In this experiment, a lead form with numerous fields shown all at once, was broken down into 3 steps. The first step only asked for a Specialty choice that expanded into a list of options. Impact on full form completions was measured.

Total Sample: 5,628 • Statistical Power at 5% MDE: 7.5%

Test #373 on Backstage.com by Stanley ZuoStanley Zuo Sep 06, 2021 Mobile Listing X.X% Sales

Stanley Tested Pattern #32: Condensed List On Backstage.com

Pattern #32: Condensed List - Variant A
Pattern #32: Condensed List - Variant B

In this experiment, listing descriptions were shortened dynamically using exposable "more" links. This way, the variation showed shorter listings and therefore more listings per screens. Impact on listing clicks (progression) along with membership starts was measured.

Total Sample: 43,469 • Statistical Power at 5% MDE: 92.2%

Test #372 on Fluke.com by Marika FranciscoMarika Francisco Aug 30, 2021 Desktop Product X.X% Sales

Marika Tested Pattern #122: Zigzag Layout On Fluke.com

Pattern #122: Zigzag Layout - Variant A
Pattern #122: Zigzag Layout - Variant B

In this experiment, product descriptions were changed to an alternating zig zag layout with images. Impact on adds-to-cart and sales was measured.

Total Sample: 18,325 • Statistical Power at 5% MDE: 3.2%

Test #369 on Getninjas.com.br by Rodolfo LugliRodolfo Lugli Aug 05, 2021 Desktop Mobile Home & Landing X.X% Leads

Rodolfo Tested Pattern #7: Social Counts On Getninjas.com.br

Pattern #7: Social Counts - Variant A
Pattern #7: Social Counts - Variant B

In this experimemt, the number of service requests per month were shown that people can expect after signing up in a given category. Impact on leads was measured (people who would be signing up to offer a particular service).

Total Sample: 4,256 • Statistical Power at 5% MDE: 14%

Test #366 on Mvideo.ru by Andrey AndreevAndrey Andreev Jul 14, 2021 Desktop Mobile Product X.X% Sales

Andrey Tested Pattern #93: Auto Next On Mvideo.ru

Pattern #93: Auto Next - Variant A
Pattern #93: Auto Next - Variant B

In this experiment, the interaction on a product page was changed. In the control (A), after clicking "add to cart", the customer would remain on the product page with an updated basket and the button changing to "Go To Cart". The variation (B) however took customers forward automatically to the cart.

Total Sample: 80,881 • Statistical Power at 5% MDE: 95%

Test #361 on Chaosgroup.com by Velin PenevVelin Penev Jun 22, 2021 Desktop Product X.X% Sales

Velin Tested Pattern #49: Above The Fold Call To Action On Chaosgroup.com

Pattern #49: Above The Fold Call To Action - Variant A
Pattern #49: Above The Fold Call To Action - Variant B

In this experiment, a pricing plan selector was shifted from the very bottom of a long product page towards (almost) the top. Impact on any transactions or sales was measured.

Total Sample: 49,030 • Statistical Power at 5% MDE: 6.5%

Test #358 on Preply.com by Gleb HodorovskiyGleb Hodorovskiy Jun 03, 2021 Desktop Home & Landing X.X% Sales

Gleb Tested Pattern #58: Full Height False Bottom On Preply.com

Pattern #58: Full Height False Bottom - Variant A
Pattern #58: Full Height False Bottom - Variant B

In this experiment, the header section of the homepage drastically focused around the call to action. This was done by removing elements and forcing a false bottom.

Total Sample: 133,252 • Statistical Power at 5% MDE: 14.8%

Test #357 on Baremetrics.com by Brian SierakowskiBrian Sierakowski Jun 02, 2021 Desktop Mobile Pricing X.X% Signups

Brian Tested Pattern #113: More Or Fewer Plans On Baremetrics.com

Pattern #113: More Or Fewer Plans - Variant A
Pattern #113: More Or Fewer Plans - Variant B

In this experiment, a single focus signup was transformed into a pricing comparison table. Impact on signups was measured.

Total Sample: 4,947 • Statistical Power at 5% MDE: 5.5%

Test #355 by Jakub LinowskiJakub Linowski May 28, 2021 Desktop Mobile Home & Landing X.X% Sales

Jakub Tested Pattern #79: Product Highlights

Pattern #79: Product Highlights - Variant A
Pattern #79: Product Highlights - Variant B

In this experiment, an extra product choice was added to the header of a homepage. Instead of only highlighting a set of four specific products, the option to build custom variety one, was added.

Total Sample: 26,902 • Statistical Power at 5% MDE: 30.7%

Test #353 on Backstage.com by Stanley ZuoStanley Zuo May 12, 2021 Desktop Product X.X% Signups

Stanley Tested Pattern #13: Centered Forms & Buttons On Backstage.com

Pattern #13: Centered Forms & Buttons - Variant A
Pattern #13: Centered Forms & Buttons - Variant B

This experiment challanged a right aligned detail page. After clicking a result of a job role on a listing page, an overlay would appear on the right with the details. The variation used a full width screen instead - effectively centering the page.

Total Sample: 30,428 • Statistical Power at 5% MDE: 53.6%

Test #352 on Us.flukecal.com by John HickeyJohn Hickey May 11, 2021 Desktop Global X.X% Leads

John Tested Pattern #123: Single Or Double Column Form Fields On Us.flukecal.com

Pattern #123: Single Or Double Column Form Fields - Variant A
Pattern #123: Single Or Double Column Form Fields - Variant B

In this experiment, single column (longer) form fields were tested against a two column layout (more compact).

Total Sample: 252 • Statistical Power at 5% MDE: 4.5%

Test #351 on Baremetrics.com by Brian SierakowskiBrian Sierakowski Apr 30, 2021 Desktop Mobile Home & Landing X.X% Signups

Brian Tested Pattern #11: Gradual Reassurance On Baremetrics.com

Pattern #11: Gradual Reassurance - Variant A
Pattern #11: Gradual Reassurance - Variant B

In this experiment, static integration logos were replaced with selectable ones that reassured users to signup. After clicking an integration logo, a comparison chart would appear showing how Baremetrics improves upon a selected payment processor, along with a call to signup. Impact on signups was measured.

Total Sample: 24,416 • Statistical Power at 5% MDE: 7%

Test #348 on Flukenetworks.com by Marika FranciscoMarika Francisco Apr 22, 2021 Desktop Home & Landing X.X% Progression

Marika Tested Pattern #97: Bigger Form Fields On Flukenetworks.com

Pattern #97: Bigger Form Fields - Variant A
Pattern #97: Bigger Form Fields - Variant B

In this simple experiment, the size of the "Get Quote" button in the top navigation was increased. 

Total Sample: 39,992 • Statistical Power at 5% MDE: 4.3%

Test #347 by Jakub LinowskiJakub Linowski Apr 07, 2021 Desktop Mobile Home & Landing X.X% Sales

Jakub Tested Pattern #26: Cart Reminder And Recently Viewed

Pattern #26: Cart Reminder And Recently Viewed - Variant A
Pattern #26: Cart Reminder And Recently Viewed - Variant B

In this experiment, when customers viewed a product and returned to the homepage, they would then see the most recently viewed one - a delicate nudge. The experiment ran with full traffic and impact on sales was measured.

Total Sample: 13,749 • Statistical Power at 5% MDE: 15.8%

Test #104 on 3dhubs.com by Rob DraaijerRob Draaijer Mar 31, 2021 Desktop Listing X.X% Leads

Rob Tested Pattern #15: Bulleted Reassurances On 3dhubs.com

Pattern #15: Bulleted Reassurances - Variant A
Pattern #15: Bulleted Reassurances - Variant B

This experiment attempted to increase the number of leads on a lead-funnel. As the first step, users were being asked to upload a file. The control showed the file types that were allowed, whereas the variation changed the copy to show a number of benefits for taking that action. The text-based benefits included the: receiving feedback, prices and lead times.

Total Sample: 50,698 • Statistical Power at 5% MDE: 43%

Test #345 on Getninjas.com.br by Rodolfo LugliRodolfo Lugli Mar 29, 2021 Desktop Home & Landing X.X% Leads

Rodolfo Tested Pattern #9: Multiple Steps On Getninjas.com.br

Pattern #9: Multiple Steps - Variant A
Pattern #9: Multiple Steps - Variant B

In this experiment, a single long form was broken into at least 3 steps. 

Total Sample: 401,145 • Statistical Power at 5% MDE: 99.9%

Test #343 on Snocks.com by Samuel HessSamuel Hess Mar 12, 2021 Desktop Mobile Product X.X% Sales

Samuel Tested Pattern #122: Zigzag Layout On Snocks.com

Pattern #122: Zigzag Layout - Variant A
Pattern #122: Zigzag Layout - Variant B

In this experiment, the content on a product page was reorganized into a zigzagging (alternating layout) along with reinforcing photos. Impact on adds-to-cart and total sales was measured.

Total Sample: 23,865 • Statistical Power at 5% MDE: 32.9%