All Latest 644 A/B Tests

MOST RECENT TESTS

Test #433 on Expertinstitute.com by Ardit VeliuArdit Veliu Sep 30, 2022 Desktop Mobile Signup X.X% Leads

Ardit Tested Pattern #20: Canned Response On Expertinstitute.com

Pattern #20: Canned Response - Variant A
Pattern #20: Canned Response - Variant B

In this experiment a dynamic canned response was added to a lead form. The contents of the copy reflected a handful of user choices from other form field selections. Impact on leads / form submissions was measured.

Total Sample: 14,490 • Statistical Power at 5% MDE: 13.3%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #434 on Learnwithhomer.com by Stanley ZuoStanley Zuo Sep 30, 2022 Mobile Signup X.X% Signups

Stanley Tested Pattern #66: Complementary Upsell On Learnwithhomer.com

Pattern #66: Complementary Upsell - Variant A
Pattern #66: Complementary Upsell - Variant B

In this experiment, an upsell modal appeared during a signup funnel. In the variation, if users did not select a monthly plan, they were nudged to an annual one. Impact on overall and annual signups was measured.

Total Sample: 6,098 • Statistical Power at 5% MDE: 8.6%

Test #432 on Snocks.com by Melina HessMelina Hess Sep 29, 2022 Mobile Global X.X% Sales

Melina Tested Pattern #94: Visible Search On Snocks.com

Pattern #94: Visible Search - Variant A
Pattern #94: Visible Search - Variant B

In this experiment, search functionality was added in the variation, globally on mobile. Impact on sales was measured

Total Sample: 1,175,864 • Statistical Power at 5% MDE: 99.9%

Test #431 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Sep 23, 2022 Desktop Mobile Pricing X.X% Sales

Daria Tested Pattern #21: What It's Worth On Volders.de

Pattern #21: What It's Worth - Variant A
Pattern #21: What It's Worth - Variant B

In this experiment, a historically higher price point was added as a crossed out anchor. The control only showed the current price. The variation showed the current price with the higher price crossed out. Impact on sales was measured.

Total Sample: 34,645 • Statistical Power at 5% MDE: 97.1%

Test #428 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Aug 16, 2022 Desktop Mobile Signup X.X% Sales

Daria Tested Pattern #26: Cart Reminder And Recently Viewed On Volders.de

Pattern #26: Cart Reminder And Recently Viewed - Variant A
Pattern #26: Cart Reminder And Recently Viewed - Variant B

In this test 1) a passive hint communicated to users that their data will be saved for 7 days for them to be able to continue their cancellation later and 2) dropped off users were targeted with an email campaign within the first 4 hours after drop off. The reminder email linked users to a shipping page (checkout page) without them requiring to fill out their personal, contract information one more time. 

Total Sample: 155,342 • Statistical Power at 5% MDE: 98.8%

Test #429 on Snocks.com by Melina HessMelina Hess Aug 16, 2022 Mobile Desktop Product X.X% Sales

Melina Tested Pattern #121: Free Shipping On Snocks.com

Pattern #121: Free Shipping - Variant A
Pattern #121: Free Shipping - Variant B

Upon clicking the Add-To-Cart button, the button label changed to a cheering message which congratulated the user on his choice and ensuring that shipping is free. Translation: "Good choice. Your shipping and returns are free."

Total Sample: 56,067 • Statistical Power at 5% MDE: 99.9%

Test #427 on Designlab.com by Daniel ShapiroDaniel Shapiro Aug 10, 2022 Desktop Mobile Checkout X.X% Sales

Daniel Tested Pattern #28: Easiest Fields First On Designlab.com

Pattern #28: Easiest Fields First - Variant A
Pattern #28: Easiest Fields First - Variant B

In this experiment, the course enrollment start date was moved from step 2 to step 1 of an enrollment / checkout flow. The test was run by Designlab - that offers design courses and education with a strong element of mentorship. Impact on progression to next step and completed transactions were measured.

Total Sample: 12,185 • Statistical Power at 5% MDE: 15.3%

Test #426 on Phorest.com by Sorcha MullisSorcha Mullis Aug 09, 2022 Desktop Mobile Home & Landing X.X% Leads

Sorcha Tested Pattern #9: Multiple Steps On Phorest.com

Pattern #9: Multiple Steps - Variant A
Pattern #9: Multiple Steps - Variant B

In this experiment, a single step popup modal was tested against a 4 step sign up funnel. Users entered the experiment on the homepage and the behavior of the two buttons, book a demo and get a quote, was adjusted. Impact on leads was measured.

Total Sample: 88,974 • Statistical Power at 5% MDE: 7.5%

Test #423 on Expertinstitute.com by Ardit VeliuArdit Veliu Jul 26, 2022 Desktop Mobile Home & Landing X.X% Leads

Ardit Tested Pattern #110: Optional Field Labels On Expertinstitute.com

Pattern #110: Optional Field Labels - Variant A
Pattern #110: Optional Field Labels - Variant B

In this experiment, an "optional" label was shown near a message form field. Impact on overall leads was measured (requesting experts).

Total Sample: 1,462 • Statistical Power at 5% MDE: 6.6%

Test #422 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Jul 22, 2022 Desktop Mobile Shopping Cart X.X% Sales

Daria Tested Pattern #3: Fewer Form Fields On Volders.de

Pattern #3: Fewer Form Fields - Variant A
Pattern #3: Fewer Form Fields - Variant B

A birthdate field was removed during a signup process for a paid contract cancellation service. Impact on progression (next step) and sales (completed transactions) was measured.

Total Sample: 20,345 • Statistical Power at 5% MDE: 16.6%

Test #421 on Amazon.com by Marika FranciscoMarika Francisco Jul 15, 2022 Desktop Mobile Product X.X% Sales

Marika Tested Pattern #43: Long Titles On Amazon.com

Pattern #43: Long Titles - Variant A
Pattern #43: Long Titles - Variant B

Fluke is an industrial manufacterer of measurement and calibration devices and they ran a test on their Amazon store for a series of their prodcts. Instead of using short product names, they tested longer and more descriptive ones. Impact was measured on sales.

Total Sample: 3,727,581 • Statistical Power at 5% MDE: 92.5%

Test #420 on Designlab.com by Daniel ShapiroDaniel Shapiro Jul 12, 2022 Desktop Mobile Product X.X% Leads

Daniel Tested Pattern #115: Pricing Comparison Table On Designlab.com

Pattern #115: Pricing Comparison Table - Variant A
Pattern #115: Pricing Comparison Table - Variant B

In this experiment, pricing plans were laid out horizontally for easier comparison. In the variation, most of the plan benefits, features and differences were also referenced using a single lable that was left-aligned. The idea was to make the variables aligned and therefore more comparable.

This pricing table appeared at the bottom of a long design program landing page. Impact on leads and applications was measured.

Total Sample: 9,521 • Statistical Power at 5% MDE: 14.3%

Test #418 on Online.metro-cc.ru by Andrey AndreevAndrey Andreev Jun 28, 2022 Desktop Mobile Home & Landing X.X% Sales

Andrey Tested Pattern #135: Product Categories On Online.metro-cc.ru

Pattern #135: Product Categories - Variant A
Pattern #135: Product Categories - Variant B

In this experiment, popular product category links were added near the bottom of the homepage of an online grocery store - Metro. Impact on completed sales was measured.

Total Sample: 78,582 • Statistical Power at 5% MDE: 12.6%

Test #417 on Cxl.com by Ognjen BoškovićOgnjen Bošković Jun 27, 2022 Desktop Mobile Signup X.X% Signups

Ognjen Tested Pattern #127: Vague Or Specific Benefits On Cxl.com

Pattern #127: Vague Or Specific Benefits - Variant A
Pattern #127: Vague Or Specific Benefits - Variant B

CXL ran an interesting experiment that (1) added more specificity on a newsletter subscription page as well as (2) reinforced it consistently. Most visibly, an abstract or vague headline statement (control) was changed to a benefit oriented one - hinting that subscribers will eventually receive valuable case studies. This message was further reinforced with supporting copy that explained where these case studies are obtained from along with the frequency of the delivery. This same message was also reinforced with an image of actual case studies. The call to action button was also adjusted to mimic the benefit.

Impact on newsletter signups was measured.

Total Sample: 8,974 • Statistical Power at 5% MDE: 28.4%

Test #416 on Designlab.com by Daniel ShapiroDaniel Shapiro Jun 20, 2022 Desktop Mobile Product X.X% Sales

Daniel Tested Pattern #105: Lead Magnets On Designlab.com

Pattern #105: Lead Magnets - Variant A
Pattern #105: Lead Magnets - Variant B

In this experiment, copy was added which communicated that students signing up for a course would receive extra design resources (the lead magnet). This was added in multiple states of the course page. Impact on lead generation and enrollment was measured.

Total Sample: 6,603 • Statistical Power at 5% MDE: 7.4%

Test #415 on Learnwithhomer.com by Stanley ZuoStanley Zuo Jun 09, 2022 Mobile Checkout X.X% Signups

Stanley Tested Pattern #3: Fewer Form Fields On Learnwithhomer.com

Pattern #3: Fewer Form Fields - Variant A
Pattern #3: Fewer Form Fields - Variant B

Do fewer confirmation form fields matter? In this experiment, redundant password and email confirmation fields were removed during a signup / checkout funnel. Impact on signups was measured.

Total Sample: 2,286 • Statistical Power at 5% MDE: 5.2%

Test #414 on Volders.de by Frederik FröhleFrederik Fröhle May 31, 2022 Desktop Mobile Checkout X.X% Sales

Frederik Tested Pattern #98: Auto Suggest On Volders.de

Pattern #98: Auto Suggest - Variant A
Pattern #98: Auto Suggest - Variant B

Does adding Google's address auto complete functionality to an address field help with higher form completions? This auto fill feature has been tested in the variation of a contract cancellation funnel. After selecting an auto completed address from a pulldown menu, the following fields were preselected: house number, zip code, city and country (potentially lowering friction?). Impact on successful form completions (contract cancellations) has been measured. Notice how the form also expanded progressively upon selecting the complete address in the variation.  

Total Sample: 128,199 • Statistical Power at 5% MDE: 97.4%

Test #413 by Jakub LinowskiJakub Linowski May 26, 2022 Desktop Mobile Shopping Cart X.X% Sales

Jakub Tested Pattern #114: Less Or More Visible Prices

Pattern #114: Less Or More Visible Prices - Variant A
Pattern #114: Less Or More Visible Prices - Variant B

Should prices be displayed during an editing state on an add-to-cart screen (after adding-to-cart from a product detail page)? In this experiment, club pricing was added and shown in three areas after customers clicked on a small "edit details" links. The control did not have this pricing information (which was only shown on a product detail screen before).

Total Sample: 5,698 • Statistical Power at 5% MDE: 59.9%

Test #412 on Volders.de by Frederik FröhleFrederik Fröhle May 16, 2022 Desktop Mobile Checkout X.X% Sales

Frederik Tested Pattern #15: Bulleted Reassurances On Volders.de

Pattern #15: Bulleted Reassurances - Variant A
Pattern #15: Bulleted Reassurances - Variant B

The Volders experimentation team assumed that adding information about how long it might take to get a cancellation confirmation near a CTA Button would result in higher cancellation requests (the paid service being offered).

The variation contained additional copy translated from German to: "Most of our users receive their cancellation confirmation from <vendor> within 14 days by email or letter."

Total Sample: 53,120 • Statistical Power at 5% MDE: 99.3%

Test #411 by Ayat ShukairyAyat Shukairy May 09, 2022 Desktop Mobile Product X.X% Sales

Ayat Tested Pattern #126: Bottom Or Left Thumbnails

Pattern #126: Bottom Or Left Thumbnails - Variant A
Pattern #126: Bottom Or Left Thumbnails - Variant B

Bottom aligned thumbnails were shifted to the left side on a product image. Doing so, also shifted the product descriptions a little higher. Impact on adds-to-cart and total transactions was measured.

Total Sample: 588,443 • Statistical Power at 5% MDE: 70.1%