All Latest 634 A/B Tests

Test #515 on by Jakub LinowskiJakub Linowski Jan 31, 2024 Desktop Mobile Home & Landing X.X% Sales

Jakub Tested Pattern #69: Autodiscounting

 - Variant A
 - Variant B

In this experiment, people who saw an offer (in an email or popup) would see a more visible site wide reinforcement of their earned discount being active. In the control, the discount was only shown during checkout. In the variation, it was shown throughout the web site on the homepage and product detail pages.

Total Sample: 2,471 Statistical Power at 5% MDE: 8.5%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #513 on Dripl.de by Jona EisenbergerJona Eisenberger Jan 23, 2024 Mobile Desktop Product X.X% Sales

Jona Tested Pattern #15: Bulleted Reassurances On Dripl.de

 - Variant A
 - Variant B

In this experiment of a floor cleaning product, a reassurance about product uses was appended to two cleaning agent upsells. In the control users saw the 2 standard upsells for laminate and tiles. And in the variation the copy "Right for me. 100x Cleanings" (uses) was appended. 

Total Sample: 69,221 Statistical Power at 5% MDE: 18.7%

Test #511 on Online.metro-cc.ru by Andrey AndreevAndrey Andreev Jan 09, 2024 Desktop Home & Landing X.X% Sales

Andrey Tested Pattern #79: Product Highlights On Online.metro-cc.ru

 - Variant A
 - Variant B

In this experiment, popular products were shown at the top of the homepage. Impact to sale was measured.

Total Sample: 760,358 Statistical Power at 5% MDE: 99.9%

Test #509 on Jared.com by Craig KistlerCraig Kistler Dec 18, 2023 Desktop Product X.X% Sales

Craig Tested Pattern #66: Complementary Upsell On Jared.com

 - Variant A
 - Variant B

In this experiment, a protection plan was launched as a modal for customers that clicked add-to-cart without choosing the upsell. Impact on adds-to-cart and sales was measured.

Total Sample: 407,785 Statistical Power at 5% MDE: 23.2%

Test #507 on Fairment.de by Jona EisenbergerJona Eisenberger Dec 11, 2023 Mobile Listing X.X% Sales

Jona Tested Pattern #133: Product Availability On Fairment.de

 - Variant A
 - Variant B

In this experiment, product availabiltiy bars were shown on products with low stock. This was shown on listing pages. Impact on adds to cart and sales was measured.

Total Sample: 42,001 Statistical Power at 5% MDE: 40.1%

Test #501 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Nov 02, 2023 Desktop Mobile Pricing X.X% Sales

Daria Tested Pattern #132: One Time Payment Copy On Volders.de

 - Variant A
 - Variant B

In this experiment, the wording "one-time payment" was appended below all three pricing plans for a contract cancelation service. Impact on orders placed was measured. (Translated from German: "Einmalige Zahlung").

Total Sample: 89,887 Statistical Power at 5% MDE: 99.9%

Test #500 on Kayoutlet.com by Craig KistlerCraig Kistler Oct 25, 2023 Mobile Product X.X% Sales

Craig Tested Pattern #7: Social Counts On Kayoutlet.com

 - Variant A
 - Variant B

This was a replication attempt similar to experiment 497. A social proof message was added below add to cart buttons on product detail pages. Impact on adds to cart and transactions was measured.

Total Sample: 106,467 Statistical Power at 5% MDE: 19.1%

Test #499 on Shmoodyapp.com by Michael McSweeneyMichael McSweeney Oct 20, 2023 Mobile Signup X.X% Sales

Michael Tested Pattern #99: Progress Bar On Shmoodyapp.com

 - Variant A
 - Variant B

In this experiment, a progress bar was added to a signup flow of a mental health app. It appeared on numerous steps of a onboarding flow. Impact on signup completions and checkouts was measured.

Total Sample: 56,735 Statistical Power at 5% MDE: 37.1%

Test #498 on by Jakub LinowskiJakub Linowski Oct 19, 2023 Desktop Mobile Product X.X% Sales

Jakub Tested Pattern #7: Social Counts

 - Variant A
 - Variant B

In this experiment, the variation tracked how many people would add something to cart over the last 24 hours and display that just below the add to cart button. Impact on adds to cart and transactions was measured.

Total Sample: 14,539 Statistical Power at 5% MDE: 17.9%

Test #497 on Jared.com by Craig KistlerCraig Kistler Oct 16, 2023 Mobile Product X.X% Sales

Craig Tested Pattern #7: Social Counts On Jared.com

 - Variant A
 - Variant B

In this experiment, a social proof message was added below add to cart buttons on product detail pages. The copy read "Hurry, X people have viewed this product". Impact on adds to cart and transactions was measured.

Total Sample: 451,850 Statistical Power at 5% MDE: 44.3%

Test #496 on Livefresh.de by Pascal DietzPascal Dietz Oct 03, 2023 Desktop Mobile Product X.X% Sales

Pascal Tested Pattern #43: Long Titles On Livefresh.de

 - Variant A
 - Variant B

In this product detail page experiment a simple subheadline was appended to a new product. The tagline read "Lose weight sustainably and healthily." (Google translated from German "Nachhaltig und gesund Gewicht verlieren."). Impact on sales and revenue was measured.

Total Sample: 68,822 Statistical Power at 5% MDE: 51.3%

Test #494 on Online.Metro-cc.ru by Andrey AndreevAndrey Andreev Sep 20, 2023 Desktop Mobile Home & Landing X.X% Sales

Andrey Tested Pattern #26: Cart Reminder And Recently Viewed On Online.Metro-cc.ru

 - Variant A
 - Variant B

In this experiment, recently purchased products were appended at the top of the homepage. The test ran for loggedin users only. Impact on add-to-cart, sales and revenue was measured.

Total Sample: 53,513 Statistical Power at 5% MDE: 78.3%

Test #492 on Formelskin.de by Alexander KriegerAlexander Krieger Sep 15, 2023 Mobile Signup X.X% Sales

Alexander Tested Pattern #131: Authority On Formelskin.de

 - Variant A
 - Variant B

This experiment was a retest of a previously tested test 491. Simiarly, a photo of the real doctor behind a skin care product was shown throughout a signup / questionnaire flow in the variation. 

Total Sample: 26,270 Statistical Power at 5% MDE: 17%

Test #491 on Formelskin.de by Alexander KriegerAlexander Krieger Sep 08, 2023 Mobile Signup X.X% Sales

Alexander Tested Pattern #131: Authority On Formelskin.de

 - Variant A
 - Variant B

In this experiment, a photo of the real doctor behind a skin care product was shown throughout a signup / questionnaire flow. In the control version, the photo with introduction was only shown on the first screen. In the variation, the photo appeared on the first and all other screens - reinforcing expertise, authority and the idea of a consultation. Impact on signups and purchases was measured.

Total Sample: 18,949 Statistical Power at 5% MDE: 13.1%

Test #488 on Metro-cc.ru by Andrey AndreevAndrey Andreev Aug 11, 2023 Mobile Product X.X% Sales

Andrey Tested Pattern #4: Testimonials On Metro-cc.ru

 - Variant A
 - Variant B

In this experiment, customer reviews were shown on product detail pages. Impact on adds to cart and sales was measured.

Total Sample: 103,655 Statistical Power at 5% MDE: 13.7%

Test #486 on Livefresh.de by Pascal DietzPascal Dietz Aug 03, 2023 Mobile Product X.X% Sales

Pascal Tested Pattern #78: Tags, Badges And Structured Information On Livefresh.de

 - Variant A
 - Variant B

Does a "Bestseller" badge increase sales? This experiment tested exactly this by adding a visible badge on a juice cleansing product detail page.

Total Sample: 416,423 Statistical Power at 5% MDE: 98.8%

Test #483 on Menufy.com by Aleksandr ElesevAleksandr Elesev Jul 17, 2023 Desktop Checkout X.X% Sales

Aleksandr Tested Pattern #124: Confirmed Selection On Menufy.com

 - Variant A
 - Variant B

In this experiment, a thanking confirmation message was appended at the top of the checkout screen of a local food delivery service. Impact on completed transactions was measured.

Total Sample: 8,626 Statistical Power at 5% MDE: 99.9%

Test #481 on Backstage.com by Stanley ZuoStanley Zuo Jul 14, 2023 Desktop Mobile Checkout X.X% Sales

Stanley Tested Pattern #15: Bulleted Reassurances On Backstage.com

 - Variant A
 - Variant B

In this experiment, selling points and benefits of a subscription were placed as bullets at the top of a checkout page. The benefits highlighted things such as: unlimited applications, access to vetted jobs and the ability to cancel anytime. Impact on sales was measured.

Total Sample: 5,616 Statistical Power at 5% MDE: 24.3%

Test #480 on Aboalarm.de by Daria KurchinskaiaDaria Kurchinskaia Jun 15, 2023 Desktop Mobile Checkout X.X% Sales

Daria Tested Pattern #15: Bulleted Reassurances On Aboalarm.de

 - Variant A
 - Variant B

In this experiment, a list of benefits were shown for each of the 3 plans on the last step of a contract cancelation service. The lowest plan only had one benefit, whereas the highest plan had 3. Impact on transactions was measured. 

Total Sample: 104,902 Statistical Power at 5% MDE: 93%

Test #479 on Aboalarm.de by Daria KurchinskaiaDaria Kurchinskaia Jun 15, 2023 Desktop Mobile Checkout X.X% Sales

Daria Tested Pattern #15: Bulleted Reassurances On Aboalarm.de

 - Variant A
 - Variant B

In this experiment, a list of benefits were shown for each of the 3 plans on the last step of a contract cancelation service. Benefit items not included in the lower plans were also shown with grayed out styles (and an "x"). Clearly the higher paid plan had all the benefits listed. Impact on transactions was measured. 

Total Sample: 104,863 Statistical Power at 5% MDE: 93%