All Latest 645 A/B Tests

Test #537 on Online.metro-cc.ru by Andrey AndreevAndrey Andreev Jun 19, 2024 Desktop Mobile Product X.X% Sales

Andrey Tested Pattern #135: Product Categories On Online.metro-cc.ru

Pattern #135: Product Categories - Variant A
Pattern #135: Product Categories - Variant B

In this experiment, a simple product category link was added on product detail pages. These links linked to listing pages with more of a similar product type. Impact on sales was measured.

Total Sample: 100,360 • Statistical Power at 5% MDE: 24.4%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #535 on 686.com by Adan ArchilaAdan Archila May 31, 2024 Desktop Listing X.X% Sales

Adan Tested Pattern #120: Supporting Theme Images On 686.com

Pattern #120: Supporting Theme Images - Variant A
Pattern #120: Supporting Theme Images - Variant B

In this experiment, a static category theme image at the top of a listing page was tested against the same page but without the extra image. Impact on sales was measured.

Total Sample: 79,626 • Statistical Power at 5% MDE: 22.8%

Test #534 on Online.metro-cc.ru by Andrey AndreevAndrey Andreev May 28, 2024 Desktop Global X.X% Sales

Andrey Tested Pattern #82: Onboarding Callouts On Online.metro-cc.ru

Pattern #82: Onboarding Callouts - Variant A
Pattern #82: Onboarding Callouts - Variant B

In this site wide experiment, logged out customers were directed to sign in with the help of a top navigation callout. The experiment was triggered for anyone who saw the callout message. After signing in, the user would 1) remain on the same screen they were on and 2) the top header slightly changed to show a logged in state (with their orders, favorites and active username). Impact on successful logins and overall purchases was measured.

Total Sample: 1,817,890 • Statistical Power at 5% MDE: 99.6%

Test #528 on 686.com by Adan ArchilaAdan Archila Apr 26, 2024 Mobile Listing X.X% Sales

Adan Tested Pattern #120: Supporting Theme Images On 686.com

Pattern #120: Supporting Theme Images - Variant A
Pattern #120: Supporting Theme Images - Variant B

In this experiment, the effect of having additional static category themed images was tested on category listing pages. Impact on sales was measured.

Total Sample: 216,092 • Statistical Power at 5% MDE: 35.2%

Test #524 on Jared.com by Craig KistlerCraig Kistler Mar 26, 2024 Mobile Listing X.X% Revenue

Craig Tested Pattern #79: Product Highlights On Jared.com

Pattern #79: Product Highlights - Variant A
Pattern #79: Product Highlights - Variant B

In this experiment, additional (discounted) products were shown at the top of category listing pages with a link to see more such products ("View All Price Drops"). Impact on overall sales was measured.

Test #523 on Livefresh.de by Pascal DietzPascal Dietz Mar 21, 2024 Mobile Product X.X% Sales

Pascal Tested Pattern #131: Authority On Livefresh.de

Pattern #131: Authority - Variant A
Pattern #131: Authority - Variant B

In this experiment, a photo of a doctor (who researched and created the a cleanising juice - the product) was appended after the add to cart button. The copy is translated as follows from "Von der promovierten Ernährungswissenschaftlerin Dr. Ulrike Fisher auf wissenschaftlicher Basis entwickelt." [German] -> "Developed by doctoral nutritionist Dr. Ulrike Fisher with a scientific basis." Impact on sales was measured.

Total Sample: 194,958 • Statistical Power at 5% MDE: 92.7%

Test #521 on Finn.com by Maksim Meged Maksim Meged Mar 14, 2024 Mobile Home & Landing X.X% Sales

Maksim Tested Pattern #26: Cart Reminder And Recently Viewed On Finn.com

Pattern #26: Cart Reminder And Recently Viewed - Variant A
Pattern #26: Cart Reminder And Recently Viewed - Variant B

This experiment was triggered by a small segment of users who completed the first step of checkout funnel (submitted email, name, phone number), but dropped from checkout and returned to cars catalogue 7 or more minutes later.

In the control, users didn't see any cart icon nor function to resume their checkout flow.

In the variation however, users saw a filled shopping cart icon with resume functionality. Clicking on the icon would guide and redirect users to their latest abandoned stage of their checkout flow.

Total Sample: 2,160 • Statistical Power at 5% MDE: 12%

Test #517 on Aboalarm.de by Daria KurchinskaiaDaria Kurchinskaia Feb 12, 2024 Desktop Checkout X.X% Sales

Daria Tested Pattern #78: Tags, Badges And Structured Information On Aboalarm.de

Pattern #78: Tags, Badges And Structured Information - Variant A
Pattern #78: Tags, Badges And Structured Information - Variant B

In this experiment, a "most popular" badge ("am beliebsten") was appended to the middle plan of a contract cancellation service. Impact on overall purchases and revenue was measured.

Total Sample: 127,712 • Statistical Power at 5% MDE: 94.9%

Test #515 by Jakub LinowskiJakub Linowski Jan 31, 2024 Desktop Mobile Home & Landing X.X% Sales

Jakub Tested Pattern #69: Autodiscounting

Pattern #69: Autodiscounting - Variant A
Pattern #69: Autodiscounting - Variant B

In this experiment, people who saw an offer (in an email or popup) would see a more visible site wide reinforcement of their earned discount being active. In the control, the discount was only shown during checkout. In the variation, it was shown throughout the web site on the homepage and product detail pages.

Total Sample: 2,471 • Statistical Power at 5% MDE: 8.5%

Test #513 on Dripl.de by Jona EisenbergerJona Eisenberger Jan 23, 2024 Mobile Desktop Product X.X% Sales

Jona Tested Pattern #15: Bulleted Reassurances On Dripl.de

Pattern #15: Bulleted Reassurances - Variant A
Pattern #15: Bulleted Reassurances - Variant B

In this experiment of a floor cleaning product, a reassurance about product uses was appended to two cleaning agent upsells. In the control users saw the 2 standard upsells for laminate and tiles. And in the variation the copy "Right for me. 100x Cleanings" (uses) was appended. 

Total Sample: 69,221 • Statistical Power at 5% MDE: 18.7%

Test #511 on Online.metro-cc.ru by Andrey AndreevAndrey Andreev Jan 09, 2024 Desktop Home & Landing X.X% Sales

Andrey Tested Pattern #79: Product Highlights On Online.metro-cc.ru

Pattern #79: Product Highlights - Variant A
Pattern #79: Product Highlights - Variant B

In this experiment, popular products were shown at the top of the homepage. Impact to sale was measured.

Total Sample: 760,358 • Statistical Power at 5% MDE: 99.9%

Test #509 on Jared.com by Craig KistlerCraig Kistler Dec 18, 2023 Desktop Product X.X% Sales

Craig Tested Pattern #66: Complementary Upsell On Jared.com

Pattern #66: Complementary Upsell - Variant A
Pattern #66: Complementary Upsell - Variant B

In this experiment, a protection plan was launched as a modal for customers that clicked add-to-cart without choosing the upsell. Impact on adds-to-cart and sales was measured.

Total Sample: 407,785 • Statistical Power at 5% MDE: 23.2%

Test #507 on Fairment.de by Jona EisenbergerJona Eisenberger Dec 11, 2023 Mobile Listing X.X% Sales

Jona Tested Pattern #133: Product Availability On Fairment.de

Pattern #133: Product Availability - Variant A
Pattern #133: Product Availability - Variant B

In this experiment, product availabiltiy bars were shown on products with low stock. This was shown on listing pages. Impact on adds to cart and sales was measured.

Total Sample: 42,001 • Statistical Power at 5% MDE: 40.1%

Test #501 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Nov 02, 2023 Desktop Mobile Pricing X.X% Sales

Daria Tested Pattern #132: One Time Payment Copy On Volders.de

Pattern #132: One Time Payment Copy - Variant A
Pattern #132: One Time Payment Copy - Variant B

In this experiment, the wording "one-time payment" was appended below all three pricing plans for a contract cancelation service. Impact on orders placed was measured. (Translated from German: "Einmalige Zahlung").

Total Sample: 89,887 • Statistical Power at 5% MDE: 99.9%

Test #500 on Kayoutlet.com by Craig KistlerCraig Kistler Oct 25, 2023 Mobile Product X.X% Sales

Craig Tested Pattern #7: Social Counts On Kayoutlet.com

Pattern #7: Social Counts - Variant A
Pattern #7: Social Counts - Variant B

This was a replication attempt similar to experiment 497. A social proof message was added below add to cart buttons on product detail pages. Impact on adds to cart and transactions was measured.

Total Sample: 106,467 • Statistical Power at 5% MDE: 19.1%

Test #499 on Shmoodyapp.com by Michael McSweeneyMichael McSweeney Oct 20, 2023 Mobile Signup X.X% Sales

Michael Tested Pattern #99: Progress Bar On Shmoodyapp.com

Pattern #99: Progress Bar - Variant A
Pattern #99: Progress Bar - Variant B

In this experiment, a progress bar was added to a signup flow of a mental health app. It appeared on numerous steps of a onboarding flow. Impact on signup completions and checkouts was measured.

Total Sample: 56,735 • Statistical Power at 5% MDE: 37.1%

Test #498 by Jakub LinowskiJakub Linowski Oct 19, 2023 Desktop Mobile Product X.X% Sales

Jakub Tested Pattern #7: Social Counts

Pattern #7: Social Counts - Variant A
Pattern #7: Social Counts - Variant B

In this experiment, the variation tracked how many people would add something to cart over the last 24 hours and display that just below the add to cart button. Impact on adds to cart and transactions was measured.

Total Sample: 14,539 • Statistical Power at 5% MDE: 17.9%

Test #497 on Jared.com by Craig KistlerCraig Kistler Oct 16, 2023 Mobile Product X.X% Sales

Craig Tested Pattern #7: Social Counts On Jared.com

Pattern #7: Social Counts - Variant A
Pattern #7: Social Counts - Variant B

In this experiment, a social proof message was added below add to cart buttons on product detail pages. The copy read "Hurry, X people have viewed this product". Impact on adds to cart and transactions was measured.

Total Sample: 451,850 • Statistical Power at 5% MDE: 44.3%

Test #496 on Livefresh.de by Pascal DietzPascal Dietz Oct 03, 2023 Desktop Mobile Product X.X% Sales

Pascal Tested Pattern #43: Long Titles On Livefresh.de

Pattern #43: Long Titles - Variant A
Pattern #43: Long Titles - Variant B

In this product detail page experiment a simple subheadline was appended to a new product. The tagline read "Lose weight sustainably and healthily." (Google translated from German "Nachhaltig und gesund Gewicht verlieren."). Impact on sales and revenue was measured.

Total Sample: 68,822 • Statistical Power at 5% MDE: 51.3%

Test #494 on Online.Metro-cc.ru by Andrey AndreevAndrey Andreev Sep 20, 2023 Desktop Mobile Home & Landing X.X% Sales

Andrey Tested Pattern #26: Cart Reminder And Recently Viewed On Online.Metro-cc.ru

Pattern #26: Cart Reminder And Recently Viewed - Variant A
Pattern #26: Cart Reminder And Recently Viewed - Variant B

In this experiment, recently purchased products were appended at the top of the homepage. The test ran for loggedin users only. Impact on add-to-cart, sales and revenue was measured.

Total Sample: 53,513 • Statistical Power at 5% MDE: 78.3%