All Latest 645 A/B Tests

Test #211 on Skype App by Ronny KohaviRonny Kohavi Nov 20, 2018 Mobile Global X.X% Progression

Ronny Tested Pattern #2: Icon Labels

Pattern #2: Icon Labels - Variant A
Pattern #2: Icon Labels - Variant B

Microsoft Skype ran an experiment for the mobile segment of the Skype application with a treatment having combined icons with corresponding labels. The control only showed icons.

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Get Access to All Test Results

Test #207 on Suzuki.be by Karl GilisKarl Gilis Nov 01, 2018 Desktop Product X.X% Progression

Karl Tested Pattern #88: Action Button On Suzuki.be

Pattern #88: Action Button - Variant A
Pattern #88: Action Button - Variant B

This test ran on the Suzuki Swift landing page. In the B variation, extra links with the following copy ‘Discover the <name of car model>’ were added for each sub product. (in Dutch: ‘Bekijk de…’).

Test #201 by Ben LabayBen Labay Sep 19, 2018 Desktop Mobile Thank You X.X% Sales

Ben Tested Pattern #7: Social Counts

Pattern #7: Social Counts - Variant A
Pattern #7: Social Counts - Variant B

In this test the upsell modal had an added text box with number of people that day who took the offer. The test hypothesis was that social proof will add motivation to take an action and the offer.
Total Sample: 59,003 • Statistical Power at 5% MDE: 47.9%

Test #197 on Reverb.com by Nicholas EvansNicholas Evans Sep 04, 2018 Desktop Product X.X% Sales

Nicholas Tested Pattern #4: Testimonials On Reverb.com

Pattern #4: Testimonials - Variant A
Pattern #4: Testimonials - Variant B

In the variation, customer reviews were exposed from a less visible tab view.
Total Sample: 223,110 • Statistical Power at 5% MDE: 85.3%

Test #195 on Yummly.com by Kimberly CheungKimberly Cheung Aug 13, 2018 Desktop Mobile Listing X.X% Signups

Kimberly Tested Pattern #78: Tags, Badges And Structured Information On Yummly.com

Pattern #78: Tags, Badges And Structured Information - Variant A
Pattern #78: Tags, Badges And Structured Information - Variant B
Total Sample: 371,274 • Statistical Power at 5% MDE: 99.9%

Test #192 on Refactoring.guru by Alexander ShvetsAlexander Shvets Aug 07, 2018 Desktop Mobile Product X.X% Sales

Alexander Tested Pattern #4: Testimonials On Refactoring.guru

Pattern #4: Testimonials - Variant A
Pattern #4: Testimonials - Variant B

In this experiment, a number of customer reviews were added at the middle of a product page.
Total Sample: 6,504 • Statistical Power at 5% MDE: 3.8%

Test #193 on Yummly.com by Marcos CiarrocchiMarcos Ciarrocchi Aug 07, 2018 Desktop Mobile Signup X.X% Signups

Marcos Tested Pattern #91: Forced Action On Yummly.com

Pattern #91: Forced Action - Variant A
Pattern #91: Forced Action - Variant B

In this experiment, the presence of an additional "skip all" text link was tested on a multiple step signup flow. The skip all link allowed users to bypass personalization questions and go straight to their app dashboard. The control (A) shows its presence, and in variant B we can see it was removed.
Total Sample: 1,335,347 • Statistical Power at 5% MDE: 99.9%

Test #188 on Thomasnet.com by Julian GaviriaJulian Gaviria Jul 11, 2018 Desktop Mobile Home & Landing X.X% Progression

Julian Tested Pattern #4: Testimonials On Thomasnet.com

Pattern #4: Testimonials - Variant A
Pattern #4: Testimonials - Variant B

In this experiment, a testimonial with a repeated call to action was placed at the bottom of a landing page.
Total Sample: 3,945 • Statistical Power at 5% MDE: 8.7%

Test #186 by Devesh KhanalDevesh Khanal Jul 02, 2018 Mobile Home & Landing X.X% Sales

Devesh Tested Pattern #14: Exposed Menu Options

Pattern #14: Exposed Menu Options - Variant A
Pattern #14: Exposed Menu Options - Variant B

In this experiment, additional product categories were added at the top of the navigation.
Total Sample: 76,974 • Statistical Power at 5% MDE: 41.4%

Test #183 on Trydesignlab.com by Daniel ShapiroDaniel Shapiro Jun 19, 2018 Desktop Mobile Product X.X% Signups

Daniel Tested Pattern #46: Pay Later On Trydesignlab.com

Pattern #46: Pay Later - Variant A
Pattern #46: Pay Later - Variant B
Total Sample: 5,495 • Statistical Power at 5% MDE: 4.9%

Test #182 on Yummly.com by Marcos CiarrocchiMarcos Ciarrocchi Jun 12, 2018 Desktop Listing X.X% Engagement

Marcos Tested Pattern #6: Customer Star Ratings On Yummly.com

Pattern #6: Customer Star Ratings - Variant A
Pattern #6: Customer Star Ratings - Variant B
Total Sample: 673,296 • Statistical Power at 5% MDE: 99.9%

Test #181 on Yummly.com by Kimberly CheungKimberly Cheung Jun 04, 2018 Desktop Mobile Signup X.X% Signups

Kimberly Tested Pattern #7: Social Counts On Yummly.com

Pattern #7: Social Counts - Variant A
Pattern #7: Social Counts - Variant B
Total Sample: 175,247 • Statistical Power at 5% MDE: 61.1%

Test #180 on Radhabeauty.com by Viljo VabritViljo Vabrit Jun 03, 2018 Mobile Global X.X% Sales

Viljo Tested Pattern #45: Benefit Bar On Radhabeauty.com

Pattern #45: Benefit Bar - Variant A
Pattern #45: Benefit Bar - Variant B
Total Sample: 17,926 • Statistical Power at 5% MDE: 12.2%

Test #177 by Devesh KhanalDevesh Khanal May 17, 2018 Desktop Shopping Cart X.X% Revenue

Devesh Tested Pattern #66: Complementary Upsell

Pattern #66: Complementary Upsell - Variant A
Pattern #66: Complementary Upsell - Variant B

Test #178 by Devesh KhanalDevesh Khanal May 17, 2018 Desktop Product X.X% Sales

Devesh Tested Pattern #66: Complementary Upsell

Pattern #66: Complementary Upsell - Variant A
Pattern #66: Complementary Upsell - Variant B
Total Sample: 63,410 • Statistical Power at 5% MDE: 9.7%

Test #176 on Kenhub.com by Niels HapkeNiels Hapke May 16, 2018 Desktop Mobile Checkout X.X% Sales

Niels Tested Pattern #4: Testimonials On Kenhub.com

Pattern #4: Testimonials - Variant A
Pattern #4: Testimonials - Variant B

In this experiment, testimonials were added on a checkout screen.
Total Sample: 3,333 • Statistical Power at 5% MDE: 49.3%

Test #175 on Thomasnet.com by Julian GaviriaJulian Gaviria May 14, 2018 Desktop Mobile Content X.X% Leads

Julian Tested Pattern #23: Inline Link Nudge On Thomasnet.com

Pattern #23: Inline Link Nudge - Variant A
Pattern #23: Inline Link Nudge - Variant B
Total Sample: 67,295 • Statistical Power at 5% MDE: 3%

Test #171 on Sjvc.edu by Phillip BarnesPhillip Barnes Apr 27, 2018 Desktop Mobile Signup X.X% Leads

Phillip Tested Pattern #63: Trust Seals On Sjvc.edu

Pattern #63: Trust Seals - Variant A
Pattern #63: Trust Seals - Variant B
Total Sample: 1,197 • Statistical Power at 5% MDE: 12.8%

Test #168 on Akademiafotografii.p... by Grzegorz JancewiczGrzegorz Jancewicz Apr 23, 2018 Desktop Listing X.X% Leads

Grzegorz Tested Pattern #45: Benefit Bar On Akademiafotografii.p...

Pattern #45: Benefit Bar - Variant A
Pattern #45: Benefit Bar - Variant B
Total Sample: 4,302 • Statistical Power at 5% MDE: 5.5%

Test #167 on Lovehoney.co.uk by Matthew CurryMatthew Curry Apr 10, 2018 Desktop Product X.X% Sales

Matthew Tested Pattern #69: Autodiscounting On Lovehoney.co.uk

Pattern #69: Autodiscounting - Variant A
Pattern #69: Autodiscounting - Variant B

The idea is that, if the customer has an active discount code for that session (either by landing on a particular page, or entering a code somewhere), instead of just showing the discount in the basket, we show it further (earlier) up the funnel and automatically discount on the product page.

The effectiveness of this depends on the discount level, I've tested it at 30% and 50%.

With a 50% Discount:
Add to Cart rate +33%
Sales rate + 24%

With a 30% Discount:
Add to Cart rate +11.6%
Sales rate + 10.2%