All Latest 645 A/B Tests

MOST RECENT TESTS

Test #181 on Yummly.com by Kimberly CheungKimberly Cheung Jun 04, 2018 Desktop Mobile Signup X.X% Signups

Kimberly Tested Pattern #7: Social Counts On Yummly.com

Pattern #7: Social Counts - Variant A
Pattern #7: Social Counts - Variant B
Total Sample: 175,247 • Statistical Power at 5% MDE: 61.1%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Get Access to All Test Results

Test #180 on Radhabeauty.com by Viljo VabritViljo Vabrit Jun 03, 2018 Mobile Global X.X% Sales

Viljo Tested Pattern #45: Benefit Bar On Radhabeauty.com

Pattern #45: Benefit Bar - Variant A
Pattern #45: Benefit Bar - Variant B
Total Sample: 17,926 • Statistical Power at 5% MDE: 12.2%

Test #176 on Kenhub.com by Niels HapkeNiels Hapke May 16, 2018 Desktop Mobile Checkout X.X% Sales

Niels Tested Pattern #4: Testimonials On Kenhub.com

Pattern #4: Testimonials - Variant A
Pattern #4: Testimonials - Variant B

In this experiment, testimonials were added on a checkout screen.
Total Sample: 3,333 • Statistical Power at 5% MDE: 49.3%

Test #174 on Examine.com by Martin WongMartin Wong May 13, 2018 Desktop Mobile Checkout X.X% Sales

Martin Tested Pattern #67: Currency & Taxes On Examine.com

Pattern #67: Currency & Taxes - Variant A
Pattern #67: Currency & Taxes - Variant B
Total Sample: 1,702 • Statistical Power at 5% MDE: 8.3%

Test #168 on Akademiafotografii.p... by Grzegorz JancewiczGrzegorz Jancewicz Apr 23, 2018 Desktop Listing X.X% Leads

Grzegorz Tested Pattern #45: Benefit Bar On Akademiafotografii.p...

Pattern #45: Benefit Bar - Variant A
Pattern #45: Benefit Bar - Variant B
Total Sample: 4,302 • Statistical Power at 5% MDE: 5.5%

Test #163 on Lovehoney.co.uk by Matthew CurryMatthew Curry Mar 27, 2018 Mobile Desktop Product X.X% Revenue

Matthew Tested Pattern #62: Urgent Next Day Delivery On Lovehoney.co.uk

Pattern #62: Urgent Next Day Delivery - Variant A
Pattern #62: Urgent Next Day Delivery - Variant B

Test #157 on Bobandlush.com by Viljo VabritViljo Vabrit Mar 06, 2018 Desktop Product X.X% Sales

Viljo Tested Pattern #62: Urgent Next Day Delivery On Bobandlush.com

Pattern #62: Urgent Next Day Delivery - Variant A
Pattern #62: Urgent Next Day Delivery - Variant B

Hypothesis - Urgency leads to higher conversions as people have less time to think and will instead follow their impulse to complete the purchase now rather than later.

The Treatment - From Monday to Friday, before 4 PM, we added the urgency message “Free next business day delivery if you order before 4 PM (UK)”. We placed the urgency message just below the product name ensuring that it fell into the reading pattern of all visitors interested in the products. To ensure the treatment was only shown at the right time and for the right people we added a condition: MON, TUE, WED, THU, FRI before 4 PM; UK IP.
Total Sample: 1,877 • Statistical Power at 5% MDE: 6.9%

Test #152 on Reverb.com by Nicholas EvansNicholas Evans Feb 13, 2018 Desktop Mobile Product X.X% Sales

Nicholas Tested Pattern #15: Bulleted Reassurances On Reverb.com

Pattern #15: Bulleted Reassurances - Variant A
Pattern #15: Bulleted Reassurances - Variant B
Total Sample: 487,851 • Statistical Power at 5% MDE: 29.3%

Test #151 on Mamaearth.ca by Michael BernsteinMichael Bernstein Feb 08, 2018 Desktop Checkout X.X% Sales

Michael Tested Pattern #45: Benefit Bar On Mamaearth.ca

Pattern #45: Benefit Bar - Variant A
Pattern #45: Benefit Bar - Variant B
Total Sample: 1,614 • Statistical Power at 5% MDE: 7.4%

Test #150 on Normanrecords.com by Nathon RaineNathon Raine Feb 07, 2018 Desktop Home & Landing X.X% Sales

Nathon Tested Pattern #45: Benefit Bar On Normanrecords.com

Pattern #45: Benefit Bar - Variant A
Pattern #45: Benefit Bar - Variant B
Total Sample: 120,700 • Statistical Power at 5% MDE: 35.2%

Test #140 on Akademiafotografii.p... by Grzegorz JancewiczGrzegorz Jancewicz Jan 05, 2018 Desktop Mobile Product X.X% Leads

Grzegorz Tested Pattern #46: Pay Later On Akademiafotografii.p...

Pattern #46: Pay Later - Variant A
Pattern #46: Pay Later - Variant B

The test was run on multiple course pages. The screenshot contains the cropped bottom part of a long screen with an exposed signup form. The variation introduced additional text above the form which states: "Free Cancellation. Payment is not required today".
Total Sample: 37,727 • Statistical Power at 5% MDE: 19.9%

Test #137 on Trydesignlab.com by Daniel ShapiroDaniel Shapiro Dec 22, 2017 Desktop Mobile Checkout X.X% Sales

Daniel Tested Pattern #46: Pay Later On Trydesignlab.com

Pattern #46: Pay Later - Variant A
Pattern #46: Pay Later - Variant B

This test was run on a 3 step checkout process. The first screen was asking for contact information, and the second screen asked for credit card details. The change was shown on both first two steps as shown on the image below.
Total Sample: 2,218 • Statistical Power at 5% MDE: 7.3%

Test #138 on Trydesignlab.com by Daniel ShapiroDaniel Shapiro Dec 22, 2017 Desktop Mobile Checkout X.X% Sales

Daniel Tested Pattern #42: Countdown Timer On Trydesignlab.com

Pattern #42: Countdown Timer - Variant A
Pattern #42: Countdown Timer - Variant B

This test was run on a 3 step checkout process. The first screen was asking for contact information, and the second screen asked for credit card details. The change was shown on both first two steps as shown on the image below.
Total Sample: 2,258 • Statistical Power at 5% MDE: 8.2%

Test #135 on Akademiafotografii.p... by Grzegorz JancewiczGrzegorz Jancewicz Dec 18, 2017 Desktop Home & Landing X.X% Leads

Grzegorz Tested Pattern #4: Testimonials On Akademiafotografii.p...

Pattern #4: Testimonials - Variant A
Pattern #4: Testimonials - Variant B
Total Sample: 3,927 • Statistical Power at 5% MDE: 5.8%

Test #136 on Missetam.nl by Marlies Wilms FloetMarlies Wilms Floet Dec 18, 2017 Desktop Product X.X% Sales

Marlies Wilms Tested Pattern #42: Countdown Timer On Missetam.nl

Pattern #42: Countdown Timer - Variant A
Pattern #42: Countdown Timer - Variant B

De Nieuwe Zaak (a Dutch ecommerce agency) tested a countdown timer on an ecommerce site. The variation showed how many hours and minutes were left in order to be eligible for next day delivery. The variation beat the control which did not have this element of urgency.

Test #133 on Bing.com by Ronny KohaviRonny Kohavi Dec 13, 2017 Desktop Mobile Listing X.X% Revenue

Ronny Tested Pattern #43: Long Titles On Bing.com

Pattern #43: Long Titles - Variant A
Pattern #43: Long Titles - Variant B

In 2012 a Microsoft employee working on Bing had an idea about changing the way the search engine displayed ad headlines. Developing it wouldn’t require much effort—just a few days of an engineer’s time—but it was one of hundreds of ideas proposed, and the program managers deemed it a low priority. So it languished for more than six months, until an engineer, who saw that the cost of writing the code for it would be small, launched a simple online controlled experiment—an A/B test—to assess its impact. Within hours the new headline variation was producing abnormally high revenue, triggering a “too good to be true” alert.

HBR, September–October 2017 Issue, https://hbr.org/2017/09/the-surprising-power-of-online-experiments

Note: This experiment was a solid success and replicated multiple times over a period of months. It worked at Bing and had a profound influence. The only reason why we atributed a 0.25 point (a "Maybe") was because we don't have the exact sample size and conversion data.

 

Test #128 on Acousticalsurfaces.c... by Nina BayattiNina Bayatti Dec 02, 2017 Desktop Signup X.X% Leads

Nina Tested Pattern #20: Canned Response On Acousticalsurfaces.c...

Pattern #20: Canned Response - Variant A
Pattern #20: Canned Response - Variant B
Total Sample: 2,951 • Statistical Power at 5% MDE: 7.1%

Test #127 on Glass.net by Mark FreedleMark Freedle Nov 23, 2017 Desktop Mobile Listing X.X% Leads

Mark Tested Pattern #42: Countdown Timer On Glass.net

Pattern #42: Countdown Timer - Variant A
Pattern #42: Countdown Timer - Variant B
Total Sample: 61,034 • Statistical Power at 5% MDE: 23.7%

Test #117 on Ssdnodes.com by Matt ConnorMatt Connor Oct 02, 2017 Desktop Shopping Cart X.X% Sales

Matt Tested Pattern #21: What It's Worth On Ssdnodes.com

Pattern #21: What It's Worth - Variant A
Pattern #21: What It's Worth - Variant B
Total Sample: 4,000 • Statistical Power at 5% MDE: 7.9%

Test #115 on Akademiafotografii.p... by Grzegorz JancewiczGrzegorz Jancewicz Sep 01, 2017 Desktop Product X.X% Leads

Grzegorz Tested Pattern #4: Testimonials On Akademiafotografii.p...

Pattern #4: Testimonials - Variant A
Pattern #4: Testimonials - Variant B
Total Sample: 13,817 • Statistical Power at 5% MDE: 9.8%