All Latest 644 A/B Tests

Screen

Device

Metric

Tag

Powered for

Only Significant

p-value < 0.05

MOST RECENT TESTS

Test #226 on Microsoft.com by Ronny Kohavi Feb 18, 2019 Desktop Product X.X% Sales

Ronny Tested Pattern #96: Single Focus Photos On Microsoft.com

Pattern #96: Single Focus Photos - Variant A

Pattern #96: Single Focus Photos - Variant B

Microsoft Store ran an experiment on the Surface Book 2 product page. The treatment showed a hero image with fewer, yet larger product photos

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Get Access to All Test Results

Test #225 by Devesh Khanal Feb 17, 2019 Desktop Product X.X% Sales

Devesh Tested Pattern #41: Sticky Call To Action

Pattern #41: Sticky Call To Action - Variant A

Pattern #41: Sticky Call To Action - Variant B

Total Sample: 53,908 • Statistical Power at 5% MDE: 35.7%

Test #221 on Microsoft.com by Ronny Kohavi Jan 27, 2019 Desktop Product X.X% Sales

Ronny Tested Pattern #49: Above The Fold Call To Action On Microsoft.com

Pattern #49: Above The Fold Call To Action - Variant A

Pattern #49: Above The Fold Call To Action - Variant B

Microsoft Store ran an experiment on the Office 365 Home product page. The treatment raised the purchase calls to action higher by removing the hero image.

Test #212 on Mt.com by Vito Mediavilla Dec 04, 2018 Desktop Mobile Product X.X% Leads

Vito Tested Pattern #60: Repeated Bottom Call To Action On Mt.com

Pattern #60: Repeated Bottom Call To Action - Variant A

Pattern #60: Repeated Bottom Call To Action - Variant B

In this experiment, a call to action was repeated at the bottom of the product page. Additional certification icons were also added for additional reassurance.

Total Sample: 26,467 • Statistical Power at 5% MDE: 9.7%

Test #207 on Suzuki.be by Karl Gilis Nov 01, 2018 Desktop Product X.X% Progression

Karl Tested Pattern #88: Action Button On Suzuki.be

This test ran on the Suzuki Swift landing page. In the B variation, extra links with the following copy ‘Discover the <name of car model>’ were added for each sub product. (in Dutch: ‘Bekijk de…’).

Test #197 on Reverb.com by Nicholas Evans Sep 04, 2018 Desktop Product X.X% Sales

Nicholas Tested Pattern #4: Testimonials On Reverb.com

In the variation, customer reviews were exposed from a less visible tab view.

Total Sample: 223,110 • Statistical Power at 5% MDE: 85.3%

Test #192 on Refactoring.guru by Alexander Shvets Aug 07, 2018 Desktop Mobile Product X.X% Sales

Alexander Tested Pattern #4: Testimonials On Refactoring.guru

In this experiment, a number of customer reviews were added at the middle of a product page.

Total Sample: 6,504 • Statistical Power at 5% MDE: 3.8%

Test #187 on Trydesignlab.com by Will Anderson Jul 09, 2018 Desktop Mobile Product X.X% Sales

Will Tested Pattern #71: Personalized Next Step On Trydesignlab.com

Pattern #71: Personalized Next Step - Variant A

Pattern #71: Personalized Next Step - Variant B

In this experiment, a lead form (with a syllabus) would transform into the next application step of "enrolling" after being submitted. More so, this surfacing of the next enrollment step was personalized and shown for users that also returned to the web site in future visits.

Total Sample: 3,326 • Statistical Power at 5% MDE: 5.5%

Test #183 on Trydesignlab.com by Daniel Shapiro Jun 19, 2018 Desktop Mobile Product X.X% Signups

Daniel Tested Pattern #46: Pay Later On Trydesignlab.com

Total Sample: 5,495 • Statistical Power at 5% MDE: 4.9%

Test #178 by Devesh Khanal May 17, 2018 Desktop Product X.X% Sales

Devesh Tested Pattern #66: Complementary Upsell

Pattern #66: Complementary Upsell - Variant A

Pattern #66: Complementary Upsell - Variant B

Total Sample: 63,410 • Statistical Power at 5% MDE: 9.7%

Test #172 on Trydesignlab.com by Daniel Shapiro May 08, 2018 Desktop Mobile Product X.X% Leads

Daniel Tested Pattern #11: Gradual Reassurance On Trydesignlab.com

Pattern #11: Gradual Reassurance - Variant A

Pattern #11: Gradual Reassurance - Variant B

Total Sample: 16,341 • Statistical Power at 5% MDE: 14.8%

Test #167 on Lovehoney.co.uk by Matthew Curry Apr 10, 2018 Desktop Product X.X% Sales

Matthew Tested Pattern #69: Autodiscounting On Lovehoney.co.uk

Pattern #69: Autodiscounting - Variant A

Pattern #69: Autodiscounting - Variant B

The idea is that, if the customer has an active discount code for that session (either by landing on a particular page, or entering a code somewhere), instead of just showing the discount in the basket, we show it further (earlier) up the funnel and automatically discount on the product page.

The effectiveness of this depends on the discount level, I've tested it at 30% and 50%.

With a 50% Discount:
Add to Cart rate +33%
Sales rate + 24%

With a 30% Discount:
Add to Cart rate +11.6%
Sales rate + 10.2%

Test #163 on Lovehoney.co.uk by Matthew Curry Mar 27, 2018 Mobile Desktop Product X.X% Revenue

Matthew Tested Pattern #62: Urgent Next Day Delivery On Lovehoney.co.uk

Pattern #62: Urgent Next Day Delivery - Variant A

Pattern #62: Urgent Next Day Delivery - Variant B

Test #160 on Examine.com by Martin Wong Mar 15, 2018 Desktop Mobile Product X.X% Sales

Martin Tested Pattern #49: Above The Fold Call To Action On Examine.com

Total Sample: 25,025 • Statistical Power at 5% MDE: 9%

Test #157 on Bobandlush.com by Viljo Vabrit Mar 06, 2018 Desktop Product X.X% Sales

Viljo Tested Pattern #62: Urgent Next Day Delivery On Bobandlush.com

Hypothesis - Urgency leads to higher conversions as people have less time to think and will instead follow their impulse to complete the purchase now rather than later.

The Treatment - From Monday to Friday, before 4 PM, we added the urgency message “Free next business day delivery if you order before 4 PM (UK)”. We placed the urgency message just below the product name ensuring that it fell into the reading pattern of all visitors interested in the products. To ensure the treatment was only shown at the right time and for the right people we added a condition: MON, TUE, WED, THU, FRI before 4 PM; UK IP.

Total Sample: 1,877 • Statistical Power at 5% MDE: 6.9%

Test #155 on Mt.com by Vito Mediavilla Feb 22, 2018 Mobile Product X.X% Leads