All Latest 633 A/B Tests

Screen

Device

Metric

Tag

Powered for

Only Significant

p-value < 0.05

MOST RECENT TESTS

Test #221 on Microsoft.com by Ronny Kohavi Jan 27, 2019 Desktop Product X.X% Sales

Ronny Tested Pattern #49: Above The Fold Call To Action On Microsoft.com

Microsoft Store ran an experiment on the Office 365 Home product page. The treatment raised the purchase calls to action higher by removing the hero image.

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Get Access to All Test Results

Test #212 on Mt.com by Vito Mediavilla Dec 04, 2018 Desktop Mobile Product X.X% Leads

Vito Tested Pattern #60: Repeated Bottom Call To Action On Mt.com

In this experiment, a call to action was repeated at the bottom of the product page. Additional certification icons were also added for additional reassurance.

Total Sample: 26,467 • Statistical Power at 5% MDE: 9.7%

Test #207 on Suzuki.be by Karl Gilis Nov 01, 2018 Desktop Product X.X% Progression

Karl Tested Pattern #88: Action Button On Suzuki.be

This test ran on the Suzuki Swift landing page. In the B variation, extra links with the following copy ‘Discover the <name of car model>’ were added for each sub product. (in Dutch: ‘Bekijk de…’).

Test #197 on Reverb.com by Nicholas Evans Sep 04, 2018 Desktop Product X.X% Sales

Nicholas Tested Pattern #4: Testimonials On Reverb.com

In the variation, customer reviews were exposed from a less visible tab view.

Total Sample: 223,110 • Statistical Power at 5% MDE: 85.3%

Test #192 on Refactoring.guru by Alexander Shvets Aug 07, 2018 Desktop Mobile Product X.X% Sales

Alexander Tested Pattern #4: Testimonials On Refactoring.guru

In this experiment, a number of customer reviews were added at the middle of a product page.

Total Sample: 6,504 • Statistical Power at 5% MDE: 3.8%

Test #187 on Trydesignlab.com by Will Anderson Jul 09, 2018 Desktop Mobile Product X.X% Sales

Will Tested Pattern #71: Personalized Next Step On Trydesignlab.com

In this experiment, a lead form (with a syllabus) would transform into the next application step of "enrolling" after being submitted. More so, this surfacing of the next enrollment step was personalized and shown for users that also returned to the web site in future visits.

Total Sample: 3,326 • Statistical Power at 5% MDE: 5.5%

Test #183 on Trydesignlab.com by Daniel Shapiro Jun 19, 2018 Desktop Mobile Product X.X% Signups

Daniel Tested Pattern #46: Pay Later On Trydesignlab.com

Total Sample: 5,495 • Statistical Power at 5% MDE: 4.9%

Test #178 on by Devesh Khanal May 17, 2018 Desktop Product X.X% Sales

Devesh Tested Pattern #66: Complementary Upsell

Total Sample: 63,410 • Statistical Power at 5% MDE: 9.7%

Test #172 on Trydesignlab.com by Daniel Shapiro May 08, 2018 Desktop Mobile Product X.X% Leads

Daniel Tested Pattern #11: Gradual Reassurance On Trydesignlab.com

Total Sample: 16,341 • Statistical Power at 5% MDE: 14.8%

Test #167 on Lovehoney.co.uk by Matthew Curry Apr 10, 2018 Desktop Product X.X% Sales

Matthew Tested Pattern #69: Autodiscounting On Lovehoney.co.uk

The idea is that, if the customer has an active discount code for that session (either by landing on a particular page, or entering a code somewhere), instead of just showing the discount in the basket, we show it further (earlier) up the funnel and automatically discount on the product page.

The effectiveness of this depends on the discount level, I've tested it at 30% and 50%.

With a 50% Discount:
Add to Cart rate +33%
Sales rate + 24%

With a 30% Discount:
Add to Cart rate +11.6%
Sales rate + 10.2%

Test #163 on Lovehoney.co.uk by Matthew Curry Mar 27, 2018 Mobile Desktop Product X.X% Revenue

Matthew Tested Pattern #62: Urgent Next Day Delivery On Lovehoney.co.uk

Test #160 on Examine.com by Martin Wong Mar 15, 2018 Desktop Mobile Product X.X% Sales

Martin Tested Pattern #49: Above The Fold Call To Action On Examine.com

Total Sample: 25,025 • Statistical Power at 5% MDE: 9%

Test #157 on Bobandlush.com by Viljo Vabrit Mar 06, 2018 Desktop Product X.X% Sales

Viljo Tested Pattern #62: Urgent Next Day Delivery On Bobandlush.com

Hypothesis - Urgency leads to higher conversions as people have less time to think and will instead follow their impulse to complete the purchase now rather than later.

The Treatment - From Monday to Friday, before 4 PM, we added the urgency message “Free next business day delivery if you order before 4 PM (UK)”. We placed the urgency message just below the product name ensuring that it fell into the reading pattern of all visitors interested in the products. To ensure the treatment was only shown at the right time and for the right people we added a condition: MON, TUE, WED, THU, FRI before 4 PM; UK IP.

Total Sample: 1,877 • Statistical Power at 5% MDE: 6.9%

Test #155 on Mt.com by Vito Mediavilla Feb 22, 2018 Mobile Product X.X% Leads