All Latest 644 A/B Tests

Screen

Device

Metric

Tag

Powered for

Only Significant

p-value < 0.05

Test #232 on Yoast.com by Sjardo Janssen Mar 15, 2019 Desktop Mobile Checkout X.X% Sales

Sjardo Tested Pattern #6: Customer Star Ratings On Yoast.com

Pattern #6: Customer Star Ratings - Variant A

Pattern #6: Customer Star Ratings - Variant B

In this experiment, the Yoast team replaced one of the bulleted reassurances on their checkout page ("4.9 out of 5 stars" vs "24/7 Free support"). Raising the question - is free support or high reviews valued more? - Thanks Sjardo & Meike for sharing!

Total Sample: 20,661 • Statistical Power at 5% MDE: 30.3%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Get Access to All Test Results

Test #230 on Goodui.org by Jakub Linowski Mar 09, 2019 Desktop Mobile Listing X.X% Engagement

Jakub Tested Pattern #56: Hover Button On Goodui.org

In this test we tested onhover buttons (variant) versus more traditional always exposed and visible ones.

Total Sample: 30,699 • Statistical Power at 5% MDE: 98.1%

Test #226 on Microsoft.com by Ronny Kohavi Feb 18, 2019 Desktop Product X.X% Sales

Ronny Tested Pattern #96: Single Focus Photos On Microsoft.com

Pattern #96: Single Focus Photos - Variant A

Pattern #96: Single Focus Photos - Variant B

Microsoft Store ran an experiment on the Surface Book 2 product page. The treatment showed a hero image with fewer, yet larger product photos

Test #225 by Devesh Khanal Feb 17, 2019 Desktop Product X.X% Sales

Devesh Tested Pattern #41: Sticky Call To Action

Pattern #41: Sticky Call To Action - Variant A

Pattern #41: Sticky Call To Action - Variant B

Total Sample: 53,908 • Statistical Power at 5% MDE: 35.7%

Test #223 on Volders.de by Alexander Krieger Feb 01, 2019 Desktop Mobile Signup X.X% Revenue

Alexander Tested Pattern #12: Payment First On Volders.de

This test deprioritized the free option (kostenlos) of cancelling a contract. It did so by placing it under the paid options as small text link / radio option.

Test #221 on Microsoft.com by Ronny Kohavi Jan 27, 2019 Desktop Product X.X% Sales

Ronny Tested Pattern #49: Above The Fold Call To Action On Microsoft.com

Pattern #49: Above The Fold Call To Action - Variant A

Pattern #49: Above The Fold Call To Action - Variant B

Microsoft Store ran an experiment on the Office 365 Home product page. The treatment raised the purchase calls to action higher by removing the hero image.

Test #220 by Alex James Jan 18, 2019 Desktop Listing X.X% Leads

Alex Tested Pattern #34: Open In A New Tab

Pattern #34: Open In A New Tab - Variant A

Pattern #34: Open In A New Tab - Variant B

This experiment measured the effect of opening new listing (job applications) in a new tab, against opening them in the same window. The experiment A-B was inversed to match the pattern (in reality, the original already opened the tabs in a new window).

Total Sample: 55,703 • Statistical Power at 5% MDE: 99.9%

Test #214 on Yummly.com by Marcos Ciarrocchi Dec 07, 2018 Mobile Signup X.X% Signups

Marcos Tested Pattern #91: Forced Action On Yummly.com

In this experiment, the highly prominent "skip" button was replaced with a less prominent text link. The copy of the skip text link also clarified the consequence of the action - losing out on personalization benefits. More so, the habitual top-right cancel icon was also removed.

Test #213 on Mt.com by Vito Mediavilla Dec 04, 2018 Desktop Mobile Signup X.X% Leads

Vito Tested Pattern #85: Benefit Button On Mt.com

In this variation, a longer button label with a clearer benefit was tested against a shorter one.

Total Sample: 26,639 • Statistical Power at 5% MDE: 12.8%

Test #209 on Vivareal.com.br by Vinicius Barros Peixoto Nov 12, 2018 Mobile Listing X.X% Leads

Vinicius Tested Pattern #34: Open In A New Tab On Vivareal.com.br

The idea of this experiment was taking advantage of mobile browser behavior. When a link is open in a new tab on mobile browsers, and users hit the back button, the tab closes and users get back exactly where they were before without any new result page load.

Total Sample: 778,955 • Statistical Power at 5% MDE: 99.9%

Test #208 on Thomasnet.com by Julian Gaviria Nov 02, 2018 Desktop Mobile Listing X.X% Leads