All Latest 634 A/B Tests

MOST RECENT TESTS

Test #242 on by Alex JamesAlex James May 27, 2019 Desktop Mobile Signup X.X% Signups

Alex Tested Pattern #7: Social Counts

 - Variant A
 - Variant B
Total Sample: 79,012 Statistical Power at 5% MDE: 56.8%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #241 on Kenhub.com by Niels HapkeNiels Hapke May 24, 2019 Desktop Mobile Signup X.X% Signups

Niels Tested Pattern #105: Lead Magnets On Kenhub.com

 - Variant A
 - Variant B

In this experiment, a free study guide ebook was promoted on a registration page.

Total Sample: 63,631 Statistical Power at 5% MDE: 87.5%

Test #237 on Goodui.org by Jakub LinowskiJakub Linowski Apr 29, 2019 Desktop Mobile Listing X.X% Sales

Jakub Tested Pattern #103: Money Back Guarantee On Goodui.org

 - Variant A
 - Variant B
Total Sample: 21,096 Statistical Power at 5% MDE: 6%

Test #235 on Thomasnet.com by Julian GaviriaJulian Gaviria Apr 02, 2019 Desktop Mobile Home & Landing X.X% Leads

Julian Tested Pattern #102: Expanded Or Condensed Layout On Thomasnet.com

 - Variant A
 - Variant B

In this experiment, the layout was condensed from a taller to a shorter one.

Total Sample: 11,567 Statistical Power at 5% MDE: 17.3%

Test #232 on Yoast.com by Sjardo JanssenSjardo Janssen Mar 15, 2019 Desktop Mobile Checkout X.X% Sales

Sjardo Tested Pattern #6: Customer Star Ratings On Yoast.com

 - Variant A
 - Variant B

In this experiment, the Yoast team replaced one of the bulleted reassurances on their checkout page ("4.9 out of 5 stars" vs "24/7 Free support"). Raising the question - is free support or high reviews valued more? - Thanks Sjardo & Meike for sharing!

Total Sample: 20,661 Statistical Power at 5% MDE: 30.3%

Test #231 on Glass.net by Mark FreedleMark Freedle Mar 14, 2019 Desktop Mobile Signup X.X% Leads

Mark Tested Pattern #20: Canned Response On Glass.net

 - Variant A
 - Variant B
Total Sample: 1,342 Statistical Power at 5% MDE: 6.4%

Test #230 on Goodui.org by Jakub LinowskiJakub Linowski Mar 09, 2019 Desktop Mobile Listing X.X% Engagement

Jakub Tested Pattern #56: Hover Button On Goodui.org

 - Variant A
 - Variant B

In this test we tested onhover buttons (variant) versus more traditional always exposed and visible ones.

Total Sample: 30,699 Statistical Power at 5% MDE: 98.1%

Test #229 on by Jakub LinowskiJakub Linowski Mar 08, 2019 Desktop Mobile Checkout X.X% Sales

Jakub Tested Pattern #99: Progress Bar

 - Variant A
 - Variant B
Total Sample: 11,214 Statistical Power at 5% MDE: 13%

Test #228 on by Jakub LinowskiJakub Linowski Mar 05, 2019 Desktop Mobile Checkout X.X% Sales

Jakub Tested Pattern #99: Progress Bar

 - Variant A
 - Variant B

The variation added a progress bar to one of the checkout steps for a ticket ordering site.

Total Sample: 12,961 Statistical Power at 5% MDE: 14.7%

Test #223 on Volders.de by Alexander KriegerAlexander Krieger Feb 01, 2019 Desktop Mobile Signup X.X% Revenue

Alexander Tested Pattern #12: Payment First On Volders.de

 - Variant A
 - Variant B

This test deprioritized the free option (kostenlos) of cancelling a contract. It did so by placing it under the paid options as small text link / radio option.

Test #218 on Yummly.com by Kimberly CheungKimberly Cheung Jan 14, 2019 Desktop Mobile Home & Landing X.X% Signups

Kimberly Tested Pattern #94: Visible Search On Yummly.com

 - Variant A
 - Variant B

Hypothesis: Anonymous users can't use global search while on the home promo page. We believe that if we show a global search bar to anonymous users, it presents a higher converting funnel (guided search) and will increase our sign-up rates significantly.

Control (A): Logged out users don't see global search bar.

Variant (B): Logged out users see global search bar. After searching for a keyword, the signup funnel starts with a more personalized reason to continue the signup process.

Total Sample: 20,122 Statistical Power at 5% MDE: 13.8%

Test #219 on Mt.com by Vito MediavillaVito Mediavilla Jan 14, 2019 Desktop Mobile Home & Landing X.X% Leads

Vito Tested Pattern #95: Clickable Product Previews On Mt.com

 - Variant A
 - Variant B
Total Sample: 4,836 Statistical Power at 5% MDE: 11%

Test #217 on Thomasnet.com by Julian GaviriaJulian Gaviria Jan 03, 2019 Desktop Mobile Home & Landing X.X% Engagement

Julian Tested Pattern #41: Sticky Call To Action On Thomasnet.com

 - Variant A
 - Variant B
Total Sample: 158,023 Statistical Power at 5% MDE: 8.9%

Test #215 on Vivareal.com.br by Vinicius Barros PeixotoVinicius Barros Peixoto Dec 21, 2018 Mobile Listing X.X% Leads

Vinicius Tested Pattern #92: Already Viewed Label On Vivareal.com.br

 - Variant A
 - Variant B

The idea of this test was to add a "Viewed" label on a listing page to indicate listings which have already been viewed by users.

Total Sample: 2,452,026 Statistical Power at 5% MDE: 99.9%

Test #214 on Yummly.com by Marcos CiarrocchiMarcos Ciarrocchi Dec 07, 2018 Mobile Signup X.X% Signups

Marcos Tested Pattern #91: Forced Action On Yummly.com

 - Variant A
 - Variant B

In this experiment, the highly prominent "skip" button was replaced with a less prominent text link. The copy of the skip text link also clarified the consequence of the action - losing out on personalization benefits. More so, the habitual top-right cancel icon was also removed.

Test #212 on Mt.com by Vito MediavillaVito Mediavilla Dec 04, 2018 Desktop Mobile Product X.X% Leads

Vito Tested Pattern #60: Repeated Bottom Call To Action On Mt.com

 - Variant A
 - Variant B

In this experiment, a call to action was repeated at the bottom of the product page. Additional certification icons were also added for additional reassurance.

Total Sample: 26,467 Statistical Power at 5% MDE: 9.7%

Test #213 on Mt.com by Vito MediavillaVito Mediavilla Dec 04, 2018 Desktop Mobile Signup X.X% Leads

Vito Tested Pattern #85: Benefit Button On Mt.com

 - Variant A
 - Variant B

In this variation, a longer button label with a clearer benefit was tested against a shorter one.

Total Sample: 26,639 Statistical Power at 5% MDE: 12.8%

Test #211 on Skype App by Ronny KohaviRonny Kohavi Nov 20, 2018 Mobile Global X.X% Progression

Ronny Tested Pattern #2: Icon Labels

 - Variant A
 - Variant B

Microsoft Skype ran an experiment for the mobile segment of the Skype application with a treatment having combined icons with corresponding labels. The control only showed icons.

Test #210 on Bomgar.com by Lee ElkinsLee Elkins Nov 13, 2018 Desktop Mobile Home & Landing X.X% Signups

Lee Tested Pattern #64: Tunnel On Bomgar.com

 - Variant A
 - Variant B

In this experiment, the header navigation links were removed on a landing page in order to provide more focus to the signup form.

Total Sample: 13,871 Statistical Power at 5% MDE: 6.3%

Test #209 on Vivareal.com.br by Vinicius Barros PeixotoVinicius Barros Peixoto Nov 12, 2018 Mobile Listing X.X% Leads

Vinicius Tested Pattern #34: Open In A New Tab On Vivareal.com.br

 - Variant A
 - Variant B

The idea of this experiment was taking advantage of mobile browser behavior. When a link is open in a new tab on mobile browsers, and users hit the back button, the tab closes and users get back exactly where they were before without any new result page load.

Total Sample: 778,955 Statistical Power at 5% MDE: 99.9%