All Latest 650 A/B Tests

MOST RECENT TESTS

Test #257 on Thomasnet.com by Julian GaviriaJulian Gaviria Sep 09, 2019 Desktop Mobile Signup X.X% Signups

Julian Tested Pattern #109: Required Field Labels On Thomasnet.com

Pattern #109: Required Field Labels - Variant A
Pattern #109: Required Field Labels - Variant B

In this followup experiment, field labels without and with a marked asterisk were tested.
Total Sample: 5,739 • Statistical Power at 5% MDE: 31.3%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Get Access to All Test Results

Test #256 by Alex JamesAlex James Aug 23, 2019 Desktop Mobile Signup X.X% Signups

Alex Tested Pattern #109: Required Field Labels

Pattern #109: Required Field Labels - Variant A
Pattern #109: Required Field Labels - Variant B

The original had no fields marked as required. The variant had all fields marked as required with an asterisk (and a reference note).
Total Sample: 116,470 • Statistical Power at 5% MDE: 75.4%

Test #255 on Thomasnet.com by Julian GaviriaJulian Gaviria Aug 22, 2019 Desktop Mobile Signup X.X% Signups

Julian Tested Pattern #109: Required Field Labels On Thomasnet.com

Pattern #109: Required Field Labels - Variant A
Pattern #109: Required Field Labels - Variant B

In this experiment, field labels without and with a marked asterisk were tested.
Total Sample: 5,977 • Statistical Power at 5% MDE: 22.6%

Test #254 on Volders.de by Alexander KriegerAlexander Krieger Aug 16, 2019 Desktop Mobile Signup X.X% Revenue

Alexander Tested Pattern #17: Least Or Most Expensive First On Volders.de

Pattern #17: Least Or Most Expensive First - Variant A
Pattern #17: Least Or Most Expensive First - Variant B

In this experiment, 4 things were adjusted in the variation: the highest pricing plan was shifted to the left, it was set as the default one, the recommendation was also adjusted to point to the highest plan, and one benefit from the lowest plan was removed (customer support).

Test #253 on Volders.de by Alexander KriegerAlexander Krieger Aug 08, 2019 Desktop Mobile Signup X.X% Revenue

Alexander Tested Pattern #17: Least Or Most Expensive First On Volders.de

Pattern #17: Least Or Most Expensive First - Variant A
Pattern #17: Least Or Most Expensive First - Variant B

In this experiment, two pricing plans were inverted to show the most expensive plan first (in the variation).

Test #252 on Thomasnet.com by Julian GaviriaJulian Gaviria Jul 30, 2019 Desktop Mobile Content X.X% Engagement

Julian Tested Pattern #107: Contrast Links & Buttons On Thomasnet.com

Pattern #107: Contrast Links & Buttons - Variant A
Pattern #107: Contrast Links & Buttons - Variant B

In this experiment a light blue bold link was tested against a darker blue bold link. 
Total Sample: 142,102 • Statistical Power at 5% MDE: 74.3%

Test #250 on Volders.de by Alexander KriegerAlexander Krieger Jul 25, 2019 Desktop Mobile Signup X.X% Signups

Alexander Tested Pattern #106: Back Buttons On Volders.de

Pattern #106: Back Buttons - Variant A
Pattern #106: Back Buttons - Variant B

In this experiment, a version without a back button was tested against a one where it was positioned in the upper left corner. This test occured on a second step of a contract cancellation service.
Total Sample: 16,894 • Statistical Power at 5% MDE: 81%

Test #251 on Goodui.org by Jakub LinowskiJakub Linowski Jul 25, 2019 Desktop Mobile Content X.X% Signups

Jakub Tested Pattern #57: Maybe Later On Goodui.org

Pattern #57: Maybe Later - Variant A
Pattern #57: Maybe Later - Variant B

In this experiment, we tested a standard modal (with 2 choices) against a "Maybe Later" one (with 3 choices). One of the choices in the variant allowed users to postpone their decision with a "maybe" which would enable a floating bar at the bottom of the screen. Clicking on any of the "Yes" options would send people to the bottom of the screen with an email signup form. Increasing signup was our primary measure. Both modals also appeared instantly after a page load.
Total Sample: 7,759 • Statistical Power at 5% MDE: 7.2%

Test #247 on Thomasnet.com by Julian GaviriaJulian Gaviria Jun 13, 2019 Desktop Mobile Content X.X% Signups

Julian Tested Pattern #41: Sticky Call To Action On Thomasnet.com

Pattern #41: Sticky Call To Action - Variant A
Pattern #41: Sticky Call To Action - Variant B

In this experiment, the same message (with a link) for signing up to a newsletter was shown in two distinct ways. The control (A) showed the signup message as inline one that preceded the content of the article at the very top. The variant showed the same signup message as a scroll-delayed sticky interaction at the bottom of the screen. The background color of the B variant was also inverted to match the style of the footer.
Total Sample: 76,900 • Statistical Power at 5% MDE: 4.6%

Test #246 on Thomasnet.com by Julian GaviriaJulian Gaviria Jun 12, 2019 Desktop Mobile Home & Landing X.X% Progression

Julian Tested Pattern #88: Action Button On Thomasnet.com

Pattern #88: Action Button - Variant A
Pattern #88: Action Button - Variant B

In this experiment, the navigation label was changed from "Network" to "Find Suppliers". The idea was to make use of an action label with a clearer benefit.
Total Sample: 956,112 • Statistical Power at 5% MDE: 47.9%

Test #243 on Goodui.org by Jakub LinowskiJakub Linowski Jun 05, 2019 Desktop Mobile Home & Landing X.X% Progression

Jakub Tested Pattern #77: Filled Or Ghost Buttons On Goodui.org

Pattern #77: Filled Or Ghost Buttons - Variant A
Pattern #77: Filled Or Ghost Buttons - Variant B

In this experiment, full red background buttons were changed to ghost buttons (red outline and transparent fill)
Total Sample: 25,046 • Statistical Power at 5% MDE: 22.6%

Test #242 by Alex JamesAlex James May 27, 2019 Desktop Mobile Signup X.X% Signups

Alex Tested Pattern #7: Social Counts

Pattern #7: Social Counts - Variant A
Pattern #7: Social Counts - Variant B
Total Sample: 79,012 • Statistical Power at 5% MDE: 56.8%

Test #241 on Kenhub.com by Niels HapkeNiels Hapke May 24, 2019 Desktop Mobile Signup X.X% Signups

Niels Tested Pattern #105: Lead Magnets On Kenhub.com

Pattern #105: Lead Magnets - Variant A
Pattern #105: Lead Magnets - Variant B

In this experiment, a free study guide ebook was promoted on a registration page.
Total Sample: 63,631 • Statistical Power at 5% MDE: 87.5%

Test #237 on Goodui.org by Jakub LinowskiJakub Linowski Apr 29, 2019 Desktop Mobile Listing X.X% Sales

Jakub Tested Pattern #103: Money Back Guarantee On Goodui.org

Pattern #103: Money Back Guarantee - Variant A
Pattern #103: Money Back Guarantee - Variant B
Total Sample: 21,096 • Statistical Power at 5% MDE: 6%

Test #235 on Thomasnet.com by Julian GaviriaJulian Gaviria Apr 02, 2019 Desktop Mobile Home & Landing X.X% Leads

Julian Tested Pattern #102: Expanded Or Condensed Layout On Thomasnet.com

Pattern #102: Expanded Or Condensed Layout - Variant A
Pattern #102: Expanded Or Condensed Layout - Variant B

In this experiment, the layout was condensed from a taller to a shorter one.
Total Sample: 11,567 • Statistical Power at 5% MDE: 17.3%

Test #232 on Yoast.com by Sjardo JanssenSjardo Janssen Mar 15, 2019 Desktop Mobile Checkout X.X% Sales

Sjardo Tested Pattern #6: Customer Star Ratings On Yoast.com

Pattern #6: Customer Star Ratings - Variant A
Pattern #6: Customer Star Ratings - Variant B

In this experiment, the Yoast team replaced one of the bulleted reassurances on their checkout page ("4.9 out of 5 stars" vs "24/7 Free support"). Raising the question - is free support or high reviews valued more? - Thanks Sjardo & Meike for sharing!
Total Sample: 20,661 • Statistical Power at 5% MDE: 30.3%

Test #231 on Glass.net by Mark FreedleMark Freedle Mar 14, 2019 Desktop Mobile Signup X.X% Leads

Mark Tested Pattern #20: Canned Response On Glass.net

Pattern #20: Canned Response - Variant A
Pattern #20: Canned Response - Variant B
Total Sample: 1,342 • Statistical Power at 5% MDE: 6.4%

Test #230 on Goodui.org by Jakub LinowskiJakub Linowski Mar 09, 2019 Desktop Mobile Listing X.X% Engagement

Jakub Tested Pattern #56: Hover Button On Goodui.org

Pattern #56: Hover Button - Variant A
Pattern #56: Hover Button - Variant B

In this test we tested onhover buttons (variant) versus more traditional always exposed and visible ones.
Total Sample: 30,699 • Statistical Power at 5% MDE: 98.1%

Test #229 by Jakub LinowskiJakub Linowski Mar 08, 2019 Desktop Mobile Checkout X.X% Sales

Jakub Tested Pattern #99: Progress Bar

Pattern #99: Progress Bar - Variant A
Pattern #99: Progress Bar - Variant B
Total Sample: 11,214 • Statistical Power at 5% MDE: 13%

Test #228 by Jakub LinowskiJakub Linowski Mar 05, 2019 Desktop Mobile Checkout X.X% Sales

Jakub Tested Pattern #99: Progress Bar

Pattern #99: Progress Bar - Variant A
Pattern #99: Progress Bar - Variant B

The variation added a progress bar to one of the checkout steps for a ticket ordering site.
Total Sample: 12,961 • Statistical Power at 5% MDE: 14.7%