All Latest 645 A/B Tests

Test #259 on Thomasnet.com by Julian GaviriaJulian Gaviria Sep 12, 2019 Desktop Mobile Signup X.X% Signups

Julian Tested Pattern #110: Optional Field Labels On Thomasnet.com

Pattern #110: Optional Field Labels - Variant A
Pattern #110: Optional Field Labels - Variant B
Total Sample: 5,890 • Statistical Power at 5% MDE: 31.9%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Get Access to All Test Results

Test #258 on Thomasnet.com by Julian GaviriaJulian Gaviria Sep 12, 2019 Desktop Mobile Signup X.X% Signups

Julian Tested Pattern #110: Optional Field Labels On Thomasnet.com

Pattern #110: Optional Field Labels - Variant A
Pattern #110: Optional Field Labels - Variant B

In this variation an optional field label was added.
Total Sample: 5,965 • Statistical Power at 5% MDE: 22.6%

Test #256 by Alex JamesAlex James Aug 23, 2019 Desktop Mobile Signup X.X% Signups

Alex Tested Pattern #109: Required Field Labels

Pattern #109: Required Field Labels - Variant A
Pattern #109: Required Field Labels - Variant B

The original had no fields marked as required. The variant had all fields marked as required with an asterisk (and a reference note).
Total Sample: 116,470 • Statistical Power at 5% MDE: 75.4%

Test #255 on Thomasnet.com by Julian GaviriaJulian Gaviria Aug 22, 2019 Desktop Mobile Signup X.X% Signups

Julian Tested Pattern #109: Required Field Labels On Thomasnet.com

Pattern #109: Required Field Labels - Variant A
Pattern #109: Required Field Labels - Variant B

In this experiment, field labels without and with a marked asterisk were tested.
Total Sample: 5,977 • Statistical Power at 5% MDE: 22.6%

Test #250 on Volders.de by Alexander KriegerAlexander Krieger Jul 25, 2019 Desktop Mobile Signup X.X% Signups

Alexander Tested Pattern #106: Back Buttons On Volders.de

Pattern #106: Back Buttons - Variant A
Pattern #106: Back Buttons - Variant B

In this experiment, a version without a back button was tested against a one where it was positioned in the upper left corner. This test occured on a second step of a contract cancellation service.
Total Sample: 16,894 • Statistical Power at 5% MDE: 81%

Test #249 on Autoscout24.com by Optimizely Optimizely Jul 17, 2019 Desktop Product X.X% Leads

Optimizely Tested Pattern #20: Canned Response On Autoscout24.com

Pattern #20: Canned Response - Variant A
Pattern #20: Canned Response - Variant B

AutoScout24 is Europe’s largest online marketplace for new and used cars. As published by Optimizely, they tested a form pre-filled with text that read: ‘Hello. I am interested in your vehicle. Please contact me. Kind regards.’ 

Test #242 by Alex JamesAlex James May 27, 2019 Desktop Mobile Signup X.X% Signups

Alex Tested Pattern #7: Social Counts

Pattern #7: Social Counts - Variant A
Pattern #7: Social Counts - Variant B
Total Sample: 79,012 • Statistical Power at 5% MDE: 56.8%

Test #241 on Kenhub.com by Niels HapkeNiels Hapke May 24, 2019 Desktop Mobile Signup X.X% Signups

Niels Tested Pattern #105: Lead Magnets On Kenhub.com

Pattern #105: Lead Magnets - Variant A
Pattern #105: Lead Magnets - Variant B

In this experiment, a free study guide ebook was promoted on a registration page.
Total Sample: 63,631 • Statistical Power at 5% MDE: 87.5%

Test #237 on Goodui.org by Jakub LinowskiJakub Linowski Apr 29, 2019 Desktop Mobile Listing X.X% Sales

Jakub Tested Pattern #103: Money Back Guarantee On Goodui.org

Pattern #103: Money Back Guarantee - Variant A
Pattern #103: Money Back Guarantee - Variant B
Total Sample: 21,096 • Statistical Power at 5% MDE: 6%

Test #231 on Glass.net by Mark FreedleMark Freedle Mar 14, 2019 Desktop Mobile Signup X.X% Leads

Mark Tested Pattern #20: Canned Response On Glass.net

Pattern #20: Canned Response - Variant A
Pattern #20: Canned Response - Variant B
Total Sample: 1,342 • Statistical Power at 5% MDE: 6.4%

Test #229 by Jakub LinowskiJakub Linowski Mar 08, 2019 Desktop Mobile Checkout X.X% Sales

Jakub Tested Pattern #99: Progress Bar

Pattern #99: Progress Bar - Variant A
Pattern #99: Progress Bar - Variant B
Total Sample: 11,214 • Statistical Power at 5% MDE: 13%

Test #228 by Jakub LinowskiJakub Linowski Mar 05, 2019 Desktop Mobile Checkout X.X% Sales

Jakub Tested Pattern #99: Progress Bar

Pattern #99: Progress Bar - Variant A
Pattern #99: Progress Bar - Variant B

The variation added a progress bar to one of the checkout steps for a ticket ordering site.
Total Sample: 12,961 • Statistical Power at 5% MDE: 14.7%

Test #227 on Volders.de by Alexander KriegerAlexander Krieger Mar 04, 2019 Desktop Signup X.X% Leads

Alexander Tested Pattern #99: Progress Bar On Volders.de

Pattern #99: Progress Bar - Variant A
Pattern #99: Progress Bar - Variant B

In this experiment, a simple progress bar was added to a 2nd step of contract cancellation funnel. The progress bar used separate styles to show which steps were completed, what the current step was, as well as the future step.

Translation of the 3 steps from German are as follows:

  1. Fill out the termination
  2. Send termination
  3. Proof of termination
Total Sample: 11,418 • Statistical Power at 5% MDE: 20.6%

Test #222 on Thomasnet.com by Julian GaviriaJulian Gaviria Feb 01, 2019 Desktop Listing X.X% Leads

Julian Tested Pattern #7: Social Counts On Thomasnet.com

Pattern #7: Social Counts - Variant A
Pattern #7: Social Counts - Variant B

In this variation, a number of social proof references were added to a signup modal.
Total Sample: 34,676 • Statistical Power at 5% MDE: 47.1%

Test #218 on Yummly.com by Kimberly CheungKimberly Cheung Jan 14, 2019 Desktop Mobile Home & Landing X.X% Signups

Kimberly Tested Pattern #94: Visible Search On Yummly.com

Pattern #94: Visible Search - Variant A
Pattern #94: Visible Search - Variant B

Hypothesis: Anonymous users can't use global search while on the home promo page. We believe that if we show a global search bar to anonymous users, it presents a higher converting funnel (guided search) and will increase our sign-up rates significantly.

Control (A): Logged out users don't see global search bar.

Variant (B): Logged out users see global search bar. After searching for a keyword, the signup funnel starts with a more personalized reason to continue the signup process.
Total Sample: 20,122 • Statistical Power at 5% MDE: 13.8%

Test #219 on Mt.com by Vito MediavillaVito Mediavilla Jan 14, 2019 Desktop Mobile Home & Landing X.X% Leads

Vito Tested Pattern #95: Clickable Product Previews On Mt.com

Pattern #95: Clickable Product Previews - Variant A
Pattern #95: Clickable Product Previews - Variant B
Total Sample: 4,836 • Statistical Power at 5% MDE: 11%

Test #217 on Thomasnet.com by Julian GaviriaJulian Gaviria Jan 03, 2019 Desktop Mobile Home & Landing X.X% Engagement

Julian Tested Pattern #41: Sticky Call To Action On Thomasnet.com

Pattern #41: Sticky Call To Action - Variant A
Pattern #41: Sticky Call To Action - Variant B
Total Sample: 158,023 • Statistical Power at 5% MDE: 8.9%

Test #215 on Vivareal.com.br by Vinicius Barros PeixotoVinicius Barros Peixoto Dec 21, 2018 Mobile Listing X.X% Leads

Vinicius Tested Pattern #92: Already Viewed Label On Vivareal.com.br

Pattern #92: Already Viewed Label - Variant A
Pattern #92: Already Viewed Label - Variant B

The idea of this test was to add a "Viewed" label on a listing page to indicate listings which have already been viewed by users.
Total Sample: 2,452,026 • Statistical Power at 5% MDE: 99.9%

Test #216 on Support.microsoft.co... by Ronny KohaviRonny Kohavi Dec 21, 2018 Desktop Home & Landing X.X% Progression

Ronny Tested Pattern #2: Icon Labels On Support.microsoft.co...

Pattern #2: Icon Labels - Variant A
Pattern #2: Icon Labels - Variant B

Microsoft ran an experiment on their Customer Satisfaction Survey at both support.microsoft.com and answers.microsoft.com (Desktop). The treatment contained two icon labels at the opposite sides of the star rating range (ex: Very Dissatisfied and Very Satisfied) - providing it with additional meaning.

Test #212 on Mt.com by Vito MediavillaVito Mediavilla Dec 04, 2018 Desktop Mobile Product X.X% Leads

Vito Tested Pattern #60: Repeated Bottom Call To Action On Mt.com

Pattern #60: Repeated Bottom Call To Action - Variant A
Pattern #60: Repeated Bottom Call To Action - Variant B

In this experiment, a call to action was repeated at the bottom of the product page. Additional certification icons were also added for additional reassurance.
Total Sample: 26,467 • Statistical Power at 5% MDE: 9.7%