All Latest 633 A/B Tests

MOST RECENT TESTS

Test #256 on by Alex JamesAlex James Aug 23, 2019 Desktop Mobile Signup X.X% Signups

Alex Tested Pattern #109: Required Field Labels

 - Variant A
 - Variant B

The original had no fields marked as required. The variant had all fields marked as required with an asterisk (and a reference note).

Total Sample: 116,470 Statistical Power at 5% MDE: 75.4%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #255 on Thomasnet.com by Julian GaviriaJulian Gaviria Aug 22, 2019 Desktop Mobile Signup X.X% Signups

Julian Tested Pattern #109: Required Field Labels On Thomasnet.com

 - Variant A
 - Variant B

In this experiment, field labels without and with a marked asterisk were tested.

Total Sample: 5,977 Statistical Power at 5% MDE: 22.6%

Test #254 on Volders.de by Alexander KriegerAlexander Krieger Aug 16, 2019 Desktop Mobile Signup X.X% Revenue

Alexander Tested Pattern #17: Least Or Most Expensive First On Volders.de

 - Variant A
 - Variant B

In this experiment, 4 things were adjusted in the variation: the highest pricing plan was shifted to the left, it was set as the default one, the recommendation was also adjusted to point to the highest plan, and one benefit from the lowest plan was removed (customer support).

Test #253 on Volders.de by Alexander KriegerAlexander Krieger Aug 08, 2019 Desktop Mobile Signup X.X% Revenue

Alexander Tested Pattern #17: Least Or Most Expensive First On Volders.de

 - Variant A
 - Variant B

In this experiment, two pricing plans were inverted to show the most expensive plan first (in the variation).

Test #250 on Volders.de by Alexander KriegerAlexander Krieger Jul 25, 2019 Desktop Mobile Signup X.X% Signups

Alexander Tested Pattern #106: Back Buttons On Volders.de

 - Variant A
 - Variant B

In this experiment, a version without a back button was tested against a one where it was positioned in the upper left corner. This test occured on a second step of a contract cancellation service.

Total Sample: 16,894 Statistical Power at 5% MDE: 81%

Test #248 on Volders.de by Alexander KriegerAlexander Krieger Jul 16, 2019 Desktop Signup X.X% Sales

Alexander Tested Pattern #20: Canned Response On Volders.de

 - Variant A
 - Variant B

In this experiment, an editable contract cancellation letter was tested against a non-editable one. The editable letter first appeared in a text state, which required a click on a link in order for it to be transformed into an editable textarea field.

Total Sample: 13,878 Statistical Power at 5% MDE: 20.5%

Test #39 on Over-blog.com by Tael PinaultTael Pinault Jul 02, 2019 Desktop Signup X.X% Signups

Tael Tested Pattern #83: Progressive Fields On Over-blog.com

 - Variant A
 - Variant B

This test explored a progressive disclosure interaction in variation B. When a user started typing text into the first field, two more fields expanded into view.

Total Sample: 10,464 Statistical Power at 5% MDE: 37.8%

Test #245 on Kenhub.com by Niels HapkeNiels Hapke Jun 11, 2019 Desktop Signup X.X% Signups

Niels Tested Pattern #19: Benefit Testimonials On Kenhub.com

 - Variant A
 - Variant B

In this experiment a more elaborate and authentic testimonial was used instead of three more generic ones from social media sites.

Total Sample: 130,872 Statistical Power at 5% MDE: 99.7%

Test #242 on by Alex JamesAlex James May 27, 2019 Desktop Mobile Signup X.X% Signups

Alex Tested Pattern #7: Social Counts

 - Variant A
 - Variant B
Total Sample: 79,012 Statistical Power at 5% MDE: 56.8%

Test #241 on Kenhub.com by Niels HapkeNiels Hapke May 24, 2019 Desktop Mobile Signup X.X% Signups

Niels Tested Pattern #105: Lead Magnets On Kenhub.com

 - Variant A
 - Variant B

In this experiment, a free study guide ebook was promoted on a registration page.

Total Sample: 63,631 Statistical Power at 5% MDE: 87.5%

Test #236 on by Alex JamesAlex James Apr 04, 2019 Desktop Signup X.X% Signups

Alex Tested Pattern #9: Multiple Steps

 - Variant A
 - Variant B

In this experiment a single screen signup process was broken into 2 separate steps: account creation & details.

Total Sample: 282,316 Statistical Power at 5% MDE: 92%

Test #233 on Annonce.cz by Ondřej IlinčevOndřej Ilinčev Apr 01, 2019 Desktop Signup X.X% Sales

Ondřej Tested Pattern #100: Postponed Registration On Annonce.cz

 - Variant A
 - Variant B

In this experiment, the old flow had a login / register / guest checkout as a second step. The variant B flow removed this step and treated everyone as a guest (and later offered to login / register). In the B version guests had a voluntary registration on the thank you page and in a confirmation email. 

 

Total Sample: 12,146 Statistical Power at 5% MDE: 74%

Test #231 on Glass.net by Mark FreedleMark Freedle Mar 14, 2019 Desktop Mobile Signup X.X% Leads

Mark Tested Pattern #20: Canned Response On Glass.net

 - Variant A
 - Variant B
Total Sample: 1,342 Statistical Power at 5% MDE: 6.4%

Test #227 on Volders.de by Alexander KriegerAlexander Krieger Mar 04, 2019 Desktop Signup X.X% Leads

Alexander Tested Pattern #99: Progress Bar On Volders.de

 - Variant A
 - Variant B

In this experiment, a simple progress bar was added to a 2nd step of contract cancellation funnel. The progress bar used separate styles to show which steps were completed, what the current step was, as well as the future step.

Translation of the 3 steps from German are as follows:

  1. Fill out the termination
  2. Send termination
  3. Proof of termination
Total Sample: 11,418 Statistical Power at 5% MDE: 20.6%

Test #223 on Volders.de by Alexander KriegerAlexander Krieger Feb 01, 2019 Desktop Mobile Signup X.X% Revenue

Alexander Tested Pattern #12: Payment First On Volders.de

 - Variant A
 - Variant B

This test deprioritized the free option (kostenlos) of cancelling a contract. It did so by placing it under the paid options as small text link / radio option.

Test #214 on Yummly.com by Marcos CiarrocchiMarcos Ciarrocchi Dec 07, 2018 Mobile Signup X.X% Signups

Marcos Tested Pattern #91: Forced Action On Yummly.com

 - Variant A
 - Variant B

In this experiment, the highly prominent "skip" button was replaced with a less prominent text link. The copy of the skip text link also clarified the consequence of the action - losing out on personalization benefits. More so, the habitual top-right cancel icon was also removed.

Test #213 on Mt.com by Vito MediavillaVito Mediavilla Dec 04, 2018 Desktop Mobile Signup X.X% Leads

Vito Tested Pattern #85: Benefit Button On Mt.com

 - Variant A
 - Variant B

In this variation, a longer button label with a clearer benefit was tested against a shorter one.

Total Sample: 26,639 Statistical Power at 5% MDE: 12.8%

Test #206 on Yummly.com by Kimberly CheungKimberly Cheung Oct 25, 2018 Desktop Mobile Signup X.X% Signups

Kimberly Tested Pattern #40: Blurred Product Background On Yummly.com

 - Variant A
 - Variant B

This test ran on the first step of a multiple signup funnel and only changed the background to a blurred image.

Total Sample: 138,106 Statistical Power at 5% MDE: 77.4%

Test #198 on Bomgar.com by Lee ElkinsLee Elkins Sep 03, 2018 Desktop Signup X.X% Signups

Lee Tested Pattern #40: Blurred Product Background On Bomgar.com

 - Variant A
 - Variant B
Total Sample: 1,893 Statistical Power at 5% MDE: 21.3%

Test #193 on Yummly.com by Marcos CiarrocchiMarcos Ciarrocchi Aug 07, 2018 Desktop Mobile Signup X.X% Signups

Marcos Tested Pattern #91: Forced Action On Yummly.com

 - Variant A
 - Variant B

In this experiment, the presence of an additional "skip all" text link was tested on a multiple step signup flow. The skip all link allowed users to bypass personalization questions and go straight to their app dashboard. The control (A) shows its presence, and in variant B we can see it was removed.

Total Sample: 1,335,347 Statistical Power at 5% MDE: 99.9%