All Latest 644 A/B Tests

Test #257 on Thomasnet.com by Julian GaviriaJulian Gaviria Sep 09, 2019 Desktop Mobile Signup X.X% Signups

Julian Tested Pattern #109: Required Field Labels On Thomasnet.com

Pattern #109: Required Field Labels - Variant A
Pattern #109: Required Field Labels - Variant B

In this followup experiment, field labels without and with a marked asterisk were tested.

Total Sample: 5,739 • Statistical Power at 5% MDE: 31.3%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #254 on Volders.de by Alexander KriegerAlexander Krieger Aug 16, 2019 Desktop Mobile Signup X.X% Revenue

Alexander Tested Pattern #17: Least Or Most Expensive First On Volders.de

Pattern #17: Least Or Most Expensive First - Variant A
Pattern #17: Least Or Most Expensive First - Variant B

In this experiment, 4 things were adjusted in the variation: the highest pricing plan was shifted to the left, it was set as the default one, the recommendation was also adjusted to point to the highest plan, and one benefit from the lowest plan was removed (customer support).

Test #40 on Adoramapix.com by Herman KleinHerman Klein Aug 13, 2019 Desktop Product X.X% Sales

Herman Tested Pattern #85: Benefit Button On Adoramapix.com

Pattern #85: Benefit Button - Variant A
Pattern #85: Benefit Button - Variant B

In this experiment only the button label changed. The control had a more immediate (next step-oriented) button label of "Upload Your Photos Now". The variation tried a more benefit-oriented button of "Order Now" (hinting at paying and obtaining the product).

Total Sample: 13,454 • Statistical Power at 5% MDE: 43.7%

Test #253 on Volders.de by Alexander KriegerAlexander Krieger Aug 08, 2019 Desktop Mobile Signup X.X% Revenue

Alexander Tested Pattern #17: Least Or Most Expensive First On Volders.de

Pattern #17: Least Or Most Expensive First - Variant A
Pattern #17: Least Or Most Expensive First - Variant B

In this experiment, two pricing plans were inverted to show the most expensive plan first (in the variation).

Test #252 on Thomasnet.com by Julian GaviriaJulian Gaviria Jul 30, 2019 Desktop Mobile Content X.X% Engagement

Julian Tested Pattern #107: Contrast Links & Buttons On Thomasnet.com

Pattern #107: Contrast Links & Buttons - Variant A
Pattern #107: Contrast Links & Buttons - Variant B

In this experiment a light blue bold link was tested against a darker blue bold link. 

Total Sample: 142,102 • Statistical Power at 5% MDE: 74.3%

Test #251 on Goodui.org by Jakub LinowskiJakub Linowski Jul 25, 2019 Desktop Mobile Content X.X% Signups

Jakub Tested Pattern #57: Maybe Later On Goodui.org

Pattern #57: Maybe Later - Variant A
Pattern #57: Maybe Later - Variant B

In this experiment, we tested a standard modal (with 2 choices) against a "Maybe Later" one (with 3 choices). One of the choices in the variant allowed users to postpone their decision with a "maybe" which would enable a floating bar at the bottom of the screen. Clicking on any of the "Yes" options would send people to the bottom of the screen with an email signup form. Increasing signup was our primary measure. Both modals also appeared instantly after a page load.

Total Sample: 7,759 • Statistical Power at 5% MDE: 7.2%

Test #248 on Volders.de by Alexander KriegerAlexander Krieger Jul 16, 2019 Desktop Signup X.X% Sales

Alexander Tested Pattern #20: Canned Response On Volders.de

Pattern #20: Canned Response - Variant A
Pattern #20: Canned Response - Variant B

In this experiment, an editable contract cancellation letter was tested against a non-editable one. The editable letter first appeared in a text state, which required a click on a link in order for it to be transformed into an editable textarea field.

Total Sample: 13,878 • Statistical Power at 5% MDE: 20.5%

Test #39 on Over-blog.com by Tael PinaultTael Pinault Jul 02, 2019 Desktop Signup X.X% Signups

Tael Tested Pattern #83: Progressive Fields On Over-blog.com

Pattern #83: Progressive Fields - Variant A
Pattern #83: Progressive Fields - Variant B

This test explored a progressive disclosure interaction in variation B. When a user started typing text into the first field, two more fields expanded into view.

Total Sample: 10,464 • Statistical Power at 5% MDE: 37.8%

Test #247 on Thomasnet.com by Julian GaviriaJulian Gaviria Jun 13, 2019 Desktop Mobile Content X.X% Signups

Julian Tested Pattern #41: Sticky Call To Action On Thomasnet.com

Pattern #41: Sticky Call To Action - Variant A
Pattern #41: Sticky Call To Action - Variant B

In this experiment, the same message (with a link) for signing up to a newsletter was shown in two distinct ways. The control (A) showed the signup message as inline one that preceded the content of the article at the very top. The variant showed the same signup message as a scroll-delayed sticky interaction at the bottom of the screen. The background color of the B variant was also inverted to match the style of the footer.

Total Sample: 76,900 • Statistical Power at 5% MDE: 4.6%

Test #246 on Thomasnet.com by Julian GaviriaJulian Gaviria Jun 12, 2019 Desktop Mobile Home & Landing X.X% Progression

Julian Tested Pattern #88: Action Button On Thomasnet.com

Pattern #88: Action Button - Variant A
Pattern #88: Action Button - Variant B

In this experiment, the navigation label was changed from "Network" to "Find Suppliers". The idea was to make use of an action label with a clearer benefit.

Total Sample: 956,112 • Statistical Power at 5% MDE: 47.9%

Test #245 on Kenhub.com by Niels HapkeNiels Hapke Jun 11, 2019 Desktop Signup X.X% Signups

Niels Tested Pattern #19: Benefit Testimonials On Kenhub.com

Pattern #19: Benefit Testimonials - Variant A
Pattern #19: Benefit Testimonials - Variant B

In this experiment a more elaborate and authentic testimonial was used instead of three more generic ones from social media sites.

Total Sample: 130,872 • Statistical Power at 5% MDE: 99.7%

Test #244 on Mt.com by Vito MediavillaVito Mediavilla Jun 06, 2019 Desktop Product X.X% Leads

Vito Tested Pattern #49: Above The Fold Call To Action On Mt.com

Pattern #49: Above The Fold Call To Action - Variant A
Pattern #49: Above The Fold Call To Action - Variant B

A product landing page with an image was tested against one without - raising the call to action above the fold.

Total Sample: 3,711 • Statistical Power at 5% MDE: 11.2%

Test #243 on Goodui.org by Jakub LinowskiJakub Linowski Jun 05, 2019 Desktop Mobile Home & Landing X.X% Progression

Jakub Tested Pattern #77: Filled Or Ghost Buttons On Goodui.org

Pattern #77: Filled Or Ghost Buttons - Variant A
Pattern #77: Filled Or Ghost Buttons - Variant B

In this experiment, full red background buttons were changed to ghost buttons (red outline and transparent fill)

Total Sample: 25,046 • Statistical Power at 5% MDE: 22.6%

Test #240 on Thomasnet.com by Julian GaviriaJulian Gaviria May 16, 2019 Desktop Listing X.X% Engagement

Julian Tested Pattern #13: Centered Forms & Buttons On Thomasnet.com

Pattern #13: Centered Forms & Buttons - Variant A
Pattern #13: Centered Forms & Buttons - Variant B

In this experiment, two different positions of the View Catalog button were compared. In version A the button was smaller and on the right. In version B the button was wider and more central. (The actual test was inverted before publishing to match the A-B of the pattern.)

Total Sample: 207,844 • Statistical Power at 5% MDE: 31.5%

Test #239 on Volders.de by Alexander KriegerAlexander Krieger May 02, 2019 Desktop Home & Landing X.X% Progression

Alexander Tested Pattern #13: Centered Forms & Buttons On Volders.de

Pattern #13: Centered Forms & Buttons - Variant A
Pattern #13: Centered Forms & Buttons - Variant B

A contract cancellation landing page was tested for the effect of a single (wider CTA area with a left aligned button) vs two column layout (narrow CTA area with a right aligned button).

Total Sample: 1,288 • Statistical Power at 5% MDE: 7.2%

Test #238 on Suzuki.be by Karl GilisKarl Gilis May 01, 2019 Desktop Home & Landing X.X% Progression

Karl Tested Pattern #104: Carousel Vs Static Grid Images On Suzuki.be

Pattern #104: Carousel Vs Static Grid Images - Variant A
Pattern #104: Carousel Vs Static Grid Images - Variant B

In this homepage experiment with a 4-slide carousel, the slides changed every 3.5 seconds and users could also choose another slide manually. In the variations, instead of the 4 slides in the carousel, static images were used to take up the same amount of space.

Test #236 by Alex JamesAlex James Apr 04, 2019 Desktop Signup X.X% Signups

Alex Tested Pattern #9: Multiple Steps

Pattern #9: Multiple Steps - Variant A
Pattern #9: Multiple Steps - Variant B

In this experiment a single screen signup process was broken into 2 separate steps: account creation & details.

Total Sample: 282,316 • Statistical Power at 5% MDE: 92%

Test #235 on Thomasnet.com by Julian GaviriaJulian Gaviria Apr 02, 2019 Desktop Mobile Home & Landing X.X% Leads

Julian Tested Pattern #102: Expanded Or Condensed Layout On Thomasnet.com

Pattern #102: Expanded Or Condensed Layout - Variant A
Pattern #102: Expanded Or Condensed Layout - Variant B

In this experiment, the layout was condensed from a taller to a shorter one.

Total Sample: 11,567 • Statistical Power at 5% MDE: 17.3%

Test #233 on Annonce.cz by Ondřej IlinčevOndřej Ilinčev Apr 01, 2019 Desktop Signup X.X% Sales

Ondřej Tested Pattern #100: Postponed Registration On Annonce.cz

Pattern #100: Postponed Registration - Variant A
Pattern #100: Postponed Registration - Variant B

In this experiment, the old flow had a login / register / guest checkout as a second step. The variant B flow removed this step and treated everyone as a guest (and later offered to login / register). In the B version guests had a voluntary registration on the thank you page and in a confirmation email. 

 

Total Sample: 12,146 • Statistical Power at 5% MDE: 74%

Test #234 by Alex JamesAlex James Apr 01, 2019 Desktop Listing X.X% Leads

Alex Tested Pattern #101: Search Keyword Highlighting

Pattern #101: Search Keyword Highlighting - Variant A
Pattern #101: Search Keyword Highlighting - Variant B

In this experiment, a different style for keyword highlighting was used.

Total Sample: 738,295 • Statistical Power at 5% MDE: 99.9%