All Latest 644 A/B Tests

MOST RECENT TESTS

Test #279 on Umbraco.com by Lars Skjold IversenLars Skjold Iversen Jan 16, 2020 Desktop Mobile Home & Landing X.X% Sales

Lars Tested Pattern #79: Product Highlights On Umbraco.com

Pattern #79: Product Highlights - Variant A
Pattern #79: Product Highlights - Variant B

In this experiment, 3 additional course links with descriptions were added to the homepage. The idea was to increase course sales aside of the Saas subscription signups.

Total Sample: 22,540 • Statistical Power at 5% MDE: 3.7%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #278 by Someone Someone Jan 15, 2020 Mobile Shopping Cart X.X% Sales

Someone Tested Pattern #64: Tunnel

Pattern #64: Tunnel - Variant A
Pattern #64: Tunnel - Variant B

In this experiment, multiple elemenets were removed from the bottom of the cart page. This was done to see if they were potential distractions that hindered the purchase process. The elements included such things as: Order details (visible on other pages), large product photo, a "what you receive" section with selling points, more reassurances, and a newsletter subscribe box.

Total Sample: 1,369 • Statistical Power at 5% MDE: 12%

Test #276 on Umbraco.com by Lars Skjold IversenLars Skjold Iversen Dec 31, 2019 Desktop Mobile Home & Landing X.X% Signups

Lars Tested Pattern #111: Field Explanations On Umbraco.com

Pattern #111: Field Explanations - Variant A
Pattern #111: Field Explanations - Variant B

In this experiment, the idea was to move away from copy that was focusing on the needs of the company ("we need your email") towards copy that hinted at a customer benefit ("create your trial").

Total Sample: 48,811 • Statistical Power at 5% MDE: 9.4%

Test #273 on Elevate App by Jesse GerminarioJesse Germinario Dec 19, 2019 Mobile Signup X.X% Signups

Jesse Tested Pattern #9: Multiple Steps

Pattern #9: Multiple Steps - Variant A
Pattern #9: Multiple Steps - Variant B

This experiment tests the impact of a different paywall screen on iOS. The current control paywall screen for 100% of iOS users was the animated pro screen. In this screen, users see an animation that gives shows glimpses of several Elevate games in action, as well as a bulleted list of key selling points for Pro. The proposed change (variant B) features a swipeable carousel of pages where each page has an image and some accompanying text explaining a different benefit of subscribing to Pro. The hypothesis is that we can lift conversion by showing users the alternate swiping paywall screen.

Total Sample: 152,381 • Statistical Power at 5% MDE: 87.2%

Test #274 by Someone Someone Dec 16, 2019 Desktop Mobile Checkout X.X% Sales

Someone Tested Pattern #1: Remove Coupon Fields

Pattern #1: Remove Coupon Fields - Variant A
Pattern #1: Remove Coupon Fields - Variant B

In this experiment, a fully visible coupon field (A) was made less visible by turning it into a default collaped link (B). Clicking on the link caused the coupon field to appear.

Total Sample: 3,851 • Statistical Power at 5% MDE: 90.2%

Test #271 on Analytics-toolkit.co... by Georgi Z. GeorgievGeorgi Z. Georgiev Nov 24, 2019 Desktop Mobile Signup X.X% Signups

Georgi Tested Pattern #4: Testimonials On Analytics-toolkit.co...

Pattern #4: Testimonials - Variant A
Pattern #4: Testimonials - Variant B

In this experiment, the test variant showed 2 testimonials on the user registration / free trial registration page at Analytics-Toolkit.com

Total Sample: 2,432 • Statistical Power at 5% MDE: 17.4%

Test #268 on Backstage.com by Stanley ZuoStanley Zuo Nov 08, 2019 Mobile Listing X.X% Sales

Stanley Tested Pattern #14: Exposed Menu Options On Backstage.com

Pattern #14: Exposed Menu Options - Variant A
Pattern #14: Exposed Menu Options - Variant B

The change in this experiment was an exposed SEO panel (B) with a number of clickable filter options. 

Total Sample: 208,456 • Statistical Power at 5% MDE: 97%

Test #267 on Backstage.com by Stanley ZuoStanley Zuo Nov 05, 2019 Mobile Checkout X.X% Sales

Stanley Tested Pattern #99: Progress Bar On Backstage.com

Pattern #99: Progress Bar - Variant A
Pattern #99: Progress Bar - Variant B

A "Step X of 4" progress bar was tested against a fully visible one that was also clickable.

Total Sample: 8,552 • Statistical Power at 5% MDE: 6%

Test #266 by Someone Someone Oct 25, 2019 Desktop Mobile Product X.X% Sales

Someone Tested Pattern #4: Testimonials

Pattern #4: Testimonials - Variant A
Pattern #4: Testimonials - Variant B

In this experiment, a product page showed customer reviews at the bottom of the page (B) instead of hiding them under a tab (A).

Total Sample: 17,139 • Statistical Power at 5% MDE: 15.4%

Test #265 on Poll-app.com by Pierre Olivier MartelPierre Olivier Martel Oct 17, 2019 Desktop Mobile Pricing X.X% Sales

Pierre Olivier Tested Pattern #112: Lower Price Frames On Poll-app.com

Pattern #112: Lower Price Frames - Variant A
Pattern #112: Lower Price Frames - Variant B

In this experiment, the $69 payment was explained as $6 per month over 1 year, and the $89 was explained as $4 per month over 2 years. 

Total Sample: 4,157 • Statistical Power at 5% MDE: 7.9%

Test #264 on Kenhub.com by Niels HapkeNiels Hapke Oct 05, 2019 Desktop Mobile Global X.X% Sales

Niels Tested Pattern #41: Sticky Call To Action On Kenhub.com

Pattern #41: Sticky Call To Action - Variant A
Pattern #41: Sticky Call To Action - Variant B

In this experiment users saw a sticky bar advertising the benefits of a Premium account across the bottom of the website, wherever they navigate. The sticky call to action appeared with a 4 second delay and was floating.

Total Sample: 50,572 • Statistical Power at 5% MDE: 20.3%

Test #263 on Goodui.org by Jakub LinowskiJakub Linowski Oct 04, 2019 Desktop Mobile Home & Landing X.X% Signups

Jakub Tested Pattern #22: Empowering Headline On Goodui.org

Pattern #22: Empowering Headline - Variant A
Pattern #22: Empowering Headline - Variant B

In this experiment a different headline was tested. "Reach Higher Conversions Faster" vs. "Learn From What We Try And Test".

Total Sample: 53,636 • Statistical Power at 5% MDE: 24.3%

Test #262 on Thomasnet.com by Julian GaviriaJulian Gaviria Oct 03, 2019 Desktop Mobile Listing X.X% Leads

Julian Tested Pattern #32: Condensed List On Thomasnet.com

Pattern #32: Condensed List - Variant A
Pattern #32: Condensed List - Variant B

In this experiment, the B version condensed the company listings. This was done by showing less of the description and introducing a "more" and "less" dynamic links that would expand and collapse the description.

Total Sample: 105,732 • Statistical Power at 5% MDE: 8.6%

Test #105 on Inktweb.nl by Martijn OudMartijn Oud Sep 23, 2019 Desktop Mobile Signup X.X% Signups

Martijn Tested Pattern #111: Field Explanations On Inktweb.nl

Pattern #111: Field Explanations - Variant A
Pattern #111: Field Explanations - Variant B

In this experiment, onhover tooltip explanations were added to selected fields (Firstname, Lastname, Phone, Email and Password). One translation example of the Firstname tooltip was the following "Enter your first name (or letter) so that we can address you in a more personal way".

Total Sample: 3,169 • Statistical Power at 5% MDE: 27.7%

Test #258 on Thomasnet.com by Julian GaviriaJulian Gaviria Sep 12, 2019 Desktop Mobile Signup X.X% Signups

Julian Tested Pattern #110: Optional Field Labels On Thomasnet.com

Pattern #110: Optional Field Labels - Variant A
Pattern #110: Optional Field Labels - Variant B

In this variation an optional field label was added.

Total Sample: 5,965 • Statistical Power at 5% MDE: 22.6%

Test #259 on Thomasnet.com by Julian GaviriaJulian Gaviria Sep 12, 2019 Desktop Mobile Signup X.X% Signups

Julian Tested Pattern #110: Optional Field Labels On Thomasnet.com

Pattern #110: Optional Field Labels - Variant A
Pattern #110: Optional Field Labels - Variant B
Total Sample: 5,890 • Statistical Power at 5% MDE: 31.9%

Test #257 on Thomasnet.com by Julian GaviriaJulian Gaviria Sep 09, 2019 Desktop Mobile Signup X.X% Signups

Julian Tested Pattern #109: Required Field Labels On Thomasnet.com

Pattern #109: Required Field Labels - Variant A
Pattern #109: Required Field Labels - Variant B

In this followup experiment, field labels without and with a marked asterisk were tested.

Total Sample: 5,739 • Statistical Power at 5% MDE: 31.3%

Test #256 by Alex JamesAlex James Aug 23, 2019 Desktop Mobile Signup X.X% Signups

Alex Tested Pattern #109: Required Field Labels

Pattern #109: Required Field Labels - Variant A
Pattern #109: Required Field Labels - Variant B

The original had no fields marked as required. The variant had all fields marked as required with an asterisk (and a reference note).

Total Sample: 116,470 • Statistical Power at 5% MDE: 75.4%

Test #255 on Thomasnet.com by Julian GaviriaJulian Gaviria Aug 22, 2019 Desktop Mobile Signup X.X% Signups

Julian Tested Pattern #109: Required Field Labels On Thomasnet.com

Pattern #109: Required Field Labels - Variant A
Pattern #109: Required Field Labels - Variant B

In this experiment, field labels without and with a marked asterisk were tested.

Total Sample: 5,977 • Statistical Power at 5% MDE: 22.6%

Test #254 on Volders.de by Alexander KriegerAlexander Krieger Aug 16, 2019 Desktop Mobile Signup X.X% Revenue

Alexander Tested Pattern #17: Least Or Most Expensive First On Volders.de

Pattern #17: Least Or Most Expensive First - Variant A
Pattern #17: Least Or Most Expensive First - Variant B

In this experiment, 4 things were adjusted in the variation: the highest pricing plan was shifted to the left, it was set as the default one, the recommendation was also adjusted to point to the highest plan, and one benefit from the lowest plan was removed (customer support).