All Latest 645 A/B Tests

Test #295 on Thomasnet.com by Julian GaviriaJulian Gaviria Apr 29, 2020 Desktop Mobile Content X.X% Engagement

Julian Tested Pattern #25: Nagging Results On Thomasnet.com

Pattern #25: Nagging Results - Variant A
Pattern #25: Nagging Results - Variant B

In this experiment, blog article pages were hidden behind a registration wall - requing a signup to access. The registration wall would appear after the first paragraph using gradual opacity to cover the rest of the article. We have published the effects of this change on registrations (signups) and on engagement (users viewing other more important company detail pages).

Total Sample: 131,642 • Statistical Power at 5% MDE: 26.7%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #294 on Umbraco.com by Lars Skjold IversenLars Skjold Iversen Apr 23, 2020 Desktop Pricing X.X% Progression

Lars Tested Pattern #115: Pricing Comparison Table On Umbraco.com

Pattern #115: Pricing Comparison Table - Variant A
Pattern #115: Pricing Comparison Table - Variant B

In this experiment, plan properties on a pricing page were horizontally aligned (for easier comparison). More so, labels and values were also broken on separate lines.

Total Sample: 18,623 • Statistical Power at 5% MDE: 5.3%

Test #289 on Prepagent.com by Arthur SparksArthur Sparks Mar 23, 2020 Desktop Pricing X.X% Revenue

Arthur Tested Pattern #17: Least Or Most Expensive First On Prepagent.com

Pattern #17: Least Or Most Expensive First - Variant A
Pattern #17: Least Or Most Expensive First - Variant B

In this experiment, the order of pricing plans was rearranged as to show the most expensive one first.

Test #290 on Prepagent.com by Arthur SparksArthur Sparks Mar 23, 2020 Desktop Pricing X.X% Sales

Arthur Tested Pattern #14: Exposed Menu Options On Prepagent.com

Pattern #14: Exposed Menu Options - Variant A
Pattern #14: Exposed Menu Options - Variant B

In this experiment, a simple pulldown menu (for US state selection) was replaced with all state options shown as selectable buttons. The states were also abbreviated.

Total Sample: 26,029 • Statistical Power at 5% MDE: 14.2%

Test #286 on Volders.de by Alexander KriegerAlexander Krieger Feb 28, 2020 Desktop Mobile Home & Landing X.X% Sales

Alexander Tested Pattern #9: Multiple Steps On Volders.de

Pattern #9: Multiple Steps - Variant A
Pattern #9: Multiple Steps - Variant B

In this experiment, a long contract cancellation landing page (control) was broken down into 4 steps with 1 final summary step (variation).

Total Sample: 67,999 • Statistical Power at 5% MDE: 75.6%

Test #281 on Backstage.com by Stanley ZuoStanley Zuo Jan 31, 2020 Desktop Listing X.X% Sales

Stanley Tested Pattern #116: Links Or Buttons On Backstage.com

Pattern #116: Links Or Buttons - Variant A
Pattern #116: Links Or Buttons - Variant B

In this experiment, multiple view detail links for a listing tile were turned into higher contrast buttons. 

Total Sample: 386,439 • Statistical Power at 5% MDE: 99.2%

Test #277 on Prepagent.com by Arthur SparksArthur Sparks Jan 03, 2020 Desktop Pricing X.X% Revenue

Arthur Tested Pattern #115: Pricing Comparison Table On Prepagent.com

Pattern #115: Pricing Comparison Table - Variant A
Pattern #115: Pricing Comparison Table - Variant B

In this experiment, side-by-side plan features were aligned and changed to a comparison table with checkmarks for easier comparison.

Test #275 on Prepagent.com by Arthur SparksArthur Sparks Dec 31, 2019 Desktop Pricing X.X% Revenue

Arthur Tested Pattern #114: Less Or More Visible Prices On Prepagent.com

Pattern #114: Less Or More Visible Prices - Variant A
Pattern #114: Less Or More Visible Prices - Variant B

In this experiment, all three prices of each plan were shifted higher for greater visibility.

Test #276 on Umbraco.com by Lars Skjold IversenLars Skjold Iversen Dec 31, 2019 Desktop Mobile Home & Landing X.X% Signups

Lars Tested Pattern #111: Field Explanations On Umbraco.com

Pattern #111: Field Explanations - Variant A
Pattern #111: Field Explanations - Variant B

In this experiment, the idea was to move away from copy that was focusing on the needs of the company ("we need your email") towards copy that hinted at a customer benefit ("create your trial").

Total Sample: 48,811 • Statistical Power at 5% MDE: 9.4%

Test #273 on Elevate App by Jesse GerminarioJesse Germinario Dec 19, 2019 Mobile Signup X.X% Signups

Jesse Tested Pattern #9: Multiple Steps

Pattern #9: Multiple Steps - Variant A
Pattern #9: Multiple Steps - Variant B

This experiment tests the impact of a different paywall screen on iOS. The current control paywall screen for 100% of iOS users was the animated pro screen. In this screen, users see an animation that gives shows glimpses of several Elevate games in action, as well as a bulleted list of key selling points for Pro. The proposed change (variant B) features a swipeable carousel of pages where each page has an image and some accompanying text explaining a different benefit of subscribing to Pro. The hypothesis is that we can lift conversion by showing users the alternate swiping paywall screen.

Total Sample: 152,381 • Statistical Power at 5% MDE: 87.2%

Test #274 by Someone Someone Dec 16, 2019 Desktop Mobile Checkout X.X% Sales

Someone Tested Pattern #1: Remove Coupon Fields

Pattern #1: Remove Coupon Fields - Variant A
Pattern #1: Remove Coupon Fields - Variant B

In this experiment, a fully visible coupon field (A) was made less visible by turning it into a default collaped link (B). Clicking on the link caused the coupon field to appear.

Total Sample: 3,851 • Statistical Power at 5% MDE: 90.2%

Test #272 on Backstage.com by Stanley ZuoStanley Zuo Dec 03, 2019 Desktop Pricing X.X% Revenue

Stanley Tested Pattern #113: More Or Fewer Plans On Backstage.com

Pattern #113: More Or Fewer Plans - Variant A
Pattern #113: More Or Fewer Plans - Variant B

In this experiment, the three pricing plans were condensed into a single recommended plan (annual), with a secondary option to choose the monthly plan.

Test #270 on Dentalplans.com by J.R. HernandezJ.R. Hernandez Nov 19, 2019 Desktop Listing X.X% Sales

J.R. Tested Pattern #37: List Or Grid View On Dentalplans.com

Pattern #37: List Or Grid View - Variant A
Pattern #37: List Or Grid View - Variant B

In this experiment, a list view was tested against a grid view.

Total Sample: 264,000 • Statistical Power at 5% MDE: 48.1%

Test #269 on Thomasnet.com by Julian GaviriaJulian Gaviria Nov 15, 2019 Desktop Home & Landing X.X% Signups

Julian Tested Pattern #14: Exposed Menu Options On Thomasnet.com

Pattern #14: Exposed Menu Options - Variant A
Pattern #14: Exposed Menu Options - Variant B

In this experiment, the variation exposed 6 of the options from the pulldown menu as tabs.

Total Sample: 6,168 • Statistical Power at 5% MDE: 6.7%

Test #268 on Backstage.com by Stanley ZuoStanley Zuo Nov 08, 2019 Mobile Listing X.X% Sales

Stanley Tested Pattern #14: Exposed Menu Options On Backstage.com

Pattern #14: Exposed Menu Options - Variant A
Pattern #14: Exposed Menu Options - Variant B

The change in this experiment was an exposed SEO panel (B) with a number of clickable filter options. 

Total Sample: 208,456 • Statistical Power at 5% MDE: 97%

Test #267 on Backstage.com by Stanley ZuoStanley Zuo Nov 05, 2019 Mobile Checkout X.X% Sales

Stanley Tested Pattern #99: Progress Bar On Backstage.com

Pattern #99: Progress Bar - Variant A
Pattern #99: Progress Bar - Variant B

A "Step X of 4" progress bar was tested against a fully visible one that was also clickable.

Total Sample: 8,552 • Statistical Power at 5% MDE: 6%

Test #266 by Someone Someone Oct 25, 2019 Desktop Mobile Product X.X% Sales

Someone Tested Pattern #4: Testimonials

Pattern #4: Testimonials - Variant A
Pattern #4: Testimonials - Variant B

In this experiment, a product page showed customer reviews at the bottom of the page (B) instead of hiding them under a tab (A).

Total Sample: 17,139 • Statistical Power at 5% MDE: 15.4%

Test #265 on Poll-app.com by Pierre Olivier MartelPierre Olivier Martel Oct 17, 2019 Desktop Mobile Pricing X.X% Sales

Pierre Olivier Tested Pattern #112: Lower Price Frames On Poll-app.com

Pattern #112: Lower Price Frames - Variant A
Pattern #112: Lower Price Frames - Variant B

In this experiment, the $69 payment was explained as $6 per month over 1 year, and the $89 was explained as $4 per month over 2 years. 

Total Sample: 4,157 • Statistical Power at 5% MDE: 7.9%

Test #263 on Goodui.org by Jakub LinowskiJakub Linowski Oct 04, 2019 Desktop Mobile Home & Landing X.X% Signups

Jakub Tested Pattern #22: Empowering Headline On Goodui.org

Pattern #22: Empowering Headline - Variant A
Pattern #22: Empowering Headline - Variant B

In this experiment a different headline was tested. "Reach Higher Conversions Faster" vs. "Learn From What We Try And Test".

Total Sample: 53,636 • Statistical Power at 5% MDE: 24.3%

Test #262 on Thomasnet.com by Julian GaviriaJulian Gaviria Oct 03, 2019 Desktop Mobile Listing X.X% Leads

Julian Tested Pattern #32: Condensed List On Thomasnet.com

Pattern #32: Condensed List - Variant A
Pattern #32: Condensed List - Variant B

In this experiment, the B version condensed the company listings. This was done by showing less of the description and introducing a "more" and "less" dynamic links that would expand and collapse the description.

Total Sample: 105,732 • Statistical Power at 5% MDE: 8.6%