All Latest 650 A/B Tests

Test #296 on Backstage.com by Stanley ZuoStanley Zuo Apr 30, 2020 Mobile Content X.X% Progression

Stanley Tested Pattern #23: Inline Link Nudge On Backstage.com

Pattern #23: Inline Link Nudge - Variant A
Pattern #23: Inline Link Nudge - Variant B

In this simple experiment, a text link to a join page was injected on an article page. The hypothesis was that more users would signup as a result of this subtle trigger.

Total Sample: 411,288 • Statistical Power at 5% MDE: 11%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #294 on Umbraco.com by Lars Skjold IversenLars Skjold Iversen Apr 23, 2020 Desktop Pricing X.X% Progression

Lars Tested Pattern #115: Pricing Comparison Table On Umbraco.com

Pattern #115: Pricing Comparison Table - Variant A
Pattern #115: Pricing Comparison Table - Variant B

In this experiment, plan properties on a pricing page were horizontally aligned (for easier comparison). More so, labels and values were also broken on separate lines.

Total Sample: 18,623 • Statistical Power at 5% MDE: 5.3%

Test #293 on Backstage.com by Stanley ZuoStanley Zuo Apr 14, 2020 Desktop Mobile Product X.X% Progression

Stanley Tested Pattern #114: Less Or More Visible Prices On Backstage.com

Pattern #114: Less Or More Visible Prices - Variant A
Pattern #114: Less Or More Visible Prices - Variant B

In this experiment on a casting call site, pricing information was shown beside the application button. This change shows the effect of setting a price expectation and being more clear that the application process is not free.

Total Sample: 115,061 • Statistical Power at 5% MDE: 9.3%

Test #292 on Backstage.com by Stanley ZuoStanley Zuo Apr 13, 2020 Desktop Mobile Listing X.X% Progression

Stanley Tested Pattern #24: Visible Availability On Backstage.com

Pattern #24: Visible Availability - Variant A
Pattern #24: Visible Availability - Variant B

The core hypothesis of this experiment was that by showing clear availability (in green text) beside each casting call, more users would apply and become premium members. The experiment reports on two metrics: application starts (the first progression metric), and premium membership sales (measured a few steps further in the funnel).

Total Sample: 180,098 • Statistical Power at 5% MDE: 16.8%

Test #288 on Kenhub.com by Niels HapkeNiels Hapke Mar 05, 2020 Desktop Mobile Home & Landing X.X% Progression

Niels Tested Pattern #117: Company Logos On Kenhub.com

Pattern #117: Company Logos - Variant A
Pattern #117: Company Logos - Variant B

In this experiment, customer logos (of universities attended by students using Kenhub) were placed on a homepage. The experiment tested for the effect on registration visits, and premium subscription starts.

Total Sample: 36,420 • Statistical Power at 5% MDE: 57.2%

Test #287 on Goodui.org by Jakub LinowskiJakub Linowski Mar 04, 2020 Desktop Mobile Pricing X.X% Progression

Jakub Tested Pattern #117: Company Logos On Goodui.org

Pattern #117: Company Logos - Variant A
Pattern #117: Company Logos - Variant B

In this experiment, a handful of customers and contributors from GoodUI were added on a pricing page to test the effect on sales.

Total Sample: 6,021 • Statistical Power at 5% MDE: 4.4%

Test #286 on Volders.de by Alexander KriegerAlexander Krieger Feb 28, 2020 Desktop Mobile Home & Landing X.X% Progression

Alexander Tested Pattern #9: Multiple Steps On Volders.de

Pattern #9: Multiple Steps - Variant A
Pattern #9: Multiple Steps - Variant B

In this experiment, a long contract cancellation landing page (control) was broken down into 4 steps with 1 final summary step (variation).

Total Sample: 67,999 • Statistical Power at 5% MDE: 75.6%

Test #285 on Ibood.com by Lukas JorissenLukas Jorissen Feb 27, 2020 Desktop Product X.X% Progression

Lukas Tested Pattern #7: Social Counts On Ibood.com

Pattern #7: Social Counts - Variant A
Pattern #7: Social Counts - Variant B

In this experiment, realtime social proof information has been added below an add-to-cart button. The variation shows how many users that have viewed, or placed a product into their basket. Translated to "4 visitors have this product in their shopping cart."

Total Sample: 137,190 • Statistical Power at 5% MDE: 36.7%

Test #284 on Thomasnet.com by Julian GaviriaJulian Gaviria Feb 19, 2020 Desktop Mobile Listing X.X% Progression

Julian Tested Pattern #78: Tags, Badges And Structured Information On Thomasnet.com

Pattern #78: Tags, Badges And Structured Information - Variant A
Pattern #78: Tags, Badges And Structured Information - Variant B

In this experiment, structured data tags were displayed on a listing page to help potential buyers make better decisions. The additional information about the listed companies included: annual revenue, employee count, and year of establishment.

Total Sample: 241,428 • Statistical Power at 5% MDE: 13%

Test #282 on Thomasnet.com by Julian GaviriaJulian Gaviria Feb 07, 2020 Desktop Mobile Listing X.X% Progression

Julian Tested Pattern #51: Shortcut Buttons On Thomasnet.com

Pattern #51: Shortcut Buttons - Variant A
Pattern #51: Shortcut Buttons - Variant B

In this experiment, a contact button was added to a listing / search results page to make it faster to contact a company. This same button was also visible on the company detail page.

Total Sample: 163,317 • Statistical Power at 5% MDE: 9.7%

Test #281 on Backstage.com by Stanley ZuoStanley Zuo Jan 31, 2020 Desktop Listing X.X% Progression

Stanley Tested Pattern #116: Links Or Buttons On Backstage.com

Pattern #116: Links Or Buttons - Variant A
Pattern #116: Links Or Buttons - Variant B

In this experiment, multiple view detail links for a listing tile were turned into higher contrast buttons. 

Total Sample: 386,439 • Statistical Power at 5% MDE: 99.2%

Test #280 on Volders.de by Alexander KriegerAlexander Krieger Jan 24, 2020 Desktop Mobile Signup X.X% Progression

Alexander Tested Pattern #3: Fewer Form Fields On Volders.de

Pattern #3: Fewer Form Fields - Variant A
Pattern #3: Fewer Form Fields - Variant B

In this experiment on a contract cancellation funnel, one field was removed - a secondary contract ID. The control and variation both had a primary "customer ID" with which to identify and cancel someone's contract with.

Total Sample: 24,301 • Statistical Power at 5% MDE: 43.6%

Test #279 on Umbraco.com by Lars Skjold IversenLars Skjold Iversen Jan 16, 2020 Desktop Mobile Home & Landing X.X% Progression

Lars Tested Pattern #79: Product Highlights On Umbraco.com

Pattern #79: Product Highlights - Variant A
Pattern #79: Product Highlights - Variant B

In this experiment, 3 additional course links with descriptions were added to the homepage. The idea was to increase course sales aside of the Saas subscription signups.

Total Sample: 22,540 • Statistical Power at 5% MDE: 3.7%

Test #278 by Someone Someone Jan 15, 2020 Mobile Shopping Cart X.X% Progression

Someone Tested Pattern #64: Tunnel

Pattern #64: Tunnel - Variant A
Pattern #64: Tunnel - Variant B

In this experiment, multiple elemenets were removed from the bottom of the cart page. This was done to see if they were potential distractions that hindered the purchase process. The elements included such things as: Order details (visible on other pages), large product photo, a "what you receive" section with selling points, more reassurances, and a newsletter subscribe box.

Total Sample: 1,369 • Statistical Power at 5% MDE: 12%

Test #270 on Dentalplans.com by J.R. HernandezJ.R. Hernandez Nov 19, 2019 Desktop Listing X.X% Progression

J.R. Tested Pattern #37: List Or Grid View On Dentalplans.com

Pattern #37: List Or Grid View - Variant A
Pattern #37: List Or Grid View - Variant B

In this experiment, a list view was tested against a grid view.

Total Sample: 264,000 • Statistical Power at 5% MDE: 48.1%

Test #269 on Thomasnet.com by Julian GaviriaJulian Gaviria Nov 15, 2019 Desktop Home & Landing X.X% Progression

Julian Tested Pattern #14: Exposed Menu Options On Thomasnet.com

Pattern #14: Exposed Menu Options - Variant A
Pattern #14: Exposed Menu Options - Variant B

In this experiment, the variation exposed 6 of the options from the pulldown menu as tabs.

Total Sample: 6,168 • Statistical Power at 5% MDE: 6.7%

Test #268 on Backstage.com by Stanley ZuoStanley Zuo Nov 08, 2019 Mobile Listing X.X% Progression

Stanley Tested Pattern #14: Exposed Menu Options On Backstage.com

Pattern #14: Exposed Menu Options - Variant A
Pattern #14: Exposed Menu Options - Variant B

The change in this experiment was an exposed SEO panel (B) with a number of clickable filter options. 

Total Sample: 208,456 • Statistical Power at 5% MDE: 97%

Test #267 on Backstage.com by Stanley ZuoStanley Zuo Nov 05, 2019 Mobile Checkout X.X% Progression

Stanley Tested Pattern #99: Progress Bar On Backstage.com

Pattern #99: Progress Bar - Variant A
Pattern #99: Progress Bar - Variant B

A "Step X of 4" progress bar was tested against a fully visible one that was also clickable.

Total Sample: 8,552 • Statistical Power at 5% MDE: 6%

Test #266 by Someone Someone Oct 25, 2019 Desktop Mobile Product X.X% Progression

Someone Tested Pattern #4: Testimonials

Pattern #4: Testimonials - Variant A
Pattern #4: Testimonials - Variant B

In this experiment, a product page showed customer reviews at the bottom of the page (B) instead of hiding them under a tab (A).

Total Sample: 17,139 • Statistical Power at 5% MDE: 15.4%

Test #264 on Kenhub.com by Niels HapkeNiels Hapke Oct 05, 2019 Desktop Mobile Global X.X% Progression

Niels Tested Pattern #41: Sticky Call To Action On Kenhub.com

Pattern #41: Sticky Call To Action - Variant A
Pattern #41: Sticky Call To Action - Variant B

In this experiment users saw a sticky bar advertising the benefits of a Premium account across the bottom of the website, wherever they navigate. The sticky call to action appeared with a 4 second delay and was floating.

Total Sample: 50,572 • Statistical Power at 5% MDE: 20.3%