All Latest 650 A/B Tests

Test #394 on Chaos.com by Velin PenevVelin Penev Jan 29, 2022 Desktop Product X.X% Progression

Velin Tested Pattern #113: More Or Fewer Plans On Chaos.com

Pattern #113: More Or Fewer Plans - Variant A
Pattern #113: More Or Fewer Plans - Variant B

In this experiment, a two plan pricing plan (standalone product with a higher collection set) was tested against a single plan one (only a collection set). Impact on clicks and total sales was measured.

Total Sample: 23,261 • Statistical Power at 5% MDE: 4.8%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #5 on Busyteacher.org by Andrei ZakhareuskiAndrei Zakhareuski Jan 23, 2022 Desktop Mobile Product X.X% Progression

Andrei Tested Pattern #21: What It's Worth On Busyteacher.org

Pattern #21: What It's Worth - Variant A
Pattern #21: What It's Worth - Variant B

The variation included a simple extention of the headline to include the full dollar worth of a discounted bundle deal. 

Total Sample: 4,895 • Statistical Power at 5% MDE: 4%

Test #392 by Jakub LinowskiJakub Linowski Dec 31, 2021 Desktop Mobile Product X.X% Progression

Jakub Tested Pattern #122: Zigzag Layout

Pattern #122: Zigzag Layout - Variant A
Pattern #122: Zigzag Layout - Variant B

In this experiment, paragraph style copy was reorganized into a zig zag layout with key sections being reinforiced with copy-as-image statements. Impact on adds-to-cart and sales was measured.

Total Sample: 41,827 • Statistical Power at 5% MDE: 28.5%

Test #391 on Backstage.com by Stanley ZuoStanley Zuo Dec 30, 2021 Desktop Mobile Listing X.X% Progression

Stanley Tested Pattern #82: Onboarding Callouts On Backstage.com

Pattern #82: Onboarding Callouts - Variant A
Pattern #82: Onboarding Callouts - Variant B

This experiment added a small nudge or callout to encourage more signups. Attention was directed towards the save function, which lead to the signup flow for anyone not signed it. Impact on signups was measured.

Total Sample: 159,569 • Statistical Power at 5% MDE: 99.5%

Test #388 by Jakub LinowskiJakub Linowski Dec 09, 2021 Mobile Listing X.X% Progression

Jakub Tested Pattern #51: Shortcut Buttons

Pattern #51: Shortcut Buttons - Variant A
Pattern #51: Shortcut Buttons - Variant B

In this experiment, extra "buy now" buttons were added on a listing page. These buttons were shortcuts to an add to cart flow, whereas the "learn more" buttons lead customers to product detail pages (visible in both control and variation). Impact on adds-to-cart and sales was measured.

Total Sample: 7,051 • Statistical Power at 5% MDE: 10.1%

Test #387 by Jakub LinowskiJakub Linowski Nov 30, 2021 Mobile Listing X.X% Progression

Jakub Tested Pattern #88: Action Button

Pattern #88: Action Button - Variant A
Pattern #88: Action Button - Variant B

In this experiment, simple "Learn More" buttons were added underneath each product thumbnail. These buttons were additional triggers that linked to product detail pages on top of the existing thumbnails and product names (that also linked to the PDPs). Impact on product visits and sales was measured.

Total Sample: 7,173 • Statistical Power at 5% MDE: 10%

Test #382 on Snocks.com by Samuel HessSamuel Hess Oct 31, 2021 Desktop Product X.X% Progression

Samuel Tested Pattern #43: Long Titles On Snocks.com

Pattern #43: Long Titles - Variant A
Pattern #43: Long Titles - Variant B

In this experiment, the variation contained a simple text change that described the quality of the product. According to Google Translate the text changed from "tight men's underpants" (control) to "are so comfortable that you don't feel them when you wear them" (variation).

Total Sample: 140,756 • Statistical Power at 5% MDE: 86.3%

Test #381 on Expertinstitute.com by Ardit VeliuArdit Veliu Oct 30, 2021 Desktop Mobile Home & Landing X.X% Progression

Ardit Tested Pattern #49: Above The Fold Call To Action On Expertinstitute.com

Pattern #49: Above The Fold Call To Action - Variant A
Pattern #49: Above The Fold Call To Action - Variant B

In this experiment, an extra button to a signup lead form was placed above the fold. In the control, the same button appeared further down on the page.

Total Sample: 2,970 • Statistical Power at 5% MDE: 4.7%

Test #378 by Jakub LinowskiJakub Linowski Oct 07, 2021 Desktop Mobile Product X.X% Progression

Jakub Tested Pattern #119: Unselected Or Selected Defaults

Pattern #119: Unselected Or Selected Defaults - Variant A
Pattern #119: Unselected Or Selected Defaults - Variant B

Here we have an experiment with a variation that preselected an option for a club duration. The control required customers to first express the choice for how many months they would like to order a product for. Whereas the variation defaulted to 12 months from the beginning.

Impact on adds-to-cart and sales was measured. The experiment unfortunately had to be stopped early due to another embedded variation that was performing poorly. And so it does not have many transactions.

Total Sample: 2,953 • Statistical Power at 5% MDE: 5.2%

Test #377 on Adoramapix.com by Jakub LinowskiJakub Linowski Sep 30, 2021 Desktop Shopping Cart X.X% Progression

Jakub Tested Pattern #121: Free Shipping On Adoramapix.com

Pattern #121: Free Shipping - Variant A
Pattern #121: Free Shipping - Variant B

In this experiment, a big "free shipping" badge was added and defaulted to when available. Impact on progression to checkouts and completed sales was measured.

Total Sample: 4,422 • Statistical Power at 5% MDE: 85.5%

Test #376 on Snocks.com by Samuel HessSamuel Hess Sep 29, 2021 Mobile Desktop Product X.X% Progression

Samuel Tested Pattern #15: Bulleted Reassurances On Snocks.com

Pattern #15: Bulleted Reassurances - Variant A
Pattern #15: Bulleted Reassurances - Variant B

In this experiment, a series of reassurances were added just below the add to cart button. These included: "free shipping", "free returns" and an "anti-hole guarantee". The test ran on the product page of an socks ecommerce company. Impact on sales was measured.

Total Sample: 136,126 • Statistical Power at 5% MDE: 79.6%

Test #375 on Backstage.com by Stanley ZuoStanley Zuo Sep 17, 2021 Mobile Listing X.X% Progression

Stanley Tested Pattern #32: Condensed List On Backstage.com

Pattern #32: Condensed List - Variant A
Pattern #32: Condensed List - Variant B

Similar to experiment 373, listing descriptions were shortened dynamically using exposable "more" links AND dates/location data was removed. This way, the variation showed shorter listings and therefore more listings per screens. Impact on listing clicks (progression) along with membership starts was measured.

Total Sample: 44,045 • Statistical Power at 5% MDE: 92.5%

Test #373 on Backstage.com by Stanley ZuoStanley Zuo Sep 06, 2021 Mobile Listing X.X% Progression

Stanley Tested Pattern #32: Condensed List On Backstage.com

Pattern #32: Condensed List - Variant A
Pattern #32: Condensed List - Variant B

In this experiment, listing descriptions were shortened dynamically using exposable "more" links. This way, the variation showed shorter listings and therefore more listings per screens. Impact on listing clicks (progression) along with membership starts was measured.

Total Sample: 43,469 • Statistical Power at 5% MDE: 92.2%

Test #372 on Fluke.com by Marika FranciscoMarika Francisco Aug 30, 2021 Desktop Product X.X% Progression

Marika Tested Pattern #122: Zigzag Layout On Fluke.com

Pattern #122: Zigzag Layout - Variant A
Pattern #122: Zigzag Layout - Variant B

In this experiment, product descriptions were changed to an alternating zig zag layout with images. Impact on adds-to-cart and sales was measured.

Total Sample: 18,325 • Statistical Power at 5% MDE: 3.2%

Test #371 by Jakub LinowskiJakub Linowski Aug 18, 2021 Desktop Listing X.X% Progression

Jakub Tested Pattern #51: Shortcut Buttons

Pattern #51: Shortcut Buttons - Variant A
Pattern #51: Shortcut Buttons - Variant B

In this experiment, the variation added an extra "Buy Now" button that linked directly into the cart process. The variation only had a learn more button linking directly to a product page.

Total Sample: 3,041 • Statistical Power at 5% MDE: 10.4%

Test #370 on Thomasnet.com by Julian GaviriaJulian Gaviria Aug 16, 2021 Desktop Mobile Listing X.X% Progression

Julian Tested Pattern #88: Action Button On Thomasnet.com

Pattern #88: Action Button - Variant A
Pattern #88: Action Button - Variant B

This experiment added the simple verb ("play") before the "factory video" label to encourage more video plays. Impact on progression / clicks was measured.

Total Sample: 259,330 • Statistical Power at 5% MDE: 49%

Test #367 on Backstage.com by Stanley ZuoStanley Zuo Jul 22, 2021 Desktop Mobile Signup X.X% Progression

Stanley Tested Pattern #124: Confirmed Selection On Backstage.com

Pattern #124: Confirmed Selection - Variant A
Pattern #124: Confirmed Selection - Variant B

In this experiment, a confiming text was added at the top of a signup modal. The text reinforced the selection from a previous listing page, giving a good reason for continuing the signup process.

Total Sample: 29,480 • Statistical Power at 5% MDE: 14.3%

Test #364 on Lotuscrafts.eu by Samuel HessSamuel Hess Jul 06, 2021 Desktop Product X.X% Progression

Samuel Tested Pattern #122: Zigzag Layout On Lotuscrafts.eu

Pattern #122: Zigzag Layout - Variant A
Pattern #122: Zigzag Layout - Variant B

In this experiment, product descriptions or qualities were added using a zigzag layout pattern composed of photos and extra copy. This sections was appended between existing copy and testimonials. Some of the copy highlights included wording such as "innovative surface", "non-slip and wide", and "skin-friendly". Impact on adds-to-cart and sales was measured.

Total Sample: 32,392 • Statistical Power at 5% MDE: 29.1%

Test #363 by Jakub LinowskiJakub Linowski Jul 05, 2021 Desktop Mobile Product X.X% Progression

Jakub Tested Pattern #30: Authentic Photos

Pattern #30: Authentic Photos - Variant A
Pattern #30: Authentic Photos - Variant B

In this experiment, six product photos from were added which showed the actual products (including unboxing images).

Total Sample: 24,655 • Statistical Power at 5% MDE: 33.9%

Test #360 by Jakub LinowskiJakub Linowski Jun 16, 2021 Desktop Product X.X% Progression

Jakub Tested Pattern #60: Repeated Bottom Call To Action

Pattern #60: Repeated Bottom Call To Action - Variant A
Pattern #60: Repeated Bottom Call To Action - Variant B

In this experiment, a comparison chart with various purchasing options was appended at the bottom of a product page. 

Total Sample: 32,070 • Statistical Power at 5% MDE: 35.1%