All Latest 634 A/B Tests

Test #413 on by Jakub LinowskiJakub Linowski May 26, 2022 Desktop Mobile Shopping Cart X.X% Progression

Jakub Tested Pattern #114: Less Or More Visible Prices

 - Variant A
 - Variant B

Should prices be displayed during an editing state on an add-to-cart screen (after adding-to-cart from a product detail page)? In this experiment, club pricing was added and shown in three areas after customers clicked on a small "edit details" links. The control did not have this pricing information (which was only shown on a product detail screen before).

Total Sample: 5,698 Statistical Power at 5% MDE: 59.9%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #411 on by Ayat ShukairyAyat Shukairy May 09, 2022 Desktop Mobile Product X.X% Progression

Ayat Tested Pattern #126: Bottom Or Left Thumbnails

 - Variant A
 - Variant B

Bottom aligned thumbnails were shifted to the left side on a product image. Doing so, also shifted the product descriptions a little higher. Impact on adds-to-cart and total transactions was measured.

Total Sample: 588,443 Statistical Power at 5% MDE: 70.1%

Test #410 on Designlab.com by Daniel ShapiroDaniel Shapiro May 05, 2022 Desktop Mobile Home & Landing X.X% Progression

Daniel Tested Pattern #29: Surfaced Content On Designlab.com

 - Variant A
 - Variant B

Does linking to a product detail page help? Or should a homepage simply focus on generic lead generation? In this homepage experiment, the presence of a component with a link to a detailed course landing page was tested. Impact on course enrollments was measured.

Total Sample: 25,443 Statistical Power at 5% MDE: 6.3%

Test #408 on by Jakub LinowskiJakub Linowski Apr 29, 2022 Desktop Mobile Product X.X% Progression

Jakub Tested Pattern #67: Currency & Taxes

 - Variant A
 - Variant B

Here is a very simple experiment where CDN currency copy was appended to prices being shown on a Canadian ecommerce site. Additional copy ("from Canada") was appended to an existing shipping message.  

Total Sample: 16,332 Statistical Power at 5% MDE: 17.9%

Test #400 on by Herman KleinHerman Klein Mar 06, 2022 Desktop Shopping Cart X.X% Progression

Herman Tested Pattern #65: Add More For Extra Incentive

 - Variant A
 - Variant B

In this experiment, customers who were not yet eligible for free shipping (with cart amounts under $39) were presented with an encouraging message to add more and earn free shipping. Impact on progression to checkout and sales was measured.

Total Sample: 3,111 Statistical Power at 5% MDE: 60.3%

Test #399 on by Jakub LinowskiJakub Linowski Feb 27, 2022 Desktop Mobile Checkout X.X% Progression

Jakub Tested Pattern #35: Floating Labels

 - Variant A
 - Variant B

In this experiment, top-aligned field labels were tested against floating labels (with labels floating inside the form field itself).

Total Sample: 2,073 Statistical Power at 5% MDE: 29.2%

Test #398 on Adoramapix.com by Herman KleinHerman Klein Feb 25, 2022 Desktop Shopping Cart X.X% Progression

Herman Tested Pattern #121: Free Shipping On Adoramapix.com

 - Variant A
 - Variant B

In this experiment, customers were eligible for free shipping (with cart amounts of $39 or more) were presented with an earned free shipping message. The variation also showed the value of the earned free shipping - (for example $10). Impact on progression to checkout and sales was measured.

Total Sample: 3,123 Statistical Power at 5% MDE: 60.5%

Test #395 on by Jakub LinowskiJakub Linowski Jan 31, 2022 Desktop Mobile Product X.X% Progression

Jakub Tested Pattern #52: How It Works

 - Variant A
 - Variant B

In this experiment, a How It Works section was appended on a product detail page just above product reviews.

Total Sample: 17,099 Statistical Power at 5% MDE: 15.8%

Test #394 on Chaos.com by Velin PenevVelin Penev Jan 29, 2022 Desktop Product X.X% Progression

Velin Tested Pattern #113: More Or Fewer Plans On Chaos.com

 - Variant A
 - Variant B

In this experiment, a two plan pricing plan (standalone product with a higher collection set) was tested against a single plan one (only a collection set). Impact on clicks and total sales was measured.

Total Sample: 23,261 Statistical Power at 5% MDE: 4.8%

Test #5 on Busyteacher.org by Andrei ZakhareuskiAndrei Zakhareuski Jan 23, 2022 Desktop Mobile Product X.X% Progression

Andrei Tested Pattern #21: What It's Worth On Busyteacher.org

 - Variant A
 - Variant B

The variation included a simple extention of the headline to include the full dollar worth of a discounted bundle deal. 

Total Sample: 4,895 Statistical Power at 5% MDE: 4%

Test #392 on by Jakub LinowskiJakub Linowski Dec 31, 2021 Desktop Mobile Product X.X% Progression

Jakub Tested Pattern #122: Zigzag Layout

 - Variant A
 - Variant B

In this experiment, paragraph style copy was reorganized into a zig zag layout with key sections being reinforiced with copy-as-image statements. Impact on adds-to-cart and sales was measured.

Total Sample: 41,827 Statistical Power at 5% MDE: 28.5%

Test #391 on Backstage.com by Stanley ZuoStanley Zuo Dec 30, 2021 Desktop Mobile Listing X.X% Progression

Stanley Tested Pattern #82: Onboarding Callouts On Backstage.com

 - Variant A
 - Variant B

This experiment added a small nudge or callout to encourage more signups. Attention was directed towards the save function, which lead to the signup flow for anyone not signed it. Impact on signups was measured.

Total Sample: 159,569 Statistical Power at 5% MDE: 99.5%

Test #388 on by Jakub LinowskiJakub Linowski Dec 09, 2021 Mobile Listing X.X% Progression

Jakub Tested Pattern #51: Shortcut Buttons

 - Variant A
 - Variant B

In this experiment, extra "buy now" buttons were added on a listing page. These buttons were shortcuts to an add to cart flow, whereas the "learn more" buttons lead customers to product detail pages (visible in both control and variation). Impact on adds-to-cart and sales was measured.

Total Sample: 7,051 Statistical Power at 5% MDE: 10.1%

Test #387 on by Jakub LinowskiJakub Linowski Nov 30, 2021 Mobile Listing X.X% Progression

Jakub Tested Pattern #88: Action Button

 - Variant A
 - Variant B

In this experiment, simple "Learn More" buttons were added underneath each product thumbnail. These buttons were additional triggers that linked to product detail pages on top of the existing thumbnails and product names (that also linked to the PDPs). Impact on product visits and sales was measured.

Total Sample: 7,173 Statistical Power at 5% MDE: 10%

Test #382 on Snocks.com by Samuel HessSamuel Hess Oct 31, 2021 Desktop Product X.X% Progression

Samuel Tested Pattern #43: Long Titles On Snocks.com

 - Variant A
 - Variant B

In this experiment, the variation contained a simple text change that described the quality of the product. According to Google Translate the text changed from "tight men's underpants" (control) to "are so comfortable that you don't feel them when you wear them" (variation).

Total Sample: 140,756 Statistical Power at 5% MDE: 86.3%

Test #381 on Expertinstitute.com by Ardit VeliuArdit Veliu Oct 30, 2021 Desktop Mobile Home & Landing X.X% Progression

Ardit Tested Pattern #49: Above The Fold Call To Action On Expertinstitute.com

 - Variant A
 - Variant B

In this experiment, an extra button to a signup lead form was placed above the fold. In the control, the same button appeared further down on the page.

Total Sample: 2,970 Statistical Power at 5% MDE: 4.7%

Test #378 on by Jakub LinowskiJakub Linowski Oct 07, 2021 Desktop Mobile Product X.X% Progression

Jakub Tested Pattern #119: Unselected Or Selected Defaults

 - Variant A
 - Variant B

Here we have an experiment with a variation that preselected an option for a club duration. The control required customers to first express the choice for how many months they would like to order a product for. Whereas the variation defaulted to 12 months from the beginning.

Impact on adds-to-cart and sales was measured. The experiment unfortunately had to be stopped early due to another embedded variation that was performing poorly. And so it does not have many transactions.

Total Sample: 2,953 Statistical Power at 5% MDE: 5.2%

Test #377 on Adoramapix.com by Jakub LinowskiJakub Linowski Sep 30, 2021 Desktop Shopping Cart X.X% Progression

Jakub Tested Pattern #121: Free Shipping On Adoramapix.com

 - Variant A
 - Variant B

In this experiment, a big "free shipping" badge was added and defaulted to when available. Impact on progression to checkouts and completed sales was measured.

Total Sample: 4,422 Statistical Power at 5% MDE: 85.5%

Test #376 on Snocks.com by Samuel HessSamuel Hess Sep 29, 2021 Mobile Desktop Product X.X% Progression

Samuel Tested Pattern #15: Bulleted Reassurances On Snocks.com

 - Variant A
 - Variant B

In this experiment, a series of reassurances were added just below the add to cart button. These included: "free shipping", "free returns" and an "anti-hole guarantee". The test ran on the product page of an socks ecommerce company. Impact on sales was measured.

Total Sample: 136,126 Statistical Power at 5% MDE: 79.6%

Test #375 on Backstage.com by Stanley ZuoStanley Zuo Sep 17, 2021 Mobile Listing X.X% Progression

Stanley Tested Pattern #32: Condensed List On Backstage.com

 - Variant A
 - Variant B

Similar to experiment 373, listing descriptions were shortened dynamically using exposable "more" links AND dates/location data was removed. This way, the variation showed shorter listings and therefore more listings per screens. Impact on listing clicks (progression) along with membership starts was measured.

Total Sample: 44,045 Statistical Power at 5% MDE: 92.5%