All Latest 645 A/B Tests

Test #419 by Jakub LinowskiJakub Linowski Jun 29, 2022 Desktop Home & Landing X.X% Sales

Jakub Tested Pattern #68: Welcome Discount

Pattern #68: Welcome Discount - Variant A
Pattern #68: Welcome Discount - Variant B

In this experiment, the presence of a discount modal (bigger discount for larger purchases) was tested on home and product pages. The trigger happend after a scroll down, a few second pause, and a mouse scroll up interaction. Impact on sales was measured.

Total Sample: 196,639 • Statistical Power at 5% MDE: 98.5%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #418 on Online.metro-cc.ru by Andrey AndreevAndrey Andreev Jun 28, 2022 Desktop Mobile Home & Landing X.X% Sales

Andrey Tested Pattern #135: Product Categories On Online.metro-cc.ru

Pattern #135: Product Categories - Variant A
Pattern #135: Product Categories - Variant B

In this experiment, popular product category links were added near the bottom of the homepage of an online grocery store - Metro. Impact on completed sales was measured.

Total Sample: 78,582 • Statistical Power at 5% MDE: 12.6%

Test #416 on Designlab.com by Daniel ShapiroDaniel Shapiro Jun 20, 2022 Desktop Mobile Product X.X% Sales

Daniel Tested Pattern #105: Lead Magnets On Designlab.com

Pattern #105: Lead Magnets - Variant A
Pattern #105: Lead Magnets - Variant B

In this experiment, copy was added which communicated that students signing up for a course would receive extra design resources (the lead magnet). This was added in multiple states of the course page. Impact on lead generation and enrollment was measured.

Total Sample: 6,603 • Statistical Power at 5% MDE: 7.4%

Test #413 by Jakub LinowskiJakub Linowski May 26, 2022 Desktop Mobile Shopping Cart X.X% Sales

Jakub Tested Pattern #114: Less Or More Visible Prices

Pattern #114: Less Or More Visible Prices - Variant A
Pattern #114: Less Or More Visible Prices - Variant B

Should prices be displayed during an editing state on an add-to-cart screen (after adding-to-cart from a product detail page)? In this experiment, club pricing was added and shown in three areas after customers clicked on a small "edit details" links. The control did not have this pricing information (which was only shown on a product detail screen before).

Total Sample: 5,698 • Statistical Power at 5% MDE: 59.9%

Test #412 on Volders.de by Frederik FröhleFrederik Fröhle May 16, 2022 Desktop Mobile Checkout X.X% Sales

Frederik Tested Pattern #15: Bulleted Reassurances On Volders.de

Pattern #15: Bulleted Reassurances - Variant A
Pattern #15: Bulleted Reassurances - Variant B

The Volders experimentation team assumed that adding information about how long it might take to get a cancellation confirmation near a CTA Button would result in higher cancellation requests (the paid service being offered).

The variation contained additional copy translated from German to: "Most of our users receive their cancellation confirmation from <vendor> within 14 days by email or letter."

Total Sample: 53,120 • Statistical Power at 5% MDE: 99.3%

Test #410 on Designlab.com by Daniel ShapiroDaniel Shapiro May 05, 2022 Desktop Mobile Home & Landing X.X% Sales

Daniel Tested Pattern #29: Surfaced Content On Designlab.com

Pattern #29: Surfaced Content - Variant A
Pattern #29: Surfaced Content - Variant B

Does linking to a product detail page help? Or should a homepage simply focus on generic lead generation? In this homepage experiment, the presence of a component with a link to a detailed course landing page was tested. Impact on course enrollments was measured.

Total Sample: 25,443 • Statistical Power at 5% MDE: 6.3%

Test #408 by Jakub LinowskiJakub Linowski Apr 29, 2022 Desktop Mobile Product X.X% Sales

Jakub Tested Pattern #67: Currency & Taxes

Pattern #67: Currency &amp; Taxes - Variant A
Pattern #67: Currency &amp; Taxes - Variant B

Here is a very simple experiment where CDN currency copy was appended to prices being shown on a Canadian ecommerce site. Additional copy ("from Canada") was appended to an existing shipping message.  

Total Sample: 16,332 • Statistical Power at 5% MDE: 17.9%

Test #407 on Snocks.com by Melina HessMelina Hess Apr 22, 2022 Desktop Mobile Product X.X% Sales

Melina Tested Pattern #125: Unit Prices On Snocks.com

Pattern #125: Unit Prices - Variant A
Pattern #125: Unit Prices - Variant B

Here is an experiment with an added "price per pair" in the context of products with multiple items (packs).A high contrast badge-like copy was appended underneath the price which translates to "only $X per item". Impact on sales was measured.

Total Sample: 112,178 • Statistical Power at 5% MDE: 56%

Test #403 by Jakub LinowskiJakub Linowski Mar 29, 2022 Desktop Mobile Checkout X.X% Sales

Jakub Tested Pattern #42: Countdown Timer

Pattern #42: Countdown Timer - Variant A
Pattern #42: Countdown Timer - Variant B

In this experiment, an urgency related message with a dynamic countdown timer was added on the final checkout screen. Impact on sales was measured.

Total Sample: 7,466 • Statistical Power at 5% MDE: 99.9%

Test #400 by Herman KleinHerman Klein Mar 06, 2022 Desktop Shopping Cart X.X% Sales

Herman Tested Pattern #65: Add More For Extra Incentive

Pattern #65: Add More For Extra Incentive - Variant A
Pattern #65: Add More For Extra Incentive - Variant B

In this experiment, customers who were not yet eligible for free shipping (with cart amounts under $39) were presented with an encouraging message to add more and earn free shipping. Impact on progression to checkout and sales was measured.

Total Sample: 3,111 • Statistical Power at 5% MDE: 60.3%

Test #398 on Adoramapix.com by Herman KleinHerman Klein Feb 25, 2022 Desktop Shopping Cart X.X% Sales

Herman Tested Pattern #121: Free Shipping On Adoramapix.com

Pattern #121: Free Shipping - Variant A
Pattern #121: Free Shipping - Variant B

In this experiment, customers were eligible for free shipping (with cart amounts of $39 or more) were presented with an earned free shipping message. The variation also showed the value of the earned free shipping - (for example $10). Impact on progression to checkout and sales was measured.

Total Sample: 3,123 • Statistical Power at 5% MDE: 60.5%

Test #397 on Snocks.com by Melina HessMelina Hess Feb 24, 2022 Mobile Product X.X% Revenue

Melina Tested Pattern #121: Free Shipping On Snocks.com

Pattern #121: Free Shipping - Variant A
Pattern #121: Free Shipping - Variant B

In this experiment a "Free Shipping & Returns" message was added just below the price information. 

Test #395 by Jakub LinowskiJakub Linowski Jan 31, 2022 Desktop Mobile Product X.X% Sales

Jakub Tested Pattern #52: How It Works

Pattern #52: How It Works - Variant A
Pattern #52: How It Works - Variant B

In this experiment, a How It Works section was appended on a product detail page just above product reviews.

Total Sample: 17,099 • Statistical Power at 5% MDE: 15.8%

Test #5 on Busyteacher.org by Andrei ZakhareuskiAndrei Zakhareuski Jan 23, 2022 Desktop Mobile Product X.X% Sales

Andrei Tested Pattern #21: What It's Worth On Busyteacher.org

Pattern #21: What It&#039;s Worth - Variant A
Pattern #21: What It&#039;s Worth - Variant B

The variation included a simple extention of the headline to include the full dollar worth of a discounted bundle deal. 

Total Sample: 4,895 • Statistical Power at 5% MDE: 4%

Test #122 on Designlab.com by Daniel ShapiroDaniel Shapiro Jan 22, 2022 Desktop Mobile Product X.X% Leads

Daniel Tested Pattern #30: Authentic Photos On Designlab.com

Pattern #30: Authentic Photos - Variant A
Pattern #30: Authentic Photos - Variant B

In this experiment for an online design course, the variation had an extra section with student work previews. The previews were not clickable but were added just below the fold. Impact on leads was measured by potential students requesting a syllabus through an online form throughout the long landing page.

Total Sample: 9,862 • Statistical Power at 5% MDE: 7.4%

Test #391 on Backstage.com by Stanley ZuoStanley Zuo Dec 30, 2021 Desktop Mobile Listing X.X% Signups

Stanley Tested Pattern #82: Onboarding Callouts On Backstage.com

Pattern #82: Onboarding Callouts - Variant A
Pattern #82: Onboarding Callouts - Variant B

This experiment added a small nudge or callout to encourage more signups. Attention was directed towards the save function, which lead to the signup flow for anyone not signed it. Impact on signups was measured.

Total Sample: 159,569 • Statistical Power at 5% MDE: 99.5%

Test #390 on Snocks.com by Melina HessMelina Hess Dec 21, 2021 Desktop Mobile Checkout X.X% Sales

Melina Tested Pattern #106: Back Buttons On Snocks.com

Pattern #106: Back Buttons - Variant A
Pattern #106: Back Buttons - Variant B

In this experiment, the variation has a "Back To Shopping Cart" link right underneath the checkout button. Impact on sales was measured.

Total Sample: 22,852 • Statistical Power at 5% MDE: 97.5%

Test #388 by Jakub LinowskiJakub Linowski Dec 09, 2021 Mobile Listing X.X% Sales

Jakub Tested Pattern #51: Shortcut Buttons

Pattern #51: Shortcut Buttons - Variant A
Pattern #51: Shortcut Buttons - Variant B

In this experiment, extra "buy now" buttons were added on a listing page. These buttons were shortcuts to an add to cart flow, whereas the "learn more" buttons lead customers to product detail pages (visible in both control and variation). Impact on adds-to-cart and sales was measured.

Total Sample: 7,051 • Statistical Power at 5% MDE: 10.1%

Test #387 by Jakub LinowskiJakub Linowski Nov 30, 2021 Mobile Listing X.X% Sales

Jakub Tested Pattern #88: Action Button

Pattern #88: Action Button - Variant A
Pattern #88: Action Button - Variant B

In this experiment, simple "Learn More" buttons were added underneath each product thumbnail. These buttons were additional triggers that linked to product detail pages on top of the existing thumbnails and product names (that also linked to the PDPs). Impact on product visits and sales was measured.

Total Sample: 7,173 • Statistical Power at 5% MDE: 10%

Test #385 on Snocks.com by Samuel HessSamuel Hess Nov 25, 2021 Mobile Product X.X% Sales

Samuel Tested Pattern #63: Trust Seals On Snocks.com

Pattern #63: Trust Seals - Variant A
Pattern #63: Trust Seals - Variant B

In this experiment, a simple Amazon badge was added on a product page. The translated (Google) language read: "Since we founded snocks, our customers have always been our focus - we are constantly trying to develop further. That's why we're extremely proud that Amazon attracted attention and that they named us Sales Partner of the Year in 2019."

Total Sample: 351,255 • Statistical Power at 5% MDE: 99.9%