All Latest 634 A/B Tests

Test #441 on by Melina HessMelina Hess Nov 23, 2022 Desktop Mobile Product X.X% Sales

Melina Tested Pattern #41: Sticky Call To Action

 - Variant A
 - Variant B

In this experiment, a floating sticky bar was added on product pages. The sticky elements only appeared after users scrolled beyond the fixed buy box  area that is relatively high on the page (visible on the control screenshot). The sticky bar contained three elements: a button to configure up to three product choices, a flavor selection pulldown, and the add-to-cart button. 

In the variation, when users clicked on the flavor pulldown three things happened. First, they were anchored back to the top of the buy box. Second, the floating sticky disappeared. And third, the flavors pulldown automatically expanded (overlapping the original primary add-to-cart button).

The control did not have any of the sticky behaviors.

Impact to total sales was measured.

Total Sample: 67,618 Statistical Power at 5% MDE: 70.8%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #437 on Vivareal.com.br by Rodrigo MauésRodrigo Maués Oct 28, 2022 Desktop Mobile Listing X.X% Leads

Rodrigo Tested Pattern #18: Single Or Alternative Buttons On Vivareal.com.br

 - Variant A
 - Variant B

In this experiment, an additional and alternative text based call to action link was added on real estate properity listings. Instead of only having "Ligar" ("Call"), "Enviar mensagem" was also appended ("Send Message"). This additional link lead to a lead-gen form. 

Total Sample: 2,595,541 Statistical Power at 5% MDE: 99.9%

Test #438 on Phorest.com by Sorcha MullisSorcha Mullis Oct 28, 2022 Desktop Mobile Home & Landing X.X% Leads

Sorcha Tested Pattern #18: Single Or Alternative Buttons On Phorest.com

 - Variant A
 - Variant B

In this experiment, the addition of a secondary CTA for lead generation in the nav and the hero were tested. The additional button invited users to watch a gated demo video (approx 12 minutes). Clicking the CTA triggered a pop-up form collecting some basic contact information before the user could access the video page. Total leads were measured with lead form submittions. 

 

Total Sample: 34,059 Statistical Power at 5% MDE: 9.2%

Test #436 on Designlab.com by Daniel ShapiroDaniel Shapiro Oct 25, 2022 Desktop Mobile Home & Landing X.X% Sales

Daniel Tested Pattern #7: Social Counts On Designlab.com

 - Variant A
 - Variant B

In this experiment, social proof and scarcity messages were shown on a course enrollment landing page. When students were starting to sign up at the beginning of each month (with greater availability), a simpler "X students have already enrolled in the next cohort" message was used.

Later in the month when fewer spots were available, a more scarce message was used with the following copy "ONLY X SPOTS available in the next cohort".

In both cases, the numbers were accurate and dynamically updated.

Total Sample: 11,214 Statistical Power at 5% MDE: 8.6%

Test #433 on Expertinstitute.com by Ardit VeliuArdit Veliu Sep 30, 2022 Desktop Mobile Signup X.X% Leads

Ardit Tested Pattern #20: Canned Response On Expertinstitute.com

 - Variant A
 - Variant B

In this experiment a dynamic canned response was added to a lead form. The contents of the copy reflected a handful of user choices from other form field selections. Impact on leads / form submissions was measured.

Total Sample: 14,490 Statistical Power at 5% MDE: 13.3%

Test #434 on Learnwithhomer.com by Stanley ZuoStanley Zuo Sep 30, 2022 Mobile Signup X.X% Signups

Stanley Tested Pattern #66: Complementary Upsell On Learnwithhomer.com

 - Variant A
 - Variant B

In this experiment, an upsell modal appeared during a signup funnel. In the variation, if users did not select a monthly plan, they were nudged to an annual one. Impact on overall and annual signups was measured.

Total Sample: 6,098 Statistical Power at 5% MDE: 8.6%

Test #432 on Snocks.com by Melina HessMelina Hess Sep 29, 2022 Mobile Global X.X% Sales

Melina Tested Pattern #94: Visible Search On Snocks.com

 - Variant A
 - Variant B

In this experiment, search functionality was added in the variation, globally on mobile. Impact on sales was measured

Total Sample: 1,175,864 Statistical Power at 5% MDE: 99.9%

Test #431 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Sep 23, 2022 Desktop Mobile Pricing X.X% Sales

Daria Tested Pattern #21: What It's Worth On Volders.de

 - Variant A
 - Variant B

In this experiment, a historically higher price point was added as a crossed out anchor. The control only showed the current price. The variation showed the current price with the higher price crossed out. Impact on sales was measured.

Total Sample: 34,645 Statistical Power at 5% MDE: 97.1%

Test #430 on by Jakub LinowskiJakub Linowski Sep 20, 2022 Desktop Product X.X% Sales

Jakub Tested Pattern #26: Cart Reminder And Recently Viewed

 - Variant A
 - Variant B

In this experiment, up to 5 recently viewed product pages would appear on the left hand navigation. The idea was to make it easier for users get back to what they were looking at in case they were browsing. These recently viewed products were not visibile in the control. Impact on adds-to-cart and completed transactions was measured.

Total Sample: 10,324 Statistical Power at 5% MDE: 28%

Test #428 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Aug 16, 2022 Desktop Mobile Signup X.X% Sales

Daria Tested Pattern #26: Cart Reminder And Recently Viewed On Volders.de

 - Variant A
 - Variant B

In this test 1) a passive hint communicated to users that their data will be saved for 7 days for them to be able to continue their cancellation later and 2) dropped off users were targeted with an email campaign within the first 4 hours after drop off. The reminder email linked users to a shipping page (checkout page) without them requiring to fill out their personal, contract information one more time. 

Total Sample: 155,342 Statistical Power at 5% MDE: 98.8%

Test #424 on by Sandis ViksnaSandis Viksna Jul 28, 2022 Desktop Shopping Cart X.X% Sales

Sandis Tested Pattern #45: Benefit Bar

 - Variant A
 - Variant B

In this experiment, 3 reassurances were shown at the bottom of cart and checkout pages. The web site was selling health and nutrition products for males and one of the reassurances included "discreet shipping". Impact on sales was measured.

Total Sample: 8,655 Statistical Power at 5% MDE: 8.6%

Test #423 on Expertinstitute.com by Ardit VeliuArdit Veliu Jul 26, 2022 Desktop Mobile Home & Landing X.X% Leads

Ardit Tested Pattern #110: Optional Field Labels On Expertinstitute.com

 - Variant A
 - Variant B

In this experiment, an "optional" label was shown near a message form field. Impact on overall leads was measured (requesting experts).

Total Sample: 1,462 Statistical Power at 5% MDE: 6.6%

Test #419 on by Jakub LinowskiJakub Linowski Jun 29, 2022 Desktop Home & Landing X.X% Sales

Jakub Tested Pattern #68: Welcome Discount

 - Variant A
 - Variant B

In this experiment, the presence of a discount modal (bigger discount for larger purchases) was tested on home and product pages. The trigger happend after a scroll down, a few second pause, and a mouse scroll up interaction. Impact on sales was measured.

Total Sample: 196,639 Statistical Power at 5% MDE: 98.5%

Test #418 on Online.metro-cc.ru by Andrey AndreevAndrey Andreev Jun 28, 2022 Desktop Mobile Home & Landing X.X% Sales

Andrey Tested Pattern #135: Product Categories On Online.metro-cc.ru

 - Variant A
 - Variant B

In this experiment, popular product category links were added near the bottom of the homepage of an online grocery store - Metro. Impact on completed sales was measured.

Total Sample: 78,582 Statistical Power at 5% MDE: 12.6%

Test #416 on Designlab.com by Daniel ShapiroDaniel Shapiro Jun 20, 2022 Desktop Mobile Product X.X% Sales

Daniel Tested Pattern #105: Lead Magnets On Designlab.com

 - Variant A
 - Variant B

In this experiment, copy was added which communicated that students signing up for a course would receive extra design resources (the lead magnet). This was added in multiple states of the course page. Impact on lead generation and enrollment was measured.

Total Sample: 6,603 Statistical Power at 5% MDE: 7.4%

Test #413 on by Jakub LinowskiJakub Linowski May 26, 2022 Desktop Mobile Shopping Cart X.X% Sales

Jakub Tested Pattern #114: Less Or More Visible Prices

 - Variant A
 - Variant B

Should prices be displayed during an editing state on an add-to-cart screen (after adding-to-cart from a product detail page)? In this experiment, club pricing was added and shown in three areas after customers clicked on a small "edit details" links. The control did not have this pricing information (which was only shown on a product detail screen before).

Total Sample: 5,698 Statistical Power at 5% MDE: 59.9%

Test #412 on Volders.de by Frederik FröhleFrederik Fröhle May 16, 2022 Desktop Mobile Checkout X.X% Sales

Frederik Tested Pattern #15: Bulleted Reassurances On Volders.de

 - Variant A
 - Variant B

The Volders experimentation team assumed that adding information about how long it might take to get a cancellation confirmation near a CTA Button would result in higher cancellation requests (the paid service being offered).

The variation contained additional copy translated from German to: "Most of our users receive their cancellation confirmation from <vendor> within 14 days by email or letter."

Total Sample: 53,120 Statistical Power at 5% MDE: 99.3%

Test #410 on Designlab.com by Daniel ShapiroDaniel Shapiro May 05, 2022 Desktop Mobile Home & Landing X.X% Sales

Daniel Tested Pattern #29: Surfaced Content On Designlab.com

 - Variant A
 - Variant B

Does linking to a product detail page help? Or should a homepage simply focus on generic lead generation? In this homepage experiment, the presence of a component with a link to a detailed course landing page was tested. Impact on course enrollments was measured.

Total Sample: 25,443 Statistical Power at 5% MDE: 6.3%

Test #408 on by Jakub LinowskiJakub Linowski Apr 29, 2022 Desktop Mobile Product X.X% Sales

Jakub Tested Pattern #67: Currency & Taxes

 - Variant A
 - Variant B

Here is a very simple experiment where CDN currency copy was appended to prices being shown on a Canadian ecommerce site. Additional copy ("from Canada") was appended to an existing shipping message.  

Total Sample: 16,332 Statistical Power at 5% MDE: 17.9%

Test #407 on Snocks.com by Melina HessMelina Hess Apr 22, 2022 Desktop Mobile Product X.X% Sales

Melina Tested Pattern #125: Unit Prices On Snocks.com

 - Variant A
 - Variant B

Here is an experiment with an added "price per pair" in the context of products with multiple items (packs).A high contrast badge-like copy was appended underneath the price which translates to "only $X per item". Impact on sales was measured.

Total Sample: 112,178 Statistical Power at 5% MDE: 56%