All Latest 640 A/B Tests

Test #444 on by Melina HessMelina Hess Nov 30, 2022 Mobile Product X.X% Sales

Melina Tested Pattern #41: Sticky Call To Action

 - Variant A
 - Variant B

In this experiment, a floating add-to-cart with two product choices (color and size) were added on a product page. This appeared only after scrolling past the existing / embedded product selection buy box.

The floating add-to-cart widget had another layer of complexity in that it allowed customers to make a size and color selection with an expandable fly out. Making a selection would also append the total price to the add-to-cart button label.

Impact on total transactions was measured.

Total Sample: 1,483,721 • Statistical Power at 5% MDE: 99.9%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #442 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Nov 27, 2022 Desktop Mobile Home & Landing X.X% Sales

Daria Tested Pattern #4: Testimonials On Volders.de

 - Variant A
 - Variant B

In this experiment, three testimonials were appended at the bottom of landing pages of a contract cancelation service (paid). These testimonials were also shown throughout the complete signup funnel (4 more steps). Impact on progression (step 2) and final completed purchases were measured.

Total Sample: 73,143 • Statistical Power at 5% MDE: 83.7%

Test #443 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Nov 27, 2022 Desktop Mobile Home & Landing X.X% Sales

Daria Tested Pattern #4: Testimonials On Volders.de

 - Variant A
 - Variant B

In this experiment, four TrustPilot reviews were appended at the bottom of landing pages of a contract cancelation service (paid). These reviews were also shown throughout the complete signup funnel (4 more steps). Impact on progression (step 2) and final completed purchases were measured.

Total Sample: 72,943 • Statistical Power at 5% MDE: 83.6%

Test #441 on by Melina HessMelina Hess Nov 23, 2022 Desktop Mobile Product X.X% Sales

Melina Tested Pattern #41: Sticky Call To Action

 - Variant A
 - Variant B

In this experiment, a floating sticky bar was added on product pages. The sticky elements only appeared after users scrolled beyond the fixed buy box  area that is relatively high on the page (visible on the control screenshot). The sticky bar contained three elements: a button to configure up to three product choices, a flavor selection pulldown, and the add-to-cart button. 

In the variation, when users clicked on the flavor pulldown three things happened. First, they were anchored back to the top of the buy box. Second, the floating sticky disappeared. And third, the flavors pulldown automatically expanded (overlapping the original primary add-to-cart button).

The control did not have any of the sticky behaviors.

Impact to total sales was measured.

Total Sample: 67,618 • Statistical Power at 5% MDE: 70.8%

Test #437 on Vivareal.com.br by Rodrigo MauésRodrigo Maués Oct 28, 2022 Desktop Mobile Listing X.X% Leads

Rodrigo Tested Pattern #18: Single Or Alternative Buttons On Vivareal.com.br

 - Variant A
 - Variant B

In this experiment, an additional and alternative text based call to action link was added on real estate properity listings. Instead of only having "Ligar" ("Call"), "Enviar mensagem" was also appended ("Send Message"). This additional link lead to a lead-gen form. 

Total Sample: 2,595,541 • Statistical Power at 5% MDE: 99.9%

Test #438 on Phorest.com by Sorcha MullisSorcha Mullis Oct 28, 2022 Desktop Mobile Home & Landing X.X% Leads

Sorcha Tested Pattern #18: Single Or Alternative Buttons On Phorest.com

 - Variant A
 - Variant B

In this experiment, the addition of a secondary CTA for lead generation in the nav and the hero were tested. The additional button invited users to watch a gated demo video (approx 12 minutes). Clicking the CTA triggered a pop-up form collecting some basic contact information before the user could access the video page. Total leads were measured with lead form submittions. 

 

Total Sample: 34,059 • Statistical Power at 5% MDE: 9.2%

Test #436 on Designlab.com by Daniel ShapiroDaniel Shapiro Oct 25, 2022 Desktop Mobile Home & Landing X.X% Sales

Daniel Tested Pattern #7: Social Counts On Designlab.com

 - Variant A
 - Variant B

In this experiment, social proof and scarcity messages were shown on a course enrollment landing page. When students were starting to sign up at the beginning of each month (with greater availability), a simpler "X students have already enrolled in the next cohort" message was used.

Later in the month when fewer spots were available, a more scarce message was used with the following copy "ONLY X SPOTS available in the next cohort".

In both cases, the numbers were accurate and dynamically updated.

Total Sample: 11,214 • Statistical Power at 5% MDE: 8.6%

Test #433 on Expertinstitute.com by Ardit VeliuArdit Veliu Sep 30, 2022 Desktop Mobile Signup X.X% Leads

Ardit Tested Pattern #20: Canned Response On Expertinstitute.com

 - Variant A
 - Variant B

In this experiment a dynamic canned response was added to a lead form. The contents of the copy reflected a handful of user choices from other form field selections. Impact on leads / form submissions was measured.

Total Sample: 14,490 • Statistical Power at 5% MDE: 13.3%

Test #434 on Learnwithhomer.com by Stanley ZuoStanley Zuo Sep 30, 2022 Mobile Signup X.X% Signups

Stanley Tested Pattern #66: Complementary Upsell On Learnwithhomer.com

 - Variant A
 - Variant B

In this experiment, an upsell modal appeared during a signup funnel. In the variation, if users did not select a monthly plan, they were nudged to an annual one. Impact on overall and annual signups was measured.

Total Sample: 6,098 • Statistical Power at 5% MDE: 8.6%

Test #432 on Snocks.com by Melina HessMelina Hess Sep 29, 2022 Mobile Global X.X% Sales

Melina Tested Pattern #94: Visible Search On Snocks.com

 - Variant A
 - Variant B

In this experiment, search functionality was added in the variation, globally on mobile. Impact on sales was measured

Total Sample: 1,175,864 • Statistical Power at 5% MDE: 99.9%

Test #431 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Sep 23, 2022 Desktop Mobile Pricing X.X% Sales

Daria Tested Pattern #21: What It's Worth On Volders.de

 - Variant A
 - Variant B

In this experiment, a historically higher price point was added as a crossed out anchor. The control only showed the current price. The variation showed the current price with the higher price crossed out. Impact on sales was measured.

Total Sample: 34,645 • Statistical Power at 5% MDE: 97.1%

Test #430 on by Jakub LinowskiJakub Linowski Sep 20, 2022 Desktop Product X.X% Sales

Jakub Tested Pattern #26: Cart Reminder And Recently Viewed

 - Variant A
 - Variant B

In this experiment, up to 5 recently viewed product pages would appear on the left hand navigation. The idea was to make it easier for users get back to what they were looking at in case they were browsing. These recently viewed products were not visibile in the control. Impact on adds-to-cart and completed transactions was measured.

Total Sample: 10,324 • Statistical Power at 5% MDE: 28%

Test #428 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Aug 16, 2022 Desktop Mobile Signup X.X% Sales

Daria Tested Pattern #26: Cart Reminder And Recently Viewed On Volders.de

 - Variant A
 - Variant B

In this test 1) a passive hint communicated to users that their data will be saved for 7 days for them to be able to continue their cancellation later and 2) dropped off users were targeted with an email campaign within the first 4 hours after drop off. The reminder email linked users to a shipping page (checkout page) without them requiring to fill out their personal, contract information one more time. 

Total Sample: 155,342 • Statistical Power at 5% MDE: 98.8%

Test #424 on by Sandis ViksnaSandis Viksna Jul 28, 2022 Desktop Shopping Cart X.X% Sales

Sandis Tested Pattern #45: Benefit Bar

 - Variant A
 - Variant B

In this experiment, 3 reassurances were shown at the bottom of cart and checkout pages. The web site was selling health and nutrition products for males and one of the reassurances included "discreet shipping". Impact on sales was measured.

Total Sample: 8,655 • Statistical Power at 5% MDE: 8.6%

Test #423 on Expertinstitute.com by Ardit VeliuArdit Veliu Jul 26, 2022 Desktop Mobile Home & Landing X.X% Leads

Ardit Tested Pattern #110: Optional Field Labels On Expertinstitute.com

 - Variant A
 - Variant B

In this experiment, an "optional" label was shown near a message form field. Impact on overall leads was measured (requesting experts).

Total Sample: 1,462 • Statistical Power at 5% MDE: 6.6%

Test #419 on by Jakub LinowskiJakub Linowski Jun 29, 2022 Desktop Home & Landing X.X% Sales

Jakub Tested Pattern #68: Welcome Discount

 - Variant A
 - Variant B

In this experiment, the presence of a discount modal (bigger discount for larger purchases) was tested on home and product pages. The trigger happend after a scroll down, a few second pause, and a mouse scroll up interaction. Impact on sales was measured.

Total Sample: 196,639 • Statistical Power at 5% MDE: 98.5%

Test #418 on Online.metro-cc.ru by Andrey AndreevAndrey Andreev Jun 28, 2022 Desktop Mobile Home & Landing X.X% Sales

Andrey Tested Pattern #135: Product Categories On Online.metro-cc.ru

 - Variant A
 - Variant B

In this experiment, popular product category links were added near the bottom of the homepage of an online grocery store - Metro. Impact on completed sales was measured.

Total Sample: 78,582 • Statistical Power at 5% MDE: 12.6%

Test #416 on Designlab.com by Daniel ShapiroDaniel Shapiro Jun 20, 2022 Desktop Mobile Product X.X% Sales

Daniel Tested Pattern #105: Lead Magnets On Designlab.com

 - Variant A
 - Variant B

In this experiment, copy was added which communicated that students signing up for a course would receive extra design resources (the lead magnet). This was added in multiple states of the course page. Impact on lead generation and enrollment was measured.

Total Sample: 6,603 • Statistical Power at 5% MDE: 7.4%

Test #413 on by Jakub LinowskiJakub Linowski May 26, 2022 Desktop Mobile Shopping Cart X.X% Sales

Jakub Tested Pattern #114: Less Or More Visible Prices

 - Variant A
 - Variant B

Should prices be displayed during an editing state on an add-to-cart screen (after adding-to-cart from a product detail page)? In this experiment, club pricing was added and shown in three areas after customers clicked on a small "edit details" links. The control did not have this pricing information (which was only shown on a product detail screen before).

Total Sample: 5,698 • Statistical Power at 5% MDE: 59.9%

Test #412 on Volders.de by Frederik FröhleFrederik Fröhle May 16, 2022 Desktop Mobile Checkout X.X% Sales

Frederik Tested Pattern #15: Bulleted Reassurances On Volders.de

 - Variant A
 - Variant B

The Volders experimentation team assumed that adding information about how long it might take to get a cancellation confirmation near a CTA Button would result in higher cancellation requests (the paid service being offered).

The variation contained additional copy translated from German to: "Most of our users receive their cancellation confirmation from <vendor> within 14 days by email or letter."

Total Sample: 53,120 • Statistical Power at 5% MDE: 99.3%