All Latest 644 A/B Tests

MOST RECENT TESTS

Test #508 on Online.metro-cc.ru by Andrey AndreevAndrey Andreev Dec 15, 2023 Mobile Product X.X% Sales

Andrey Tested Pattern #93: Auto Next On Online.metro-cc.ru

Pattern #93: Auto Next - Variant A
Pattern #93: Auto Next - Variant B

In this experiment, the variation transformed an add-to-cart button into a shopping cart one (making it a two step process). The control had an interaction where an add-to-cart button would become a quantity selection and then a shopping cart action (a three step process). In a way, the control kept users in a "dead-end" quantity selection middle state that required clicking the back button to continue the sale. Whereas the variation automatically moved users into the next step (closer towards being able to make a purchase). Impact on sales was measured. 

Total Sample: 514,834 • Statistical Power at 5% MDE: 29.5%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #499 on Shmoodyapp.com by Michael McSweeneyMichael McSweeney Oct 20, 2023 Mobile Signup X.X% Sales

Michael Tested Pattern #99: Progress Bar On Shmoodyapp.com

Pattern #99: Progress Bar - Variant A
Pattern #99: Progress Bar - Variant B

In this experiment, a progress bar was added to a signup flow of a mental health app. It appeared on numerous steps of a onboarding flow. Impact on signup completions and checkouts was measured.

Total Sample: 56,735 • Statistical Power at 5% MDE: 37.1%

Test #495 on Formelskin.de by Alexander KriegerAlexander Krieger Sep 25, 2023 Mobile Signup X.X% Sales

Alexander Tested Pattern #9: Multiple Steps On Formelskin.de

Pattern #9: Multiple Steps - Variant A
Pattern #9: Multiple Steps - Variant B

In this experiment two sections of a form on a single page (control) were broken out into 2 separate steps (variation). The two sections included personal details and shipping information. This step appeared after having received a product recommendation when filling out a questionnaire for a skin care product. Impact on next step progression and purchases was measured.

Total Sample: 8,280 • Statistical Power at 5% MDE: 14.1%

Test #490 by Jakub LinowskiJakub Linowski Aug 17, 2023 Desktop Mobile X.X% Sales

Jakub Tested Pattern #9: Multiple Steps

Pattern #9: Multiple Steps - Variant A
Pattern #9: Multiple Steps - Variant B

In this experiment, a section dedicated to choosing gift messages was taken out and separated into its own individial step. The change happened on the first step of a checkout flow and increased the flow by an additional step. Impact on checkouts and total sales was measured.

Total Sample: 3,955 • Statistical Power at 5% MDE: 62.8%

Test #460 on Backstage.com by Stanley ZuoStanley Zuo Mar 21, 2023 Mobile Listing X.X% Sales

Stanley Tested Pattern #41: Sticky Call To Action On Backstage.com

Pattern #41: Sticky Call To Action - Variant A
Pattern #41: Sticky Call To Action - Variant B

In this experiment, a floating top navigation was shown with a "Join" button. In the control, the navigation was only visible at the top of the page. Also keep in mind that signup starts were also triggered throughout multiple CTAs throughout the page and from particular job detail pages. The a/b test ran on a listing page of Backstage - a casting call job site. Impact on signups  and checkouts was measured.

Total Sample: 1,057,325 • Statistical Power at 5% MDE: 41.8%

Test #457 by Jakub LinowskiJakub Linowski Feb 26, 2023 Desktop Mobile Shopping Cart X.X% Sales

Jakub Tested Pattern #64: Tunnel

Pattern #64: Tunnel - Variant A
Pattern #64: Tunnel - Variant B

In this experiment, hamburger navigation with product links, was removed from the shopping / add to cart page. Tertiary links (How It Works, Guarantee details and support pages) were also removed, while a "Secure Checkout" message was brought into the header. This header version was already present on the next checkout page, and was copied over one step earlier. The idea was to keep customers within the checkout flow. Impact on sales was measured.

Total Sample: 4,452 • Statistical Power at 5% MDE: 48.6%

Test #456 on Aboalarm.de by Daria KurchinskaiaDaria Kurchinskaia Feb 23, 2023 Desktop Mobile Signup X.X% Sales

Daria Tested Pattern #28: Easiest Fields First On Aboalarm.de

Pattern #28: Easiest Fields First - Variant A
Pattern #28: Easiest Fields First - Variant B

In this experiment, a more difficult step of a contract cancelation service flow was rearranged toward a later step. In the variation, the easier step (hypothetically) with personal details and address fields was placed as the first step. Whereas the step with contract or account numbers (hypothetically more difficult) were placed as the second step.

Total Sample: 138,384 • Statistical Power at 5% MDE: 92.7%

Test #453 by Jakub LinowskiJakub Linowski Jan 31, 2023 Desktop Mobile Shopping Cart X.X% Sales

Jakub Tested Pattern #64: Tunnel

Pattern #64: Tunnel - Variant A
Pattern #64: Tunnel - Variant B

In this experiment, the footer was simplified with multiple elements being removed (catalog request, secondary links, and a guarantee). Additionally, a more prominent phone number was also displayed.

Total Sample: 17,107 • Statistical Power at 5% MDE: 91.8%

Test #450 on Trenyrkarna.cz by Ondřej IlinčevOndřej Ilinčev Jan 20, 2023 Desktop Shopping Cart X.X% Sales

Ondřej Tested Pattern #64: Tunnel On Trenyrkarna.cz

Pattern #64: Tunnel - Variant A
Pattern #64: Tunnel - Variant B

In this experiment, top category navigation (with fly out categories) was removed from the shopping cart of an online store. Impact on sale transactions was measured.

Total Sample: 4,737 • Statistical Power at 5% MDE: 25.4%

Test #444 by Melina HessMelina Hess Nov 30, 2022 Mobile Product X.X% Sales

Melina Tested Pattern #41: Sticky Call To Action

Pattern #41: Sticky Call To Action - Variant A
Pattern #41: Sticky Call To Action - Variant B

In this experiment, a floating add-to-cart with two product choices (color and size) were added on a product page. This appeared only after scrolling past the existing / embedded product selection buy box.

The floating add-to-cart widget had another layer of complexity in that it allowed customers to make a size and color selection with an expandable fly out. Making a selection would also append the total price to the add-to-cart button label.

Impact on total transactions was measured.

Total Sample: 1,483,721 • Statistical Power at 5% MDE: 99.9%

Test #441 by Melina HessMelina Hess Nov 23, 2022 Desktop Mobile Product X.X% Sales

Melina Tested Pattern #41: Sticky Call To Action

Pattern #41: Sticky Call To Action - Variant A
Pattern #41: Sticky Call To Action - Variant B

In this experiment, a floating sticky bar was added on product pages. The sticky elements only appeared after users scrolled beyond the fixed buy box  area that is relatively high on the page (visible on the control screenshot). The sticky bar contained three elements: a button to configure up to three product choices, a flavor selection pulldown, and the add-to-cart button. 

In the variation, when users clicked on the flavor pulldown three things happened. First, they were anchored back to the top of the buy box. Second, the floating sticky disappeared. And third, the flavors pulldown automatically expanded (overlapping the original primary add-to-cart button).

The control did not have any of the sticky behaviors.

Impact to total sales was measured.

Total Sample: 67,618 • Statistical Power at 5% MDE: 70.8%

Test #435 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Oct 17, 2022 Desktop Mobile Checkout X.X% Sales

Daria Tested Pattern #9: Multiple Steps On Volders.de

Pattern #9: Multiple Steps - Variant A
Pattern #9: Multiple Steps - Variant B

In this experiment, a question about a customer's reason for purchase was broken out into a separate step and moved earlier in the process. In the control, this question was asked in the final checkout step along with a plan selection (Step 4 of 4). In the variation, this question was shifted as a standalone first step (Step 1 of 5). Impact on completed purchases was measured (for a contract cancellation service in this case).

Total Sample: 139,608 • Statistical Power at 5% MDE: 98.2%

Test #432 on Snocks.com by Melina HessMelina Hess Sep 29, 2022 Mobile Global X.X% Sales

Melina Tested Pattern #94: Visible Search On Snocks.com

Pattern #94: Visible Search - Variant A
Pattern #94: Visible Search - Variant B

In this experiment, search functionality was added in the variation, globally on mobile. Impact on sales was measured

Total Sample: 1,175,864 • Statistical Power at 5% MDE: 99.9%

Test #430 by Jakub LinowskiJakub Linowski Sep 20, 2022 Desktop Product X.X% Sales

Jakub Tested Pattern #26: Cart Reminder And Recently Viewed

Pattern #26: Cart Reminder And Recently Viewed - Variant A
Pattern #26: Cart Reminder And Recently Viewed - Variant B

In this experiment, up to 5 recently viewed product pages would appear on the left hand navigation. The idea was to make it easier for users get back to what they were looking at in case they were browsing. These recently viewed products were not visibile in the control. Impact on adds-to-cart and completed transactions was measured.

Total Sample: 10,324 • Statistical Power at 5% MDE: 28%

Test #428 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Aug 16, 2022 Desktop Mobile Signup X.X% Sales

Daria Tested Pattern #26: Cart Reminder And Recently Viewed On Volders.de

Pattern #26: Cart Reminder And Recently Viewed - Variant A
Pattern #26: Cart Reminder And Recently Viewed - Variant B

In this test 1) a passive hint communicated to users that their data will be saved for 7 days for them to be able to continue their cancellation later and 2) dropped off users were targeted with an email campaign within the first 4 hours after drop off. The reminder email linked users to a shipping page (checkout page) without them requiring to fill out their personal, contract information one more time. 

Total Sample: 155,342 • Statistical Power at 5% MDE: 98.8%

Test #426 on Phorest.com by Sorcha MullisSorcha Mullis Aug 09, 2022 Desktop Mobile Home & Landing X.X% Leads

Sorcha Tested Pattern #9: Multiple Steps On Phorest.com

Pattern #9: Multiple Steps - Variant A
Pattern #9: Multiple Steps - Variant B

In this experiment, a single step popup modal was tested against a 4 step sign up funnel. Users entered the experiment on the homepage and the behavior of the two buttons, book a demo and get a quote, was adjusted. Impact on leads was measured.

Total Sample: 88,974 • Statistical Power at 5% MDE: 7.5%

Test #425 by Jakub LinowskiJakub Linowski Aug 03, 2022 Desktop Product X.X% Sales

Jakub Tested Pattern #41: Sticky Call To Action

Pattern #41: Sticky Call To Action - Variant A
Pattern #41: Sticky Call To Action - Variant B

In this experiment, the complete buy box on a product detail page, floated along as users scrolled through the long screen. The variation made sure the product choice and order now button was always visible. Impact on adds-to-cart and sales was measured.

Total Sample: 8,174 • Statistical Power at 5% MDE: 13.8%

Test #414 on Volders.de by Frederik FröhleFrederik Fröhle May 31, 2022 Desktop Mobile Checkout X.X% Sales

Frederik Tested Pattern #98: Auto Suggest On Volders.de

Pattern #98: Auto Suggest - Variant A
Pattern #98: Auto Suggest - Variant B

Does adding Google's address auto complete functionality to an address field help with higher form completions? This auto fill feature has been tested in the variation of a contract cancellation funnel. After selecting an auto completed address from a pulldown menu, the following fields were preselected: house number, zip code, city and country (potentially lowering friction?). Impact on successful form completions (contract cancellations) has been measured. Notice how the form also expanded progressively upon selecting the complete address in the variation.  

Total Sample: 128,199 • Statistical Power at 5% MDE: 97.4%

Test #405 on Learnwithhomer.com by Stanley ZuoStanley Zuo Apr 07, 2022 Mobile Signup X.X% Sales

Stanley Tested Pattern #119: Unselected Or Selected Defaults On Learnwithhomer.com

Pattern #119: Unselected Or Selected Defaults - Variant A
Pattern #119: Unselected Or Selected Defaults - Variant B

In this experiment, the annual plan was preselected instead of starting off unselected in the control. Impact on two metrics was measured: any plan and annual plan purchases.

Total Sample: 14,903 • Statistical Power at 5% MDE: 26.2%

Test #390 on Snocks.com by Melina HessMelina Hess Dec 21, 2021 Desktop Mobile Checkout X.X% Sales

Melina Tested Pattern #106: Back Buttons On Snocks.com

Pattern #106: Back Buttons - Variant A
Pattern #106: Back Buttons - Variant B

In this experiment, the variation has a "Back To Shopping Cart" link right underneath the checkout button. Impact on sales was measured.

Total Sample: 22,852 • Statistical Power at 5% MDE: 97.5%