All Latest 639 A/B Tests

MOST RECENT TESTS

Test #490 on by Jakub LinowskiJakub Linowski Aug 17, 2023 Desktop Mobile X.X% Sales

Jakub Tested Pattern #9: Multiple Steps

 - Variant A
 - Variant B

In this experiment, a section dedicated to choosing gift messages was taken out and separated into its own individial step. The change happened on the first step of a checkout flow and increased the flow by an additional step. Impact on checkouts and total sales was measured.

Total Sample: 3,955 Statistical Power at 5% MDE: 62.8%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #460 on Backstage.com by Stanley ZuoStanley Zuo Mar 21, 2023 Mobile Listing X.X% Sales

Stanley Tested Pattern #41: Sticky Call To Action On Backstage.com

 - Variant A
 - Variant B

In this experiment, a floating top navigation was shown with a "Join" button. In the control, the navigation was only visible at the top of the page. Also keep in mind that signup starts were also triggered throughout multiple CTAs throughout the page and from particular job detail pages. The a/b test ran on a listing page of Backstage - a casting call job site. Impact on signups  and checkouts was measured.

Total Sample: 1,057,325 Statistical Power at 5% MDE: 41.8%

Test #457 on by Jakub LinowskiJakub Linowski Feb 26, 2023 Desktop Mobile Shopping Cart X.X% Sales

Jakub Tested Pattern #64: Tunnel

 - Variant A
 - Variant B

In this experiment, hamburger navigation with product links, was removed from the shopping / add to cart page. Tertiary links (How It Works, Guarantee details and support pages) were also removed, while a "Secure Checkout" message was brought into the header. This header version was already present on the next checkout page, and was copied over one step earlier. The idea was to keep customers within the checkout flow. Impact on sales was measured.

Total Sample: 4,452 Statistical Power at 5% MDE: 48.6%

Test #456 on Aboalarm.de by Daria KurchinskaiaDaria Kurchinskaia Feb 23, 2023 Desktop Mobile Signup X.X% Sales

Daria Tested Pattern #28: Easiest Fields First On Aboalarm.de

 - Variant A
 - Variant B

In this experiment, a more difficult step of a contract cancelation service flow was rearranged toward a later step. In the variation, the easier step (hypothetically) with personal details and address fields was placed as the first step. Whereas the step with contract or account numbers (hypothetically more difficult) were placed as the second step.

Total Sample: 138,384 Statistical Power at 5% MDE: 92.7%

Test #453 on by Jakub LinowskiJakub Linowski Jan 31, 2023 Desktop Mobile Shopping Cart X.X% Sales

Jakub Tested Pattern #64: Tunnel

 - Variant A
 - Variant B

In this experiment, the footer was simplified with multiple elements being removed (catalog request, secondary links, and a guarantee). Additionally, a more prominent phone number was also displayed.

Total Sample: 17,107 Statistical Power at 5% MDE: 91.8%

Test #450 on Trenyrkarna.cz by Ondřej IlinčevOndřej Ilinčev Jan 20, 2023 Desktop Shopping Cart X.X% Sales

Ondřej Tested Pattern #64: Tunnel On Trenyrkarna.cz

 - Variant A
 - Variant B

In this experiment, top category navigation (with fly out categories) was removed from the shopping cart of an online store. Impact on sale transactions was measured.

Total Sample: 4,737 Statistical Power at 5% MDE: 25.4%

Test #444 on by Melina HessMelina Hess Nov 30, 2022 Mobile Product X.X% Sales

Melina Tested Pattern #41: Sticky Call To Action

 - Variant A
 - Variant B

In this experiment, a floating add-to-cart with two product choices (color and size) were added on a product page. This appeared only after scrolling past the existing / embedded product selection buy box.

The floating add-to-cart widget had another layer of complexity in that it allowed customers to make a size and color selection with an expandable fly out. Making a selection would also append the total price to the add-to-cart button label.

Impact on total transactions was measured.

Total Sample: 1,483,721 Statistical Power at 5% MDE: 99.9%

Test #441 on by Melina HessMelina Hess Nov 23, 2022 Desktop Mobile Product X.X% Sales

Melina Tested Pattern #41: Sticky Call To Action

 - Variant A
 - Variant B

In this experiment, a floating sticky bar was added on product pages. The sticky elements only appeared after users scrolled beyond the fixed buy box  area that is relatively high on the page (visible on the control screenshot). The sticky bar contained three elements: a button to configure up to three product choices, a flavor selection pulldown, and the add-to-cart button. 

In the variation, when users clicked on the flavor pulldown three things happened. First, they were anchored back to the top of the buy box. Second, the floating sticky disappeared. And third, the flavors pulldown automatically expanded (overlapping the original primary add-to-cart button).

The control did not have any of the sticky behaviors.

Impact to total sales was measured.

Total Sample: 67,618 Statistical Power at 5% MDE: 70.8%

Test #435 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Oct 17, 2022 Desktop Mobile Checkout X.X% Sales

Daria Tested Pattern #9: Multiple Steps On Volders.de

 - Variant A
 - Variant B

In this experiment, a question about a customer's reason for purchase was broken out into a separate step and moved earlier in the process. In the control, this question was asked in the final checkout step along with a plan selection (Step 4 of 4). In the variation, this question was shifted as a standalone first step (Step 1 of 5). Impact on completed purchases was measured (for a contract cancellation service in this case).

Total Sample: 139,608 Statistical Power at 5% MDE: 98.2%

Test #432 on Snocks.com by Melina HessMelina Hess Sep 29, 2022 Mobile Global X.X% Sales

Melina Tested Pattern #94: Visible Search On Snocks.com

 - Variant A
 - Variant B

In this experiment, search functionality was added in the variation, globally on mobile. Impact on sales was measured

Total Sample: 1,175,864 Statistical Power at 5% MDE: 99.9%

Test #430 on by Jakub LinowskiJakub Linowski Sep 20, 2022 Desktop Product X.X% Sales

Jakub Tested Pattern #26: Cart Reminder And Recently Viewed

 - Variant A
 - Variant B

In this experiment, up to 5 recently viewed product pages would appear on the left hand navigation. The idea was to make it easier for users get back to what they were looking at in case they were browsing. These recently viewed products were not visibile in the control. Impact on adds-to-cart and completed transactions was measured.

Total Sample: 10,324 Statistical Power at 5% MDE: 28%

Test #428 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Aug 16, 2022 Desktop Mobile Signup X.X% Sales

Daria Tested Pattern #26: Cart Reminder And Recently Viewed On Volders.de

 - Variant A
 - Variant B

In this test 1) a passive hint communicated to users that their data will be saved for 7 days for them to be able to continue their cancellation later and 2) dropped off users were targeted with an email campaign within the first 4 hours after drop off. The reminder email linked users to a shipping page (checkout page) without them requiring to fill out their personal, contract information one more time. 

Total Sample: 155,342 Statistical Power at 5% MDE: 98.8%

Test #426 on Phorest.com by Sorcha MullisSorcha Mullis Aug 09, 2022 Desktop Mobile Home & Landing X.X% Leads

Sorcha Tested Pattern #9: Multiple Steps On Phorest.com

 - Variant A
 - Variant B

In this experiment, a single step popup modal was tested against a 4 step sign up funnel. Users entered the experiment on the homepage and the behavior of the two buttons, book a demo and get a quote, was adjusted. Impact on leads was measured.

Total Sample: 88,974 Statistical Power at 5% MDE: 7.5%

Test #425 on by Jakub LinowskiJakub Linowski Aug 03, 2022 Desktop Product X.X% Sales

Jakub Tested Pattern #41: Sticky Call To Action

 - Variant A
 - Variant B

In this experiment, the complete buy box on a product detail page, floated along as users scrolled through the long screen. The variation made sure the product choice and order now button was always visible. Impact on adds-to-cart and sales was measured.

Total Sample: 8,174 Statistical Power at 5% MDE: 13.8%

Test #414 on Volders.de by Frederik FröhleFrederik Fröhle May 31, 2022 Desktop Mobile Checkout X.X% Sales

Frederik Tested Pattern #98: Auto Suggest On Volders.de

 - Variant A
 - Variant B

Does adding Google's address auto complete functionality to an address field help with higher form completions? This auto fill feature has been tested in the variation of a contract cancellation funnel. After selecting an auto completed address from a pulldown menu, the following fields were preselected: house number, zip code, city and country (potentially lowering friction?). Impact on successful form completions (contract cancellations) has been measured. Notice how the form also expanded progressively upon selecting the complete address in the variation.  

Total Sample: 128,199 Statistical Power at 5% MDE: 97.4%

Test #405 on Learnwithhomer.com by Stanley ZuoStanley Zuo Apr 07, 2022 Mobile Signup X.X% Sales

Stanley Tested Pattern #119: Unselected Or Selected Defaults On Learnwithhomer.com

 - Variant A
 - Variant B

In this experiment, the annual plan was preselected instead of starting off unselected in the control. Impact on two metrics was measured: any plan and annual plan purchases.

Total Sample: 14,903 Statistical Power at 5% MDE: 26.2%

Test #390 on Snocks.com by Melina HessMelina Hess Dec 21, 2021 Desktop Mobile Checkout X.X% Sales

Melina Tested Pattern #106: Back Buttons On Snocks.com

 - Variant A
 - Variant B

In this experiment, the variation has a "Back To Shopping Cart" link right underneath the checkout button. Impact on sales was measured.

Total Sample: 22,852 Statistical Power at 5% MDE: 97.5%

Test #388 on by Jakub LinowskiJakub Linowski Dec 09, 2021 Mobile Listing X.X% Sales

Jakub Tested Pattern #51: Shortcut Buttons

 - Variant A
 - Variant B

In this experiment, extra "buy now" buttons were added on a listing page. These buttons were shortcuts to an add to cart flow, whereas the "learn more" buttons lead customers to product detail pages (visible in both control and variation). Impact on adds-to-cart and sales was measured.

Total Sample: 7,051 Statistical Power at 5% MDE: 10.1%

Test #378 on by Jakub LinowskiJakub Linowski Oct 07, 2021 Desktop Mobile Product X.X% Sales

Jakub Tested Pattern #119: Unselected Or Selected Defaults

 - Variant A
 - Variant B

Here we have an experiment with a variation that preselected an option for a club duration. The control required customers to first express the choice for how many months they would like to order a product for. Whereas the variation defaulted to 12 months from the beginning.

Impact on adds-to-cart and sales was measured. The experiment unfortunately had to be stopped early due to another embedded variation that was performing poorly. And so it does not have many transactions.

Total Sample: 2,953 Statistical Power at 5% MDE: 5.2%

Test #374 on Expertinstitute.com by Ardit VeliuArdit Veliu Sep 15, 2021 Desktop Mobile Home & Landing X.X% Leads

Ardit Tested Pattern #9: Multiple Steps On Expertinstitute.com

 - Variant A
 - Variant B

In this experiment, a lead form with numerous fields shown all at once, was broken down into 3 steps. The first step only asked for a Specialty choice that expanded into a list of options. Impact on full form completions was measured.

Total Sample: 5,628 Statistical Power at 5% MDE: 7.5%