All Latest 633 A/B Tests

MOST RECENT TESTS

Test #453 on by Jakub LinowskiJakub Linowski Jan 31, 2023 Desktop Mobile Shopping Cart X.X% Sales

Jakub Tested Pattern #64: Tunnel

 - Variant A
 - Variant B

In this experiment, the footer was simplified with multiple elements being removed (catalog request, secondary links, and a guarantee). Additionally, a more prominent phone number was also displayed.

Total Sample: 17,107 Statistical Power at 5% MDE: 91.8%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #450 on Trenyrkarna.cz by Ondřej IlinčevOndřej Ilinčev Jan 20, 2023 Desktop Shopping Cart X.X% Sales

Ondřej Tested Pattern #64: Tunnel On Trenyrkarna.cz

 - Variant A
 - Variant B

In this experiment, top category navigation (with fly out categories) was removed from the shopping cart of an online store. Impact on sale transactions was measured.

Total Sample: 4,737 Statistical Power at 5% MDE: 25.4%

Test #444 on by Melina HessMelina Hess Nov 30, 2022 Mobile Product X.X% Sales

Melina Tested Pattern #41: Sticky Call To Action

 - Variant A
 - Variant B

In this experiment, a floating add-to-cart with two product choices (color and size) were added on a product page. This appeared only after scrolling past the existing / embedded product selection buy box.

The floating add-to-cart widget had another layer of complexity in that it allowed customers to make a size and color selection with an expandable fly out. Making a selection would also append the total price to the add-to-cart button label.

Impact on total transactions was measured.

Total Sample: 1,483,721 Statistical Power at 5% MDE: 99.9%

Test #441 on by Melina HessMelina Hess Nov 23, 2022 Desktop Mobile Product X.X% Sales

Melina Tested Pattern #41: Sticky Call To Action

 - Variant A
 - Variant B

In this experiment, a floating sticky bar was added on product pages. The sticky elements only appeared after users scrolled beyond the fixed buy box  area that is relatively high on the page (visible on the control screenshot). The sticky bar contained three elements: a button to configure up to three product choices, a flavor selection pulldown, and the add-to-cart button. 

In the variation, when users clicked on the flavor pulldown three things happened. First, they were anchored back to the top of the buy box. Second, the floating sticky disappeared. And third, the flavors pulldown automatically expanded (overlapping the original primary add-to-cart button).

The control did not have any of the sticky behaviors.

Impact to total sales was measured.

Total Sample: 67,618 Statistical Power at 5% MDE: 70.8%

Test #435 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Oct 17, 2022 Desktop Mobile Checkout X.X% Sales

Daria Tested Pattern #9: Multiple Steps On Volders.de

 - Variant A
 - Variant B

In this experiment, a question about a customer's reason for purchase was broken out into a separate step and moved earlier in the process. In the control, this question was asked in the final checkout step along with a plan selection (Step 4 of 4). In the variation, this question was shifted as a standalone first step (Step 1 of 5). Impact on completed purchases was measured (for a contract cancellation service in this case).

Total Sample: 139,608 Statistical Power at 5% MDE: 98.2%

Test #432 on Snocks.com by Melina HessMelina Hess Sep 29, 2022 Mobile Global X.X% Sales

Melina Tested Pattern #94: Visible Search On Snocks.com

 - Variant A
 - Variant B

In this experiment, search functionality was added in the variation, globally on mobile. Impact on sales was measured

Total Sample: 1,175,864 Statistical Power at 5% MDE: 99.9%

Test #430 on by Jakub LinowskiJakub Linowski Sep 20, 2022 Desktop Product X.X% Sales

Jakub Tested Pattern #26: Cart Reminder And Recently Viewed

 - Variant A
 - Variant B

In this experiment, up to 5 recently viewed product pages would appear on the left hand navigation. The idea was to make it easier for users get back to what they were looking at in case they were browsing. These recently viewed products were not visibile in the control. Impact on adds-to-cart and completed transactions was measured.

Total Sample: 10,324 Statistical Power at 5% MDE: 28%

Test #428 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Aug 16, 2022 Desktop Mobile Signup X.X% Sales

Daria Tested Pattern #26: Cart Reminder And Recently Viewed On Volders.de

 - Variant A
 - Variant B

In this test 1) a passive hint communicated to users that their data will be saved for 7 days for them to be able to continue their cancellation later and 2) dropped off users were targeted with an email campaign within the first 4 hours after drop off. The reminder email linked users to a shipping page (checkout page) without them requiring to fill out their personal, contract information one more time. 

Total Sample: 155,342 Statistical Power at 5% MDE: 98.8%

Test #426 on Phorest.com by Sorcha MullisSorcha Mullis Aug 09, 2022 Desktop Mobile Home & Landing X.X% Leads

Sorcha Tested Pattern #9: Multiple Steps On Phorest.com

 - Variant A
 - Variant B

In this experiment, a single step popup modal was tested against a 4 step sign up funnel. Users entered the experiment on the homepage and the behavior of the two buttons, book a demo and get a quote, was adjusted. Impact on leads was measured.

Total Sample: 88,974 Statistical Power at 5% MDE: 7.5%

Test #425 on by Jakub LinowskiJakub Linowski Aug 03, 2022 Desktop Product X.X% Sales

Jakub Tested Pattern #41: Sticky Call To Action

 - Variant A
 - Variant B

In this experiment, the complete buy box on a product detail page, floated along as users scrolled through the long screen. The variation made sure the product choice and order now button was always visible. Impact on adds-to-cart and sales was measured.

Total Sample: 8,174 Statistical Power at 5% MDE: 13.8%

Test #414 on Volders.de by Frederik FröhleFrederik Fröhle May 31, 2022 Desktop Mobile Checkout X.X% Sales

Frederik Tested Pattern #98: Auto Suggest On Volders.de

 - Variant A
 - Variant B

Does adding Google's address auto complete functionality to an address field help with higher form completions? This auto fill feature has been tested in the variation of a contract cancellation funnel. After selecting an auto completed address from a pulldown menu, the following fields were preselected: house number, zip code, city and country (potentially lowering friction?). Impact on successful form completions (contract cancellations) has been measured. Notice how the form also expanded progressively upon selecting the complete address in the variation.  

Total Sample: 128,199 Statistical Power at 5% MDE: 97.4%

Test #405 on Learnwithhomer.com by Stanley ZuoStanley Zuo Apr 07, 2022 Mobile Signup X.X% Sales

Stanley Tested Pattern #119: Unselected Or Selected Defaults On Learnwithhomer.com

 - Variant A
 - Variant B

In this experiment, the annual plan was preselected instead of starting off unselected in the control. Impact on two metrics was measured: any plan and annual plan purchases.

Total Sample: 14,903 Statistical Power at 5% MDE: 26.2%

Test #390 on Snocks.com by Melina HessMelina Hess Dec 21, 2021 Desktop Mobile Checkout X.X% Sales

Melina Tested Pattern #106: Back Buttons On Snocks.com

 - Variant A
 - Variant B

In this experiment, the variation has a "Back To Shopping Cart" link right underneath the checkout button. Impact on sales was measured.

Total Sample: 22,852 Statistical Power at 5% MDE: 97.5%

Test #388 on by Jakub LinowskiJakub Linowski Dec 09, 2021 Mobile Listing X.X% Sales

Jakub Tested Pattern #51: Shortcut Buttons

 - Variant A
 - Variant B

In this experiment, extra "buy now" buttons were added on a listing page. These buttons were shortcuts to an add to cart flow, whereas the "learn more" buttons lead customers to product detail pages (visible in both control and variation). Impact on adds-to-cart and sales was measured.

Total Sample: 7,051 Statistical Power at 5% MDE: 10.1%

Test #378 on by Jakub LinowskiJakub Linowski Oct 07, 2021 Desktop Mobile Product X.X% Sales

Jakub Tested Pattern #119: Unselected Or Selected Defaults

 - Variant A
 - Variant B

Here we have an experiment with a variation that preselected an option for a club duration. The control required customers to first express the choice for how many months they would like to order a product for. Whereas the variation defaulted to 12 months from the beginning.

Impact on adds-to-cart and sales was measured. The experiment unfortunately had to be stopped early due to another embedded variation that was performing poorly. And so it does not have many transactions.

Total Sample: 2,953 Statistical Power at 5% MDE: 5.2%

Test #374 on Expertinstitute.com by Ardit VeliuArdit Veliu Sep 15, 2021 Desktop Mobile Home & Landing X.X% Leads

Ardit Tested Pattern #9: Multiple Steps On Expertinstitute.com

 - Variant A
 - Variant B

In this experiment, a lead form with numerous fields shown all at once, was broken down into 3 steps. The first step only asked for a Specialty choice that expanded into a list of options. Impact on full form completions was measured.

Total Sample: 5,628 Statistical Power at 5% MDE: 7.5%

Test #366 on Mvideo.ru by Andrey AndreevAndrey Andreev Jul 14, 2021 Desktop Mobile Product X.X% Sales

Andrey Tested Pattern #93: Auto Next On Mvideo.ru

 - Variant A
 - Variant B

In this experiment, the interaction on a product page was changed. In the control (A), after clicking "add to cart", the customer would remain on the product page with an updated basket and the button changing to "Go To Cart". The variation (B) however took customers forward automatically to the cart.

Total Sample: 80,881 Statistical Power at 5% MDE: 95%

Test #351 on Baremetrics.com by Brian SierakowskiBrian Sierakowski Apr 30, 2021 Desktop Mobile Home & Landing X.X% Signups

Brian Tested Pattern #11: Gradual Reassurance On Baremetrics.com

 - Variant A
 - Variant B

In this experiment, static integration logos were replaced with selectable ones that reassured users to signup. After clicking an integration logo, a comparison chart would appear showing how Baremetrics improves upon a selected payment processor, along with a call to signup. Impact on signups was measured.

Total Sample: 24,416 Statistical Power at 5% MDE: 7%

Test #333 on Expertinstitute.com by Ardit VeliuArdit Veliu Dec 31, 2020 Desktop Mobile Home & Landing X.X% Leads

Ardit Tested Pattern #11: Gradual Reassurance On Expertinstitute.com

 - Variant A
 - Variant B

In this experiment, the variation broke up a lead form into two parts. In the first step users were asked for their state followed by a standard contact form on a second step. All of the states were shown as selectable options. In the control version, the landing page only showed a button which lead to the full form. The experiment measured impact on lead form submissions.

Total Sample: 11,258 Statistical Power at 5% MDE: 6.1%

Test #313 on Trydesignlab.com by Daniel ShapiroDaniel Shapiro Aug 19, 2020 Desktop Mobile Home & Landing X.X% Leads

Daniel Tested Pattern #11: Gradual Reassurance On Trydesignlab.com

 - Variant A
 - Variant B

In this experiment, instead of showing a single-focused lead form (for the UX Academy Program), users were asked to express a wider set of choices first (for the UX Academy or shortter set of skill-based courses). The experiment measured overall leads for both types of programs.

Total Sample: 13,231 Statistical Power at 5% MDE: 15.7%