All Latest 631 A/B Tests

Test #577 on by Jakub LinowskiJakub Linowski Feb 19, 2025 Desktop Product X.X% Progression

Jakub Tested Pattern #48: Video Testimonials

 - Variant A
 - Variant B

In this experiment, a video unboxing and product overview video was added on product pages. After pressing play, the video expanded to a full column width (taking over the middle column while replacing the 5 small square photo tiles and growing in height.) The video in its play state also contained a prominent (X) icon that allowed users to stop and revert to the original state.

Impact on adds to cart and sales was measured.

Total Sample: 18,143 Statistical Power at 5% MDE: 27%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #575 on Finn.com by Tim KarcherTim Karcher Feb 12, 2025 Desktop Listing X.X% Progression

Tim Tested Pattern #34: Open In A New Tab On Finn.com

 - Variant A
 - Variant B

In this experiment, product listing were either opened in the same window (control) or opened in a new tab and focused on (variation). Impact on signups and sales was measured.

Total Sample: 37,497 Statistical Power at 5% MDE: 9.1%

Test #576 on Finn.com by Tim KarcherTim Karcher Feb 12, 2025 Mobile Listing X.X% Progression

Tim Tested Pattern #34: Open In A New Tab On Finn.com

 - Variant A
 - Variant B
Total Sample: 66,947 Statistical Power at 5% MDE: 12.5%

Test #574 on Myer.com.au by Jay KimJay Kim Jan 30, 2025 Mobile Product X.X% Progression

Jay Tested Pattern #41: Sticky Call To Action On Myer.com.au

 - Variant A
 - Variant B

In this experiment published by Jay Kim, a sticky add to cart button was added on mobile product detail pages. It appeared after the scroll depth past the original add to cart button. Impact to adds to cart and completed sales was measured.

Total Sample: 524,153 Statistical Power at 5% MDE: 93.9%

Test #573 on Online.metro-cc.ru by Andrey AndreevAndrey Andreev Jan 28, 2025 Mobile Shopping Cart X.X% Progression

Andrey Tested Pattern #41: Sticky Call To Action On Online.metro-cc.ru

 - Variant A
 - Variant B

In this experiment, a floating "Place order" button was added to a mobile shopping cart screen. Impact on progression (to checkout) and sales was measured.

Total Sample: 6,527 Statistical Power at 5% MDE: 41.6%

Test #572 on by Deborah O'MalleyDeborah O'Malley Jan 27, 2025 Mobile Product X.X% Progression

Deborah Tested Pattern #41: Sticky Call To Action

 - Variant A
 - Variant B

In this experiment from GuessTheTest.com, the control variation only showed the primary add-to-cart button further down on the page. The variation however turned the button into a floating one once users scrolled on the page. Impact on button clicks was measured.

Total Sample: 1,320 Statistical Power at 5% MDE: 21.5%

Test #571 on by Jakub LinowskiJakub Linowski Jan 03, 2025 Desktop Mobile Product X.X% Progression

Jakub Tested Pattern #30: Authentic Photos

 - Variant A
 - Variant B

In this experiment, additional customer product photos were shown at the top of the product page. A "Show More Photos" button was also added which launched a modal with additional and larger images. Impact on sales was measured.

Total Sample: 71,218 Statistical Power at 5% MDE: 51.1%

Test #568 on by Jakub LinowskiJakub Linowski Dec 22, 2024 X.X% Progression

Jakub Tested Pattern #80: Persistent Filters

 - Variant A
 - Variant B

In this experiment, "persistence" of 4 product choices was added in the variation. When users made a product selection such as: duration, payment terms, starting month or shipping frequency, their choices were remembered and defaulted on next visits, reloads or when viewing other products. Impact on adds to cart and sales was measured.

Total Sample: 81,116 Statistical Power at 5% MDE: 39.9%

Test #566 on Banter.com by Craig KistlerCraig Kistler Dec 11, 2024 Desktop Product X.X% Progression

Craig Tested Pattern #66: Complementary Upsell On Banter.com

 - Variant A
 - Variant B

In this experiment, a modal based message was shown to encourage extra products being added as complementary upsells. In the control, the promotion text appeared at the bottom as red text ("Buy one, get up to two 50% Off"). Whereas in the variation, specific products were shown on the modal (post add-to-cart). Impact on adds-to-cart, sales and average revenue was measured.

Sample size too small for 80% power

Test #565 on Umbraco.com by Lars Skjold IversenLars Skjold Iversen Nov 30, 2024 Desktop Home & Landing X.X% Progression

Lars Tested Pattern #129: Right Or Left Aligned Forms On Umbraco.com

 - Variant A
 - Variant B

In this experiment, the right vs left position of a form (at the bottom of a landing page) was a/b tested. Impact on progression and form completion was measured.

Total Sample: 5,654 Statistical Power at 5% MDE: 3.9%

Test #564 on Hellostake.com by Louis AlstonLouis Alston Nov 26, 2024 Desktop Mobile Home & Landing X.X% Progression

Louis Tested Pattern #114: Less Or More Visible Prices On Hellostake.com

 - Variant A
 - Variant B

In this a/b test, the headline was changed to reflect pricing information (informing that trades are starting at $3). Impact on progression and signups was measured.

Total Sample: 25,321 Statistical Power at 5% MDE: 6.5%

Test #563 on Expertinstitute.com by Ardit VeliuArdit Veliu Nov 19, 2024 Desktop Home & Landing X.X% Progression

Ardit Tested Pattern #108: Frequently Asked Questions On Expertinstitute.com

 - Variant A
 - Variant B

In this RETEST experiment, a Frequently Asked Questions section was added near the bottom of a short lead gen form. This test ran on one of Expert Institute's landing pages for their expert witness seeking services. Impact on leads was measured. It was also triggered by users who scrolled at least 100px downwards towards the FAQ section.

Total Sample: 6,383 Statistical Power at 5% MDE: 9.3%

Test #562 on by Jakub LinowskiJakub Linowski Nov 13, 2024 Desktop Mobile Checkout X.X% Progression

Jakub Tested Pattern #99: Progress Bar

 - Variant A
 - Variant B

In this experiment, a 3 step progress bar was added starting on a checkout funnel (dedicated add-to-cart page, greeting card step and order summary). The progress bar also showed any completed steps as a "checked off" state. More so, users were able to use the progress bar as a navigation item to any previously completed and currently active steps. Impact on sales was measured.

Total Sample: 2,313 Statistical Power at 5% MDE: 28.2%

Test #559 on Tourradar.com by Clemens GraveClemens Grave Oct 18, 2024 Desktop Listing X.X% Progression

Clemens Tested Pattern #137: Visible Filters On Tourradar.com

 - Variant A
 - Variant B

In this experiment, left column filters (control) were tested against top-aligned and sticky filters (variation). The hypothesis was to increase their visibility. Impact on their use and progression to next step (product/tour detail page) were measured.

Total Sample: 36,470 Statistical Power at 5% MDE: 99.5%

Test #556 on Snocks.com by Melina HessMelina Hess Oct 08, 2024 Mobile Product X.X% Progression

Melina Tested Pattern #65: Add More For Extra Incentive On Snocks.com

 - Variant A
 - Variant B

In this experiment, free shipping was a/b tested against free shipping with a 30€ purchase or higher. Hence, in the variation, customers needed to reach a cart amount total in order to be eligible for the free shipping. 

Total Sample: 692,708 Statistical Power at 5% MDE: 99.9%

Test #552 on Tourradar.com by Clemens GraveClemens Grave Sep 19, 2024 Product X.X% Progression

Clemens Tested Pattern #15: Bulleted Reassurances On Tourradar.com

 - Variant A
 - Variant B

In this experiment, existing reassurance copy was changed. The control contained copy that focused on splitting payments and paying over time. The variation challenged this and showed a best price guarantee and the ability to earn credits. 

Total Sample: 432,567 Statistical Power at 5% MDE: 44.7%

Test #551 on Tourradar.com by Clemens GraveClemens Grave Sep 04, 2024 Desktop Product X.X% Progression

Clemens Tested Pattern #139: Page Level Navigation On Tourradar.com

 - Variant A
 - Variant B

In this experiment, a floating page level navigation was added near the top of the product pages (under the main image). The navigation links included: Overview, Itinerary, What's Included, Operator, Reviews and Availability. Impact on next step progression was measured.

Total Sample: 70,793 Statistical Power at 5% MDE: 99.9%

Test #550 on Online.metro-cc.ru by Andrey AndreevAndrey Andreev Aug 14, 2024 Mobile Listing X.X% Progression

Andrey Tested Pattern #137: Visible Filters On Online.metro-cc.ru

 - Variant A
 - Variant B

Instead of only displaying sort and filters at the top of a listing page, the variation always had them visible with a sticky/floating interaction. Impact on sales was measured.

Total Sample: 312,289 Statistical Power at 5% MDE: 35.2%

Test #549 on Kay.com by Craig KistlerCraig Kistler Aug 13, 2024 Desktop Listing X.X% Progression

Craig Tested Pattern #138: Visible Payment Options On Kay.com

 - Variant A
 - Variant B

In this experiment, an inline panel was shown as a product tile. The panel informed customers about a variety of payment options (with detailed information being further presented after a button click). Impact on sales was measured.

Total Sample: 342,123 Statistical Power at 5% MDE: 22.7%

Test #545 on Banter.com by Craig KistlerCraig Kistler Jul 29, 2024 Desktop Product X.X% Progression

Craig Tested Pattern #66: Complementary Upsell On Banter.com

 - Variant A
 - Variant B

In this experiment, a modal based message was shown to encourage extra products being added as complementary upsells. In the control, the promotion text appeared at the bottom as red text ("Buy one, get up to two 50% Off"). Whereas in the variation, specific products were shown on the modal (post add-to-cart). Impact on adds-to-cart, sales and average revenue was measured.

Sample size too small for 80% power