All Latest 633 A/B Tests

MOST RECENT TESTS

Test #465 on by Melina HessMelina Hess Apr 22, 2023 Desktop Mobile Product X.X% Revenue

Melina Tested Pattern #15: Bulleted Reassurances

 - Variant A
 - Variant B

In this product detail page experiment, a number of reassurances were brought out visually in a lined or bulleted way. The 4 reassurances included: free shipping and returns; Made in Germany, 90 Day Returns; and Buy With Invoice (popular in Germany). Impact on revenue per user was measured. The control contained very feint copy (smaller and more subtle) about free shipping.

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #462 on by Jakub LinowskiJakub Linowski Mar 24, 2023 Desktop Mobile Product X.X% Sales

Jakub Tested Pattern #128: Standard Or Superscript Price Format

 - Variant A
 - Variant B

In this experiment, standard $ signs and cents were formatted into a smaller superscript. Impact on add-to-cart and sales was measured.

Total Sample: 46,737 Statistical Power at 5% MDE: 35%

Test #461 on Snocks.com by Melina HessMelina Hess Mar 23, 2023 Mobile Product X.X% Sales

Melina Tested Pattern #15: Bulleted Reassurances On Snocks.com

 - Variant A
 - Variant B

In this product detail page experiment, reassurances under the add-to-cart button were rearranged. The control contained copy about: free shipping and free returns formatted as two gray boxes, with a variety of payment methods and their corresponding logos underneath.

The variation used a more convention bulleted, line-by-line format. It also contained free shipping and returns, but also elaborated with "100 day returns", an "anti-hole guarantee", and "purchase with invoice" (perhaps more popular in Germany?). Impact on sales was measured.

Total Sample: 522,417 Statistical Power at 5% MDE: 99.9%

Test #451 on Fluke.com by Marika FranciscoMarika Francisco Jan 25, 2023 Desktop Product X.X% Sales

Marika Tested Pattern #115: Pricing Comparison Table On Fluke.com

 - Variant A
 - Variant B

In this experiment, a product comparison table was added in the middle of a product detail page. The comparison table contained products from the same class or family of products. Clicking on the photo thumbnails also allowed customers to visit the specific detail page. Impact on adds to cart and transactions was measured.

Total Sample: 52,560 Statistical Power at 5% MDE: 4.8%

Test #449 on Snocks.com by Melina HessMelina Hess Dec 31, 2022 Desktop Mobile Product X.X% Sales

Melina Tested Pattern #78: Tags, Badges And Structured Information On Snocks.com

 - Variant A
 - Variant B

In this experiment, bestselling product colors were highlighted with a bestseller badge on product listing and product detail pages.

Total Sample: 508,908 Statistical Power at 5% MDE: 99.9%

Test #448 on Zapimoveis.com.br by Vinicius Barros PeixotoVinicius Barros Peixoto Dec 30, 2022 Desktop Mobile Product X.X% Leads

Vinicius Tested Pattern #21: What It's Worth On Zapimoveis.com.br

 - Variant A
 - Variant B

In this experiment, the B variation showed the relative discount in price from within the most recent 12 months as a percentage point. A tooltip was also shown which explained this message on hover. 

Total Sample: 136,584 Statistical Power at 5% MDE: 97.6%

Test #446 on by Jakub LinowskiJakub Linowski Dec 15, 2022 Desktop Mobile Product X.X% Sales

Jakub Tested Pattern #18: Single Or Alternative Buttons

 - Variant A
 - Variant B

In this experiment, an additional purchase trigger was added - the ability to start by chosing a gift announcement option. In the control, this same question was asked further in the purchase funnel (after adding to cart). In the variation, this question was surfaced earlier as an alternative way of starting the purchase. Impact on total sales was measured.

Total Sample: 100,016 Statistical Power at 5% MDE: 57.5%

Test #444 on by Melina HessMelina Hess Nov 30, 2022 Mobile Product X.X% Sales

Melina Tested Pattern #41: Sticky Call To Action

 - Variant A
 - Variant B

In this experiment, a floating add-to-cart with two product choices (color and size) were added on a product page. This appeared only after scrolling past the existing / embedded product selection buy box.

The floating add-to-cart widget had another layer of complexity in that it allowed customers to make a size and color selection with an expandable fly out. Making a selection would also append the total price to the add-to-cart button label.

Impact on total transactions was measured.

Total Sample: 1,483,721 Statistical Power at 5% MDE: 99.9%

Test #441 on by Melina HessMelina Hess Nov 23, 2022 Desktop Mobile Product X.X% Sales

Melina Tested Pattern #41: Sticky Call To Action

 - Variant A
 - Variant B

In this experiment, a floating sticky bar was added on product pages. The sticky elements only appeared after users scrolled beyond the fixed buy box  area that is relatively high on the page (visible on the control screenshot). The sticky bar contained three elements: a button to configure up to three product choices, a flavor selection pulldown, and the add-to-cart button. 

In the variation, when users clicked on the flavor pulldown three things happened. First, they were anchored back to the top of the buy box. Second, the floating sticky disappeared. And third, the flavors pulldown automatically expanded (overlapping the original primary add-to-cart button).

The control did not have any of the sticky behaviors.

Impact to total sales was measured.

Total Sample: 67,618 Statistical Power at 5% MDE: 70.8%

Test #430 on by Jakub LinowskiJakub Linowski Sep 20, 2022 Desktop Product X.X% Sales

Jakub Tested Pattern #26: Cart Reminder And Recently Viewed

 - Variant A
 - Variant B

In this experiment, up to 5 recently viewed product pages would appear on the left hand navigation. The idea was to make it easier for users get back to what they were looking at in case they were browsing. These recently viewed products were not visibile in the control. Impact on adds-to-cart and completed transactions was measured.

Total Sample: 10,324 Statistical Power at 5% MDE: 28%

Test #429 on Snocks.com by Melina HessMelina Hess Aug 16, 2022 Mobile Desktop Product X.X% Sales

Melina Tested Pattern #121: Free Shipping On Snocks.com

 - Variant A
 - Variant B

Upon clicking the Add-To-Cart button, the button label changed to a cheering message which congratulated the user on his choice and ensuring that shipping is free. Translation: "Good choice. Your shipping and returns are free."

Total Sample: 56,067 Statistical Power at 5% MDE: 99.9%

Test #425 on by Jakub LinowskiJakub Linowski Aug 03, 2022 Desktop Product X.X% Sales

Jakub Tested Pattern #41: Sticky Call To Action

 - Variant A
 - Variant B

In this experiment, the complete buy box on a product detail page, floated along as users scrolled through the long screen. The variation made sure the product choice and order now button was always visible. Impact on adds-to-cart and sales was measured.

Total Sample: 8,174 Statistical Power at 5% MDE: 13.8%

Test #421 on Amazon.com by Marika FranciscoMarika Francisco Jul 15, 2022 Desktop Mobile Product X.X% Sales

Marika Tested Pattern #43: Long Titles On Amazon.com

 - Variant A
 - Variant B

Fluke is an industrial manufacterer of measurement and calibration devices and they ran a test on their Amazon store for a series of their prodcts. Instead of using short product names, they tested longer and more descriptive ones. Impact was measured on sales.

Total Sample: 3,727,581 Statistical Power at 5% MDE: 92.5%

Test #420 on Designlab.com by Daniel ShapiroDaniel Shapiro Jul 12, 2022 Desktop Mobile Product X.X% Leads

Daniel Tested Pattern #115: Pricing Comparison Table On Designlab.com

 - Variant A
 - Variant B

In this experiment, pricing plans were laid out horizontally for easier comparison. In the variation, most of the plan benefits, features and differences were also referenced using a single lable that was left-aligned. The idea was to make the variables aligned and therefore more comparable.

This pricing table appeared at the bottom of a long design program landing page. Impact on leads and applications was measured.

Total Sample: 9,521 Statistical Power at 5% MDE: 14.3%

Test #416 on Designlab.com by Daniel ShapiroDaniel Shapiro Jun 20, 2022 Desktop Mobile Product X.X% Sales

Daniel Tested Pattern #105: Lead Magnets On Designlab.com

 - Variant A
 - Variant B

In this experiment, copy was added which communicated that students signing up for a course would receive extra design resources (the lead magnet). This was added in multiple states of the course page. Impact on lead generation and enrollment was measured.

Total Sample: 6,603 Statistical Power at 5% MDE: 7.4%

Test #411 on by Ayat ShukairyAyat Shukairy May 09, 2022 Desktop Mobile Product X.X% Sales

Ayat Tested Pattern #126: Bottom Or Left Thumbnails

 - Variant A
 - Variant B

Bottom aligned thumbnails were shifted to the left side on a product image. Doing so, also shifted the product descriptions a little higher. Impact on adds-to-cart and total transactions was measured.

Total Sample: 588,443 Statistical Power at 5% MDE: 70.1%

Test #408 on by Jakub LinowskiJakub Linowski Apr 29, 2022 Desktop Mobile Product X.X% Sales

Jakub Tested Pattern #67: Currency & Taxes

 - Variant A
 - Variant B

Here is a very simple experiment where CDN currency copy was appended to prices being shown on a Canadian ecommerce site. Additional copy ("from Canada") was appended to an existing shipping message.  

Total Sample: 16,332 Statistical Power at 5% MDE: 17.9%

Test #407 on Snocks.com by Melina HessMelina Hess Apr 22, 2022 Desktop Mobile Product X.X% Sales

Melina Tested Pattern #125: Unit Prices On Snocks.com

 - Variant A
 - Variant B

Here is an experiment with an added "price per pair" in the context of products with multiple items (packs).A high contrast badge-like copy was appended underneath the price which translates to "only $X per item". Impact on sales was measured.

Total Sample: 112,178 Statistical Power at 5% MDE: 56%

Test #406 on Chaos.com by Velin PenevVelin Penev Apr 12, 2022 Desktop Mobile Product X.X% Sales

Velin Tested Pattern #112: Lower Price Frames On Chaos.com

 - Variant A
 - Variant B

In this experiment, the pricing of three software plans was consistently framed into a more comparable monthly context. Whereas the control version only showed the total prices for each plan, the variation showed both the total and monthly prices. Impact on sales was measured. 

Total Sample: 69,080 Statistical Power at 5% MDE: 8.5%

Test #402 on Snocks.com by Melina HessMelina Hess Mar 25, 2022 Mobile Product X.X% Sales

Melina Tested Pattern #103: Money Back Guarantee On Snocks.com

 - Variant A
 - Variant B

In this experiment on a product screen, a message about quality (from the founders) was reformatted to reinforce the "Anti Hole Guarantee" (in the context of socks). The founders were also made more prominent and their signature added as well. Interestingly, the actual guarantee copy (and its important detail about a 6 month product replacement) was present further down on both the control and variation.

The translation of the new copy (Google Translate) reads:

Since we founded SNOCKS, the quality and comfort of our products have always been our top priority. Why? When we both were still studying, we didn't have the money to buy new socks & underpants every 1-2 months. That's why it was clear to us from the start that we wanted to sell quality products that are also extremely comfortable. And just in case something goes wrong, we have decided to introduce an anti-hole guarantee.

Total Sample: 456,229 Statistical Power at 5% MDE: 99.1%