All Latest 633 A/B Tests

MOST RECENT TESTS

Test #419 on by Jakub LinowskiJakub Linowski Jun 29, 2022 Desktop Home & Landing X.X% Sales

Jakub Tested Pattern #68: Welcome Discount

 - Variant A
 - Variant B

In this experiment, the presence of a discount modal (bigger discount for larger purchases) was tested on home and product pages. The trigger happend after a scroll down, a few second pause, and a mouse scroll up interaction. Impact on sales was measured.

Total Sample: 196,639 Statistical Power at 5% MDE: 98.5%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #418 on Online.metro-cc.ru by Andrey AndreevAndrey Andreev Jun 28, 2022 Desktop Mobile Home & Landing X.X% Sales

Andrey Tested Pattern #135: Product Categories On Online.metro-cc.ru

 - Variant A
 - Variant B

In this experiment, popular product category links were added near the bottom of the homepage of an online grocery store - Metro. Impact on completed sales was measured.

Total Sample: 78,582 Statistical Power at 5% MDE: 12.6%

Test #417 on Cxl.com by Ognjen BoškovićOgnjen Bošković Jun 27, 2022 Desktop Mobile Signup X.X% Signups

Ognjen Tested Pattern #127: Vague Or Specific Benefits On Cxl.com

 - Variant A
 - Variant B

CXL ran an interesting experiment that (1) added more specificity on a newsletter subscription page as well as (2) reinforced it consistently. Most visibly, an abstract or vague headline statement (control) was changed to a benefit oriented one - hinting that subscribers will eventually receive valuable case studies. This message was further reinforced with supporting copy that explained where these case studies are obtained from along with the frequency of the delivery. This same message was also reinforced with an image of actual case studies. The call to action button was also adjusted to mimic the benefit.

Impact on newsletter signups was measured.

Total Sample: 8,974 Statistical Power at 5% MDE: 28.4%

Test #416 on Designlab.com by Daniel ShapiroDaniel Shapiro Jun 20, 2022 Desktop Mobile Product X.X% Sales

Daniel Tested Pattern #105: Lead Magnets On Designlab.com

 - Variant A
 - Variant B

In this experiment, copy was added which communicated that students signing up for a course would receive extra design resources (the lead magnet). This was added in multiple states of the course page. Impact on lead generation and enrollment was measured.

Total Sample: 6,603 Statistical Power at 5% MDE: 7.4%

Test #414 on Volders.de by Frederik FröhleFrederik Fröhle May 31, 2022 Desktop Mobile Checkout X.X% Sales

Frederik Tested Pattern #98: Auto Suggest On Volders.de

 - Variant A
 - Variant B

Does adding Google's address auto complete functionality to an address field help with higher form completions? This auto fill feature has been tested in the variation of a contract cancellation funnel. After selecting an auto completed address from a pulldown menu, the following fields were preselected: house number, zip code, city and country (potentially lowering friction?). Impact on successful form completions (contract cancellations) has been measured. Notice how the form also expanded progressively upon selecting the complete address in the variation.  

Total Sample: 128,199 Statistical Power at 5% MDE: 97.4%

Test #413 on by Jakub LinowskiJakub Linowski May 26, 2022 Desktop Mobile Shopping Cart X.X% Sales

Jakub Tested Pattern #114: Less Or More Visible Prices

 - Variant A
 - Variant B

Should prices be displayed during an editing state on an add-to-cart screen (after adding-to-cart from a product detail page)? In this experiment, club pricing was added and shown in three areas after customers clicked on a small "edit details" links. The control did not have this pricing information (which was only shown on a product detail screen before).

Total Sample: 5,698 Statistical Power at 5% MDE: 59.9%

Test #412 on Volders.de by Frederik FröhleFrederik Fröhle May 16, 2022 Desktop Mobile Checkout X.X% Sales

Frederik Tested Pattern #15: Bulleted Reassurances On Volders.de

 - Variant A
 - Variant B

The Volders experimentation team assumed that adding information about how long it might take to get a cancellation confirmation near a CTA Button would result in higher cancellation requests (the paid service being offered).

The variation contained additional copy translated from German to: "Most of our users receive their cancellation confirmation from <vendor> within 14 days by email or letter."

Total Sample: 53,120 Statistical Power at 5% MDE: 99.3%

Test #411 on by Ayat ShukairyAyat Shukairy May 09, 2022 Desktop Mobile Product X.X% Sales

Ayat Tested Pattern #126: Bottom Or Left Thumbnails

 - Variant A
 - Variant B

Bottom aligned thumbnails were shifted to the left side on a product image. Doing so, also shifted the product descriptions a little higher. Impact on adds-to-cart and total transactions was measured.

Total Sample: 588,443 Statistical Power at 5% MDE: 70.1%

Test #410 on Designlab.com by Daniel ShapiroDaniel Shapiro May 05, 2022 Desktop Mobile Home & Landing X.X% Sales

Daniel Tested Pattern #29: Surfaced Content On Designlab.com

 - Variant A
 - Variant B

Does linking to a product detail page help? Or should a homepage simply focus on generic lead generation? In this homepage experiment, the presence of a component with a link to a detailed course landing page was tested. Impact on course enrollments was measured.

Total Sample: 25,443 Statistical Power at 5% MDE: 6.3%

Test #409 on Expertinstitute.com by Ardit VeliuArdit Veliu Apr 30, 2022 Desktop Mobile Signup X.X% Leads

Ardit Tested Pattern #20: Canned Response On Expertinstitute.com

 - Variant A
 - Variant B

In this experiment, the copy of an input textarea on a lead form was used to summarize a user's choices. Instead of showing using a generic "Message" statement, the variation use the following formula: "I'm a [Lead Type] located in [State] looking for [Expert Type]. Looking forward to hearing from you today. Thank You." Impact of leads was measured.

Total Sample: 2,831 Statistical Power at 5% MDE: 6.9%

Test #408 on by Jakub LinowskiJakub Linowski Apr 29, 2022 Desktop Mobile Product X.X% Sales

Jakub Tested Pattern #67: Currency & Taxes

 - Variant A
 - Variant B

Here is a very simple experiment where CDN currency copy was appended to prices being shown on a Canadian ecommerce site. Additional copy ("from Canada") was appended to an existing shipping message.  

Total Sample: 16,332 Statistical Power at 5% MDE: 17.9%

Test #407 on Snocks.com by Melina HessMelina Hess Apr 22, 2022 Desktop Mobile Product X.X% Sales

Melina Tested Pattern #125: Unit Prices On Snocks.com

 - Variant A
 - Variant B

Here is an experiment with an added "price per pair" in the context of products with multiple items (packs).A high contrast badge-like copy was appended underneath the price which translates to "only $X per item". Impact on sales was measured.

Total Sample: 112,178 Statistical Power at 5% MDE: 56%

Test #406 on Chaos.com by Velin PenevVelin Penev Apr 12, 2022 Desktop Mobile Product X.X% Sales

Velin Tested Pattern #112: Lower Price Frames On Chaos.com

 - Variant A
 - Variant B

In this experiment, the pricing of three software plans was consistently framed into a more comparable monthly context. Whereas the control version only showed the total prices for each plan, the variation showed both the total and monthly prices. Impact on sales was measured. 

Total Sample: 69,080 Statistical Power at 5% MDE: 8.5%

Test #404 on Expertinstitute.com by Ardit VeliuArdit Veliu Mar 31, 2022 Desktop Mobile Signup X.X% Leads

Ardit Tested Pattern #97: Bigger Form Fields On Expertinstitute.com

 - Variant A
 - Variant B

In this experiment, the button width was extended to the full width of the form above. Impact on leads was measured.

Total Sample: 6,408 Statistical Power at 5% MDE: 8.6%

Test #403 on by Jakub LinowskiJakub Linowski Mar 29, 2022 Desktop Mobile Checkout X.X% Sales

Jakub Tested Pattern #42: Countdown Timer

 - Variant A
 - Variant B

In this experiment, an urgency related message with a dynamic countdown timer was added on the final checkout screen. Impact on sales was measured.

Total Sample: 7,466 Statistical Power at 5% MDE: 99.9%

Test #401 on Learnwithhomer.com by Stanley ZuoStanley Zuo Mar 11, 2022 Desktop Home & Landing X.X% Signups

Stanley Tested Pattern #58: Full Height False Bottom On Learnwithhomer.com

 - Variant A
 - Variant B

Here the experiment contained a layout change. Standard content components with varied heights were tested against sections with false bottoms. That is the conntent blocks in the variation took up 100% height of the viewport. This change was also applied throughout the rest of the content blocks. Impact on overall signup rates was measured.

Total Sample: 5,602 Statistical Power at 5% MDE: 5.8%

Test #400 on by Herman KleinHerman Klein Mar 06, 2022 Desktop Shopping Cart X.X% Sales

Herman Tested Pattern #65: Add More For Extra Incentive

 - Variant A
 - Variant B

In this experiment, customers who were not yet eligible for free shipping (with cart amounts under $39) were presented with an encouraging message to add more and earn free shipping. Impact on progression to checkout and sales was measured.

Total Sample: 3,111 Statistical Power at 5% MDE: 60.3%

Test #399 on by Jakub LinowskiJakub Linowski Feb 27, 2022 Desktop Mobile Checkout X.X% Progression

Jakub Tested Pattern #35: Floating Labels

 - Variant A
 - Variant B

In this experiment, top-aligned field labels were tested against floating labels (with labels floating inside the form field itself).

Total Sample: 2,073 Statistical Power at 5% MDE: 29.2%

Test #398 on Adoramapix.com by Herman KleinHerman Klein Feb 25, 2022 Desktop Shopping Cart X.X% Sales

Herman Tested Pattern #121: Free Shipping On Adoramapix.com

 - Variant A
 - Variant B

In this experiment, customers were eligible for free shipping (with cart amounts of $39 or more) were presented with an earned free shipping message. The variation also showed the value of the earned free shipping - (for example $10). Impact on progression to checkout and sales was measured.

Total Sample: 3,123 Statistical Power at 5% MDE: 60.5%

Test #396 on Depositphotos.com by Gleb HodorovskiyGleb Hodorovskiy Feb 13, 2022 Desktop Listing X.X% Revenue

Gleb Tested Pattern #124: Confirmed Selection On Depositphotos.com

 - Variant A
 - Variant B

In this experiment from Conversionrate.store, the framing of the registration message was changed from a generic account creation one to a specific image selected by the user. The experiment started on a listing page of a stock photography / illustration site. The control showed a more generic message with benefits for signing up and making the purchase. Whereas the variation repeated the actual image that customers clicked on from listing pages - establishing continuity as well as providing a reason for signing up. Impact on sales was measured.