All Latest 644 A/B Tests

MOST RECENT TESTS

Test #359 on Snocks.com by Samuel HessSamuel Hess Jun 11, 2021 Desktop Mobile Product X.X% Sales

Samuel Tested Pattern #43: Long Titles On Snocks.com

Pattern #43: Long Titles - Variant A
Pattern #43: Long Titles - Variant B

In this experiment, product titles were extended with descriptive copy. Instead of just showing the product name, "with organic cotton" was appended on product and category/listing pages. Impact to adds to cart and sales was measured.

Total Sample: 129,030 • Statistical Power at 5% MDE: 79.9%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #354 on Mvideo.ru by Andrey AndreevAndrey Andreev May 25, 2021 Desktop Mobile Product X.X% Revenue

Andrey Tested Pattern #69: Autodiscounting On Mvideo.ru

Pattern #69: Autodiscounting - Variant A
Pattern #69: Autodiscounting - Variant B

In this experiment, the discounted price was shown along with an active promotion. The control only showed that the relative -8% discount was present with the pre-discounted price.

Test #353 on Backstage.com by Stanley ZuoStanley Zuo May 12, 2021 Desktop Product X.X% Signups

Stanley Tested Pattern #13: Centered Forms & Buttons On Backstage.com

Pattern #13: Centered Forms & Buttons - Variant A
Pattern #13: Centered Forms & Buttons - Variant B

This experiment challanged a right aligned detail page. After clicking a result of a job role on a listing page, an overlay would appear on the right with the details. The variation used a full width screen instead - effectively centering the page.

Total Sample: 30,428 • Statistical Power at 5% MDE: 53.6%

Test #343 on Snocks.com by Samuel HessSamuel Hess Mar 12, 2021 Desktop Mobile Product X.X% Sales

Samuel Tested Pattern #122: Zigzag Layout On Snocks.com

Pattern #122: Zigzag Layout - Variant A
Pattern #122: Zigzag Layout - Variant B

In this experiment, the content on a product page was reorganized into a zigzagging (alternating layout) along with reinforcing photos. Impact on adds-to-cart and total sales was measured.

Total Sample: 23,865 • Statistical Power at 5% MDE: 32.9%

Test #86 on Vivareal.com.br by Rodrigo MauésRodrigo Maués Feb 28, 2021 Mobile Desktop Product X.X% Leads

Rodrigo Tested Pattern #3: Fewer Form Fields On Vivareal.com.br

Pattern #3: Fewer Form Fields - Variant A
Pattern #3: Fewer Form Fields - Variant B

In this experiment, the telephone field was removed from a lead form on a property page. Impact on leads was measured.

Total Sample: 101,914 • Statistical Power at 5% MDE: 43.5%

Test #331 by Jakub LinowskiJakub Linowski Dec 30, 2020 Desktop Mobile Product X.X% Sales

Jakub Tested Pattern #121: Free Shipping

Pattern #121: Free Shipping - Variant A
Pattern #121: Free Shipping - Variant B

In this little experiment, an extra "Free Shipping" message was added on a product page. It's prominence was increased by using white copy on a darker blue background. Impact on adds-to-cart and total sales was measured.

Total Sample: 96,962 • Statistical Power at 5% MDE: 83.6%

Test #332 by Jakub LinowskiJakub Linowski Dec 30, 2020 Desktop Mobile Product X.X% Sales

Jakub Tested Pattern #121: Free Shipping

Pattern #121: Free Shipping - Variant A
Pattern #121: Free Shipping - Variant B

In this experiment, an extra "Free Shipping" message was added on a product page - at the top of the buy box with an add-to-cart call to action. It's prominence was increased by using white copy on a darker blue background. Impact on adds-to-cart and total sales was measured.

Total Sample: 97,277 • Statistical Power at 5% MDE: 83.7%

Test #324 by Jakub LinowskiJakub Linowski Oct 30, 2020 Desktop Mobile Product X.X% Revenue

Jakub Tested Pattern #17: Least Or Most Expensive First

Pattern #17: Least Or Most Expensive First - Variant A
Pattern #17: Least Or Most Expensive First - Variant B

This experiment tested the order of purchase plans. The control version sorted the purchase options by the least expensive while the variation sorted them by the most expensive first. Impact on sales and revenue was measured.

Test #322 on Thomasnet.com by Kyle PhillipsKyle Phillips Oct 27, 2020 Desktop Mobile Product X.X% Signups

Kyle Tested Pattern #82: Onboarding Callouts On Thomasnet.com

Pattern #82: Onboarding Callouts - Variant A
Pattern #82: Onboarding Callouts - Variant B

This experiment variation prompted users to save (bookmark) a company profile on a company detail page. Clicking on the save feature while logged out, would prompt a registration modal. Hence the save feature acted as an extra reason to signup. The number of people engaging or interacting with the feature was measured, as well as registrations.

Total Sample: 279,729 • Statistical Power at 5% MDE: 23.6%

Test #99 on Vivareal.com.br by Rodrigo MauésRodrigo Maués Sep 23, 2020 Desktop Mobile Product X.X% Leads

Rodrigo Tested Pattern #24: Visible Availability On Vivareal.com.br

Pattern #24: Visible Availability - Variant A
Pattern #24: Visible Availability - Variant B

In this experiment, a lead form on a listing page showed whether an agent was recently online or not. The diplayed had two statuses: either indicating that someone is online now, or the most recent time they were online in minutes.

Total Sample: 249,954 • Statistical Power at 5% MDE: 88.6%

Test #314 on Zapimoveis.com.br by Vinicius Barros PeixotoVinicius Barros Peixoto Aug 21, 2020 Desktop Mobile Product X.X% Leads

Vinicius Tested Pattern #43: Long Titles On Zapimoveis.com.br

Pattern #43: Long Titles - Variant A
Pattern #43: Long Titles - Variant B

In this experiment, a dynamic page title was generated and added at the top of the screen. The first few words from a property description were used to dynamically generate these titles. The effect on leads was measured.

Total Sample: 6,533,516 • Statistical Power at 5% MDE: 99.9%

Test #312 by Jakub LinowskiJakub Linowski Aug 14, 2020 Desktop Mobile Product X.X% Sales

Jakub Tested Pattern #83: Progressive Fields

Pattern #83: Progressive Fields - Variant A
Pattern #83: Progressive Fields - Variant B

In this experiment, we tested a visible "Shipping Frequency" (A) option against a progressively displayed one (B) that would only appear after someone first chose a duration option. Thus in variation B, the buy box component would initially appear with fewer fields and smaller. The experiment measured initial progression and actual sales. 

Note on the data: the experiment was run a little shorter than usual, as one of the variations triggered a stop rule to protect losses (so the effect might be somewhat inflated from a lower power).

Total Sample: 2,861 • Statistical Power at 5% MDE: 5.6%

Test #304 on Backstage.com by Stanley ZuoStanley Zuo Jun 29, 2020 Mobile Product X.X% Signups

Stanley Tested Pattern #97: Bigger Form Fields On Backstage.com

Pattern #97: Bigger Form Fields - Variant A
Pattern #97: Bigger Form Fields - Variant B

In this experiment, larger "Apply" buttons were shown on a casting detail page. The application funnel would take users through a series of steps leading to a paid membership subscription. The experiment measured initial progression and account signups (email signups).

Total Sample: 109,372 • Statistical Power at 5% MDE: 95.4%

Test #301 on Zapimoveis.com.br by Vinicius Barros PeixotoVinicius Barros Peixoto May 31, 2020 Desktop Mobile Product X.X% Leads

Vinicius Tested Pattern #21: What It's Worth On Zapimoveis.com.br

Pattern #21: What It's Worth - Variant A
Pattern #21: What It's Worth - Variant B

In this experiment, the B variation property prices were framed using higher and crossed out price points from 12 months ago - achieving a relative discount. A tooltip was also shown which explained the higher price point on hover. The example in the screenshot translates to "2% less compared to 12 months ago". This high-power experiment measured the number of leads that were generated on property (product) screens.

Total Sample: 136,682 • Statistical Power at 5% MDE: 97.6%

Test #293 on Backstage.com by Stanley ZuoStanley Zuo Apr 14, 2020 Desktop Mobile Product X.X% Sales

Stanley Tested Pattern #114: Less Or More Visible Prices On Backstage.com

Pattern #114: Less Or More Visible Prices - Variant A
Pattern #114: Less Or More Visible Prices - Variant B

In this experiment on a casting call site, pricing information was shown beside the application button. This change shows the effect of setting a price expectation and being more clear that the application process is not free.

Total Sample: 115,061 • Statistical Power at 5% MDE: 9.3%

Test #285 on Ibood.com by Lukas JorissenLukas Jorissen Feb 27, 2020 Desktop Product X.X% Sales

Lukas Tested Pattern #7: Social Counts On Ibood.com

Pattern #7: Social Counts - Variant A
Pattern #7: Social Counts - Variant B

In this experiment, realtime social proof information has been added below an add-to-cart button. The variation shows how many users that have viewed, or placed a product into their basket. Translated to "4 visitors have this product in their shopping cart."

Total Sample: 137,190 • Statistical Power at 5% MDE: 36.7%

Test #266 by Someone Someone Oct 25, 2019 Desktop Mobile Product X.X% Sales

Someone Tested Pattern #4: Testimonials

Pattern #4: Testimonials - Variant A
Pattern #4: Testimonials - Variant B

In this experiment, a product page showed customer reviews at the bottom of the page (B) instead of hiding them under a tab (A).

Total Sample: 17,139 • Statistical Power at 5% MDE: 15.4%

Test #40 on Adoramapix.com by Herman KleinHerman Klein Aug 13, 2019 Desktop Product X.X% Sales

Herman Tested Pattern #85: Benefit Button On Adoramapix.com

Pattern #85: Benefit Button - Variant A
Pattern #85: Benefit Button - Variant B

In this experiment only the button label changed. The control had a more immediate (next step-oriented) button label of "Upload Your Photos Now". The variation tried a more benefit-oriented button of "Order Now" (hinting at paying and obtaining the product).

Total Sample: 13,454 • Statistical Power at 5% MDE: 43.7%

Test #249 on Autoscout24.com by Optimizely Optimizely Jul 17, 2019 Desktop Product X.X% Leads

Optimizely Tested Pattern #20: Canned Response On Autoscout24.com

Pattern #20: Canned Response - Variant A
Pattern #20: Canned Response - Variant B

AutoScout24 is Europe’s largest online marketplace for new and used cars. As published by Optimizely, they tested a form pre-filled with text that read: ‘Hello. I am interested in your vehicle. Please contact me. Kind regards.’ 

Test #244 on Mt.com by Vito MediavillaVito Mediavilla Jun 06, 2019 Desktop Product X.X% Leads

Vito Tested Pattern #49: Above The Fold Call To Action On Mt.com

Pattern #49: Above The Fold Call To Action - Variant A
Pattern #49: Above The Fold Call To Action - Variant B

A product landing page with an image was tested against one without - raising the call to action above the fold.

Total Sample: 3,711 • Statistical Power at 5% MDE: 11.2%