All Latest 644 A/B Tests

Test #351 on Baremetrics.com by Brian SierakowskiBrian Sierakowski Apr 30, 2021 Desktop Mobile Home & Landing X.X% Progression

Brian Tested Pattern #11: Gradual Reassurance On Baremetrics.com

Pattern #11: Gradual Reassurance - Variant A
Pattern #11: Gradual Reassurance - Variant B

In this experiment, static integration logos were replaced with selectable ones that reassured users to signup. After clicking an integration logo, a comparison chart would appear showing how Baremetrics improves upon a selected payment processor, along with a call to signup. Impact on signups was measured.

Total Sample: 24,416 • Statistical Power at 5% MDE: 7%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #349 on Backstage.com by Stanley ZuoStanley Zuo Apr 27, 2021 Mobile Global X.X% Progression

Stanley Tested Pattern #49: Above The Fold Call To Action On Backstage.com

Pattern #49: Above The Fold Call To Action - Variant A
Pattern #49: Above The Fold Call To Action - Variant B

In this experiment, a simple "Join Now" button was added inside an expanded mobile navigation menu. Clicking the button would start a multiple step membership subscription process. Impact on subscription starts and final sales were tracked.

Total Sample: 950,043 • Statistical Power at 5% MDE: 37.9%

Test #348 on Flukenetworks.com by Marika FranciscoMarika Francisco Apr 22, 2021 Desktop Home & Landing X.X% Progression

Marika Tested Pattern #97: Bigger Form Fields On Flukenetworks.com

Pattern #97: Bigger Form Fields - Variant A
Pattern #97: Bigger Form Fields - Variant B

In this simple experiment, the size of the "Get Quote" button in the top navigation was increased. 

Total Sample: 39,992 • Statistical Power at 5% MDE: 4.3%

Test #347 by Jakub LinowskiJakub Linowski Apr 07, 2021 Desktop Mobile Home & Landing X.X% Progression

Jakub Tested Pattern #26: Cart Reminder And Recently Viewed

Pattern #26: Cart Reminder And Recently Viewed - Variant A
Pattern #26: Cart Reminder And Recently Viewed - Variant B

In this experiment, when customers viewed a product and returned to the homepage, they would then see the most recently viewed one - a delicate nudge. The experiment ran with full traffic and impact on sales was measured.

Total Sample: 13,749 • Statistical Power at 5% MDE: 15.8%

Test #346 by Stanley ZuoStanley Zuo Mar 30, 2021 Desktop Mobile Home & Landing X.X% Progression

Stanley Tested Pattern #117: Company Logos

Pattern #117: Company Logos - Variant A
Pattern #117: Company Logos - Variant B

In this homepage experiment, company logos were added to the homepage. These were example clients or companies that Backstage works with and offers casting (job) listing from. Impact on the registration flow and membership checkouts was measured.

Total Sample: 221,012 • Statistical Power at 5% MDE: 43.1%

Test #345 on Getninjas.com.br by Rodolfo LugliRodolfo Lugli Mar 29, 2021 Desktop Home & Landing X.X% Progression

Rodolfo Tested Pattern #9: Multiple Steps On Getninjas.com.br

Pattern #9: Multiple Steps - Variant A
Pattern #9: Multiple Steps - Variant B

In this experiment, a single long form was broken into at least 3 steps. 

Total Sample: 401,145 • Statistical Power at 5% MDE: 99.9%

Test #343 on Snocks.com by Samuel HessSamuel Hess Mar 12, 2021 Desktop Mobile Product X.X% Progression

Samuel Tested Pattern #122: Zigzag Layout On Snocks.com

Pattern #122: Zigzag Layout - Variant A
Pattern #122: Zigzag Layout - Variant B

In this experiment, the content on a product page was reorganized into a zigzagging (alternating layout) along with reinforcing photos. Impact on adds-to-cart and total sales was measured.

Total Sample: 23,865 • Statistical Power at 5% MDE: 32.9%

Test #336 on Backstage.com by Stanley ZuoStanley Zuo Jan 28, 2021 Desktop Mobile Listing X.X% Progression

Stanley Tested Pattern #51: Shortcut Buttons On Backstage.com

Pattern #51: Shortcut Buttons - Variant A
Pattern #51: Shortcut Buttons - Variant B

In this experiment, a listing page was expanded to show two actions (apply and view details) instead of a single one (view details only). This variation enabled users with a shortcut action to apply for roles one step earlier (and start membership flows for new users). 

Total Sample: 159,634 • Statistical Power at 5% MDE: 16.1%

Test #337 on Backstage.com by Stanley ZuoStanley Zuo Jan 28, 2021 Desktop Mobile Listing X.X% Progression

Stanley Tested Pattern #51: Shortcut Buttons On Backstage.com

Pattern #51: Shortcut Buttons - Variant A
Pattern #51: Shortcut Buttons - Variant B

In this experiment, a listing page was expanded to show two actions (apply and view details) instead of a single one (view details only). In the variant, the "view detail" links were replaced with "apply links" starting a job application (and membership flows) sooner. 

Total Sample: 159,693 • Statistical Power at 5% MDE: 16.1%

Test #335 by Jakub LinowskiJakub Linowski Jan 27, 2021 Desktop Mobile Home & Landing X.X% Progression

Jakub Tested Pattern #32: Condensed List

Pattern #32: Condensed List - Variant A
Pattern #32: Condensed List - Variant B

The variation here has more condensed product tiles being shown on a homepage. Two pieces of information were removed: product descriptions and past selections. Impact on product page visits and total sales was measured.

Total Sample: 20,943 • Statistical Power at 5% MDE: 19.2%

Test #334 on Thomasnet.com by Kyle PhillipsKyle Phillips Jan 25, 2021 Desktop Mobile Global X.X% Progression

Kyle Tested Pattern #2: Icon Labels On Thomasnet.com

Pattern #2: Icon Labels - Variant A
Pattern #2: Icon Labels - Variant B

This experiment measured the impact of adding text labels to three icon-only nav items. 

Total Sample: 60,466 • Statistical Power at 5% MDE: 11.1%

Test #331 by Jakub LinowskiJakub Linowski Dec 30, 2020 Desktop Mobile Product X.X% Progression

Jakub Tested Pattern #121: Free Shipping

Pattern #121: Free Shipping - Variant A
Pattern #121: Free Shipping - Variant B

In this little experiment, an extra "Free Shipping" message was added on a product page. It's prominence was increased by using white copy on a darker blue background. Impact on adds-to-cart and total sales was measured.

Total Sample: 96,962 • Statistical Power at 5% MDE: 83.6%

Test #332 by Jakub LinowskiJakub Linowski Dec 30, 2020 Desktop Mobile Product X.X% Progression

Jakub Tested Pattern #121: Free Shipping

Pattern #121: Free Shipping - Variant A
Pattern #121: Free Shipping - Variant B

In this experiment, an extra "Free Shipping" message was added on a product page - at the top of the buy box with an add-to-cart call to action. It's prominence was increased by using white copy on a darker blue background. Impact on adds-to-cart and total sales was measured.

Total Sample: 97,277 • Statistical Power at 5% MDE: 83.7%

Test #330 on Backstage.com by Stanley ZuoStanley Zuo Dec 29, 2020 Desktop Content X.X% Progression

Stanley Tested Pattern #116: Links Or Buttons On Backstage.com

Pattern #116: Links Or Buttons - Variant A
Pattern #116: Links Or Buttons - Variant B

In this simple experiment on a content page, links were turned into more prominent buttons. The experiment measured clicks and signups.

Total Sample: 98,697 • Statistical Power at 5% MDE: 68.9%

Test #329 on Snocks.com by Samuel HessSamuel Hess Dec 23, 2020 Mobile Home & Landing X.X% Progression

Samuel Tested Pattern #14: Exposed Menu Options On Snocks.com

Pattern #14: Exposed Menu Options - Variant A
Pattern #14: Exposed Menu Options - Variant B

In this homepage experiment, a series of product categories were shown more visible near the top of the screen (instead of only being shown inside the hamburger menu). They linked up to corresponding listing pages with such items as: gifts, short socks, long socks, and underwear. Impact on adds-to-cart and total sales was measured.

Total Sample: 58,462 • Statistical Power at 5% MDE: 40.4%

Test #328 on Umbraco.com by Lars Skjold IversenLars Skjold Iversen Dec 21, 2020 Desktop Mobile Home & Landing X.X% Progression

Lars Tested Pattern #60: Repeated Bottom Call To Action On Umbraco.com

Pattern #60: Repeated Bottom Call To Action - Variant A
Pattern #60: Repeated Bottom Call To Action - Variant B

In this experiment, a trial signup section was added at the bottom of Umbraco's long homepage (CMS business). The experiment measured the impact on trial signups.

Total Sample: 57,371 • Statistical Power at 5% MDE: 8.4%

Test #327 on Backstage.com by Stanley ZuoStanley Zuo Nov 26, 2020 Desktop Signup X.X% Progression

Stanley Tested Pattern #120: Supporting Theme Images On Backstage.com

Pattern #120: Supporting Theme Images - Variant A
Pattern #120: Supporting Theme Images - Variant B

In this experiment, an aspirational photo was shown on the right side panel - reinforcing the theme of casting calls. The experiment measured progression to the next step and completed signups.

Total Sample: 11,009 • Statistical Power at 5% MDE: 53%

Test #326 on Thomasnet.com by Kyle PhillipsKyle Phillips Nov 25, 2020 Desktop Mobile Content X.X% Progression

Kyle Tested Pattern #41: Sticky Call To Action On Thomasnet.com

Pattern #41: Sticky Call To Action - Variant A
Pattern #41: Sticky Call To Action - Variant B

In this simple experiment on an article page, the variation slid out a sticky call to action linking to the next article. The sliding interaction triggered after some scrolling threshold (around 1000px or so). Afterwards, the sticky call to action maintained its floating position. The experiment measured clicks on this "next article" button.

Total Sample: 7,999 • Statistical Power at 5% MDE: 4.1%

Test #325 on Snocks.com by Samuel HessSamuel Hess Nov 24, 2020 Desktop Global X.X% Progression

Samuel Tested Pattern #45: Benefit Bar On Snocks.com

Pattern #45: Benefit Bar - Variant A
Pattern #45: Benefit Bar - Variant B

In this experiment, a set of reassurances and reviews were added in the header of this ecommerce website. Translating from German, these read: "Anti Hole Guarantee", "Free Shipping" and "X Ratings out of Y Reviews".

Total Sample: 11,349 • Statistical Power at 5% MDE: 14%

Test #322 on Thomasnet.com by Kyle PhillipsKyle Phillips Oct 27, 2020 Desktop Mobile Product X.X% Progression

Kyle Tested Pattern #82: Onboarding Callouts On Thomasnet.com

Pattern #82: Onboarding Callouts - Variant A
Pattern #82: Onboarding Callouts - Variant B

This experiment variation prompted users to save (bookmark) a company profile on a company detail page. Clicking on the save feature while logged out, would prompt a registration modal. Hence the save feature acted as an extra reason to signup. The number of people engaging or interacting with the feature was measured, as well as registrations.

Total Sample: 279,729 • Statistical Power at 5% MDE: 23.6%