All Latest 644 A/B Tests

MOST RECENT TESTS

Test #336 on Backstage.com by Stanley ZuoStanley Zuo Jan 28, 2021 Desktop Mobile Listing X.X% Sales

Stanley Tested Pattern #51: Shortcut Buttons On Backstage.com

Pattern #51: Shortcut Buttons - Variant A
Pattern #51: Shortcut Buttons - Variant B

In this experiment, a listing page was expanded to show two actions (apply and view details) instead of a single one (view details only). This variation enabled users with a shortcut action to apply for roles one step earlier (and start membership flows for new users). 

Total Sample: 159,634 • Statistical Power at 5% MDE: 16.1%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #337 on Backstage.com by Stanley ZuoStanley Zuo Jan 28, 2021 Desktop Mobile Listing X.X% Sales

Stanley Tested Pattern #51: Shortcut Buttons On Backstage.com

Pattern #51: Shortcut Buttons - Variant A
Pattern #51: Shortcut Buttons - Variant B

In this experiment, a listing page was expanded to show two actions (apply and view details) instead of a single one (view details only). In the variant, the "view detail" links were replaced with "apply links" starting a job application (and membership flows) sooner. 

Total Sample: 159,693 • Statistical Power at 5% MDE: 16.1%

Test #335 by Jakub LinowskiJakub Linowski Jan 27, 2021 Desktop Mobile Home & Landing X.X% Sales

Jakub Tested Pattern #32: Condensed List

Pattern #32: Condensed List - Variant A
Pattern #32: Condensed List - Variant B

The variation here has more condensed product tiles being shown on a homepage. Two pieces of information were removed: product descriptions and past selections. Impact on product page visits and total sales was measured.

Total Sample: 20,943 • Statistical Power at 5% MDE: 19.2%

Test #334 on Thomasnet.com by Kyle PhillipsKyle Phillips Jan 25, 2021 Desktop Mobile Global X.X% Progression

Kyle Tested Pattern #2: Icon Labels On Thomasnet.com

Pattern #2: Icon Labels - Variant A
Pattern #2: Icon Labels - Variant B

This experiment measured the impact of adding text labels to three icon-only nav items. 

Total Sample: 60,466 • Statistical Power at 5% MDE: 11.1%

Test #333 on Expertinstitute.com by Ardit VeliuArdit Veliu Dec 31, 2020 Desktop Mobile Home & Landing X.X% Leads

Ardit Tested Pattern #11: Gradual Reassurance On Expertinstitute.com

Pattern #11: Gradual Reassurance - Variant A
Pattern #11: Gradual Reassurance - Variant B

In this experiment, the variation broke up a lead form into two parts. In the first step users were asked for their state followed by a standard contact form on a second step. All of the states were shown as selectable options. In the control version, the landing page only showed a button which lead to the full form. The experiment measured impact on lead form submissions.

Total Sample: 11,258 • Statistical Power at 5% MDE: 6.1%

Test #332 by Jakub LinowskiJakub Linowski Dec 30, 2020 Desktop Mobile Product X.X% Sales

Jakub Tested Pattern #121: Free Shipping

Pattern #121: Free Shipping - Variant A
Pattern #121: Free Shipping - Variant B

In this experiment, an extra "Free Shipping" message was added on a product page - at the top of the buy box with an add-to-cart call to action. It's prominence was increased by using white copy on a darker blue background. Impact on adds-to-cart and total sales was measured.

Total Sample: 97,277 • Statistical Power at 5% MDE: 83.7%

Test #331 by Jakub LinowskiJakub Linowski Dec 30, 2020 Desktop Mobile Product X.X% Sales

Jakub Tested Pattern #121: Free Shipping

Pattern #121: Free Shipping - Variant A
Pattern #121: Free Shipping - Variant B

In this little experiment, an extra "Free Shipping" message was added on a product page. It's prominence was increased by using white copy on a darker blue background. Impact on adds-to-cart and total sales was measured.

Total Sample: 96,962 • Statistical Power at 5% MDE: 83.6%

Test #330 on Backstage.com by Stanley ZuoStanley Zuo Dec 29, 2020 Desktop Content X.X% Signups

Stanley Tested Pattern #116: Links Or Buttons On Backstage.com

Pattern #116: Links Or Buttons - Variant A
Pattern #116: Links Or Buttons - Variant B

In this simple experiment on a content page, links were turned into more prominent buttons. The experiment measured clicks and signups.

Total Sample: 98,697 • Statistical Power at 5% MDE: 68.9%

Test #328 on Umbraco.com by Lars Skjold IversenLars Skjold Iversen Dec 21, 2020 Desktop Mobile Home & Landing X.X% Signups

Lars Tested Pattern #60: Repeated Bottom Call To Action On Umbraco.com

Pattern #60: Repeated Bottom Call To Action - Variant A
Pattern #60: Repeated Bottom Call To Action - Variant B

In this experiment, a trial signup section was added at the bottom of Umbraco's long homepage (CMS business). The experiment measured the impact on trial signups.

Total Sample: 57,371 • Statistical Power at 5% MDE: 8.4%

Test #98 on 3dhubs.com by Rob DraaijerRob Draaijer Nov 30, 2020 Desktop Mobile Listing X.X% Leads

Rob Tested Pattern #24: Visible Availability On 3dhubs.com

Pattern #24: Visible Availability - Variant A
Pattern #24: Visible Availability - Variant B

In this experiment, the variation showed a listing's owner online status as a badge, instead of showing their average "response time". More specifically, an "Online Now" badge was shown beside individual listings of a 3D printing marketplace site. The experiment measured completed quote / lead requests (a few steps further).

Total Sample: 48,771 • Statistical Power at 5% MDE: 44%

Test #10 on Tradegecko.com by Syed AtiF HusainSyed AtiF Husain Nov 30, 2020 Desktop Home & Landing X.X% Leads

Syed AtiF Tested Pattern #10: Postponed Modal Forms On Tradegecko.com

Pattern #10: Postponed Modal Forms - Variant A
Pattern #10: Postponed Modal Forms - Variant B

In this experiment, 3 form fields were removed (postponed to a next step) from the homepage leaving only a "Start Trail" button. When users clicked on the "Start A Free 14 Day Trial" button, in both the control and variation they've seen the same next registration page with all of the fields. The registration page repeated the same fields with their corresponding values, as well as asked for a password as an extra field. The experiment measured successful leads.

Total Sample: 6,529 • Statistical Power at 5% MDE: 5.7%

Test #327 on Backstage.com by Stanley ZuoStanley Zuo Nov 26, 2020 Desktop Signup X.X% Signups

Stanley Tested Pattern #120: Supporting Theme Images On Backstage.com

Pattern #120: Supporting Theme Images - Variant A
Pattern #120: Supporting Theme Images - Variant B

In this experiment, an aspirational photo was shown on the right side panel - reinforcing the theme of casting calls. The experiment measured progression to the next step and completed signups.

Total Sample: 11,009 • Statistical Power at 5% MDE: 53%

Test #326 on Thomasnet.com by Kyle PhillipsKyle Phillips Nov 25, 2020 Desktop Mobile Content X.X% Progression

Kyle Tested Pattern #41: Sticky Call To Action On Thomasnet.com

Pattern #41: Sticky Call To Action - Variant A
Pattern #41: Sticky Call To Action - Variant B

In this simple experiment on an article page, the variation slid out a sticky call to action linking to the next article. The sliding interaction triggered after some scrolling threshold (around 1000px or so). Afterwards, the sticky call to action maintained its floating position. The experiment measured clicks on this "next article" button.

Total Sample: 7,999 • Statistical Power at 5% MDE: 4.1%

Test #325 on Snocks.com by Samuel HessSamuel Hess Nov 24, 2020 Desktop Global X.X% Sales

Samuel Tested Pattern #45: Benefit Bar On Snocks.com

Pattern #45: Benefit Bar - Variant A
Pattern #45: Benefit Bar - Variant B

In this experiment, a set of reassurances and reviews were added in the header of this ecommerce website. Translating from German, these read: "Anti Hole Guarantee", "Free Shipping" and "X Ratings out of Y Reviews".

Total Sample: 11,349 • Statistical Power at 5% MDE: 14%

Test #324 by Jakub LinowskiJakub Linowski Oct 30, 2020 Desktop Mobile Product X.X% Revenue

Jakub Tested Pattern #17: Least Or Most Expensive First

Pattern #17: Least Or Most Expensive First - Variant A
Pattern #17: Least Or Most Expensive First - Variant B

This experiment tested the order of purchase plans. The control version sorted the purchase options by the least expensive while the variation sorted them by the most expensive first. Impact on sales and revenue was measured.

Test #322 on Thomasnet.com by Kyle PhillipsKyle Phillips Oct 27, 2020 Desktop Mobile Product X.X% Signups

Kyle Tested Pattern #82: Onboarding Callouts On Thomasnet.com

Pattern #82: Onboarding Callouts - Variant A
Pattern #82: Onboarding Callouts - Variant B

This experiment variation prompted users to save (bookmark) a company profile on a company detail page. Clicking on the save feature while logged out, would prompt a registration modal. Hence the save feature acted as an extra reason to signup. The number of people engaging or interacting with the feature was measured, as well as registrations.

Total Sample: 279,729 • Statistical Power at 5% MDE: 23.6%

Test #320 by Jakub LinowskiJakub Linowski Oct 20, 2020 Desktop Checkout X.X% Sales

Jakub Tested Pattern #49: Above The Fold Call To Action

Pattern #49: Above The Fold Call To Action - Variant A
Pattern #49: Above The Fold Call To Action - Variant B

An extra "Place Order" button was duplicated above the fold on this checkout page. The control had a similar button further down at the bottom of the screen. The impact on total sales was measured from this change.

Total Sample: 2,908 • Statistical Power at 5% MDE: 97.4%

Test #319 on Backstage.com by Stanley ZuoStanley Zuo Sep 30, 2020 Desktop Pricing X.X% Revenue

Stanley Tested Pattern #113: More Or Fewer Plans On Backstage.com

Pattern #113: More Or Fewer Plans - Variant A
Pattern #113: More Or Fewer Plans - Variant B

In this experiment, a 3 plan vs 2 plan pricing page was shown to potential customers. Impact on sales and revenue were measured.

Test #318 on Thomasnet.com by Kyle PhillipsKyle Phillips Sep 29, 2020 Desktop Mobile Content X.X% Signups

Kyle Tested Pattern #60: Repeated Bottom Call To Action On Thomasnet.com

Pattern #60: Repeated Bottom Call To Action - Variant A
Pattern #60: Repeated Bottom Call To Action - Variant B

In this experiment, a simple link to a newsletter signup landing page was added at the bottom of an article. The newsletter landing page then encouraged users to provide their email address for future article updates.

Total Sample: 108,360 • Statistical Power at 5% MDE: 4.8%

Test #316 on Trydesignlab.com by Daniel ShapiroDaniel Shapiro Sep 24, 2020 Desktop Mobile Home & Landing X.X% Signups

Daniel Tested Pattern #22: Empowering Headline On Trydesignlab.com

Pattern #22: Empowering Headline - Variant A
Pattern #22: Empowering Headline - Variant B

In this experiment, the headline was changed to focus more on the end-goal of the UX Academy program - that of landing your first UI/UX role.

Total Sample: 25,634 • Statistical Power at 5% MDE: 25.6%