All Latest 634 A/B Tests

Test #337 on Backstage.com by Stanley ZuoStanley Zuo Jan 28, 2021 Desktop Mobile Listing X.X% Progression

Stanley Tested Pattern #51: Shortcut Buttons On Backstage.com

 - Variant A
 - Variant B

In this experiment, a listing page was expanded to show two actions (apply and view details) instead of a single one (view details only). In the variant, the "view detail" links were replaced with "apply links" starting a job application (and membership flows) sooner. 

Total Sample: 159,693 Statistical Power at 5% MDE: 16.1%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #336 on Backstage.com by Stanley ZuoStanley Zuo Jan 28, 2021 Desktop Mobile Listing X.X% Progression

Stanley Tested Pattern #51: Shortcut Buttons On Backstage.com

 - Variant A
 - Variant B

In this experiment, a listing page was expanded to show two actions (apply and view details) instead of a single one (view details only). This variation enabled users with a shortcut action to apply for roles one step earlier (and start membership flows for new users). 

Total Sample: 159,634 Statistical Power at 5% MDE: 16.1%

Test #335 on by Jakub LinowskiJakub Linowski Jan 27, 2021 Desktop Mobile Home & Landing X.X% Progression

Jakub Tested Pattern #32: Condensed List

 - Variant A
 - Variant B

The variation here has more condensed product tiles being shown on a homepage. Two pieces of information were removed: product descriptions and past selections. Impact on product page visits and total sales was measured.

Total Sample: 20,943 Statistical Power at 5% MDE: 19.2%

Test #334 on Thomasnet.com by Kyle PhillipsKyle Phillips Jan 25, 2021 Desktop Mobile Global X.X% Progression

Kyle Tested Pattern #2: Icon Labels On Thomasnet.com

 - Variant A
 - Variant B

This experiment measured the impact of adding text labels to three icon-only nav items. 

Total Sample: 60,466 Statistical Power at 5% MDE: 11.1%

Test #331 on by Jakub LinowskiJakub Linowski Dec 30, 2020 Desktop Mobile Product X.X% Progression

Jakub Tested Pattern #121: Free Shipping

 - Variant A
 - Variant B

In this little experiment, an extra "Free Shipping" message was added on a product page. It's prominence was increased by using white copy on a darker blue background. Impact on adds-to-cart and total sales was measured.

Total Sample: 96,962 Statistical Power at 5% MDE: 83.6%

Test #332 on by Jakub LinowskiJakub Linowski Dec 30, 2020 Desktop Mobile Product X.X% Progression

Jakub Tested Pattern #121: Free Shipping

 - Variant A
 - Variant B

In this experiment, an extra "Free Shipping" message was added on a product page - at the top of the buy box with an add-to-cart call to action. It's prominence was increased by using white copy on a darker blue background. Impact on adds-to-cart and total sales was measured.

Total Sample: 97,277 Statistical Power at 5% MDE: 83.7%

Test #330 on Backstage.com by Stanley ZuoStanley Zuo Dec 29, 2020 Desktop Content X.X% Progression

Stanley Tested Pattern #116: Links Or Buttons On Backstage.com

 - Variant A
 - Variant B

In this simple experiment on a content page, links were turned into more prominent buttons. The experiment measured clicks and signups.

Total Sample: 98,697 Statistical Power at 5% MDE: 68.9%

Test #329 on Snocks.com by Samuel HessSamuel Hess Dec 23, 2020 Mobile Home & Landing X.X% Progression

Samuel Tested Pattern #14: Exposed Menu Options On Snocks.com

 - Variant A
 - Variant B

In this homepage experiment, a series of product categories were shown more visible near the top of the screen (instead of only being shown inside the hamburger menu). They linked up to corresponding listing pages with such items as: gifts, short socks, long socks, and underwear. Impact on adds-to-cart and total sales was measured.

Total Sample: 58,462 Statistical Power at 5% MDE: 40.4%

Test #328 on Umbraco.com by Lars Skjold IversenLars Skjold Iversen Dec 21, 2020 Desktop Mobile Home & Landing X.X% Progression

Lars Tested Pattern #60: Repeated Bottom Call To Action On Umbraco.com

 - Variant A
 - Variant B

In this experiment, a trial signup section was added at the bottom of Umbraco's long homepage (CMS business). The experiment measured the impact on trial signups.

Total Sample: 57,371 Statistical Power at 5% MDE: 8.4%

Test #327 on Backstage.com by Stanley ZuoStanley Zuo Nov 26, 2020 Desktop Signup X.X% Progression

Stanley Tested Pattern #120: Supporting Theme Images On Backstage.com

 - Variant A
 - Variant B

In this experiment, an aspirational photo was shown on the right side panel - reinforcing the theme of casting calls. The experiment measured progression to the next step and completed signups.

Total Sample: 11,009 Statistical Power at 5% MDE: 53%

Test #326 on Thomasnet.com by Kyle PhillipsKyle Phillips Nov 25, 2020 Desktop Mobile Content X.X% Progression

Kyle Tested Pattern #41: Sticky Call To Action On Thomasnet.com

 - Variant A
 - Variant B

In this simple experiment on an article page, the variation slid out a sticky call to action linking to the next article. The sliding interaction triggered after some scrolling threshold (around 1000px or so). Afterwards, the sticky call to action maintained its floating position. The experiment measured clicks on this "next article" button.

Total Sample: 7,999 Statistical Power at 5% MDE: 4.1%

Test #325 on Snocks.com by Samuel HessSamuel Hess Nov 24, 2020 Desktop Global X.X% Progression

Samuel Tested Pattern #45: Benefit Bar On Snocks.com

 - Variant A
 - Variant B

In this experiment, a set of reassurances and reviews were added in the header of this ecommerce website. Translating from German, these read: "Anti Hole Guarantee", "Free Shipping" and "X Ratings out of Y Reviews".

Total Sample: 11,349 Statistical Power at 5% MDE: 14%

Test #322 on Thomasnet.com by Kyle PhillipsKyle Phillips Oct 27, 2020 Desktop Mobile Product X.X% Progression

Kyle Tested Pattern #82: Onboarding Callouts On Thomasnet.com

 - Variant A
 - Variant B

This experiment variation prompted users to save (bookmark) a company profile on a company detail page. Clicking on the save feature while logged out, would prompt a registration modal. Hence the save feature acted as an extra reason to signup. The number of people engaging or interacting with the feature was measured, as well as registrations.

Total Sample: 279,729 Statistical Power at 5% MDE: 23.6%

Test #321 on Elevate App App by Jesse GerminarioJesse Germinario Oct 23, 2020 Mobile X.X% Progression

Jesse Tested Pattern #11: Gradual Reassurance

 - Variant A
 - Variant B

This experiment aimed to increase the number of application ratings from within the Elevate app. Success was measured by the number of users going towards Google Play to create the rating. The control version prompted users if they wanted to rate the app with a simple yes and no answer. The variation however presented the rating choice right away in the form of 5 stars - enabling users to express their choice sooner.

Total Sample: 8,400 Statistical Power at 5% MDE: 15.7%

Test #319 on Backstage.com by Stanley ZuoStanley Zuo Sep 30, 2020 Desktop Pricing X.X% Progression

Stanley Tested Pattern #113: More Or Fewer Plans On Backstage.com

 - Variant A
 - Variant B

In this experiment, a 3 plan vs 2 plan pricing page was shown to potential customers. Impact on sales and revenue were measured.

Test #318 on Thomasnet.com by Kyle PhillipsKyle Phillips Sep 29, 2020 Desktop Mobile Content X.X% Progression

Kyle Tested Pattern #60: Repeated Bottom Call To Action On Thomasnet.com

 - Variant A
 - Variant B

In this experiment, a simple link to a newsletter signup landing page was added at the bottom of an article. The newsletter landing page then encouraged users to provide their email address for future article updates.

Total Sample: 108,360 Statistical Power at 5% MDE: 4.8%

Test #312 on by Jakub LinowskiJakub Linowski Aug 14, 2020 Desktop Mobile Product X.X% Progression

Jakub Tested Pattern #83: Progressive Fields

 - Variant A
 - Variant B

In this experiment, we tested a visible "Shipping Frequency" (A) option against a progressively displayed one (B) that would only appear after someone first chose a duration option. Thus in variation B, the buy box component would initially appear with fewer fields and smaller. The experiment measured initial progression and actual sales. 

Note on the data: the experiment was run a little shorter than usual, as one of the variations triggered a stop rule to protect losses (so the effect might be somewhat inflated from a lower power).

Total Sample: 2,861 Statistical Power at 5% MDE: 5.6%

Test #311 on Backstage.com by Stanley ZuoStanley Zuo Aug 11, 2020 Desktop Mobile Home & Landing X.X% Progression

Stanley Tested Pattern #118: Category Images On Backstage.com

 - Variant A
 - Variant B

In this experiment, category links (linking to casting call search results) were replaced with tile images. In addition, 2 levels of categories were also replaced with a single text link for each tile. Finally, the font size of the link titles was also increased.

Total Sample: 120,519 Statistical Power at 5% MDE: 19.7%

Test #310 on Backstage.com by Stanley ZuoStanley Zuo Jul 25, 2020 Mobile Listing X.X% Progression

Stanley Tested Pattern #77: Filled Or Ghost Buttons On Backstage.com

 - Variant A
 - Variant B

In this experiment, the style of a button leading to view detailed casting calls on a listing page was changed. In the A version the style was a filled high contrast blue background, and the B variation had a feint "ghost button" style. 

Total Sample: 594,997 Statistical Power at 5% MDE: 23.9%

Test #309 on Thomasnet.com by Julian GaviriaJulian Gaviria Jul 24, 2020 Desktop Listing X.X% Progression

Julian Tested Pattern #72: Priming Step On Thomasnet.com

 - Variant A
 - Variant B

In this experiment, an extra step was prepended at the beginning of a multiple step signup modal flow. The signup modal would appear on listing pages after requests to contact a listed company. The idea was to prime users with benefits of signing up in order to increase their motivation to do so. The experiment measured the impact on the initial progression (to the step with the email form).

Total Sample: 6,146 Statistical Power at 5% MDE: 12.5%