All Latest 645 A/B Tests

MOST RECENT TESTS

Test #373 on Backstage.com by Stanley ZuoStanley Zuo Sep 06, 2021 Mobile Listing X.X% Sales

Stanley Tested Pattern #32: Condensed List On Backstage.com

Pattern #32: Condensed List - Variant A
Pattern #32: Condensed List - Variant B

In this experiment, listing descriptions were shortened dynamically using exposable "more" links. This way, the variation showed shorter listings and therefore more listings per screens. Impact on listing clicks (progression) along with membership starts was measured.

Total Sample: 43,469 • Statistical Power at 5% MDE: 92.2%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #371 by Jakub LinowskiJakub Linowski Aug 18, 2021 Desktop Listing X.X% Sales

Jakub Tested Pattern #51: Shortcut Buttons

Pattern #51: Shortcut Buttons - Variant A
Pattern #51: Shortcut Buttons - Variant B

In this experiment, the variation added an extra "Buy Now" button that linked directly into the cart process. The variation only had a learn more button linking directly to a product page.

Total Sample: 3,041 • Statistical Power at 5% MDE: 10.4%

Test #370 on Thomasnet.com by Julian GaviriaJulian Gaviria Aug 16, 2021 Desktop Mobile Listing X.X% Progression

Julian Tested Pattern #88: Action Button On Thomasnet.com

Pattern #88: Action Button - Variant A
Pattern #88: Action Button - Variant B

This experiment added the simple verb ("play") before the "factory video" label to encourage more video plays. Impact on progression / clicks was measured.

Total Sample: 259,330 • Statistical Power at 5% MDE: 49%

Test #78 on Mvideo.ru by Andrey AndreevAndrey Andreev May 06, 2021 Desktop Mobile Listing X.X% Sales

Andrey Tested Pattern #90: Out Of Stock Or In Stock Products On Mvideo.ru

Pattern #90: Out Of Stock Or In Stock Products - Variant A
Pattern #90: Out Of Stock Or In Stock Products - Variant B

In this experiment, products which were out of stock were removed from listing pages and replaced with in stock ones (not visible in the screenshot). 

Total Sample: 5,404,332 • Statistical Power at 5% MDE: 99.9%

Test #104 on 3dhubs.com by Rob DraaijerRob Draaijer Mar 31, 2021 Desktop Listing X.X% Leads

Rob Tested Pattern #15: Bulleted Reassurances On 3dhubs.com

Pattern #15: Bulleted Reassurances - Variant A
Pattern #15: Bulleted Reassurances - Variant B

This experiment attempted to increase the number of leads on a lead-funnel. As the first step, users were being asked to upload a file. The control showed the file types that were allowed, whereas the variation changed the copy to show a number of benefits for taking that action. The text-based benefits included the: receiving feedback, prices and lead times.

Total Sample: 50,698 • Statistical Power at 5% MDE: 43%

Test #342 on Backstage.com by Stanley ZuoStanley Zuo Feb 28, 2021 Desktop Mobile Listing X.X% Engagement

Stanley Tested Pattern #25: Nagging Results On Backstage.com

Pattern #25: Nagging Results - Variant A
Pattern #25: Nagging Results - Variant B

In this experiment, a registration wall was added on a listing page of casting call profiles. The registration wall appeared after the first 9 listings or so and encouraged users to sign up. Impact on registrations was measured, along with an engagement metric of "posting a job".

Total Sample: 7,934 • Statistical Power at 5% MDE: 5.9%

Test #336 on Backstage.com by Stanley ZuoStanley Zuo Jan 28, 2021 Desktop Mobile Listing X.X% Sales

Stanley Tested Pattern #51: Shortcut Buttons On Backstage.com

Pattern #51: Shortcut Buttons - Variant A
Pattern #51: Shortcut Buttons - Variant B

In this experiment, a listing page was expanded to show two actions (apply and view details) instead of a single one (view details only). This variation enabled users with a shortcut action to apply for roles one step earlier (and start membership flows for new users). 

Total Sample: 159,634 • Statistical Power at 5% MDE: 16.1%

Test #337 on Backstage.com by Stanley ZuoStanley Zuo Jan 28, 2021 Desktop Mobile Listing X.X% Sales

Stanley Tested Pattern #51: Shortcut Buttons On Backstage.com

Pattern #51: Shortcut Buttons - Variant A
Pattern #51: Shortcut Buttons - Variant B

In this experiment, a listing page was expanded to show two actions (apply and view details) instead of a single one (view details only). In the variant, the "view detail" links were replaced with "apply links" starting a job application (and membership flows) sooner. 

Total Sample: 159,693 • Statistical Power at 5% MDE: 16.1%

Test #98 on 3dhubs.com by Rob DraaijerRob Draaijer Nov 30, 2020 Desktop Mobile Listing X.X% Leads

Rob Tested Pattern #24: Visible Availability On 3dhubs.com

Pattern #24: Visible Availability - Variant A
Pattern #24: Visible Availability - Variant B

In this experiment, the variation showed a listing's owner online status as a badge, instead of showing their average "response time". More specifically, an "Online Now" badge was shown beside individual listings of a 3D printing marketplace site. The experiment measured completed quote / lead requests (a few steps further).

Total Sample: 48,771 • Statistical Power at 5% MDE: 44%

Test #310 on Backstage.com by Stanley ZuoStanley Zuo Jul 25, 2020 Mobile Listing X.X% Sales

Stanley Tested Pattern #77: Filled Or Ghost Buttons On Backstage.com

Pattern #77: Filled Or Ghost Buttons - Variant A
Pattern #77: Filled Or Ghost Buttons - Variant B

In this experiment, the style of a button leading to view detailed casting calls on a listing page was changed. In the A version the style was a filled high contrast blue background, and the B variation had a feint "ghost button" style. 

Total Sample: 594,997 • Statistical Power at 5% MDE: 23.9%

Test #309 on Thomasnet.com by Julian GaviriaJulian Gaviria Jul 24, 2020 Desktop Listing X.X% Progression

Julian Tested Pattern #72: Priming Step On Thomasnet.com

Pattern #72: Priming Step - Variant A
Pattern #72: Priming Step - Variant B

In this experiment, an extra step was prepended at the beginning of a multiple step signup modal flow. The signup modal would appear on listing pages after requests to contact a listed company. The idea was to prime users with benefits of signing up in order to increase their motivation to do so. The experiment measured the impact on the initial progression (to the step with the email form).

Total Sample: 6,146 • Statistical Power at 5% MDE: 12.5%

Test #299 on Backstage.com by Stanley ZuoStanley Zuo May 22, 2020 Desktop Mobile Listing X.X% Sales

Stanley Tested Pattern #60: Repeated Bottom Call To Action On Backstage.com

Pattern #60: Repeated Bottom Call To Action - Variant A
Pattern #60: Repeated Bottom Call To Action - Variant B

In this experiment, at the bottom of a search results screen, a membership join button was added along with 3 encouraging reasons. The experiment measured membership funnel starts, as well as paid membership transactions (sales).

Total Sample: 106,791 • Statistical Power at 5% MDE: 12.2%

Test #298 on Zapimoveis.com.br by Vinicius Barros PeixotoVinicius Barros Peixoto May 14, 2020 Desktop Mobile Listing X.X% Leads

Vinicius Tested Pattern #36: Fewer Or More Results On Zapimoveis.com.br

Pattern #36: Fewer Or More Results - Variant A
Pattern #36: Fewer Or More Results - Variant B

In this experiment on a listing page, the search was expanded to show more listings (variation B). Conditionally, if there were fewer than 36 results, set basic filters such as number bedrooms and bathrooms were expanded and appended to the results. Hence if someone chose 2 bedrooms and 2 bathrooms in variation A, they would only see listing with that filter. In variation B however they would first see the filtered results, and later they would also see results with 3 or more of each. 

Total Sample: 1,907,053 • Statistical Power at 5% MDE: 99.9%

Test #292 on Backstage.com by Stanley ZuoStanley Zuo Apr 13, 2020 Desktop Mobile Listing X.X% Sales

Stanley Tested Pattern #24: Visible Availability On Backstage.com

Pattern #24: Visible Availability - Variant A
Pattern #24: Visible Availability - Variant B

The core hypothesis of this experiment was that by showing clear availability (in green text) beside each casting call, more users would apply and become premium members. The experiment reports on two metrics: application starts (the first progression metric), and premium membership sales (measured a few steps further in the funnel).

Total Sample: 180,098 • Statistical Power at 5% MDE: 16.8%

Test #284 on Thomasnet.com by Julian GaviriaJulian Gaviria Feb 19, 2020 Desktop Mobile Listing X.X% Leads

Julian Tested Pattern #78: Tags, Badges And Structured Information On Thomasnet.com

Pattern #78: Tags, Badges And Structured Information - Variant A
Pattern #78: Tags, Badges And Structured Information - Variant B

In this experiment, structured data tags were displayed on a listing page to help potential buyers make better decisions. The additional information about the listed companies included: annual revenue, employee count, and year of establishment.

Total Sample: 241,428 • Statistical Power at 5% MDE: 13%

Test #282 on Thomasnet.com by Julian GaviriaJulian Gaviria Feb 07, 2020 Desktop Mobile Listing X.X% Leads

Julian Tested Pattern #51: Shortcut Buttons On Thomasnet.com

Pattern #51: Shortcut Buttons - Variant A
Pattern #51: Shortcut Buttons - Variant B

In this experiment, a contact button was added to a listing / search results page to make it faster to contact a company. This same button was also visible on the company detail page.

Total Sample: 163,317 • Statistical Power at 5% MDE: 9.7%

Test #281 on Backstage.com by Stanley ZuoStanley Zuo Jan 31, 2020 Desktop Listing X.X% Sales

Stanley Tested Pattern #116: Links Or Buttons On Backstage.com

Pattern #116: Links Or Buttons - Variant A
Pattern #116: Links Or Buttons - Variant B

In this experiment, multiple view detail links for a listing tile were turned into higher contrast buttons. 

Total Sample: 386,439 • Statistical Power at 5% MDE: 99.2%

Test #270 on Dentalplans.com by J.R. HernandezJ.R. Hernandez Nov 19, 2019 Desktop Listing X.X% Sales

J.R. Tested Pattern #37: List Or Grid View On Dentalplans.com

Pattern #37: List Or Grid View - Variant A
Pattern #37: List Or Grid View - Variant B

In this experiment, a list view was tested against a grid view.

Total Sample: 264,000 • Statistical Power at 5% MDE: 48.1%

Test #268 on Backstage.com by Stanley ZuoStanley Zuo Nov 08, 2019 Mobile Listing X.X% Sales

Stanley Tested Pattern #14: Exposed Menu Options On Backstage.com

Pattern #14: Exposed Menu Options - Variant A
Pattern #14: Exposed Menu Options - Variant B

The change in this experiment was an exposed SEO panel (B) with a number of clickable filter options. 

Total Sample: 208,456 • Statistical Power at 5% MDE: 97%

Test #262 on Thomasnet.com by Julian GaviriaJulian Gaviria Oct 03, 2019 Desktop Mobile Listing X.X% Leads

Julian Tested Pattern #32: Condensed List On Thomasnet.com

Pattern #32: Condensed List - Variant A
Pattern #32: Condensed List - Variant B

In this experiment, the B version condensed the company listings. This was done by showing less of the description and introducing a "more" and "less" dynamic links that would expand and collapse the description.

Total Sample: 105,732 • Statistical Power at 5% MDE: 8.6%