All Latest 634 A/B Tests

Test #327 on Backstage.com by Stanley ZuoStanley Zuo Nov 26, 2020 Desktop Signup X.X% Signups

Stanley Tested Pattern #120: Supporting Theme Images On Backstage.com

 - Variant A
 - Variant B

In this experiment, an aspirational photo was shown on the right side panel - reinforcing the theme of casting calls. The experiment measured progression to the next step and completed signups.

Total Sample: 11,009 Statistical Power at 5% MDE: 53%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #326 on Thomasnet.com by Kyle PhillipsKyle Phillips Nov 25, 2020 Desktop Mobile Content X.X% Progression

Kyle Tested Pattern #41: Sticky Call To Action On Thomasnet.com

 - Variant A
 - Variant B

In this simple experiment on an article page, the variation slid out a sticky call to action linking to the next article. The sliding interaction triggered after some scrolling threshold (around 1000px or so). Afterwards, the sticky call to action maintained its floating position. The experiment measured clicks on this "next article" button.

Total Sample: 7,999 Statistical Power at 5% MDE: 4.1%

Test #325 on Snocks.com by Samuel HessSamuel Hess Nov 24, 2020 Desktop Global X.X% Sales

Samuel Tested Pattern #45: Benefit Bar On Snocks.com

 - Variant A
 - Variant B

In this experiment, a set of reassurances and reviews were added in the header of this ecommerce website. Translating from German, these read: "Anti Hole Guarantee", "Free Shipping" and "X Ratings out of Y Reviews".

Total Sample: 11,349 Statistical Power at 5% MDE: 14%

Test #322 on Thomasnet.com by Kyle PhillipsKyle Phillips Oct 27, 2020 Desktop Mobile Product X.X% Signups

Kyle Tested Pattern #82: Onboarding Callouts On Thomasnet.com

 - Variant A
 - Variant B

This experiment variation prompted users to save (bookmark) a company profile on a company detail page. Clicking on the save feature while logged out, would prompt a registration modal. Hence the save feature acted as an extra reason to signup. The number of people engaging or interacting with the feature was measured, as well as registrations.

Total Sample: 279,729 Statistical Power at 5% MDE: 23.6%

Test #320 on by Jakub LinowskiJakub Linowski Oct 20, 2020 Desktop Checkout X.X% Sales

Jakub Tested Pattern #49: Above The Fold Call To Action

 - Variant A
 - Variant B

An extra "Place Order" button was duplicated above the fold on this checkout page. The control had a similar button further down at the bottom of the screen. The impact on total sales was measured from this change.

Total Sample: 2,908 Statistical Power at 5% MDE: 97.4%

Test #319 on Backstage.com by Stanley ZuoStanley Zuo Sep 30, 2020 Desktop Pricing X.X% Revenue

Stanley Tested Pattern #113: More Or Fewer Plans On Backstage.com

 - Variant A
 - Variant B

In this experiment, a 3 plan vs 2 plan pricing page was shown to potential customers. Impact on sales and revenue were measured.

Test #318 on Thomasnet.com by Kyle PhillipsKyle Phillips Sep 29, 2020 Desktop Mobile Content X.X% Signups

Kyle Tested Pattern #60: Repeated Bottom Call To Action On Thomasnet.com

 - Variant A
 - Variant B

In this experiment, a simple link to a newsletter signup landing page was added at the bottom of an article. The newsletter landing page then encouraged users to provide their email address for future article updates.

Total Sample: 108,360 Statistical Power at 5% MDE: 4.8%

Test #99 on Vivareal.com.br by Rodrigo MauésRodrigo Maués Sep 23, 2020 Desktop Mobile Product X.X% Leads

Rodrigo Tested Pattern #24: Visible Availability On Vivareal.com.br

 - Variant A
 - Variant B

In this experiment, a lead form on a listing page showed whether an agent was recently online or not. The diplayed had two statuses: either indicating that someone is online now, or the most recent time they were online in minutes.

Total Sample: 249,954 Statistical Power at 5% MDE: 88.6%

Test #315 on Backstage.com by Stanley ZuoStanley Zuo Aug 22, 2020 Mobile Signup X.X% Sales

Stanley Tested Pattern #7: Social Counts On Backstage.com

 - Variant A
 - Variant B

In this experiment, a dynamic number of job postings was displayed during the signup process - reinforcing the value of signing up for membership access.

Total Sample: 43,751 Statistical Power at 5% MDE: 22.6%

Test #314 on Zapimoveis.com.br by Vinicius Barros PeixotoVinicius Barros Peixoto Aug 21, 2020 Desktop Mobile Product X.X% Leads

Vinicius Tested Pattern #43: Long Titles On Zapimoveis.com.br

 - Variant A
 - Variant B

In this experiment, a dynamic page title was generated and added at the top of the screen. The first few words from a property description were used to dynamically generate these titles. The effect on leads was measured.

Total Sample: 6,533,516 Statistical Power at 5% MDE: 99.9%

Test #308 on Umbraco.com by Lars Skjold IversenLars Skjold Iversen Jul 23, 2020 Desktop Home & Landing X.X% Signups

Lars Tested Pattern #4: Testimonials On Umbraco.com

 - Variant A
 - Variant B

In this experiment, three testimonials were added mid way though on a CMS landing page. At the end of the customer testimonials an additional trial signup button was also added - which was also the primary metric. 

Total Sample: 107,783 Statistical Power at 5% MDE: 12.1%

Test #306 on Backstage.com by Stanley ZuoStanley Zuo Jul 09, 2020 Desktop Mobile Pricing X.X% Sales

Stanley Tested Pattern #69: Autodiscounting On Backstage.com

 - Variant A
 - Variant B

In this experiment, the only change was an added message at the top of the pricing screen, clarifying that there is an active discount on a yearly plan. The discount was already communicated with a strike-through price on the control version as well. The variation simply emphasized this aggressively. 

Total Sample: 15,106 Statistical Power at 5% MDE: 17.8%

Test #303 on Thomasnet.com by Julian GaviriaJulian Gaviria Jun 26, 2020 Desktop Mobile Global X.X% Leads

Julian Tested Pattern #14: Exposed Menu Options On Thomasnet.com

 - Variant A
 - Variant B

In this experiment variation, the saved suppliers feature was surfaced in the global navigation.It was already possible to save supplier companies from listing and specific company pages. This experiment aimed to increase the saving functions visibility and possibly increase more leads. 

Total Sample: 3,326,087 Statistical Power at 5% MDE: 62.5%

Test #301 on Zapimoveis.com.br by Vinicius Barros PeixotoVinicius Barros Peixoto May 31, 2020 Desktop Mobile Product X.X% Leads

Vinicius Tested Pattern #21: What It's Worth On Zapimoveis.com.br

 - Variant A
 - Variant B

In this experiment, the B variation property prices were framed using higher and crossed out price points from 12 months ago - achieving a relative discount. A tooltip was also shown which explained the higher price point on hover. The example in the screenshot translates to "2% less compared to 12 months ago". This high-power experiment measured the number of leads that were generated on property (product) screens.

Total Sample: 136,682 Statistical Power at 5% MDE: 97.6%

Test #299 on Backstage.com by Stanley ZuoStanley Zuo May 22, 2020 Desktop Mobile Listing X.X% Sales

Stanley Tested Pattern #60: Repeated Bottom Call To Action On Backstage.com

 - Variant A
 - Variant B

In this experiment, at the bottom of a search results screen, a membership join button was added along with 3 encouraging reasons. The experiment measured membership funnel starts, as well as paid membership transactions (sales).

Total Sample: 106,791 Statistical Power at 5% MDE: 12.2%

Test #298 on Zapimoveis.com.br by Vinicius Barros PeixotoVinicius Barros Peixoto May 14, 2020 Desktop Mobile Listing X.X% Leads

Vinicius Tested Pattern #36: Fewer Or More Results On Zapimoveis.com.br

 - Variant A
 - Variant B

In this experiment on a listing page, the search was expanded to show more listings (variation B). Conditionally, if there were fewer than 36 results, set basic filters such as number bedrooms and bathrooms were expanded and appended to the results. Hence if someone chose 2 bedrooms and 2 bathrooms in variation A, they would only see listing with that filter. In variation B however they would first see the filtered results, and later they would also see results with 3 or more of each. 

Total Sample: 1,907,053 Statistical Power at 5% MDE: 99.9%

Test #297 on Trydesignlab.com by Daniel ShapiroDaniel Shapiro May 04, 2020 Desktop Home & Landing X.X% Sales

Daniel Tested Pattern #41: Sticky Call To Action On Trydesignlab.com

 - Variant A
 - Variant B

In this experiment, a sticky "Enroll" button was shown on a course landing page. The button lead to a payment funnel to allow enrolling/paying for a course. The exeperiment measured inital progression into this funnel as well as the deeper completed sales metric.

Total Sample: 19,954 Statistical Power at 5% MDE: 9.2%

Test #296 on Backstage.com by Stanley ZuoStanley Zuo Apr 30, 2020 Mobile Content X.X% Sales

Stanley Tested Pattern #23: Inline Link Nudge On Backstage.com

 - Variant A
 - Variant B

In this simple experiment, a text link to a join page was injected on an article page. The hypothesis was that more users would signup as a result of this subtle trigger.

Total Sample: 411,288 Statistical Power at 5% MDE: 11%

Test #293 on Backstage.com by Stanley ZuoStanley Zuo Apr 14, 2020 Desktop Mobile Product X.X% Sales

Stanley Tested Pattern #114: Less Or More Visible Prices On Backstage.com

 - Variant A
 - Variant B

In this experiment on a casting call site, pricing information was shown beside the application button. This change shows the effect of setting a price expectation and being more clear that the application process is not free.

Total Sample: 115,061 Statistical Power at 5% MDE: 9.3%

Test #292 on Backstage.com by Stanley ZuoStanley Zuo Apr 13, 2020 Desktop Mobile Listing X.X% Sales

Stanley Tested Pattern #24: Visible Availability On Backstage.com

 - Variant A
 - Variant B

The core hypothesis of this experiment was that by showing clear availability (in green text) beside each casting call, more users would apply and become premium members. The experiment reports on two metrics: application starts (the first progression metric), and premium membership sales (measured a few steps further in the funnel).

Total Sample: 180,098 Statistical Power at 5% MDE: 16.8%