All Latest 632 A/B Tests

MOST RECENT TESTS

Test #309 on Thomasnet.com by Julian GaviriaJulian Gaviria Jul 24, 2020 Desktop Listing X.X% Progression

Julian Tested Pattern #72: Priming Step On Thomasnet.com

 - Variant A
 - Variant B

In this experiment, an extra step was prepended at the beginning of a multiple step signup modal flow. The signup modal would appear on listing pages after requests to contact a listed company. The idea was to prime users with benefits of signing up in order to increase their motivation to do so. The experiment measured the impact on the initial progression (to the step with the email form).

Total Sample: 6,146 Statistical Power at 5% MDE: 12.5%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #308 on Umbraco.com by Lars Skjold IversenLars Skjold Iversen Jul 23, 2020 Desktop Home & Landing X.X% Signups

Lars Tested Pattern #4: Testimonials On Umbraco.com

 - Variant A
 - Variant B

In this experiment, three testimonials were added mid way though on a CMS landing page. At the end of the customer testimonials an additional trial signup button was also added - which was also the primary metric. 

Total Sample: 107,783 Statistical Power at 5% MDE: 12.1%

Test #307 on Volders.de by Michal FiechMichal Fiech Jul 17, 2020 Desktop Thank You X.X% Progression

Michal Tested Pattern #77: Filled Or Ghost Buttons On Volders.de

 - Variant A
 - Variant B

This experiment measured a shallow click goal on a button that would encourage to repeated the action that was just completed (in this case a contract cancellation). In the control version (A) a thank-you screen shows a filled button style, and the variant (B) there was a ghost button. As a note, I also flipped the A-B in this experiment for the purpose of matching it to our ghost button pattern, which means that Volders in fact was starting  out with a ghost button to begin with. 

Total Sample: 53,618 Statistical Power at 5% MDE: 41.7%

Test #306 on Backstage.com by Stanley ZuoStanley Zuo Jul 09, 2020 Desktop Mobile Pricing X.X% Sales

Stanley Tested Pattern #69: Autodiscounting On Backstage.com

 - Variant A
 - Variant B

In this experiment, the only change was an added message at the top of the pricing screen, clarifying that there is an active discount on a yearly plan. The discount was already communicated with a strike-through price on the control version as well. The variation simply emphasized this aggressively. 

Total Sample: 15,106 Statistical Power at 5% MDE: 17.8%

Test #305 on Volders.de by Michal FiechMichal Fiech Jun 30, 2020 Mobile Desktop Home & Landing X.X% Sales

Michal Tested Pattern #94: Visible Search On Volders.de

 - Variant A
 - Variant B

In this experiment, a search input field (to look for companies) along with most popular links (also company names) were displayed on the homepage of a leading contract cancellation service. The control (A) version instead had a button that sent users to a next page where the same selection could be made - only later. The measurable success criteria were the number of paid cancellations - a few steps down the funnel.

Total Sample: 10,350 Statistical Power at 5% MDE: 10.6%

Test #304 on Backstage.com by Stanley ZuoStanley Zuo Jun 29, 2020 Mobile Product X.X% Signups

Stanley Tested Pattern #97: Bigger Form Fields On Backstage.com

 - Variant A
 - Variant B

In this experiment, larger "Apply" buttons were shown on a casting detail page. The application funnel would take users through a series of steps leading to a paid membership subscription. The experiment measured initial progression and account signups (email signups).

Total Sample: 109,372 Statistical Power at 5% MDE: 95.4%

Test #61 on by Someone Someone Jun 26, 2020 Desktop Checkout X.X% Sales

Someone Tested Pattern #9: Multiple Steps

 - Variant A
 - Variant B

In this experiment, a single screen checkout was turned into a series of smaller steps in variation B. This was achieved by showing fewer fields on the first step, and shifting the remaining ones into a 3 step modal popup. The experiment measured successful transactions (sales).

Total Sample: 8,886 Statistical Power at 5% MDE: 11.1%

Test #303 on Thomasnet.com by Julian GaviriaJulian Gaviria Jun 26, 2020 Desktop Mobile Global X.X% Leads

Julian Tested Pattern #14: Exposed Menu Options On Thomasnet.com

 - Variant A
 - Variant B

In this experiment variation, the saved suppliers feature was surfaced in the global navigation.It was already possible to save supplier companies from listing and specific company pages. This experiment aimed to increase the saving functions visibility and possibly increase more leads. 

Total Sample: 3,326,087 Statistical Power at 5% MDE: 62.5%

Test #302 on Volders.de by Michal FiechMichal Fiech Jun 09, 2020 Desktop Mobile Signup X.X% Sales

Michal Tested Pattern #83: Progressive Fields On Volders.de

 - Variant A
 - Variant B

In this experiment a long form (A) was replaced with a progressive form interaction (B). Most of the form fields would appear in a grey-disabled style, until the prerequioste fields were first filled out.

Total Sample: 35,884 Statistical Power at 5% MDE: 51.2%

Test #301 on Zapimoveis.com.br by Vinicius Barros PeixotoVinicius Barros Peixoto May 31, 2020 Desktop Mobile Product X.X% Leads

Vinicius Tested Pattern #21: What It's Worth On Zapimoveis.com.br

 - Variant A
 - Variant B

In this experiment, the B variation property prices were framed using higher and crossed out price points from 12 months ago - achieving a relative discount. A tooltip was also shown which explained the higher price point on hover. The example in the screenshot translates to "2% less compared to 12 months ago". This high-power experiment measured the number of leads that were generated on property (product) screens.

Total Sample: 136,682 Statistical Power at 5% MDE: 97.6%

Test #300 on Volders.de by Michal FiechMichal Fiech May 25, 2020 Desktop Mobile Signup X.X% Progression

Michal Tested Pattern #3: Fewer Form Fields On Volders.de

 - Variant A
 - Variant B

In this experiment, a password field was removed on a contract cancellation form (Volders).

In the control version, users were required to enter their email address and a password. If the email address was associated with an existing account, then the password was used to authenticate the user (and validated). When users entered a new email address, then the password field was used to create a new account. 

In the variation, the password field was removed, as the authentication happened after the conversion itself using other backend mechanisms.

 

Total Sample: 23,356 Statistical Power at 5% MDE: 83.8%

Test #299 on Backstage.com by Stanley ZuoStanley Zuo May 22, 2020 Desktop Mobile Listing X.X% Sales

Stanley Tested Pattern #60: Repeated Bottom Call To Action On Backstage.com

 - Variant A
 - Variant B

In this experiment, at the bottom of a search results screen, a membership join button was added along with 3 encouraging reasons. The experiment measured membership funnel starts, as well as paid membership transactions (sales).

Total Sample: 106,791 Statistical Power at 5% MDE: 12.2%

Test #298 on Zapimoveis.com.br by Vinicius Barros PeixotoVinicius Barros Peixoto May 14, 2020 Desktop Mobile Listing X.X% Leads

Vinicius Tested Pattern #36: Fewer Or More Results On Zapimoveis.com.br

 - Variant A
 - Variant B

In this experiment on a listing page, the search was expanded to show more listings (variation B). Conditionally, if there were fewer than 36 results, set basic filters such as number bedrooms and bathrooms were expanded and appended to the results. Hence if someone chose 2 bedrooms and 2 bathrooms in variation A, they would only see listing with that filter. In variation B however they would first see the filtered results, and later they would also see results with 3 or more of each. 

Total Sample: 1,907,053 Statistical Power at 5% MDE: 99.9%

Test #297 on Trydesignlab.com by Daniel ShapiroDaniel Shapiro May 04, 2020 Desktop Home & Landing X.X% Sales

Daniel Tested Pattern #41: Sticky Call To Action On Trydesignlab.com

 - Variant A
 - Variant B

In this experiment, a sticky "Enroll" button was shown on a course landing page. The button lead to a payment funnel to allow enrolling/paying for a course. The exeperiment measured inital progression into this funnel as well as the deeper completed sales metric.

Total Sample: 19,954 Statistical Power at 5% MDE: 9.2%

Test #296 on Backstage.com by Stanley ZuoStanley Zuo Apr 30, 2020 Mobile Content X.X% Sales

Stanley Tested Pattern #23: Inline Link Nudge On Backstage.com

 - Variant A
 - Variant B

In this simple experiment, a text link to a join page was injected on an article page. The hypothesis was that more users would signup as a result of this subtle trigger.

Total Sample: 411,288 Statistical Power at 5% MDE: 11%

Test #295 on Thomasnet.com by Julian GaviriaJulian Gaviria Apr 29, 2020 Desktop Mobile Content X.X% Engagement

Julian Tested Pattern #25: Nagging Results On Thomasnet.com

 - Variant A
 - Variant B

In this experiment, blog article pages were hidden behind a registration wall - requing a signup to access. The registration wall would appear after the first paragraph using gradual opacity to cover the rest of the article. We have published the effects of this change on registrations (signups) and on engagement (users viewing other more important company detail pages).

Total Sample: 131,642 Statistical Power at 5% MDE: 26.7%

Test #294 on Umbraco.com by Lars Skjold IversenLars Skjold Iversen Apr 23, 2020 Desktop Pricing X.X% Progression

Lars Tested Pattern #115: Pricing Comparison Table On Umbraco.com

 - Variant A
 - Variant B

In this experiment, plan properties on a pricing page were horizontally aligned (for easier comparison). More so, labels and values were also broken on separate lines.

Total Sample: 18,623 Statistical Power at 5% MDE: 5.3%

Test #293 on Backstage.com by Stanley ZuoStanley Zuo Apr 14, 2020 Desktop Mobile Product X.X% Sales

Stanley Tested Pattern #114: Less Or More Visible Prices On Backstage.com

 - Variant A
 - Variant B

In this experiment on a casting call site, pricing information was shown beside the application button. This change shows the effect of setting a price expectation and being more clear that the application process is not free.

Total Sample: 115,061 Statistical Power at 5% MDE: 9.3%

Test #292 on Backstage.com by Stanley ZuoStanley Zuo Apr 13, 2020 Desktop Mobile Listing X.X% Sales

Stanley Tested Pattern #24: Visible Availability On Backstage.com

 - Variant A
 - Variant B

The core hypothesis of this experiment was that by showing clear availability (in green text) beside each casting call, more users would apply and become premium members. The experiment reports on two metrics: application starts (the first progression metric), and premium membership sales (measured a few steps further in the funnel).

Total Sample: 180,098 Statistical Power at 5% MDE: 16.8%

Test #291 on Elevate App by Jesse GerminarioJesse Germinario Mar 30, 2020 Mobile Signup X.X% Signups

Jesse Tested Pattern #91: Forced Action

 - Variant A
 - Variant B

This experiment was ran on the initial onboarding screens of the Elevate App - right after installing and launching the app for the first time. The change was the removal of subtle "skip" links that fast tracked users to the signup/login screen (Get Started). Hence in the variation, all users had to scroll through the 4 introductory messages before being asked to create an account.

Total Sample: 135,631 Statistical Power at 5% MDE: 96.6%