All Latest 634 A/B Tests

Test #123 on Getninjas.com.br by Rodolfo LugliRodolfo Lugli Feb 11, 2022 Mobile Thank You X.X% Engagement

Rodolfo Tested Pattern #71: Personalized Next Step On Getninjas.com.br

 - Variant A
 - Variant B

In this experiment, two app download buttons were tested against a single OS personalized one. In the control, both branded App Store and Google Play buttons were shown statically. Whereas in the variation a single download (stylized consistently with site wide button styles) button was shown depending on the user's operating system. Impact on application download was measured.

Total Sample: 21,910 Statistical Power at 5% MDE: 77%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #342 on Backstage.com by Stanley ZuoStanley Zuo Feb 28, 2021 Desktop Mobile Listing X.X% Engagement

Stanley Tested Pattern #25: Nagging Results On Backstage.com

 - Variant A
 - Variant B

In this experiment, a registration wall was added on a listing page of casting call profiles. The registration wall appeared after the first 9 listings or so and encouraged users to sign up. Impact on registrations was measured, along with an engagement metric of "posting a job".

Total Sample: 7,934 Statistical Power at 5% MDE: 5.9%

Test #303 on Thomasnet.com by Julian GaviriaJulian Gaviria Jun 26, 2020 Desktop Mobile Global X.X% Engagement

Julian Tested Pattern #14: Exposed Menu Options On Thomasnet.com

 - Variant A
 - Variant B

In this experiment variation, the saved suppliers feature was surfaced in the global navigation.It was already possible to save supplier companies from listing and specific company pages. This experiment aimed to increase the saving functions visibility and possibly increase more leads. 

Total Sample: 3,326,087 Statistical Power at 5% MDE: 62.5%

Test #295 on Thomasnet.com by Julian GaviriaJulian Gaviria Apr 29, 2020 Desktop Mobile Content X.X% Engagement

Julian Tested Pattern #25: Nagging Results On Thomasnet.com

 - Variant A
 - Variant B

In this experiment, blog article pages were hidden behind a registration wall - requing a signup to access. The registration wall would appear after the first paragraph using gradual opacity to cover the rest of the article. We have published the effects of this change on registrations (signups) and on engagement (users viewing other more important company detail pages).

Total Sample: 131,642 Statistical Power at 5% MDE: 26.7%

Test #252 on Thomasnet.com by Julian GaviriaJulian Gaviria Jul 30, 2019 Desktop Mobile Content X.X% Engagement

Julian Tested Pattern #107: Contrast Links & Buttons On Thomasnet.com

 - Variant A
 - Variant B

In this experiment a light blue bold link was tested against a darker blue bold link. 

Total Sample: 142,102 Statistical Power at 5% MDE: 74.3%

Test #251 on Goodui.org by Jakub LinowskiJakub Linowski Jul 25, 2019 Desktop Mobile Content X.X% Engagement

Jakub Tested Pattern #57: Maybe Later On Goodui.org

 - Variant A
 - Variant B

In this experiment, we tested a standard modal (with 2 choices) against a "Maybe Later" one (with 3 choices). One of the choices in the variant allowed users to postpone their decision with a "maybe" which would enable a floating bar at the bottom of the screen. Clicking on any of the "Yes" options would send people to the bottom of the screen with an email signup form. Increasing signup was our primary measure. Both modals also appeared instantly after a page load.

Total Sample: 7,759 Statistical Power at 5% MDE: 7.2%

Test #247 on Thomasnet.com by Julian GaviriaJulian Gaviria Jun 13, 2019 Desktop Mobile Content X.X% Engagement

Julian Tested Pattern #41: Sticky Call To Action On Thomasnet.com

 - Variant A
 - Variant B

In this experiment, the same message (with a link) for signing up to a newsletter was shown in two distinct ways. The control (A) showed the signup message as inline one that preceded the content of the article at the very top. The variant showed the same signup message as a scroll-delayed sticky interaction at the bottom of the screen. The background color of the B variant was also inverted to match the style of the footer.

Total Sample: 76,900 Statistical Power at 5% MDE: 4.6%

Test #240 on Thomasnet.com by Julian GaviriaJulian Gaviria May 16, 2019 Desktop Listing X.X% Engagement

Julian Tested Pattern #13: Centered Forms & Buttons On Thomasnet.com

 - Variant A
 - Variant B

In this experiment, two different positions of the View Catalog button were compared. In version A the button was smaller and on the right. In version B the button was wider and more central. (The actual test was inverted before publishing to match the A-B of the pattern.)

Total Sample: 207,844 Statistical Power at 5% MDE: 31.5%

Test #230 on Goodui.org by Jakub LinowskiJakub Linowski Mar 09, 2019 Desktop Mobile Listing X.X% Engagement

Jakub Tested Pattern #56: Hover Button On Goodui.org

 - Variant A
 - Variant B

In this test we tested onhover buttons (variant) versus more traditional always exposed and visible ones.

Total Sample: 30,699 Statistical Power at 5% MDE: 98.1%

Test #219 on Mt.com by Vito MediavillaVito Mediavilla Jan 14, 2019 Desktop Mobile Home & Landing X.X% Engagement

Vito Tested Pattern #95: Clickable Product Previews On Mt.com

 - Variant A
 - Variant B
Total Sample: 4,836 Statistical Power at 5% MDE: 11%

Test #217 on Thomasnet.com by Julian GaviriaJulian Gaviria Jan 03, 2019 Desktop Mobile Home & Landing X.X% Engagement

Julian Tested Pattern #41: Sticky Call To Action On Thomasnet.com

 - Variant A
 - Variant B
Total Sample: 158,023 Statistical Power at 5% MDE: 8.9%

Test #205 on Msn.com by Ronny KohaviRonny Kohavi Oct 25, 2018 Desktop Home & Landing X.X% Engagement

Ronny Tested Pattern #36: Fewer Or More Results On Msn.com

 - Variant A
 - Variant B

In this experiment, the carousel items were increased from 12 to 16.

Test #182 on Yummly.com by Marcos CiarrocchiMarcos Ciarrocchi Jun 12, 2018 Desktop Listing X.X% Engagement

Marcos Tested Pattern #6: Customer Star Ratings On Yummly.com

 - Variant A
 - Variant B
Total Sample: 673,296 Statistical Power at 5% MDE: 99.9%

Test #173 on Rollbar.com by Mike SmithMike Smith May 08, 2018 Desktop Mobile Home & Landing X.X% Engagement

Mike Tested Pattern #22: Empowering Headline On Rollbar.com

 - Variant A
 - Variant B
Total Sample: 8,433 Statistical Power at 5% MDE: 4.3%