All Latest 634 A/B Tests

Screen

Device

Metric

Tag

Powered for

Only Significant

p-value < 0.05

Test #218 on Yummly.com by Kimberly Cheung Jan 14, 2019 Desktop Mobile Home & Landing X.X% Signups

Kimberly Tested Pattern #94: Visible Search On Yummly.com

Hypothesis: Anonymous users can't use global search while on the home promo page. We believe that if we show a global search bar to anonymous users, it presents a higher converting funnel (guided search) and will increase our sign-up rates significantly.

Control (A): Logged out users don't see global search bar.

Variant (B): Logged out users see global search bar. After searching for a keyword, the signup funnel starts with a more personalized reason to continue the signup process.

Total Sample: 20,122 • Statistical Power at 5% MDE: 13.8%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Get Access to All Test Results

Test #214 on Yummly.com by Marcos Ciarrocchi Dec 07, 2018 Mobile Signup X.X% Signups

Marcos Tested Pattern #91: Forced Action On Yummly.com

In this experiment, the highly prominent "skip" button was replaced with a less prominent text link. The copy of the skip text link also clarified the consequence of the action - losing out on personalization benefits. More so, the habitual top-right cancel icon was also removed.

Test #210 on Bomgar.com by Lee Elkins Nov 13, 2018 Desktop Mobile Home & Landing X.X% Signups

Lee Tested Pattern #64: Tunnel On Bomgar.com

In this experiment, the header navigation links were removed on a landing page in order to provide more focus to the signup form.

Total Sample: 13,871 • Statistical Power at 5% MDE: 6.3%

Test #206 on Yummly.com by Kimberly Cheung Oct 25, 2018 Desktop Mobile Signup X.X% Signups

Kimberly Tested Pattern #40: Blurred Product Background On Yummly.com

This test ran on the first step of a multiple signup funnel and only changed the background to a blurred image.

Total Sample: 138,106 • Statistical Power at 5% MDE: 77.4%

Test #198 on Bomgar.com by Lee Elkins Sep 03, 2018 Desktop Signup X.X% Signups

Lee Tested Pattern #40: Blurred Product Background On Bomgar.com

Total Sample: 1,893 • Statistical Power at 5% MDE: 21.3%

Test #195 on Yummly.com by Kimberly Cheung Aug 13, 2018 Desktop Mobile Listing X.X% Signups

Kimberly Tested Pattern #78: Tags, Badges And Structured Information On Yummly.com

Total Sample: 371,274 • Statistical Power at 5% MDE: 99.9%

Test #194 on Bomgar.com by Lee Elkins Aug 08, 2018 Desktop Mobile Home & Landing X.X% Signups

Lee Tested Pattern #22: Empowering Headline On Bomgar.com

Total Sample: 12,953 • Statistical Power at 5% MDE: 6.6%

Test #193 on Yummly.com by Marcos Ciarrocchi Aug 07, 2018 Desktop Mobile Signup X.X% Signups

Marcos Tested Pattern #91: Forced Action On Yummly.com

In this experiment, the presence of an additional "skip all" text link was tested on a multiple step signup flow. The skip all link allowed users to bypass personalization questions and go straight to their app dashboard. The control (A) shows its presence, and in variant B we can see it was removed.

Total Sample: 1,335,347 • Statistical Power at 5% MDE: 99.9%

Test #189 on Yummly.com by Kimberly Cheung Jul 23, 2018 Desktop Mobile Signup X.X% Signups

Kimberly Tested Pattern #7: Social Counts On Yummly.com

Total Sample: 259,933 • Statistical Power at 5% MDE: 98.8%

Test #184 on Bomgar.com by Lee Elkins Jun 25, 2018 Desktop Mobile Home & Landing X.X% Signups

Lee Tested Pattern #15: Bulleted Reassurances On Bomgar.com

Total Sample: 5,455 • Statistical Power at 5% MDE: 5.7%

Test #183 on Trydesignlab.com by Daniel Shapiro Jun 19, 2018 Desktop Mobile Product X.X% Signups

Daniel Tested Pattern #46: Pay Later On Trydesignlab.com

Total Sample: 5,495 • Statistical Power at 5% MDE: 4.9%

Test #181 on Yummly.com by Kimberly Cheung Jun 04, 2018 Desktop Mobile Signup X.X% Signups

Kimberly Tested Pattern #7: Social Counts On Yummly.com

Total Sample: 175,247 • Statistical Power at 5% MDE: 61.1%

Test #179 on Yummly.com by Marcos Ciarrocchi May 18, 2018 Desktop Listing X.X% Signups

Marcos Tested Pattern #13: Centered Forms & Buttons On Yummly.com

Total Sample: 391,537 • Statistical Power at 5% MDE: 99.5%

Test #173 on Rollbar.com by Mike Smith May 08, 2018 Desktop Mobile Home & Landing X.X% Signups

Mike Tested Pattern #22: Empowering Headline On Rollbar.com

Total Sample: 8,433 • Statistical Power at 5% MDE: 4.3%

Test #166 on Olark.com by Sunir Shah Apr 05, 2018 Desktop Mobile Home & Landing X.X% Signups

Sunir Tested Pattern #12: Payment First On Olark.com

Olark's homepage was directing users towards a free forever trial - with followup nudges to upgrade some days later. The variation B of the experiment tested a more aggressive push to a pricing page page - with options to select a paid plan, as well as provide a credit card (getting the payment conversation out of the way sooner).

Total Sample: 61,916 • Statistical Power at 5% MDE: 7.3%

Test #165 on Bomgar.com by Lee Elkins Apr 04, 2018 Desktop Signup X.X% Signups