Access GoodUI through our MCP serverNEW Learn more Access GoodUI through our MCP serverNEW

All Latest 650 A/B Tests

MOST RECENT TESTS

Test #133 on Bing.com by Ronny KohaviRonny Kohavi Dec 13, 2017 Desktop Mobile Listing X.X% Revenue

Ronny Tested Pattern #43: Long Titles On Bing.com

Pattern #43: Long Titles - Variant A
Pattern #43: Long Titles - Variant B

In 2012 a Microsoft employee working on Bing had an idea about changing the way the search engine displayed ad headlines. Developing it wouldn’t require much effort—just a few days of an engineer’s time—but it was one of hundreds of ideas proposed, and the program managers deemed it a low priority. So it languished for more than six months, until an engineer, who saw that the cost of writing the code for it would be small, launched a simple online controlled experiment—an A/B test—to assess its impact. Within hours the new headline variation was producing abnormally high revenue, triggering a “too good to be true” alert.

HBR, September–October 2017 Issue, https://hbr.org/2017/09/the-surprising-power-of-online-experiments

Note: This experiment was a solid success and replicated multiple times over a period of months. It worked at Bing and had a profound influence. The only reason why we atributed a 0.25 point (a "Maybe") was because we don't have the exact sample size and conversion data.

 

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #132 on Sjvc.edu by Phillip BarnesPhillip Barnes Dec 12, 2017 Mobile Home & Landing X.X% Leads

Phillip Tested Pattern #41: Sticky Call To Action On Sjvc.edu

Pattern #41: Sticky Call To Action - Variant A
Pattern #41: Sticky Call To Action - Variant B

In this test, a footer with a button to a lead form was turned into a floating one.

Total Sample: 1,437 • Statistical Power at 5% MDE: 4.2%

Test #131 on Kenhub.com by Niels HapkeNiels Hapke Dec 11, 2017 Desktop Home & Landing X.X% Signups

Niels Tested Pattern #11: Gradual Reassurance On Kenhub.com

Pattern #11: Gradual Reassurance - Variant A
Pattern #11: Gradual Reassurance - Variant B
Total Sample: 121,291 • Statistical Power at 5% MDE: 98.8%

Test #130 on Kenhub.com by Niels HapkeNiels Hapke Dec 07, 2017 Desktop Pricing X.X% Revenue

Niels Tested Pattern #17: Least Or Most Expensive First On Kenhub.com

Pattern #17: Least Or Most Expensive First - Variant A
Pattern #17: Least Or Most Expensive First - Variant B

In this experiment, the plans were sorted by the most expensive first, left to right (variation).

Test #128 on Acousticalsurfaces.c... by Nina BayattiNina Bayatti Dec 02, 2017 Desktop Signup X.X% Leads

Nina Tested Pattern #20: Canned Response On Acousticalsurfaces.c...

Pattern #20: Canned Response - Variant A
Pattern #20: Canned Response - Variant B
Total Sample: 2,951 • Statistical Power at 5% MDE: 7.1%

Test #127 on Glass.net by Mark FreedleMark Freedle Nov 23, 2017 Desktop Mobile Listing X.X% Leads

Mark Tested Pattern #42: Countdown Timer On Glass.net

Pattern #42: Countdown Timer - Variant A
Pattern #42: Countdown Timer - Variant B
Total Sample: 61,034 • Statistical Power at 5% MDE: 23.7%

Test #126 on Rollbar.com by Mike SmithMike Smith Nov 22, 2017 Desktop Signup X.X% Signups

Mike Tested Pattern #40: Blurred Product Background On Rollbar.com

Pattern #40: Blurred Product Background - Variant A
Pattern #40: Blurred Product Background - Variant B
Total Sample: 2,205 • Statistical Power at 5% MDE: 12.4%

Test #125 on Vivareal.com.br by Rodrigo MauésRodrigo Maués Nov 13, 2017 Desktop Product X.X% Leads

Rodrigo Tested Pattern #38: Faded Background Form On Vivareal.com.br

Pattern #38: Faded Background Form - Variant A
Pattern #38: Faded Background Form - Variant B
Total Sample: 2,628,666 • Statistical Power at 5% MDE: 99.9%

Test #120 on Vivareal.com.br by Rodrigo MauésRodrigo Maués Nov 01, 2017 Desktop Listing X.X% Leads

Rodrigo Tested Pattern #37: List Or Grid View On Vivareal.com.br

Pattern #37: List Or Grid View - Variant A
Pattern #37: List Or Grid View - Variant B
Total Sample: 1,847,933 • Statistical Power at 5% MDE: 99.9%

Test #119 on Vivareal.com.br by Rodrigo MauésRodrigo Maués Nov 01, 2017 Desktop Listing X.X% Leads

Rodrigo Tested Pattern #36: Fewer Or More Results On Vivareal.com.br

Pattern #36: Fewer Or More Results - Variant A
Pattern #36: Fewer Or More Results - Variant B
Total Sample: 2,452,622 • Statistical Power at 5% MDE: 99.9%

Test #117 on Ssdnodes.com by Matt ConnorMatt Connor Oct 02, 2017 Desktop Shopping Cart X.X% Sales

Matt Tested Pattern #21: What It's Worth On Ssdnodes.com

Pattern #21: What It's Worth - Variant A
Pattern #21: What It's Worth - Variant B
Total Sample: 4,000 • Statistical Power at 5% MDE: 7.9%

Test #116 on Homewarrantyreviews.... by Sumantha ShankaranarayanaSumantha Shankaranarayana Oct 01, 2017 Desktop Home & Landing X.X% Leads

Sumantha Tested Pattern #33: Example Situations On Homewarrantyreviews....

Pattern #33: Example Situations - Variant A
Pattern #33: Example Situations - Variant B
Total Sample: 5,581 • Statistical Power at 5% MDE: 8.2%

Test #118 on Akademiafotografii.p... by Grzegorz JancewiczGrzegorz Jancewicz Oct 01, 2017 Desktop Product X.X% Leads

Grzegorz Tested Pattern #34: Open In A New Tab On Akademiafotografii.p...

Pattern #34: Open In A New Tab - Variant A
Pattern #34: Open In A New Tab - Variant B

The only change in this test was the way links were opened on this course listings page. Variation A opened links in the same window, and variation B opened them in a new tab (and focusing on the newly opened screen).

Total Sample: 3,923 • Statistical Power at 5% MDE: 5.8%

Test #113 on Vivareal.com.br by Rodrigo MauésRodrigo Maués Sep 01, 2017 Desktop Listing X.X% Leads

Rodrigo Tested Pattern #31: Instant Filter Results On Vivareal.com.br

Pattern #31: Instant Filter Results - Variant A
Pattern #31: Instant Filter Results - Variant B
Total Sample: 238,851 • Statistical Power at 5% MDE: 99.7%

Test #115 on Akademiafotografii.p... by Grzegorz JancewiczGrzegorz Jancewicz Sep 01, 2017 Desktop Product X.X% Leads

Grzegorz Tested Pattern #4: Testimonials On Akademiafotografii.p...

Pattern #4: Testimonials - Variant A
Pattern #4: Testimonials - Variant B
Total Sample: 13,817 • Statistical Power at 5% MDE: 9.8%

Test #114 on Akademiafotografii.p... by Grzegorz JancewiczGrzegorz Jancewicz Sep 01, 2017 Desktop Listing X.X% Leads

Grzegorz Tested Pattern #32: Condensed List On Akademiafotografii.p...

Pattern #32: Condensed List - Variant A
Pattern #32: Condensed List - Variant B
Total Sample: 3,363 • Statistical Power at 5% MDE: 5.3%

Test #100 on Vivareal.com.br by Rodrigo MauésRodrigo Maués Aug 01, 2017 Mobile Product X.X% Leads

Rodrigo Tested Pattern #20: Canned Response On Vivareal.com.br

Pattern #20: Canned Response - Variant A
Pattern #20: Canned Response - Variant B
Total Sample: 203,704 • Statistical Power at 5% MDE: 52.6%

Test #93 on Examine.com by Martin WongMartin Wong Aug 01, 2017 Desktop Home & Landing X.X% Sales

Martin Tested Pattern #29: Surfaced Content On Examine.com

Pattern #29: Surfaced Content - Variant A
Pattern #29: Surfaced Content - Variant B
Total Sample: 102,416 • Statistical Power at 5% MDE: 6.1%

Test #112 on Glass.net by Mark FreedleMark Freedle Jul 14, 2017 Desktop Mobile Signup X.X% Leads

Mark Tested Pattern #20: Canned Response On Glass.net

Pattern #20: Canned Response - Variant A
Pattern #20: Canned Response - Variant B
Total Sample: 13,197 • Statistical Power at 5% MDE: 21.2%

Test #111 on Trydesignlab.com by Daniel ShapiroDaniel Shapiro Jul 01, 2017 Desktop Home & Landing X.X% Signups

Daniel Tested Pattern #58: Full Height False Bottom On Trydesignlab.com

Pattern #58: Full Height False Bottom - Variant A
Pattern #58: Full Height False Bottom - Variant B
Total Sample: 7,003 • Statistical Power at 5% MDE: 10.3%