All Latest 634 A/B Tests

MOST RECENT TESTS

Test #235 on Thomasnet.com by Julian GaviriaJulian Gaviria Apr 02, 2019 Desktop Mobile Home & Landing X.X% Leads

Julian Tested Pattern #102: Expanded Or Condensed Layout On Thomasnet.com

- Variant A
- Variant B

In this experiment, the layout was condensed from a taller to a shorter one.
Total Sample: 11,567 • Statistical Power at 5% MDE: 17.3%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Get Access to All Test Results

Test #234 on by Alex JamesAlex James Apr 01, 2019 Desktop Listing X.X% Leads

Alex Tested Pattern #101: Search Keyword Highlighting

- Variant A
- Variant B

In this experiment, a different style for keyword highlighting was used.
Total Sample: 738,295 • Statistical Power at 5% MDE: 99.9%

Test #233 on Annonce.cz by Ondřej IlinčevOndřej Ilinčev Apr 01, 2019 Desktop Signup X.X% Sales

Ondřej Tested Pattern #100: Postponed Registration On Annonce.cz

- Variant A
- Variant B

In this experiment, the old flow had a login / register / guest checkout as a second step. The variant B flow removed this step and treated everyone as a guest (and later offered to login / register). In the B version guests had a voluntary registration on the thank you page and in a confirmation email. 

 
Total Sample: 12,146 • Statistical Power at 5% MDE: 74%

Test #232 on Yoast.com by Sjardo JanssenSjardo Janssen Mar 15, 2019 Desktop Mobile Checkout X.X% Sales

Sjardo Tested Pattern #6: Customer Star Ratings On Yoast.com

- Variant A
- Variant B

In this experiment, the Yoast team replaced one of the bulleted reassurances on their checkout page ("4.9 out of 5 stars" vs "24/7 Free support"). Raising the question - is free support or high reviews valued more? - Thanks Sjardo & Meike for sharing!
Total Sample: 20,661 • Statistical Power at 5% MDE: 30.3%

Test #231 on Glass.net by Mark FreedleMark Freedle Mar 14, 2019 Desktop Mobile Signup X.X% Leads

Mark Tested Pattern #20: Canned Response On Glass.net

- Variant A
- Variant B
Total Sample: 1,342 • Statistical Power at 5% MDE: 6.4%

Test #230 on Goodui.org by Jakub LinowskiJakub Linowski Mar 09, 2019 Desktop Mobile Listing X.X% Engagement

Jakub Tested Pattern #56: Hover Button On Goodui.org

- Variant A
- Variant B

In this test we tested onhover buttons (variant) versus more traditional always exposed and visible ones.
Total Sample: 30,699 • Statistical Power at 5% MDE: 98.1%

Test #229 on by Jakub LinowskiJakub Linowski Mar 08, 2019 Desktop Mobile Checkout X.X% Sales

Jakub Tested Pattern #99: Progress Bar

- Variant A
- Variant B
Total Sample: 11,214 • Statistical Power at 5% MDE: 13%

Test #228 on by Jakub LinowskiJakub Linowski Mar 05, 2019 Desktop Mobile Checkout X.X% Sales

Jakub Tested Pattern #99: Progress Bar

- Variant A
- Variant B

The variation added a progress bar to one of the checkout steps for a ticket ordering site.
Total Sample: 12,961 • Statistical Power at 5% MDE: 14.7%

Test #227 on Volders.de by Alexander KriegerAlexander Krieger Mar 04, 2019 Desktop Signup X.X% Leads

Alexander Tested Pattern #99: Progress Bar On Volders.de

- Variant A
- Variant B

In this experiment, a simple progress bar was added to a 2nd step of contract cancellation funnel. The progress bar used separate styles to show which steps were completed, what the current step was, as well as the future step.

Translation of the 3 steps from German are as follows:

  1. Fill out the termination
  2. Send termination
  3. Proof of termination
Total Sample: 11,418 • Statistical Power at 5% MDE: 20.6%

Test #226 on Microsoft.com by Ronny KohaviRonny Kohavi Feb 18, 2019 Desktop Product X.X% Sales

Ronny Tested Pattern #96: Single Focus Photos On Microsoft.com

- Variant A
- Variant B

Microsoft Store ran an experiment on the Surface Book 2 product page. The treatment showed a hero image with fewer, yet larger product photos

Test #225 on by Devesh KhanalDevesh Khanal Feb 17, 2019 Desktop Product X.X% Sales

Devesh Tested Pattern #41: Sticky Call To Action

- Variant A
- Variant B
Total Sample: 53,908 • Statistical Power at 5% MDE: 35.7%

Test #224 on by Alex JamesAlex James Feb 11, 2019 Desktop Home & Landing X.X% Progression

Alex Tested Pattern #3: Fewer Form Fields

- Variant A
- Variant B

This experiment reduced the search form by removing the distance field.
Total Sample: 97,074 • Statistical Power at 5% MDE: 99.9%

Test #222 on Thomasnet.com by Julian GaviriaJulian Gaviria Feb 01, 2019 Desktop Listing X.X% Leads

Julian Tested Pattern #7: Social Counts On Thomasnet.com

- Variant A
- Variant B

In this variation, a number of social proof references were added to a signup modal.
Total Sample: 34,676 • Statistical Power at 5% MDE: 47.1%

Test #223 on Volders.de by Alexander KriegerAlexander Krieger Feb 01, 2019 Desktop Mobile Signup X.X% Revenue

Alexander Tested Pattern #12: Payment First On Volders.de

- Variant A
- Variant B

This test deprioritized the free option (kostenlos) of cancelling a contract. It did so by placing it under the paid options as small text link / radio option.

Test #221 on Microsoft.com by Ronny KohaviRonny Kohavi Jan 27, 2019 Desktop Product X.X% Sales

Ronny Tested Pattern #49: Above The Fold Call To Action On Microsoft.com

- Variant A
- Variant B

Microsoft Store ran an experiment on the Office 365 Home product page. The treatment raised the purchase calls to action higher by removing the hero image.

Test #220 on by Alex JamesAlex James Jan 18, 2019 Desktop Listing X.X% Leads

Alex Tested Pattern #34: Open In A New Tab

- Variant A
- Variant B

This experiment measured the effect of opening new listing (job applications) in a new tab, against opening them in the same window. The experiment A-B was inversed to match the pattern (in reality, the original already opened the tabs in a new window).
Total Sample: 55,703 • Statistical Power at 5% MDE: 99.9%

Test #218 on Yummly.com by Kimberly CheungKimberly Cheung Jan 14, 2019 Desktop Mobile Home & Landing X.X% Signups

Kimberly Tested Pattern #94: Visible Search On Yummly.com

- Variant A
- Variant B

Hypothesis: Anonymous users can't use global search while on the home promo page. We believe that if we show a global search bar to anonymous users, it presents a higher converting funnel (guided search) and will increase our sign-up rates significantly.

Control (A): Logged out users don't see global search bar.

Variant (B): Logged out users see global search bar. After searching for a keyword, the signup funnel starts with a more personalized reason to continue the signup process.
Total Sample: 20,122 • Statistical Power at 5% MDE: 13.8%

Test #219 on Mt.com by Vito MediavillaVito Mediavilla Jan 14, 2019 Desktop Mobile Home & Landing X.X% Leads

Vito Tested Pattern #95: Clickable Product Previews On Mt.com

- Variant A
- Variant B
Total Sample: 4,836 • Statistical Power at 5% MDE: 11%

Test #217 on Thomasnet.com by Julian GaviriaJulian Gaviria Jan 03, 2019 Desktop Mobile Home & Landing X.X% Engagement

Julian Tested Pattern #41: Sticky Call To Action On Thomasnet.com

- Variant A
- Variant B
Total Sample: 158,023 • Statistical Power at 5% MDE: 8.9%

Test #215 on Vivareal.com.br by Vinicius Barros PeixotoVinicius Barros Peixoto Dec 21, 2018 Mobile Listing X.X% Leads

Vinicius Tested Pattern #92: Already Viewed Label On Vivareal.com.br

- Variant A
- Variant B

The idea of this test was to add a "Viewed" label on a listing page to indicate listings which have already been viewed by users.
Total Sample: 2,452,026 • Statistical Power at 5% MDE: 99.9%