All Latest 645 A/B Tests

Test #339 on Expertinstitute.com by Ardit VeliuArdit Veliu Feb 23, 2021 Desktop Home & Landing X.X% Leads

Ardit Tested Pattern #33: Example Situations On Expertinstitute.com

Pattern #33: Example Situations - Variant A
Pattern #33: Example Situations - Variant B

In this experiment, a number of use cases examples were added to illustrate the situations in which experts could help. 

Total Sample: 8,388 • Statistical Power at 5% MDE: 7.6%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #338 on Umbraco.com by Lars Skjold IversenLars Skjold Iversen Jan 29, 2021 Desktop Mobile Home & Landing X.X% Signups

Lars Tested Pattern #63: Trust Seals On Umbraco.com

Pattern #63: Trust Seals - Variant A
Pattern #63: Trust Seals - Variant B

In this experiment, the variation added three G2 badges or awards. The intent was to measure the impact of this change on signups for Umbraco.

Total Sample: 8,133 • Statistical Power at 5% MDE: 5.5%

Test #336 on Backstage.com by Stanley ZuoStanley Zuo Jan 28, 2021 Desktop Mobile Listing X.X% Sales

Stanley Tested Pattern #51: Shortcut Buttons On Backstage.com

Pattern #51: Shortcut Buttons - Variant A
Pattern #51: Shortcut Buttons - Variant B

In this experiment, a listing page was expanded to show two actions (apply and view details) instead of a single one (view details only). This variation enabled users with a shortcut action to apply for roles one step earlier (and start membership flows for new users). 

Total Sample: 159,634 • Statistical Power at 5% MDE: 16.1%

Test #334 on Thomasnet.com by Kyle PhillipsKyle Phillips Jan 25, 2021 Desktop Mobile Global X.X% Progression

Kyle Tested Pattern #2: Icon Labels On Thomasnet.com

Pattern #2: Icon Labels - Variant A
Pattern #2: Icon Labels - Variant B

This experiment measured the impact of adding text labels to three icon-only nav items. 

Total Sample: 60,466 • Statistical Power at 5% MDE: 11.1%

Test #331 by Jakub LinowskiJakub Linowski Dec 30, 2020 Desktop Mobile Product X.X% Sales

Jakub Tested Pattern #121: Free Shipping

Pattern #121: Free Shipping - Variant A
Pattern #121: Free Shipping - Variant B

In this little experiment, an extra "Free Shipping" message was added on a product page. It's prominence was increased by using white copy on a darker blue background. Impact on adds-to-cart and total sales was measured.

Total Sample: 96,962 • Statistical Power at 5% MDE: 83.6%

Test #332 by Jakub LinowskiJakub Linowski Dec 30, 2020 Desktop Mobile Product X.X% Sales

Jakub Tested Pattern #121: Free Shipping

Pattern #121: Free Shipping - Variant A
Pattern #121: Free Shipping - Variant B

In this experiment, an extra "Free Shipping" message was added on a product page - at the top of the buy box with an add-to-cart call to action. It's prominence was increased by using white copy on a darker blue background. Impact on adds-to-cart and total sales was measured.

Total Sample: 97,277 • Statistical Power at 5% MDE: 83.7%

Test #329 on Snocks.com by Samuel HessSamuel Hess Dec 23, 2020 Mobile Home & Landing X.X% Sales

Samuel Tested Pattern #14: Exposed Menu Options On Snocks.com

Pattern #14: Exposed Menu Options - Variant A
Pattern #14: Exposed Menu Options - Variant B

In this homepage experiment, a series of product categories were shown more visible near the top of the screen (instead of only being shown inside the hamburger menu). They linked up to corresponding listing pages with such items as: gifts, short socks, long socks, and underwear. Impact on adds-to-cart and total sales was measured.

Total Sample: 58,462 • Statistical Power at 5% MDE: 40.4%

Test #328 on Umbraco.com by Lars Skjold IversenLars Skjold Iversen Dec 21, 2020 Desktop Mobile Home & Landing X.X% Signups

Lars Tested Pattern #60: Repeated Bottom Call To Action On Umbraco.com

Pattern #60: Repeated Bottom Call To Action - Variant A
Pattern #60: Repeated Bottom Call To Action - Variant B

In this experiment, a trial signup section was added at the bottom of Umbraco's long homepage (CMS business). The experiment measured the impact on trial signups.

Total Sample: 57,371 • Statistical Power at 5% MDE: 8.4%

Test #327 on Backstage.com by Stanley ZuoStanley Zuo Nov 26, 2020 Desktop Signup X.X% Signups

Stanley Tested Pattern #120: Supporting Theme Images On Backstage.com

Pattern #120: Supporting Theme Images - Variant A
Pattern #120: Supporting Theme Images - Variant B

In this experiment, an aspirational photo was shown on the right side panel - reinforcing the theme of casting calls. The experiment measured progression to the next step and completed signups.

Total Sample: 11,009 • Statistical Power at 5% MDE: 53%

Test #326 on Thomasnet.com by Kyle PhillipsKyle Phillips Nov 25, 2020 Desktop Mobile Content X.X% Progression

Kyle Tested Pattern #41: Sticky Call To Action On Thomasnet.com

Pattern #41: Sticky Call To Action - Variant A
Pattern #41: Sticky Call To Action - Variant B

In this simple experiment on an article page, the variation slid out a sticky call to action linking to the next article. The sliding interaction triggered after some scrolling threshold (around 1000px or so). Afterwards, the sticky call to action maintained its floating position. The experiment measured clicks on this "next article" button.

Total Sample: 7,999 • Statistical Power at 5% MDE: 4.1%

Test #325 on Snocks.com by Samuel HessSamuel Hess Nov 24, 2020 Desktop Global X.X% Sales

Samuel Tested Pattern #45: Benefit Bar On Snocks.com

Pattern #45: Benefit Bar - Variant A
Pattern #45: Benefit Bar - Variant B

In this experiment, a set of reassurances and reviews were added in the header of this ecommerce website. Translating from German, these read: "Anti Hole Guarantee", "Free Shipping" and "X Ratings out of Y Reviews".

Total Sample: 11,349 • Statistical Power at 5% MDE: 14%

Test #322 on Thomasnet.com by Kyle PhillipsKyle Phillips Oct 27, 2020 Desktop Mobile Product X.X% Signups

Kyle Tested Pattern #82: Onboarding Callouts On Thomasnet.com

Pattern #82: Onboarding Callouts - Variant A
Pattern #82: Onboarding Callouts - Variant B

This experiment variation prompted users to save (bookmark) a company profile on a company detail page. Clicking on the save feature while logged out, would prompt a registration modal. Hence the save feature acted as an extra reason to signup. The number of people engaging or interacting with the feature was measured, as well as registrations.

Total Sample: 279,729 • Statistical Power at 5% MDE: 23.6%

Test #320 by Jakub LinowskiJakub Linowski Oct 20, 2020 Desktop Checkout X.X% Sales

Jakub Tested Pattern #49: Above The Fold Call To Action

Pattern #49: Above The Fold Call To Action - Variant A
Pattern #49: Above The Fold Call To Action - Variant B

An extra "Place Order" button was duplicated above the fold on this checkout page. The control had a similar button further down at the bottom of the screen. The impact on total sales was measured from this change.

Total Sample: 2,908 • Statistical Power at 5% MDE: 97.4%

Test #319 on Backstage.com by Stanley ZuoStanley Zuo Sep 30, 2020 Desktop Pricing X.X% Revenue

Stanley Tested Pattern #113: More Or Fewer Plans On Backstage.com

Pattern #113: More Or Fewer Plans - Variant A
Pattern #113: More Or Fewer Plans - Variant B

In this experiment, a 3 plan vs 2 plan pricing page was shown to potential customers. Impact on sales and revenue were measured.

Test #318 on Thomasnet.com by Kyle PhillipsKyle Phillips Sep 29, 2020 Desktop Mobile Content X.X% Signups

Kyle Tested Pattern #60: Repeated Bottom Call To Action On Thomasnet.com

Pattern #60: Repeated Bottom Call To Action - Variant A
Pattern #60: Repeated Bottom Call To Action - Variant B

In this experiment, a simple link to a newsletter signup landing page was added at the bottom of an article. The newsletter landing page then encouraged users to provide their email address for future article updates.

Total Sample: 108,360 • Statistical Power at 5% MDE: 4.8%

Test #99 on Vivareal.com.br by Rodrigo MauésRodrigo Maués Sep 23, 2020 Desktop Mobile Product X.X% Leads

Rodrigo Tested Pattern #24: Visible Availability On Vivareal.com.br

Pattern #24: Visible Availability - Variant A
Pattern #24: Visible Availability - Variant B

In this experiment, a lead form on a listing page showed whether an agent was recently online or not. The diplayed had two statuses: either indicating that someone is online now, or the most recent time they were online in minutes.

Total Sample: 249,954 • Statistical Power at 5% MDE: 88.6%

Test #315 on Backstage.com by Stanley ZuoStanley Zuo Aug 22, 2020 Mobile Signup X.X% Sales

Stanley Tested Pattern #7: Social Counts On Backstage.com

Pattern #7: Social Counts - Variant A
Pattern #7: Social Counts - Variant B

In this experiment, a dynamic number of job postings was displayed during the signup process - reinforcing the value of signing up for membership access.

Total Sample: 43,751 • Statistical Power at 5% MDE: 22.6%

Test #314 on Zapimoveis.com.br by Vinicius Barros PeixotoVinicius Barros Peixoto Aug 21, 2020 Desktop Mobile Product X.X% Leads

Vinicius Tested Pattern #43: Long Titles On Zapimoveis.com.br

Pattern #43: Long Titles - Variant A
Pattern #43: Long Titles - Variant B

In this experiment, a dynamic page title was generated and added at the top of the screen. The first few words from a property description were used to dynamically generate these titles. The effect on leads was measured.

Total Sample: 6,533,516 • Statistical Power at 5% MDE: 99.9%

Test #308 on Umbraco.com by Lars Skjold IversenLars Skjold Iversen Jul 23, 2020 Desktop Home & Landing X.X% Signups

Lars Tested Pattern #4: Testimonials On Umbraco.com

Pattern #4: Testimonials - Variant A
Pattern #4: Testimonials - Variant B

In this experiment, three testimonials were added mid way though on a CMS landing page. At the end of the customer testimonials an additional trial signup button was also added - which was also the primary metric. 

Total Sample: 107,783 • Statistical Power at 5% MDE: 12.1%

Test #306 on Backstage.com by Stanley ZuoStanley Zuo Jul 09, 2020 Desktop Mobile Pricing X.X% Sales

Stanley Tested Pattern #69: Autodiscounting On Backstage.com

Pattern #69: Autodiscounting - Variant A
Pattern #69: Autodiscounting - Variant B

In this experiment, the only change was an added message at the top of the pricing screen, clarifying that there is an active discount on a yearly plan. The discount was already communicated with a strike-through price on the control version as well. The variation simply emphasized this aggressively. 

Total Sample: 15,106 • Statistical Power at 5% MDE: 17.8%