All Latest 644 A/B Tests

Test #442 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Nov 27, 2022 Desktop Mobile Home & Landing X.X% Progression

Daria Tested Pattern #4: Testimonials On Volders.de

Pattern #4: Testimonials - Variant A
Pattern #4: Testimonials - Variant B

In this experiment, three testimonials were appended at the bottom of landing pages of a contract cancelation service (paid). These testimonials were also shown throughout the complete signup funnel (4 more steps). Impact on progression (step 2) and final completed purchases were measured.

Total Sample: 73,143 • Statistical Power at 5% MDE: 83.7%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #443 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Nov 27, 2022 Desktop Mobile Home & Landing X.X% Progression

Daria Tested Pattern #4: Testimonials On Volders.de

Pattern #4: Testimonials - Variant A
Pattern #4: Testimonials - Variant B

In this experiment, four TrustPilot reviews were appended at the bottom of landing pages of a contract cancelation service (paid). These reviews were also shown throughout the complete signup funnel (4 more steps). Impact on progression (step 2) and final completed purchases were measured.

Total Sample: 72,943 • Statistical Power at 5% MDE: 83.6%

Test #430 by Jakub LinowskiJakub Linowski Sep 20, 2022 Desktop Product X.X% Progression

Jakub Tested Pattern #26: Cart Reminder And Recently Viewed

Pattern #26: Cart Reminder And Recently Viewed - Variant A
Pattern #26: Cart Reminder And Recently Viewed - Variant B

In this experiment, up to 5 recently viewed product pages would appear on the left hand navigation. The idea was to make it easier for users get back to what they were looking at in case they were browsing. These recently viewed products were not visibile in the control. Impact on adds-to-cart and completed transactions was measured.

Total Sample: 10,324 • Statistical Power at 5% MDE: 28%

Test #427 on Designlab.com by Daniel ShapiroDaniel Shapiro Aug 10, 2022 Desktop Mobile Checkout X.X% Progression

Daniel Tested Pattern #28: Easiest Fields First On Designlab.com

Pattern #28: Easiest Fields First - Variant A
Pattern #28: Easiest Fields First - Variant B

In this experiment, the course enrollment start date was moved from step 2 to step 1 of an enrollment / checkout flow. The test was run by Designlab - that offers design courses and education with a strong element of mentorship. Impact on progression to next step and completed transactions were measured.

Total Sample: 12,185 • Statistical Power at 5% MDE: 15.3%

Test #425 by Jakub LinowskiJakub Linowski Aug 03, 2022 Desktop Product X.X% Progression

Jakub Tested Pattern #41: Sticky Call To Action

Pattern #41: Sticky Call To Action - Variant A
Pattern #41: Sticky Call To Action - Variant B

In this experiment, the complete buy box on a product detail page, floated along as users scrolled through the long screen. The variation made sure the product choice and order now button was always visible. Impact on adds-to-cart and sales was measured.

Total Sample: 8,174 • Statistical Power at 5% MDE: 13.8%

Test #422 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Jul 22, 2022 Desktop Mobile Shopping Cart X.X% Progression

Daria Tested Pattern #3: Fewer Form Fields On Volders.de

Pattern #3: Fewer Form Fields - Variant A
Pattern #3: Fewer Form Fields - Variant B

A birthdate field was removed during a signup process for a paid contract cancellation service. Impact on progression (next step) and sales (completed transactions) was measured.

Total Sample: 20,345 • Statistical Power at 5% MDE: 16.6%

Test #419 by Jakub LinowskiJakub Linowski Jun 29, 2022 Desktop Home & Landing X.X% Progression

Jakub Tested Pattern #68: Welcome Discount

Pattern #68: Welcome Discount - Variant A
Pattern #68: Welcome Discount - Variant B

In this experiment, the presence of a discount modal (bigger discount for larger purchases) was tested on home and product pages. The trigger happend after a scroll down, a few second pause, and a mouse scroll up interaction. Impact on sales was measured.

Total Sample: 196,639 • Statistical Power at 5% MDE: 98.5%

Test #413 by Jakub LinowskiJakub Linowski May 26, 2022 Desktop Mobile Shopping Cart X.X% Progression

Jakub Tested Pattern #114: Less Or More Visible Prices

Pattern #114: Less Or More Visible Prices - Variant A
Pattern #114: Less Or More Visible Prices - Variant B

Should prices be displayed during an editing state on an add-to-cart screen (after adding-to-cart from a product detail page)? In this experiment, club pricing was added and shown in three areas after customers clicked on a small "edit details" links. The control did not have this pricing information (which was only shown on a product detail screen before).

Total Sample: 5,698 • Statistical Power at 5% MDE: 59.9%

Test #411 by Ayat ShukairyAyat Shukairy May 09, 2022 Desktop Mobile Product X.X% Progression

Ayat Tested Pattern #126: Bottom Or Left Thumbnails

Pattern #126: Bottom Or Left Thumbnails - Variant A
Pattern #126: Bottom Or Left Thumbnails - Variant B

Bottom aligned thumbnails were shifted to the left side on a product image. Doing so, also shifted the product descriptions a little higher. Impact on adds-to-cart and total transactions was measured.

Total Sample: 588,443 • Statistical Power at 5% MDE: 70.1%

Test #410 on Designlab.com by Daniel ShapiroDaniel Shapiro May 05, 2022 Desktop Mobile Home & Landing X.X% Progression

Daniel Tested Pattern #29: Surfaced Content On Designlab.com

Pattern #29: Surfaced Content - Variant A
Pattern #29: Surfaced Content - Variant B

Does linking to a product detail page help? Or should a homepage simply focus on generic lead generation? In this homepage experiment, the presence of a component with a link to a detailed course landing page was tested. Impact on course enrollments was measured.

Total Sample: 25,443 • Statistical Power at 5% MDE: 6.3%

Test #408 by Jakub LinowskiJakub Linowski Apr 29, 2022 Desktop Mobile Product X.X% Progression

Jakub Tested Pattern #67: Currency & Taxes

Pattern #67: Currency & Taxes - Variant A
Pattern #67: Currency & Taxes - Variant B

Here is a very simple experiment where CDN currency copy was appended to prices being shown on a Canadian ecommerce site. Additional copy ("from Canada") was appended to an existing shipping message.  

Total Sample: 16,332 • Statistical Power at 5% MDE: 17.9%

Test #400 by Herman KleinHerman Klein Mar 06, 2022 Desktop Shopping Cart X.X% Progression

Herman Tested Pattern #65: Add More For Extra Incentive

Pattern #65: Add More For Extra Incentive - Variant A
Pattern #65: Add More For Extra Incentive - Variant B

In this experiment, customers who were not yet eligible for free shipping (with cart amounts under $39) were presented with an encouraging message to add more and earn free shipping. Impact on progression to checkout and sales was measured.

Total Sample: 3,111 • Statistical Power at 5% MDE: 60.3%

Test #399 by Jakub LinowskiJakub Linowski Feb 27, 2022 Desktop Mobile Checkout X.X% Progression

Jakub Tested Pattern #35: Floating Labels

Pattern #35: Floating Labels - Variant A
Pattern #35: Floating Labels - Variant B

In this experiment, top-aligned field labels were tested against floating labels (with labels floating inside the form field itself).

Total Sample: 2,073 • Statistical Power at 5% MDE: 29.2%

Test #398 on Adoramapix.com by Herman KleinHerman Klein Feb 25, 2022 Desktop Shopping Cart X.X% Progression

Herman Tested Pattern #121: Free Shipping On Adoramapix.com

Pattern #121: Free Shipping - Variant A
Pattern #121: Free Shipping - Variant B

In this experiment, customers were eligible for free shipping (with cart amounts of $39 or more) were presented with an earned free shipping message. The variation also showed the value of the earned free shipping - (for example $10). Impact on progression to checkout and sales was measured.

Total Sample: 3,123 • Statistical Power at 5% MDE: 60.5%

Test #395 by Jakub LinowskiJakub Linowski Jan 31, 2022 Desktop Mobile Product X.X% Progression

Jakub Tested Pattern #52: How It Works

Pattern #52: How It Works - Variant A
Pattern #52: How It Works - Variant B

In this experiment, a How It Works section was appended on a product detail page just above product reviews.

Total Sample: 17,099 • Statistical Power at 5% MDE: 15.8%

Test #394 on Chaos.com by Velin PenevVelin Penev Jan 29, 2022 Desktop Product X.X% Progression

Velin Tested Pattern #113: More Or Fewer Plans On Chaos.com

Pattern #113: More Or Fewer Plans - Variant A
Pattern #113: More Or Fewer Plans - Variant B

In this experiment, a two plan pricing plan (standalone product with a higher collection set) was tested against a single plan one (only a collection set). Impact on clicks and total sales was measured.

Total Sample: 23,261 • Statistical Power at 5% MDE: 4.8%

Test #5 on Busyteacher.org by Andrei ZakhareuskiAndrei Zakhareuski Jan 23, 2022 Desktop Mobile Product X.X% Progression

Andrei Tested Pattern #21: What It's Worth On Busyteacher.org

Pattern #21: What It's Worth - Variant A
Pattern #21: What It's Worth - Variant B

The variation included a simple extention of the headline to include the full dollar worth of a discounted bundle deal. 

Total Sample: 4,895 • Statistical Power at 5% MDE: 4%

Test #392 by Jakub LinowskiJakub Linowski Dec 31, 2021 Desktop Mobile Product X.X% Progression

Jakub Tested Pattern #122: Zigzag Layout

Pattern #122: Zigzag Layout - Variant A
Pattern #122: Zigzag Layout - Variant B

In this experiment, paragraph style copy was reorganized into a zig zag layout with key sections being reinforiced with copy-as-image statements. Impact on adds-to-cart and sales was measured.

Total Sample: 41,827 • Statistical Power at 5% MDE: 28.5%

Test #391 on Backstage.com by Stanley ZuoStanley Zuo Dec 30, 2021 Desktop Mobile Listing X.X% Progression

Stanley Tested Pattern #82: Onboarding Callouts On Backstage.com

Pattern #82: Onboarding Callouts - Variant A
Pattern #82: Onboarding Callouts - Variant B

This experiment added a small nudge or callout to encourage more signups. Attention was directed towards the save function, which lead to the signup flow for anyone not signed it. Impact on signups was measured.

Total Sample: 159,569 • Statistical Power at 5% MDE: 99.5%

Test #388 by Jakub LinowskiJakub Linowski Dec 09, 2021 Mobile Listing X.X% Progression

Jakub Tested Pattern #51: Shortcut Buttons

Pattern #51: Shortcut Buttons - Variant A
Pattern #51: Shortcut Buttons - Variant B

In this experiment, extra "buy now" buttons were added on a listing page. These buttons were shortcuts to an add to cart flow, whereas the "learn more" buttons lead customers to product detail pages (visible in both control and variation). Impact on adds-to-cart and sales was measured.

Total Sample: 7,051 • Statistical Power at 5% MDE: 10.1%