Pattern #124: Confirmed Selection

Pattern #124  Tested 5 timesFirst tested by Stanley Zuo Recently tested by Melina Hess on Dec 29, 2024

Based on 5 Tests, Members See How Likely Version B Wins Or Loses And By How Much

LOSSES
-5
-4
-3
-2
-1
FLAT
+1
+2
+3
+4
+5
WINS

Measured by the sum of negative and positive tests.

A B
Confirmed Selection (Variant A) Confirmed Selection (Variant B)

Expected Median Effects Of B

X.X%

Progression

(2 tests)

-

Leads

X.X%

Signups

(1 tests)

-

Engagement

X.X%

Sales

(4 tests)

X.X%

Revenue

(1 tests)

-

Retention

-

Referrals

X.X%

ANY PRIMARY

(5 tests)

All 4 tests meta-analyzed: +X.X% combined effect (p-val X.XXXX)

48.3% of 90% cumulative power target at 1% MDE from 4 tests Help replicate this with an A/B test

90%
↓
3.8% 5.5% 27.2% 22%
Signup 1 test: X%
Checkout 2 tests: X%
Product 1 test: X%

Tested on

Tests

Pattern #124: Confirmed Selection
Was Tested On Menufy.com by Aleksandr Elesev

Added

Isolated

Test # 483 on Menufy.com by Aleksandr ElesevAleksandr Elesev Jul 17, 2023 Test link

Find Out How It Did With 8,626 Visitors

  • Measured by completed sales

  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
Zoom In
Confirmed Selection - Variant A A
Confirmed Selection - Variant B B
Checkout
Statistical Power by Minimum Detectable Effect
10%0.5%
27.2%1%
78.8%2%
80%2%
99.9%5%
99.9%10%

In this experiment, a thanking confirmation message was appended at the top of the checkout screen of a local food delivery service. Impact on completed transactions was measured.

Get Access To See The Test Results

The Same Pattern Was Also Tested Here

Added

Isolated

Test # 569 on Snocks.com by Melina HessMelina Hess Dec 29, 2024 Test link

Find Out How It Did With 269,954 Visitors

  • Measured by completed orders

  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
Zoom In
Confirmed Selection - Variant A A
Confirmed Selection - Variant B B
Product
Statistical Power by Minimum Detectable Effect
9%0.5%
22%1%
65.8%2%
80%2.4%
99.9%5%
99.9%10%
99.9%15%
99.9%20%

In this experiment, reaffirming copy was added for particular size options in the variation. Here is the translation from German:

"These boxer shorts will be kept. Did you know that our boxer shorts in S are rarely returned? They fit perfectly - see for yourself!"

Impact on sales was measured.

Get Access To See The Test Results

Replaced

Isolated

Test # 482 by Jakub LinowskiJakub Linowski Jul 13, 2023 Test link

Find Out How It Did With 3,379 Visitors

  • Measured by visits to checkout

  • Measured by completed sales

  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
Zoom In
Confirmed Selection - Variant A A
Confirmed Selection - Variant B B
Checkout
Statistical Power by Minimum Detectable Effect
4%0.5%
5.5%1%
10.8%2%
44%5%
80%7.7%
95.5%10%
99.9%15%
99.9%20%

In this experiment, the choice of adding a product to cart was confirmed with a further positive message in the header of the next step (on the add to cart page). Once users left the product detail page, instead of simply stating the product name, the title was rephrased as "Product [X] Makes a Great Gift. They'll Love It!". I view this as a higher "intensity" experiment, given that the add-to-cart page was in some way already confirming the choice. Impact on sales was measured.

Get Access To See The Test Results

Replaced

Isolated

Test # 396 on Depositphotos.co... by Gleb HodorovskiyGleb Hodorovskiy Feb 13, 2022 Test link

Find Out How It Did With 801,039 Visitors

  • Measured by completed registrations

  • Measured by average revenue per user

Zoom In
Confirmed Selection - Variant A A
Confirmed Selection - Variant B B
Listing

In this experiment from Conversionrate.store, the framing of the registration message was changed from a generic account creation one to a specific image selected by the user. The experiment started on a listing page of a stock photography / illustration site. The control showed a more generic message with benefits for signing up and making the purchase. Whereas the variation repeated the actual image that customers clicked on from listing pages - establishing continuity as well as providing a reason for signing up. Impact on sales was measured.

Get Access To See The Test Results

Added

Isolated

Test # 367 on Backstage.com by Stanley ZuoStanley Zuo Jul 22, 2021 Test link

Find Out How It Did With 29,480 Visitors

  • Measured by any plan selections

  • Measured by any plan purchase

  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
Zoom In
Confirmed Selection - Variant A A
Confirmed Selection - Variant B B
Signup
Statistical Power by Minimum Detectable Effect
3%0.5%
3.8%1%
5.5%2%
14.3%5%
42.3%10%
74.5%15%
80%16.1%
93.2%20%

In this experiment, a confiming text was added at the top of a signup modal. The text reinforced the selection from a previous listing page, giving a good reason for continuing the signup process.

Get Access To See The Test Results

For each pattern, we measure three key data points derived from related tests:

REPEATABILITY - this is a measure of how often a given pattern has generated a positive or negative effect. The higher this number, the more likely the pattern will continue to repeat.

SHALLOW MEDIAN - this is a median effect measured with low intent actions such as initiating the first step of a lengthier process

DEEP MEDIAN - this is derived from the highest intent metrics that we have for a given test such as fully completed signups or sales.