Pattern #82: Onboarding Callouts

Pattern #82  Tested 3 timesFirst tested by Kyle Phillips Recently tested by Andrey Andreev on May 28, 2024

Based on 3 Tests, Members See How Likely Version B Wins Or Loses And By How Much

LOSSES
-5
-4
-3
-2
-1
FLAT
+1
+2
+3
+4
+5
WINS

Measured by the sum of negative and positive tests.

A B
Onboarding Callouts (Variant A) Onboarding Callouts (Variant B)

Expected Median Effects Of B

?

Progression

(3 tests)

-

Leads

?

Signups

(2 tests)

-

Engagement

?

Sales

(1 tests)

-

Revenue

-

Retention

-

Referrals

?

ANY PRIMARY

(3 tests)

Tested on

Tests

Pattern #82: Onboarding Callouts
Was Tested On Thomasnet.com by Kyle Phillips

Added

Isolated

Test # 322 on Thomasnet.com by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Kyle Phillips    Oct 27, 2020 Test link

Find Out How It Did With 279,729 Visitors

  • Measured by clicks on the save button

  • Measured by registrations

Zoom In
  • ?% (? successes out of ? visits)
  • ?% (? successes out of ? visits)
A
  • ?% (? successes out of ? visits)
  • ?% (? successes out of ? visits)
B
Product

This experiment variation prompted users to save (bookmark) a company profile on a company detail page. Clicking on the save feature while logged out, would prompt a registration modal. Hence the save feature acted as an extra reason to signup. The number of people engaging or interacting with the feature was measured, as well as registrations.

Get Access To See The Test Results

The Same Pattern Was Also Tested Here

Added

Isolated

Test # 534 on Online.metro-cc.... by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Andrey Andreev    May 28, 2024 Test link

Find Out How It Did With 1,817,890 Visitors

  • Measured by successful logins

  • Measured by completed purchases

Zoom In
  • ?% (? successes out of ? visits)
  • ?% (? successes out of ? visits)
A
  • ?% (? successes out of ? visits)
  • ?% (? successes out of ? visits)
B
Global

In this site wide experiment, logged out customers were directed to sign in with the help of a top navigation callout. The experiment was triggered for anyone who saw the callout message. After signing in, the user would 1) remain on the same screen they were on and 2) the top header slightly changed to show a logged in state (with their orders, favorites and active username). Impact on successful logins and overall purchases was measured.

Get Access To See The Test Results

Added

Isolated

Test # 391 on Backstage.com by $conducted_test->test->user_->first_name . ' ' . $conducted_test->test->user_->last_name Stanley Zuo    Dec 30, 2021 Test link

Find Out How It Did With 159,569 Visitors

  • Measured by save button clicks

  • Measured by trial account creations

Zoom In
  • ?% (? successes out of ? visits)
  • ?% (? successes out of ? visits)
A
  • ?% (? successes out of ? visits)
  • ?% (? successes out of ? visits)
B
Listing

This experiment added a small nudge or callout to encourage more signups. Attention was directed towards the save function, which lead to the signup flow for anyone not signed it. Impact on signups was measured.

Get Access To See The Test Results

For each pattern, we measure three key data points derived from related tests:

REPEATABILITY - this is a measure of how often a given pattern has generated a positive or negative effect. The higher this number, the more likely the pattern will continue to repeat.

SHALLOW MEDIAN - this is a median effect measured with low intent actions such as initiating the first step of a lengthier process

DEEP MEDIAN - this is derived from the highest intent metrics that we have for a given test such as fully completed signups or sales.