Pattern #33: Example Situations

Pattern #33  Tested 3 timesFirst tested by Sumantha Shankaranarayana Recently tested by Sorcha Mullis on Dec 14, 2022

Based on 3 Tests, Members See How Likely Version B Wins Or Loses And By How Much

LOSSES
-5
-4
-3
-2
-1
FLAT
+1
+2
+3
+4
+5
WINS

Measured by the sum of negative and positive tests.

A B
Example Situations (Variant A) Example Situations (Variant B)

Expected Median Effects Of B

X.X%

Progression

(1 tests)

X.X%

Leads

(3 tests)

-

Signups

-

Engagement

-

Sales

-

Revenue

-

Retention

-

Referrals

X.X%

ANY PRIMARY

(3 tests)

All 3 tests meta-analyzed: +X.X% combined effect (p-val X.XXXX)

12.5% of 90% cumulative power target at 2% MDE from 3 tests Help replicate this with an A/B test

90%
↓
4.2% 4% 4.8%
Home & Landing 3 tests: X%

Tested on

Tests

Pattern #33: Example Situations
Was Tested On Homewarrantyreviews.com by Sumantha Shankaranarayana

Replaced

Isolated

Test # 116 on Homewarrantyrevi... by Sumantha ShankaranarayanaS. S. Oct 01, 2017 Test link

Find Out How It Did With 5,581 Visitors

  • Measured by visits to next page

  • Measured by form completion on next screen

  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
Zoom In
Example Situations - Variant A A
Example Situations - Variant B B
Home & Landing
Statistical Power by Minimum Detectable Effect
3%0.5%
3.3%1%
4.2%2%
8.2%5%
20.2%10%
38.5%15%
59.5%20%
80%25.8%

Get Access To See The Test Results

The Same Pattern Was Also Tested Here

Added

Isolated

Test # 445 on Phorest.com by Sorcha MullisSorcha Mullis Dec 14, 2022 Test link

Find Out How It Did With 132,525 Visitors

  • Measured by completed lead forms

  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
Zoom In
Example Situations - Variant A A
Example Situations - Variant B B
Home & Landing
Statistical Power by Minimum Detectable Effect
3%0.5%
3.5%1%
4.8%2%
11.1%5%
30.9%10%
58.2%15%
80%19.6%
81.5%20%

In this experiment, five clickable feature elements were surfaced on the homepage. Clicking on any of these 5 elements launched an Instastory-like short 10 second video in full screen. These videos explained the key features of the software being offered. Impact on lead generation was measured.

Get Access To See The Test Results

Added

Isolated

Test # 339 on Expertinstitute.... by Ardit VeliuArdit Veliu Feb 23, 2021 Test link

Find Out How It Did With 8,388 Visitors

  • Measured by form submits

  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
Zoom In
Example Situations - Variant A A
Example Situations - Variant B B
Home & Landing
Statistical Power by Minimum Detectable Effect
3%0.5%
3.2%1%
4%2%
7.6%5%
17.9%10%
33.9%15%
53.1%20%
80%28%

In this experiment, a number of use cases examples were added to illustrate the situations in which experts could help. 

Get Access To See The Test Results

For each pattern, we measure three key data points derived from related tests:

REPEATABILITY - this is a measure of how often a given pattern has generated a positive or negative effect. The higher this number, the more likely the pattern will continue to repeat.

SHALLOW MEDIAN - this is a median effect measured with low intent actions such as initiating the first step of a lengthier process

DEEP MEDIAN - this is derived from the highest intent metrics that we have for a given test such as fully completed signups or sales.