Pattern #16: Welcome Mat - Partial

Pattern #16  Tested 3 timesFirst tested by Nicholas Evans Recently tested by Ardit Veliu on Apr 29, 2021

Based on 3 Tests, Members See How Likely Version B Wins Or Loses And By How Much

LOSSES
-5
-4
-3
-2
-1
FLAT
+1
+2
+3
+4
+5
WINS

Measured by the sum of negative and positive tests.

A B
Welcome Mat - Partial (Variant A) Welcome Mat - Partial (Variant B)

Expected Median Effects Of B

-

Progression

X.X%

Leads

(1 tests)

X.X%

Signups

(2 tests)

-

Engagement

-

Sales

-

Revenue

-

Retention

-

Referrals

X.X%

ANY PRIMARY

(3 tests)

All 3 tests meta-analyzed: +X.X% combined effect (p-val X.XXXX)

12.1% of 90% cumulative power target at 2% MDE from 3 tests Help replicate this with an A/B test

90%
↓
6% 3.6% 3%
Product 1 test: X%
Home & Landing 1 test: X%
Content 1 test: X%

Tested on

Tests

Pattern #16: Welcome Mat - Partial
Was Tested On Goodui.org by Jakub Linowski

Replaced

Isolated

Test # 76 on Goodui.org by Jakub LinowskiJakub Linowski Feb 01, 2017 Test link

Find Out How It Did With 11,330 Visitors

  • Measured by successful email signups

  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
Zoom In
Welcome Mat - Partial - Variant A A
Welcome Mat - Partial - Variant B B
Home & Landing
Statistical Power by Minimum Detectable Effect
3%0.5%
3%1%
3.6%2%
5.9%5%
11.8%10%
20.9%15%
32.9%20%
80%38.5%

Get Access To See The Test Results

The Same Pattern Was Also Tested Here

Added

Isolated

Test # 350 on Expertinstitute.... by Ardit VeliuArdit Veliu Apr 29, 2021 Test link

Find Out How It Did With 34,967 Visitors

  • Measured by total quote leads submitted

  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
Zoom In
Welcome Mat - Partial - Variant A A
Welcome Mat - Partial - Variant B B
Content
Statistical Power by Minimum Detectable Effect
3%0.5%
2.7%1%
3%2%
3.9%5%
5.9%10%
8.4%15%
11.6%20%
80%83.6%

In this experiment, a lead generating section with a call to action (welcome mat) was appended at the top of article pages. Impact on lead generation was measured.

Get Access To See The Test Results

Added

Isolated

Test # 149 on Reverb.com by Nicholas EvansNicholas Evans Feb 07, 2018 Test link

Find Out How It Did With 726,088 Visitors

  • Measured by form submits on next page

  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
Zoom In
Welcome Mat - Partial - Variant A A
Welcome Mat - Partial - Variant B B
Product
Statistical Power by Minimum Detectable Effect
3%0.5%
4%1%
6%2%
17.2%5%
51.6%10%
80%14.2%
84.2%15%
97.4%20%

Get Access To See The Test Results

For each pattern, we measure three key data points derived from related tests:

REPEATABILITY - this is a measure of how often a given pattern has generated a positive or negative effect. The higher this number, the more likely the pattern will continue to repeat.

SHALLOW MEDIAN - this is a median effect measured with low intent actions such as initiating the first step of a lengthier process

DEEP MEDIAN - this is derived from the highest intent metrics that we have for a given test such as fully completed signups or sales.