Pattern #109: Required Field Labels

Pattern #109  Tested 3 timesTested by Julian Gaviria on Sep 09, 2019

Based on 3 Tests, Members See How Likely Version B Wins Or Loses And By How Much

LOSSES
-5
-4
-3
-2
-1
FLAT
+1
+2
+3
+4
+5
WINS

Measured by the sum of negative and positive tests.

A B
Required Field Labels (Variant A) Required Field Labels (Variant B)

Expected Median Effects Of B

-

Progression

-

Leads

X.X%

Signups

(3 tests)

-

Engagement

-

Sales

-

Revenue

-

Retention

-

Referrals

X.X%

ANY PRIMARY

(3 tests)

All 3 tests meta-analyzed: +X.X% combined effect (p-val X.XXXX)

30.7% of 90% cumulative power target at 2% MDE from 3 tests Help replicate this with an A/B test

90%
↓
7% 18.6% 8.5%
Signup 3 tests: X%

Tested on

Tests

Pattern #109: Required Field Labels
Was Tested On Thomasnet.com by Julian Gaviria

Added

Isolated

Test # 255 on Thomasnet.com by Julian GaviriaJulian Gaviria Aug 22, 2019 Test link

Find Out How It Did With 5,977 Visitors

  • Measured by successful registrations

  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
Zoom In
Required Field Labels - Variant A A
Required Field Labels - Variant B B
Signup
Statistical Power by Minimum Detectable Effect
3%0.5%
4.3%1%
7%2%
22.6%5%
67%10%
80%11.7%
94.7%15%
99.7%20%

In this experiment, field labels without and with a marked asterisk were tested.

Get Access To See The Test Results

The Same Pattern Was Also Tested Here

Replaced

Isolated

Test # 257 on Thomasnet.com by Julian GaviriaJulian Gaviria Sep 09, 2019 Test link

Find Out How It Did With 5,739 Visitors

  • Measured by successful registrations

  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
Zoom In
Required Field Labels - Variant A A
Required Field Labels - Variant B B
Signup
Statistical Power by Minimum Detectable Effect
4%0.5%
4.8%1%
8.5%2%
31.3%5%
80%9.6%
83.4%10%
99.2%15%
99.9%20%

In this followup experiment, field labels without and with a marked asterisk were tested.

Get Access To See The Test Results

Added

Isolated

Test # 256 by Alex JamesAlex James Aug 23, 2019 Test link

Find Out How It Did With 116,470 Visitors

  • Measured by successful registrations

  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
Zoom In
Required Field Labels - Variant A A
Required Field Labels - Variant B B
Signup
Statistical Power by Minimum Detectable Effect
5%0.5%
7.7%1%
18.6%2%
75.4%5%
80%5.3%
99.9%10%
99.9%15%
99.9%20%

The original had no fields marked as required. The variant had all fields marked as required with an asterisk (and a reference note).

Get Access To See The Test Results

Leaks

Leak #85 from Etsy.com   |   Aug 16, 2022 Product

Etsy A/B Tests And Implements Required Field Labels

A - May 31, 2022
B - May 31, 2022 IMPLEMENTED

Here is a very tiny a/b test from Etsy - the marking of two product personalization fields as required using red asterisks. If I had to guess, I'd expect the effect on checkouts was probably very small if any (based on the pattern data so far). View Leak

+0.5 Evidence

For each pattern, we measure three key data points derived from related tests:

REPEATABILITY - this is a measure of how often a given pattern has generated a positive or negative effect. The higher this number, the more likely the pattern will continue to repeat.

SHALLOW MEDIAN - this is a median effect measured with low intent actions such as initiating the first step of a lengthier process

DEEP MEDIAN - this is derived from the highest intent metrics that we have for a given test such as fully completed signups or sales.