Pattern #19: Benefit Testimonials

Pattern #19  Tested 5 timesFirst tested by Jaymie Friesen Recently tested by Jakub Linowski on May 26, 2023

Based on 5 Tests, Members See How Likely Version B Wins Or Loses And By How Much

LOSSES
-5
-4
-3
-2
-1
FLAT
+1
+2
+3
+4
+5
WINS

Measured by the sum of negative and positive tests.

A B
Benefit Testimonials (Variant A) Benefit Testimonials (Variant B)

Expected Median Effects Of B

X.X%

Progression

(5 tests)

-

Leads

X.X%

Signups

(2 tests)

-

Engagement

X.X%

Sales

(3 tests)

-

Revenue

-

Retention

-

Referrals

X.X%

ANY PRIMARY

(5 tests)

All 5 tests meta-analyzed: +X.X% combined effect (p-val X.XXXX)

69.3% of 90% cumulative power target at 2% MDE from 5 tests Help replicate this with an A/B test

90%
↓
9.4% 11.8% 46.9% 5% 23.8%
Home & Landing 3 tests: X%
Product 1 test: X%
Signup 1 test: X%

Tested on

Tests

Pattern #19: Benefit Testimonials
Was Tested On Kenhub.com by Niels Hapke

Replaced

Isolated

Test # 245 on Kenhub.com by Niels HapkeNiels Hapke Jun 11, 2019 Test link

Find Out How It Did With 130,872 Visitors

  • Measured by successful registrations

  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
Zoom In
Benefit Testimonials - Variant A A
Benefit Testimonials - Variant B B
Signup
Statistical Power by Minimum Detectable Effect
7%0.5%
15.5%1%
46.9%2%
80%3%
99.7%5%
99.9%10%
99.9%15%
99.9%20%

In this experiment a more elaborate and authentic testimonial was used instead of three more generic ones from social media sites.

Get Access To See The Test Results

The Same Pattern Was Also Tested Here

Replaced

Isolated

Test # 473 by Jakub LinowskiJakub Linowski May 26, 2023 Test link

Find Out How It Did With 66,794 Visitors

  • Measured by viewed products

  • Measured by adds to cart

  • Measured by completed purchases

  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
Zoom In
Benefit Testimonials - Variant A A
Benefit Testimonials - Variant B B
Home & Landing
Statistical Power by Minimum Detectable Effect
5%0.5%
9.1%1%
23.8%2%
80%4.5%
87.4%5%
99.9%10%
99.9%15%
99.9%20%

In this experiment, very short form testimonials (with a reference to over 3 million customers) were replaced with 3 more elaborate ones. These elaborate or benefit testimonials contained: highlighted statements, star reviews, emphasized location, tag summaries and photos of the purchased product. The control also contained a 3 testimonial carousel interaction.

This test appeared at the bottom of a longer homepage with additional product listings above.

Get Access To See The Test Results

Added

Isolated

Test # 308 on Umbraco.com by Lars Skjold IversenLars Skjold Iversen Jul 23, 2020 Test link

Find Out How It Did With 107,783 Visitors

  • Measured by signup trial starts

  • Measured by completed signups

  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
Zoom In
Benefit Testimonials - Variant A A
Benefit Testimonials - Variant B B
Home & Landing
Statistical Power by Minimum Detectable Effect
3%0.5%
3.6%1%
5%2%
12.1%5%
34.4%10%
63.6%15%
80%18.4%
86.1%20%

In this experiment, three testimonials were added mid way though on a CMS landing page. At the end of the customer testimonials an additional trial signup button was also added - which was also the primary metric. 

Get Access To See The Test Results

Added

Isolated

Test # 83 on Onlinefaxes.com by Jaymie FriesenJaymie Friesen Mar 02, 2017 Test link

Find Out How It Did With 25,482 Visitors

  • Measured by starting the signup funnel

  • Measured by visits to thank you page (3 more steps)

  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
Zoom In
Benefit Testimonials - Variant A A
Benefit Testimonials - Variant B B
Home & Landing
Statistical Power by Minimum Detectable Effect
4%0.5%
5.1%1%
9.4%2%
36%5%
80%8.8%
88.7%10%
99.7%15%
99.9%20%

Get Access To See The Test Results

Added

Isolated

Test # 48 on Adoramapix.com by Herman KleinHerman Klein Oct 01, 2016 Test link

Find Out How It Did With 19,399 Visitors

  • Measured by customization starts

  • Measured by visits to thank you page

  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
Zoom In
Benefit Testimonials - Variant A A
Benefit Testimonials - Variant B B
Product
Statistical Power by Minimum Detectable Effect
4%0.5%
5.8%1%
11.8%2%
48.7%5%
80%7.3%
96.9%10%
99.9%15%
99.9%20%

Get Access To See The Test Results

For each pattern, we measure three key data points derived from related tests:

REPEATABILITY - this is a measure of how often a given pattern has generated a positive or negative effect. The higher this number, the more likely the pattern will continue to repeat.

SHALLOW MEDIAN - this is a median effect measured with low intent actions such as initiating the first step of a lengthier process

DEEP MEDIAN - this is derived from the highest intent metrics that we have for a given test such as fully completed signups or sales.