Pattern #43: Long Ad Titles

Pattern Author: Ronny Kohavi - General Manager, Analysis and Experimentation @ Microsoft

Based on 1 Tests, Members See How Likely This Pattern Will Win Or Lose And Its (?) Median Effect

Almost Certain Loser
-5
-4
-3
-2
-1
0
+1
+2
+3
+4
+5
Almost Certain Winner
Long Ad Titles

Pattern Properties

  1. Replace: Short Link Titles With Longer Ones Benefits

    The idea is to move ad text to the title line to make it longer. The additional copy could be something specific about the offer, or a key benefit.

Pattern #43: Long Ad Titles
Was Tested On Bing.com

  • Test 133
  • Tested on Bing.com by Ron Kohavi
  • Dec 13, 2017

Maybe It Worked Here

Screen: Search Results

Source: exp-platform.com/2017abtestingtutorial/

In 2012 a Microsoft employee working on Bing had an idea about changing the way the search engine displayed ad headlines. Developing it wouldn’t require much effort—just a few days of an engineer’s time—but it was one of hundreds of ideas proposed, and the program managers deemed it a low priority. So it languished for more than six months, until an engineer, who saw that the cost of writing the code for it would be small, launched a simple online controlled experiment—an A/B test—to assess its impact. Within hours the new headline variation was producing abnormally high revenue, triggering a “too good to be true” alert.

HBR, September–October 2017 Issue, https://hbr.org/2017/09/the-surprising-power-of-online-experiments

Note: This experiment was a solid success and replicated multiple times over a period of months. It worked at Bing and had a profound influence. The only reason why we atributed a 0.25 point (a "Maybe") was because we don't have the exact sample size and conversion data.

 

For each pattern, we measure three key data points derived from related tests:

REPEATABILITY - this is a measure of how often a given pattern has generated a positive or negative effect. The higher this number, the more likely the pattern will continue to repeat.

SHALLOW MEDIAN - this is a median effect measured with low intent actions such as initiating the first step of a lengthier process

DEEP MEDIAN - this is derived from the highest intent metrics that we have for a given test such as fully completed signups or sales.

Avoid What Failed & Repeat What Worked With
Access to All 71 Patterns Based on 117 Tests

Total sample size of all tests is based on 14,812,909 visitors - that's a lot of testing time

  • Annual Billing — Save 17%
  • Monthly Billing
Guided

Order Now

$1200 / mo $1000 / mo


Unlimited User Accounts

License to use the data for unlimited users.
Users can also join discussions.

+2 Coaching Calls / mo

Guidance on testing ideas, strategy and analysis.

Team

Order Now

$120 / mo$100 / mo


5 User Accounts

License to use the data for up to 5 users.
Users can also join discussions.

Solo

Order Now

$60 / mo $50 / mo


1 User Account

License to use the data for 1 user.
Users can also join discussions.

ALL PLANS COME WITH

Access To All Tests

Get access to all tests data and patterns that we already published.

5+ New Tests Each Month

We publish about 5 new A/B tests each month as we obtain new test data.

Cancel Anytime

If you find that Fastforward isn't for you, you can cancel at any time.

100% Secure Payment with Visa