Pattern #43: Long Ad Titles

Pattern Author: Ronny Kohavi - Technical Fellow and VP, Analysis and Experimentation @ Microsoft

Based on 1 Tests, Members See How Likely This Pattern Will Win Or Lose And Its (?) Median Effect

Almost Certain Loser
-5
-4
-3
-2
-1
0
+1
+2
+3
+4
+5
Almost Certain Winner
Long Ad Titles
  1. Replace: Short Link Titles With Longer Ones Benefits

    The idea is to move ad text to the title line to make it longer. The additional copy could be something specific about the offer, or a key benefit.

Median Effects

?

Engagement

Ex: Any Action / Visit

?

Conversions

Ex: Signups, Leads

?

Sales

Ex: Transactions, Upsells

?

Revenue

Ex: AOV, LTV

(1 tests)

?

Retention

Ex: Return Visits

?

Referrals

Ex: Social Shares

Tests

Pattern #43: Long Ad Titles
Was Tested On Bing.com by Ronny Kohavi

Maybe It Worked Here

Screen: Listing

Source: exp-platform.com/2017abtestingtutorial/

In 2012 a Microsoft employee working on Bing had an idea about changing the way the search engine displayed ad headlines. Developing it wouldn’t require much effort—just a few days of an engineer’s time—but it was one of hundreds of ideas proposed, and the program managers deemed it a low priority. So it languished for more than six months, until an engineer, who saw that the cost of writing the code for it would be small, launched a simple online controlled experiment—an A/B test—to assess its impact. Within hours the new headline variation was producing abnormally high revenue, triggering a “too good to be true” alert.

HBR, September–October 2017 Issue, https://hbr.org/2017/09/the-surprising-power-of-online-experiments

Note: This experiment was a solid success and replicated multiple times over a period of months. It worked at Bing and had a profound influence. The only reason why we atributed a 0.25 point (a "Maybe") was because we don't have the exact sample size and conversion data.

 

For each pattern, we measure three key data points derived from related tests:

REPEATABILITY - this is a measure of how often a given pattern has generated a positive or negative effect. The higher this number, the more likely the pattern will continue to repeat.

SHALLOW MEDIAN - this is a median effect measured with low intent actions such as initiating the first step of a lengthier process

DEEP MEDIAN - this is derived from the highest intent metrics that we have for a given test such as fully completed signups or sales.

Avoid What Failed & Repeat What Worked With
Access to All 91 Patterns Based on 175 Tests

Total sample size of all tests is based on 21,050,098 visitors - that's a lot of testing time

  • Annual Billing SAVE 17%
  • Monthly Billing SAVE 0%
Company-Wide

Order Now

$2400 / year
$2880 / year


Unlimited User Accounts

License to use the data by unlimited users.

Team

Order Now

$1200 / year
$1440 / year


5 User Accounts

License to use the data by up to 5 users.

Solo

Order Now

$600 / year
$720 / year


1 User Account

License to use the data by 1 user.

All Plans Come With

Access To All Tests

Get access to all tests data and patterns that we already published.

5+ New Tests Each Month

We publish about 5 new A/B tests each month as we obtain new test data.

Cancel Anytime

If you find that Fastforward isn't for you, you can cancel at any time.

100% Secure Payment with Visa