Netflix A/B Tests 4 Secondary Choices - All Of Which Get Rejected

Leak #49 from   |   Mar 23, 2020 Home & Landing

In this experiment, Netflix displayed secondary buttons in at least 4 different ways. The idea might have been related to something written about by Online Dialogue - what Bart Schutz calls Hobson's +1 Choice Effect. Instead of inviting a single call to action, secondary actions such as "learn more" or "add to favorites" are shown as well. The closest patterns of our own that might be related to this include Shortcut Buttons, Gradual Reassurance and More Results. I think it's a really interesting experiment design which shows how their engineering team tried to cover a similar idea from a number of angles. Nevertheless, a few months down the road this experiment looks like it was rejected.

IMPLEMENTED Confirmed Mar 23, 2020
A - Oct 31, 2019 Screenshot
B - Oct 31, 2019 Screenshot
C - Oct 31, 2019 Screenshot
D - Oct 31, 2019 Screenshot
E - Oct 31, 2019 Screenshot

Highlighted UI Changes From This Leak

  1. Secondary: Why Netflix (Artificial Choice)

    In this variation, the second button "Why Netflix" has been added below the primary one which simply acted as a duplicate link to the next signup screen. Given that both buttons link to the same screen, we can call this button somewhat of an artificial one. :)

  2. Secondary: Learn More

    In the C variation, the secondary button "Learn More" linked further down to the bottom of the screen with a FAQ section.

  3. Secondary: Learn More (Artificial Choice)

    The D variation was the same as C, except the "Learn More" button linked to the next signup screen (same as the primary button).

  4. Secondary Button Above The Primary One

    Finally, in the E variation, the secondary "Learn More" button was simply positioned above the primary signup one.


  • Ronny Kohavi

    Ronny Kohavi 2 years ago 10

    See single option aversion (PDF at