Netflix A/B Tests 4 Secondary Choices - All Of Which Get Rejected

Leak #49 from Netflix.com   |   Mar 23, 2020 Home & Landing

In this experiment, Netflix displayed secondary buttons in at least 4 different ways. The idea might have been related to something written about by Online Dialogue - what Bart Schutz calls Hobson's +1 Choice Effect. Instead of inviting a single call to action, secondary actions such as "learn more" or "add to favorites" are shown as well. The closest patterns of our own that might be related to this include Shortcut Buttons, Gradual Reassurance and More Results. I think it's a really interesting experiment design which shows how their engineering team tried to cover a similar idea from a number of angles. Nevertheless, a few months down the road this experiment looks like it was rejected.

IMPLEMENTED Confirmed Mar 23, 2020
A - Oct 31, 2019 Screenshot
B - Oct 31, 2019 Screenshot
C - Oct 31, 2019 Screenshot
D - Oct 31, 2019 Screenshot
E - Oct 31, 2019 Screenshot

Highlighted UI Changes From This Leak

  1. Secondary: Why Netflix (Artificial Choice)

    In this variation, the second button "Why Netflix" has been added below the primary one which simply acted as a duplicate link to the next signup screen. Given that both buttons link to the same screen, we can call this button somewhat of an artificial one. :)

  2. Secondary: Learn More

    In the C variation, the secondary button "Learn More" linked further down to the bottom of the screen with a FAQ section.

  3. Secondary: Learn More (Artificial Choice)

    The D variation was the same as C, except the "Learn More" button linked to the next signup screen (same as the primary button).

  4. Secondary Button Above The Primary One

    Finally, in the E variation, the secondary "Learn More" button was simply positioned above the primary signup one.

Previous Netflix's Home & Landing Leaks:

Leak 5

Netflix Rejected All These Tested Homepage Variations - Perhaps This One Will Help?   |   Leak 5   |   Apr 12, 2019

Leak 7

Netflix A/B Tested These 4 Button Labels With "Try It Now" Possibly Leading   |   Leak 7   |   Apr 27, 2019

Leak 18

Netflix A/B Tests And Rejects Free Trials   |   Leak 18   |   Aug 6, 2019

Leak 20

Netflix Finally Succeeds With These 3 Tested Home Page Patterns   |   Leak 20   |   Aug 22, 2019

Leak 22

Netflix A/B Tests And Rejects Secondary Sign-in And Sign-up Calls To Action   |   Leak 22   |   Sep 3, 2019

Leak 31

Netflix Designs Their Button A Little Too Big   |   Leak 31   |   Nov 4, 2019

Leak 45

Netflix A/B Tests Upfront Email Capture At The Start Of Their Signup Flow And It Succeeds   |   Leak 45   |   Feb 18, 2020

Leak 53

Netflix A/B Tests Displaying A Password Field Which Fails And Gets Rejected   |   Leak 53   |   May 25, 2020

Other Leaks From Netflix.com Include:

Leak 35

Netflix A/B Tests A Single Plan Recommendation And It Becomes Rejected   |   Leak 35   |   Dec 2, 2019

Leak 40

Netflix Keeps Its Older Price Comparison Table And Rejects Their New Layout In This A/B Test   |   Leak 40   |   Jan 10, 2020

Comments

  • Ronny Kohavi

    Ronny Kohavi 2 years ago ↑1↓0

    See single option aversion https://www.jstor.org/stable/10.1086/671343 (PDF at http://web.missouri.edu/~segerti/capstone/SingleOptionAversion.pdf)

    Reply