Netflix A/B Tested These 4 Button Labels With "Join Now" Possibly Leading

Leak 7   |   Leaked from Netflix.com   |   Apr 27, 2019 Screen: Home & Landing

Netflix has just finished testing at least four interesting button labels on their homepage. Since one of them looks like it has been implemented, we just might be able to make a few subtle comparisons which I'll attempt to capture in this leak. Of course, given the fact that we're just scratching the surface of this experiment, please take this analysis as highly hypothetical (ripe for further experimentation). 

UPDATE May 6, 2019: It seems like version C might be served to all traffic, suggesting a slightly different outcome (I might have done the analysis too quickly. Will check again in the future.)

A - Apr 15, 2019 Screenshot
IMPLEMENTED Confirmed Apr 25, 2019
B - Apr 15, 2019 Screenshot
C - Apr 15, 2019 Screenshot
D - Apr 15, 2019 Screenshot

Highlighted UI Changes From This Leak

  1. Join Netflix Vs. Join Now

    This "Join Netflix" button label might have been the control version but I'm not sure. Since "Join Now" seems to have been implemented, we have a single word comparison between A ("Netflix") and B ("Now"). If in fact B is slightly better, then maybe, just maybe, immediacy related messaging could be subtly stronger than a company name reference. 

  2. Join Now

    A few weeks after detecting this experiment setup, we observed that this was the implemented variant. Hence, we're going to assume that this was the best version.

  3. Try It Now Vs. Join Now

    We have another simple word comparison between C ("Try It") and B ("Join"). If B was in fact better than C, then it would suggest that "Joining" might be stronger than "Trying". Although joining is more committal, perhaps it does seem to carry a little more value in that users will not be "kicked out" should they wish to continue with the service. Whereas, trying feels more "temporary". Let's also not forget the bigger context of the landing page which already emphasizes the option to "cancel" very prominently. Taken together, "not temporary" and "being able to cancel at any time" seem like a powerful combination. If only, of course, B is really better than C. :)

  4. Get Netflix Vs. Join Now

    If in fact "Get" is in any way inferrior to "Join", could it mean that "joining" is more consistent with its subscription model? Wherease "getting" is wording that's possibly better suited for single purchase products? I'm only speculating here and would consider this as a hypothesis for future experiments.

    It would also be super amazing to see the actual test result comapring A ("Join Netflix") and D ("Get Netflix") to answer the above more tightly. Of course that's only for Netflix to know - unless they see this and share their results (wishful thinking)? :)


Why I Respect This Experiment

Button experiments have occasionally been a symbol of ridicule. Some consider them as a waste of time leading towards low impact or insignificant results. Some consider them as wasted efforts compared to potentially more meaningful experiments. The contrary fact that Netflix actually ran such experiments, to me suggests that no experiment is considered taboo by their team. Sometimes even small changes can have impact. And this line of thinking is completely inline with our own evolving understanding of various button patterns (some better than others and some flat of course).

Thank you Netflix for running this!

Comments

  • Jasper

    Jasper 2 months ago 10

    Really cool to see these kind of a/b tests. Its really helpful information. I was thinking B or C would be the best, since people want to start watching movies "now" and they also want to join "now". And i feel the same Danit is saying about the Joining, they want to join an already large group of people who use Netflix. So your joining a group of people with the same interest.

  • Danit

    Danit 3 months ago 10

    I think the word "Join" appeals to users because it has a social connotation.
    It implies the opportunity to be part of a community - it might provide a sense of belonging, which is a basic human need many people seek