Netflix A/B Tested These 4 Button Labels With "Join Now" Possibly Leading
Netflix has just finished testing at least four interesting button labels on their homepage. Since one of them looks like it has been implemented, we just might be able to make a few subtle comparisons which I'll attempt to capture in this leak. Of course, given the fact that we're just scratching the surface of this experiment, please take this analysis as highly hypothetical (ripe for further experimentation).
UPDATE May 6, 2019: It seems like version C might be served to all traffic, suggesting a slightly different outcome (I might have done the analysis too quickly. Will check again in the future.)
Highlighted UI Changes From This Leak
Join Netflix Vs. Join Now
This "Join Netflix" button label might have been the control version but I'm not sure. Since "Join Now" seems to have been implemented, we have a single word comparison between A ("Netflix") and B ("Now"). If in fact B is slightly better, then maybe, just maybe, immediacy related messaging could be subtly stronger than a company name reference.
A few weeks after detecting this experiment setup, we observed that this was the implemented variant. Hence, we're going to assume that this was the best version.
Try It Now Vs. Join Now
We have another simple word comparison between C ("Try It") and B ("Join"). If B was in fact better than C, then it would suggest that "Joining" might be stronger than "Trying". Although joining is more committal, perhaps it does seem to carry a little more value in that users will not be "kicked out" should they wish to continue with the service. Whereas, trying feels more "temporary". Let's also not forget the bigger context of the landing page which already emphasizes the option to "cancel" very prominently. Taken together, "not temporary" and "being able to cancel at any time" seem like a powerful combination. If only, of course, B is really better than C. :)
Get Netflix Vs. Join Now
If in fact "Get" is in any way inferrior to "Join", could it mean that "joining" is more consistent with its subscription model? Wherease "getting" is wording that's possibly better suited for single purchase products? I'm only speculating here and would consider this as a hypothesis for future experiments.
It would also be super amazing to see the actual test result comapring A ("Join Netflix") and D ("Get Netflix") to answer the above more tightly. Of course that's only for Netflix to know - unless they see this and share their results (wishful thinking)? :)
Why I Respect This Experiment
Button experiments have occasionally been a symbol of ridicule. Some consider them as a waste of time leading towards low impact or insignificant results. Some consider them as wasted efforts compared to potentially more meaningful experiments. The contrary fact that Netflix actually ran such experiments, to me suggests that no experiment is considered taboo by their team. Sometimes even small changes can have impact. And this line of thinking is completely inline with our own evolving understanding of various button patterns (some better than others and some flat of course).
Thank you Netflix for running this!