Netflix A/B Tests Displaying A Password Field Which Fails And Gets Rejected
It looks like Netflix has been iterating on showing additional fields upfront on their homepage. After they succeeded at displaying an email address upfront, this experiment now takes next step of showing a password field. The result of the leaked experiment however suggests a negative outcome as they reverted back to the control version - without the visible password.
Highlighted UI Changes From This Leak
Adding A Password Field
The first and most visible change is the additional password field in the B variation. Judging by the copy, we're going to assume that it's a dynamic sign-in or sign-up interaction with the email address being checked for whether it exists or not. It's important to note that this is a duplicated field as the password field is already shown on further steps in the signup funnel (not visible in these screenshots).
Interestingly, Volders (a contract cancellation service) recently tested something very similar (with complete tests results) where they also removed a password field from their signup funnel.
Adding An Email Newsletter Opt-out
In the B variation we also see an extra checkbox which acts as an opt out to signing up for email notifications and newsletters. Please also keep in mind that this also is a duplication from a deeper screen on the signup funnel.
It probably have been a cleaner experiment without this change which easily could have dragged the experiment down by suggesting additional and possibly unwanted emails would be sent out.
Finally, the B variation also pushed down the primary call to action button onto a new line. This is probably a minor change, but an extra confouding variable nevertheless.