Netflix A/B Tests And Rejects Free Trials

Leak #18 from Netflix.com   |   Aug 6, 2019 Home & Landing

Netflix has been offering a 30 day free trial for Canadians. Sometime in May however we spotted something way more interesting - they began a/b testing different trial durations (30, 14 and 7 days), as well as an immediate payment-first approach (without a trial). Fast track another few months and it now looks like the experiment ended with the payment first pattern being implemented to 100% of the Canadian audience. Free trial no more.

A - May 15, 2019 Screenshot
B - May 15, 2019 Screenshot
C - May 15, 2019 Screenshot
IMPLEMENTED Confirmed Aug 6, 2019
D - May 15, 2019 Screenshot

Highlighted UI Changes From This Leak

  1. Varying Free Trial Durations

    The experiment tested a wide range of free trial durations ranging from 30, 14, 7 and 0 days. Both the control and the variations asked users for a credit card during signup. The last variation (D) however charged users immediately - and looks like it was also rolled out.

    Overall I really like the test setup of this intensity-type experiment with all of the variations exploring a single trait or property (with various intensities).

    0.5 Repeatability has been assigned to Pattern #12: Payment First as evidence that it's getting better

    Repeatability is a net count of evidence for or against a pattern. It’s how we can predict which patterns are better than others. :)

Comments

  • Ernest

    Ernest 5 years ago 00

    Just went on Netflix in incognito mode. Button says "TRY 30 DAYS FREE >." Seems they're not done testing.

    • Jakub

      Jakub 5 years ago 00

      Are you outside of Canada? This is only for the Canadian segment.

      • Ernest

        Ernest 5 years ago 00

        Ah, that must be it. I missed that detail.