Airbnb Rejects Auto Suggested Dates In This Failed A/B Test

Leak #60 from   |   Sep 3, 2020 Product

Not all experiments succeed including this one that Airbnb recently ran on their property pages. Their a/b test was automatically suggesting a next available date below the "Check Availability" button when someone didn't already have any chosen dates. A month later however, the experiment completed and was rejected. Believing that this idea's potential hasn't yet been exhausted I designed an a/b test iteration (Figma) along with a detailed screencast (Youtube) for a potential follow up experiment (using GoodUI Catalyst Canvas). Hoping this will demonstrate how to clearly explore and design A/B tests using evidence based patterns. :)

A - Aug 19, 2020 Screenshot
REJECTED Confirmed Sep 3, 2020
B - Aug 19, 2020 Screenshot

Highlighted UI Changes From This Leak

  1. Automatic Date Suggestion

    Here we see that the variation has a date suggestion with the next available date (possibly for 2 nights or so). It encourages users to use those suggested dates with a subtle link underneath. So why didn't this work out?

    One reason why that this might not have worked (as it was rejected) was because it might be too difficult to guess and match someone's desired check-in date and duration. Perhaps some these dates are simply too variable to guess and predict. 0If this is so, then the usefulness of showing fixed dates might deteriorate.

How About This Follow-Up Experiment?

Just because Airbnb's experiment didn't deliver a positive effect or an implementation decision, I still think the idea of: showing date options and/or clarifying the next available dates might have some potential. And so I designed this followup up experiment in Figma along with a detailed screen cast of how to structure your a/b tests. Hoping this inspires ...

Google A/B Test