Airbnb Discovers A Better Way To Display Customer Ratings In This A/B Test

Leak 26   |   Leaked from Airbnb.com   |   Oct 3, 2019 Screen: Product Confirmed A/B Test

In this tiny and well isolated experiment, Airbnb tried a slightly different approach to displaying customer ratings on their property (product) pages. From the observed implementation decision one month later, we learned that using decimal places for customer rating averages seems to be the better approach. :)

A - Sep 3, 2019 Screenshot
IMPLEMENTED Confirmed Oct 3, 2019
B - Sep 3, 2019 Screenshot

Highlighted UI Changes From This Leak

  1. Towards Decimal Averages For Customer Ratings

    In the control (A) version we can see that Airbnb has used a full set of 5 stars to denote the average rating. By looking at older screenshots of other property pages, we also can tell that these 5 stars did convey the average amount visually by using a combination of darker green stars vs lighter gray ones. In contrast, the B variation uses a numerical representation to show the averages with one or two decimal places - and this version has been rolled out.

    Why might decimal places be better? One simplest explanation I could think of as to why actual numbers might be better is that they make it easier to compare reviews that have similar ratings (especially when those averages are similar to each other - ex: 4.62 and 4.87).

    The second potential reason is that high ratings look too perfect. Just looking at the screenshot above, for example, the A version shows what appears like a perfect 5/5 stars, whereas it's a 4.92 average rating. This numerical average could be perceived as more authentic. :)

    This is very similar to Pattern #6: Customer Star Ratings

Comments

  • Fernanda

    Fernanda 1 month ago 00

    I don't think is a good thing... someone can think looking in a fast way that the product only have 1 star.