Discover UI Design Decisions Of Leading Companies,
From Leaked Screenshots & A/B Tests.

Two months ago I managed to detect that Amazon was actually a/b testing the removal of the second "Buy Now" button on a specific product page. Finally today it looks like the experiment has completed with both buttons still being displayed - like in the old days. View Leak

I recently found a 2019 product page screenshot from Etsy's which was just asking for a comparison with the newer 2020 version from today. By doing this little exercise we can see how the screen has evolved with numerous UI changes. View Leak

In this experiment captured from a random toothpaste product page, Amazon was testing two separate purchase related defaults. Some visitors would be defaulted to a "One-time purchase" buy box (control version). While others would first see a preselected subscription option, encouraging a recurring product purchase (variation B). View Leak

In this experiment, Netflix displayed secondary buttons in at least 4 different ways. The idea might have been related to something written about by Online Dialogue - what Bart Schutz calls Hobson's +1 Choice Effect. View Leak

In this simple experiment, Airbnb tested the display of customer rarings averages on their search results screen. The tested copy was a form of social proof which showed the average rating of the properties for a given city (defined by the search). View Leak

Bol, a leading online retailer in the Netherlands, just completed a beautiful experiment with 8 add-to-cart button variations. Two potentially leading variations (C & G) were then retested in a followup A/B test. Today, we have finally detected that the C version ("In winkelwagen" translated to "Add to cart") has been implemented on all product pages. View Leak

In this experiment Booking added a second search bar on their homepage. The intention might have been to provide users a way to filter more destinations by country. Could this scenario have created an akward uncertainty about which form field to engage with? Whether this explanation is true or not, I'm not sure. What we do know however that in the end, the control version with the single search bar prevailed.  View Leak

Once upon a time, Netflix had asked users to enter their email address far down in the signup flow. The email field appeared after a user moved from the hompage, through a series of confirmation steps, and a plan selection - what might officially be counted as the fifth step. All this was challenged with the following experiment where the email field was placed right at the forefront. More so, it also looks like this changed generated some positive impact as it was also rolled out. View Leak

Etsy just completed a cascade experiment with 3 version of their global search bar. Interestingly the cascaded version with all inherited changes, took the lead. View Leak

This was a simple calendar picker experiment that Aribnb recently ran and eventually rolled out. The A/B test contains at least 4 simple UI changes which might have contributed to a positive outcome. And so this also shows us what a slightly better UI component looks like. View Leak

In this listing page experiment, Airbnb tried an very similar search interaction that is already present on Booking.com's homepage. Instead of only asking users for their destination, the search interaction was extended to prompt for travel dates and number of guests. View Leak

Netflix has been experimenting with the layout of their pricing plans. Challenging the more traditional pricing comparison table, instead they a/b tested three self-contained pricing plan tiles. This newer version however ended up being rejected as we noticed. View Leak

At first glance some A/B tests may seem like they are beautifully simple with a single controlled variable. Yet as we look at the experiment in more detail, more and more subtle differences start coming into view. This I think is the case with Amazon's most recent experiment where it seems that only a dropdown menu was shifted into a hamburger one - which ended up winning.  View Leak

When Bol ran their list vs grid view experiment, they also included a variation that tested for another very simple change: the exposure of menu options. That is, the listing page variant was designed to check if three more visible pull down options would be better or worse than just showing them hidden inside the pulldown. View Leak

Interestingly, Google was also discovered a/b testing the list vs grid pattern just as Bol finished a similar experiment this month. The Google experiment ran on one of their shopping results pages with the specific query for "flowers". And the outcome? Looks like the grid beat the list and was rolled out in this case.  View Leak