Discover UI Design Decisions Of Leading Companies,
From Leaked Screenshots & A/B Tests.


Showing 81 results

Leak #41 from   |   Jan 20, 2020 Listing

Airbnb A/B Tests Search With Automatic Prompts Which Gets Rejected

In this listing page experiment, Airbnb tried an very similar search interaction that is already present on's homepage. Instead of only asking users for their destination, the search interaction was extended to prompt for travel dates and number of guests. View Leak

Leak #40 from   |   Jan 10, 2020 Pricing

Netflix Keeps Its Older Price Comparison Table And Rejects Their New Layout In This A/B Test

Netflix has been experimenting with the layout of their pricing plans. Challenging the more traditional pricing comparison table, instead they a/b tested three self-contained pricing plan tiles. This newer version however ended up being rejected as we noticed. View Leak

Leak #39 from   |   Jan 3, 2020 Listing

Booking A/B Tests Ghost Buttons, But Their Hopes Fade Away As Predicted

Ghosts buttons. We know that this style of buttons is net negative based on this evidence-based pattern. So when Booking ran this experiment, it's no surpirse that it was eventually rejected - a predictable replication. View Leak

Leak #38 from   |   Dec 27, 2019 Global

Amazon's Loaded Hamburger Menu Beats Its Older Dropdown Version In This A/B Test

At first glance some A/B tests may seem like they are beautifully simple with a single controlled variable. Yet as we look at the experiment in more detail, more and more subtle differences start coming into view. This I think is the case with Amazon's most recent experiment where it seems that only a dropdown menu was shifted into a hamburger one - which ended up winning.  View Leak

Leak #37 from   |   Dec 17, 2019 Listing

Bol Discovers Something Better Than Classic Pulldown Menus, As Expected

When Bol ran their list vs grid view experiment, they also included a variation that tested for another very simple change: the exposure of menu options. That is, the listing page variant was designed to check if three more visible pull down options would be better or worse than just showing them hidden inside the pulldown. View Leak

Leak #36 from   |   Dec 10, 2019 Signup

Booking Fails To Replicate Airbnb's Signup, This Experiment Suggests

It seems to me that Booking may have attempted a replication or imitation of one of their competitors - I'll let you be the judge. View Leak

Leak #35 from   |   Dec 2, 2019 Pricing

Netflix A/B Tests A Single Plan Recommendation And It Becomes Rejected

Netflix made a bold move and challenged their traditional 3 plan pricing page with a preselected single plan recommendation. Clear and equally balanced choice vs. a single mid tier plan.  View Leak

Leak #34 from   |   Nov 25, 2019 Listing

Google Also A/B Tests The List Vs The Grid

Interestingly, Google was also discovered a/b testing the list vs grid pattern just as Bol finished a similar experiment this month. The Google experiment ran on one of their shopping results pages with the specific query for "flowers". And the outcome? Looks like the grid beat the list and was rolled out in this case.  View Leak

Leak #33 from   |   Nov 18, 2019 Listing

Bol A/B Tests The UI Classic: List Vs Grid View With A Clear Decision

I am continously interested in answering and providing guidance on whether grid or list views are in general better, worse or indifferent as one of many UI patterns. To my surprise, the leading Dutch web shop, has recently a/b tested this classic pattern which we were super lucky to detect on their red wine product listing pages. After anticipating their leaked design decision, we eventually learned that lists fared better for Bol. This of course we'll now use as additional and emerging evidence to tip the scales of probability (and hopefully better predict similar future experiments). View Leak

Leak #32 from   |   Nov 11, 2019 Home & Landing

Airbnb A/B Tests And Rejects Both Of These Social Proof Statements

Many companies have already tried and tested the pattern of displaying numerical social proof in some form or another. In this leaked experiment from Airbnb on their host signup landing page, we managed to detect two social proof statements that were eventually rejected. Here are some potential explanations as to possibly why they failed to deliver on an improvement. View Leak

Leak #31 from   |   Nov 4, 2019 Home & Landing

Netflix Designs Their Button A Little Too Big

Sooner or later design properties should reach optimums for their given contexts. That is, UI elements will become just right - not too big and not too small, or not too high and not to low, etc. In this leak, it seems that Netflix has approached such an optimum when they tested various button sizes on their landing page. Given that form elements and buttons should generally be bigger, it was inevitable for this new evidence to appear as an example of a button being simply too big - as seen in this beautiful experiment. :) View Leak

Leak #30 from   |   Oct 28, 2019 Home & Landing

Booking's Homepage Carousel Experiment Fails As Predicted

I was glad to detect this carousel experiment that Booking recently ran on their homepage because we've already seen similar experiments fail in the past. Instead of showing 5 location tiles, they tested a version that only showed 3 tiles at a time with an ability to slide for more - a carousel. It wasn't the automatic slider type that would unleash the wrath of Karl Gilis, but it was a user-invoked slider nevertheless. View Leak

Leak #29 from   |   Oct 21, 2019 Product

Amazon Discovers That Product Availability And Price Belong Closer Towards Add To Cart

In this little experiment, Amazon shifted the availability information towards the center of its product pages. Instead of showing if and when a product will ship near the right add to cart tile, it was moved towards above the product description. Some weeks later, we discovered that this change was rejected in favor of showing the same information in the top right. View Leak

Leak #28 from   |   Oct 16, 2019 Global

Booking Sticks To A Traditional Breadcrumb After Rejecting A Multi-Dimensional One

Booking ran a sitewide experiment where they tested two types of breadcrumb navigation. The contending variation showed a breadcrumb with two dimensions: displaying the geographical hierarchy of how deep someone is within the site (as expected), and also displaying a menu (on-click) with a secondary dimension of stay types for each level. It might have seemed like a nice idea but it didn't cut it. As the a/b test completed, the idea was rejected in favor of the old-school breadcrumb approach. Nice try booking. :) View Leak

Leak #27 from   |   Oct 7, 2019 Product

Zalando Rejects Black Add-To-Cart Buttons

Zalando has run a simple experiment on their product page where they challenged their existing orange add-to-cart button against a black one. One month later, the black button was rejected even though the black one had a higher contrast ratio.   View Leak

Leak #26 from   |   Oct 3, 2019 Product

Airbnb Discovers A Better Way To Display Customer Ratings In This A/B Test

In this tiny and well isolated experiment, Airbnb tried a slightly different approach to displaying customer ratings on their property (product) pages. From the observed implementation decision one month later, we learned that using decimal places for customer rating averages seems to be the better approach. :)  View Leak

Leak #25 from   |   Sep 23, 2019 Product

Amazon's Beautifully Designed And Failed Three vs. Two Column Layout Experiment

Amazon found the courage to run a beautiful a/b test where they put their old three column product page layout against a new two column one. Although the two column layout was arguably more beautiful with the addition of white space, margins and shadows, it was nevertheless rejected. Before it was removed we decided to leak it here to ensure its beautiful failure continues to teach us well into the future. View Leak

Leak #24 from   |   Sep 13, 2019 Home & Landing

Airbnb A/B Tests And Rejects Full Height Photos On Their Landing Page

Airbnb ran an experiment where they increased the size of the header photo on their Host signup landing page. And interestingly it was rejected a month later.   View Leak

Leak #23 from   |   Sep 9, 2019 Product

Zalando Rejects A/B Tested Company Logos And A Smaller Add-To-Cart Button

Zalando (Germany) has been experimenting with at least two interesting cascade variations on their product page. Both of these variations seem to have been rejected which is consistent with other evidence in favor of larger buttons.  View Leak

Leak #22 from   |   Sep 3, 2019 Home & Landing

Netflix A/B Tests And Rejects Secondary Sign-in And Sign-up Calls To Action

Netflix ran an experiment with two variations of appended secondary links underneath the main "Try It Now" button. One variation had a link urging users to sign-in, while another urged to resume signing-up. Both ideas didn't make it and have been rejected. Here are my thoughts and comparisons on this ... View Leak