All Latest 604 A/B Tests

In this experiment, two changes were made to the checkout page: (1) the minimum basket size requirement was made more visible using a floating element, and (2) an additional threshold was introduced to encourage customers to add more items by offering a free gift at a higher spend level. The impact on sales was measured.

In this experiment, the shopping cart screen displayed a floating “Checkout” button with common credit card icons that directed users to the checkout page when clicked. In the control, the button was positioned inline, further down the page. Impact on progression to checkout and total sales were measured.

In this experiment, a pricing table (an upsell of product protection plan coverages) was tested against a more explicit yes/no toggle for including the plan. Impact was measured on adds to cart, orders, and upsell rate, using an 80/20 traffic split.

In this experiment (1) the zip code field position was moved up, right below the last name. And (2) entering the zip code would populate the state and city using an autofill API call. Impact on orders completed was measured.

In this experiment, two types of UI filters for metal color choices were shown at the top of product listing pages. One variant only used category labels, while the other variant combined images with labels to reinforce the categories. Impact on filter usage, adds to cart and sales were measured.

In this experiment, search results and listing pages received two additional filters to remove out-of-stock and in-store-only items. This reduced the number of results shown by default. The impact on add-to-cart actions, checkout flows, and completed sales was measured.

In this experiment, a floating "Confirm Payment" button was added on the last step of a checkout flow. The button appeared with two different states: in a disabled state, leading to the terms and conditions; and in an active state, after checking off the terms. Impact on purchases was measured. 

In this experiment, a single inline page navigation (going to external sub pages) was replaced with multiple and repeated inline page navigations (that linked within the same page). Impact on adds to cart an sales was measured.

In this experiment, a breadcrumb was replaced with a circular progress bar (arguably more visible overall steps; but less visible/clickable accomplished steps). Impact on transactions was measured.

In this experiment, a horizontally scrolling set of products was replaced with an expanded and more visible grid of suggested products - enabling more discovery. 

In this experiment, reassuring copy was added to 2 of 3 pricing plans of a contract cancellation service. The copy was translated as "Cancellation guarantee. Contract cost coverage and legal support included!” Impact on overall revenue was measured.

In this experiment, the homepage interaction, headline, and images were changed. In the control, users saw a single “Shop Now” button leading directly to products. In the variation, users first chose between “Reset” or “Lose Weight,” then selected a duration in days before being shown products. The impact on sales was measured.

A progress bar was added to the top of a 3 step signup funnel. Impact on completed orders was measured.

A number of copy tweaks were made in this experiment which was flipped around to match the earliest availability pattern (one of the more visible changes). Under this assumption, one of the more visible copy changes is the switch from using a blue "Most popular" tag towards a green "Available from [DATE]". The "available from" is also visible in the other version, except further down and with lower contrast. Impact on adds to cart and orders was measured.

In this experiment, two sets of distractions were removed from the checkout put. First, the search bar was removed from the top of the screen. Second, a series of app links were also removed from the bottom of the screen near the footer. Impact on checkouts was measured.

In this experiment, returning users to a homepage would be shown recently viewed items. Impact on adds to cart and sales was measured. The experiment was triggered to returning users.

In this experiment, a 3 step progress bar was added at the top of an ecommerce checkout funnel. Impact on checkout progression and completed sales was measured.

In this experiment, price was made more visible using size and a higher contrast color. Impact on progressions and bookings was measured.

In this experiment, a Klarna buy now pay later badge was added to the PDP. It was added above the paypal buy now pay later badge in the benefit section below the ATC button. Impact on adds to cart and sales was measured.

In this experiment, a 3 step progress bar was added at the top of a signup funnel. Impact on signups was measured.