All Latest 571 A/B Tests

In this experiment, a floating page level navigation was added near the top of the product pages (under the main image). The navigation links included: Overview, Itinerary, What's Included, Operator, Reviews and Availability. Impact on next step progression was measured.

Instead of only displaying sort and filters at the top of a listing page, the variation always had them visible with a sticky/floating interaction. Impact on sales was measured.

In this experiment, a modal based message was shown to encourage extra products being added as complementary upsells. In the control, the promotion text appeared at the bottom as red text ("Buy one, get up to two 50% Off"). Whereas in the variation, specific products were shown on the modal (post add-to-cart). Impact on adds-to-cart, sales and average revenue was measured.

In this experiment, some side column filters were made more visible by being automatically expanded. These included: brand, price and country of manufacturing.

In this experiment, two different navigation defaults were tested against each other. In one version we can see 1) more popular products with 2) more categorical granularity being shown in the expanded state. In the other version we see "new products" being shown as the default (with a lot fewer product subcategories) to choose from.

In this experiment, a simple product category link was added on product detail pages. These links linked to listing pages with more of a similar product type. Impact on sales was measured.

In this experiment, the order of the checkout flow was rearranged. In the control the first step of the checkout flow started with the shipping information step, followed by greeting card selection. In the variation this was rearranged (hypothesis was that the greeting card step was easier). Impact on sales was measured.

In this site wide experiment, logged out customers were directed to sign in with the help of a top navigation callout. The experiment was triggered for anyone who saw the callout message. After signing in, the user would 1) remain on the same screen they were on and 2) the top header slightly changed to show a logged in state (with their orders, favorites and active username). Impact on successful logins and overall purchases was measured.

In this experiment, the presence of a search bar was tested against no search bar altogether. The control had search, and in the variation it was removed. The variation also exposed items from within the "Need Help?" menu, making "My account", "Contact Us" and "FAQ" more prominent.

(Here the AB test is inverted / flipped to match the pattern). 

In this experiment, infinite scrolling was a/b tested against a paginated one.

In this experiment, the order of two signup pages was tested: plan selection vs account creation. Here we have the account creation step first in the control and the the plan selection step first in the variation. (I flipped these around to match the pattern). Impact on signups was measured.

In this experiment, conversational filters were tested at the top of some listing pages. Instead of showing one set of product filters, customers were shown three sets of product questions. After selecting each answer, product results would narrow and update further down on the page. Impact on adds to cart and sales were measured.

In this experiment, a club subscription duration was preselected to 6 in the variation. The control had no club durations preselected. As a result of preselecting a club duration, a more visible price also appeared at the top (sooner in the variation). Impact on sales was measured.

In this experiment, additional (discounted) products were shown at the top of category listing pages with a link to see more such products ("View All Price Drops"). Impact on overall sales was measured.

This experiment was triggered by a small segment of users who completed the first step of checkout funnel (submitted email, name, phone number), but dropped from checkout and returned to cars catalogue 7 or more minutes later.

In the control, users didn't see any cart icon nor function to resume their checkout flow.

In the variation however, users saw a filled shopping cart icon with resume functionality. Clicking on the icon would guide and redirect users to their latest abandoned stage of their checkout flow.

This experiment started when users would click add-to-cart on product detail pages. In both the control and variation, a modal would appear. In the control the modal contained a "View Cart and Checkout" button that lead users to the cart page. In the variation the modal showed separate "View Cart" and "Checkout" buttons. The a/b test variation also introduced a green confirmation message about the product being added to cart. Impact on transactions and revenue was measured.
 

This was an experiment on the first checkout step where users would enter shipping information. The control had a longer footer with more additional sitewide links and a call to action to a newsletter. The variation removed these elements and kept the focus on the shipping information task. Impact on progression to next step and sales was measured.

In this experiment, the click area of job listing tiles was expanded to the size of the full job tile. In the control, the click area was smaller - mostly only the job headline, along with additional "view more" links on the right hand column. Clicking the tile or headline would open up a new job details page in both control and variation. Impact on progression and membership sales was measured.

In this experiment, users that did not complete a purchase and came back to the homepage were shown two different experiences. In the control, users would see the homepage (essentially having to restart the purchase flow). Whereas, in the variation, users would be shown a "Welcome Back" summary view with the completed steps shown as completed and a quick resume button to the next incomplete step. Impact on completed sales was measured.

In this experiment, the variation transformed an add-to-cart button into a shopping cart one (making it a two step process). The control had an interaction where an add-to-cart button would become a quantity selection and then a shopping cart action (a three step process). In a way, the control kept users in a "dead-end" quantity selection middle state that required clicking the back button to continue the sale. Whereas the variation automatically moved users into the next step (closer towards being able to make a purchase). Impact on sales was measured.