All Latest 620 A/B Tests

In this experiment, a two column grid layout (control) was tested against single column layout (variation) with the product information shown to the right. Please note that the screenshot shows repeated products only because it's been sourced from a Figma design file. In reality, the products in the variation were equally diverse as in the control. 

(We've also flipped the A and B to match up with our grid pattern.)

In this experiment, instead of showing 12 products per screen (with pagination), the variation showed all 42 products on a single screen. Impact on adds to cart and completed orders was measured.

This was a replication attempt similar to experiment 497. A social proof message was added below add to cart buttons on product detail pages. Impact on adds to cart and transactions was measured.

In this experiment, a progress bar was added to a signup flow of a mental health app. It appeared on numerous steps of a onboarding flow. Impact on signup completions and checkouts was measured.

In this experiment, the variation tracked how many people would add something to cart over the last 24 hours and display that just below the add to cart button. Impact on adds to cart and transactions was measured.

In this experiment, a social proof message was added below add to cart buttons on product detail pages. The copy read "Hurry, X people have viewed this product". Impact on adds to cart and transactions was measured.

In this experiment two sections of a form on a single page (control) were broken out into 2 separate steps (variation). The two sections included personal details and shipping information. This step appeared after having received a product recommendation when filling out a questionnaire for a skin care product. Impact on next step progression and purchases was measured.

In this experiment, recently purchased products were appended at the top of the homepage. The test ran for loggedin users only. Impact on add-to-cart, sales and revenue was measured.

This experiment was technically a larger multi-change one that ran on product detail pages. The control showed an automatically applied coupon (for people opening up targeted emails with the discount). The variation attempted to make this better with the following changes:

1. The discount message shifted closer towards the price / product selection (above the buy box)
2. Removal of sale discount amounts and messages tied to longer durations (6 and 12 month duration)
3. Reinforcing that the actual discount will be calculated further on checkout

In this experiment, a section dedicated to choosing gift messages was taken out and separated into its own individial step. The change happened on the first step of a checkout flow and increased the flow by an additional step. Impact on checkouts and total sales was measured.

In this experiment, the size of a saving amount badge was enlarged. Instead of typical badge, the variation stretched the height of the saving information to the full height of the duration selector. The font size was also increased. Impact on overall sales was measured.

In this experiment, customer reviews were shown on product detail pages. Impact on adds to cart and sales was measured.

In this experiment, selling points and benefits of a subscription were placed as bullets at the top of a checkout page. The benefits highlighted things such as: unlimited applications, access to vetted jobs and the ability to cancel anytime. Impact on sales was measured.

In this experiment, the choice of adding a product to cart was confirmed with a further positive message in the header of the next step (on the add to cart page). Once users left the product detail page, instead of simply stating the product name, the title was rephrased as "Product [X] Makes a Great Gift. They'll Love It!". I view this as a higher "intensity" experiment, given that the add-to-cart page was in some way already confirming the choice. Impact on sales was measured.

In this experiment, 9 Twitter card style testimonials were appended onto the homepage of Rollbar. These were image / screenshots recreations without links to the actual tweets.

In this experiment, a video testimonial was added mid page onto a signup / lead form page.

In this experiment, three video testimonials were added mid page onto a signup / lead form page.

In this homepage experiment a number of changes were introduced - with perhaps the most prominent one being the replacement of a video component with customer review badges. Additional copy changes included reinforcement of the latest version number (v11) throughout the page, as well as a dedicated (v11) section in the middle of the page. Impact on demo signups was measured.

In this experiment, an FAQ section was added near the bottom of a long product page. The reviews were collapsed by default, but expandable upon clicking. Impact on adds-to-cart and sales was measured.

In this experiment, copy was added which showed the number of experts (in a given category) a person may gain access to after filling out a lead form. The context of this is a lead-gen landing page where people are seeking experts for legal purposes. Impact on initial progression (of a multi step form) and completed leads was measured.