All Latest 534 A/B Tests

In this experiment, the wording "one-time payment" was appended below all three pricing plans for a contract cancelation service. Impact on orders placed was measured. (Translated from German: "Einmalige Zahlung").

In this experiment, the variation tracked how many people would add something to cart over the last 24 hours and display that just below the add to cart button. Impact on adds to cart and transactions was measured.

In this product detail page experiment a simple subheadline was appended to a new product. The tagline read "Lose weight sustainably and healthily." (Google translated from German "Nachhaltig und gesund Gewicht verlieren."). Impact on sales and revenue was measured.

In this experiment, recently purchased products were appended at the top of the homepage. The test ran for loggedin users only. Impact on add-to-cart, sales and revenue was measured.

This experiment was technically a larger multi-change one that ran on product detail pages. The control showed an automatically applied coupon (for people opening up targeted emails with the discount). The variation attempted to make this better with the following changes:

1. The discount message shifted closer towards the price / product selection (above the buy box)
2. Removal of sale discount amounts and messages tied to longer durations (6 and 12 month duration)
3. Reinforcing that the actual discount will be calculated further on checkout

In this experiment, a section dedicated to choosing gift messages was taken out and separated into its own individial step. The change happened on the first step of a checkout flow and increased the flow by an additional step. Impact on checkouts and total sales was measured.

In this experiment, the size of a saving amount badge was enlarged. Instead of typical badge, the variation stretched the height of the saving information to the full height of the duration selector. The font size was also increased. Impact on overall sales was measured.

In this experiment, the variation remembered and prefilled user's size choices for the duration of the session. The control version always started with a fixed product size value (ex: XS). The variation prefilled them between products or screen refreshes. Impact on sales was measured.

In this experiment, a thanking confirmation message was appended at the top of the checkout screen of a local food delivery service. Impact on completed transactions was measured.

In this experiment, selling points and benefits of a subscription were placed as bullets at the top of a checkout page. The benefits highlighted things such as: unlimited applications, access to vetted jobs and the ability to cancel anytime. Impact on sales was measured.

In this experiment, the choice of adding a product to cart was confirmed with a further positive message in the header of the next step (on the add to cart page). Once users left the product detail page, instead of simply stating the product name, the title was rephrased as "Product [X] Makes a Great Gift. They'll Love It!". I view this as a higher "intensity" experiment, given that the add-to-cart page was in some way already confirming the choice. Impact on sales was measured.

In this experiment, a list of benefits were shown for each of the 3 plans on the last step of a contract cancelation service. Benefit items not included in the lower plans were also shown with grayed out styles (and an "x"). Clearly the higher paid plan had all the benefits listed. Impact on transactions was measured. 

In this experiment, a list of benefits were shown for each of the 3 plans on the last step of a contract cancelation service. The lowest plan only had one benefit, whereas the highest plan had 3. Impact on transactions was measured. 

In this experiment, product color swatches were replaced with real product photos. Whereas the control showed the colors as more abstract circles. Impact on sales was measured.

Are more (popular) product results better than none at all? In this experiment, popular products were shown during an empty search result. Impact on sales was measured.

In this experiment, 9 Twitter card style testimonials were appended onto the homepage of Rollbar. These were image / screenshots recreations without links to the actual tweets.

In this experiment, very short form testimonials (with a reference to over 3 million customers) were replaced with 3 more elaborate ones. These elaborate or benefit testimonials contained: highlighted statements, star reviews, emphasized location, tag summaries and photos of the purchased product. The control also contained a 3 testimonial carousel interaction.

This test appeared at the bottom of a longer homepage with additional product listings above.

In this experiment, a video testimonial was added mid page onto a signup / lead form page.

In this experiment, three video testimonials were added mid page onto a signup / lead form page.

In this homepage experiment a number of changes were introduced - with perhaps the most prominent one being the replacement of a video component with customer review badges. Additional copy changes included reinforcement of the latest version number (v11) throughout the page, as well as a dedicated (v11) section in the middle of the page. Impact on demo signups was measured.