All Latest 615 A/B Tests

In this experiment, the active availability of a casting call (job offer) was was made more prominent using green text: "Accepting applications". The variation also made the "Join Now" button more prominent as an alternative path to signing up for a membership. The experiment reports on three metrics: clicks on apply, application starts and premium membership sales (measured a few steps further in the funnel).

In this experiment, an inline panel was shown as a product tile. The panel informed customers about a variety of payment options (with detailed information being further presented after a button click). Impact on sales was measured.

In this experiment, the earliest availability dates were displayed underneath product tiles on listing pages. This was a/b tested on a car rental service website. Impact on product adds-to-cart as well as transactions was measured.

In this experiment, product availabiltiy bars were shown on products with low stock. This was shown on listing pages. Impact on adds to cart and sales was measured.

Are more (popular) product results better than none at all? In this experiment, popular products were shown during an empty search result. Impact on sales was measured.

In this experiment from Conversionrate.store, the framing of the registration message was changed from a generic account creation one to a specific image selected by the user. The experiment started on a listing page of a stock photography / illustration site. The control showed a more generic message with benefits for signing up and making the purchase. Whereas the variation repeated the actual image that customers clicked on from listing pages - establishing continuity as well as providing a reason for signing up. Impact on sales was measured.

This experiment added a small nudge or callout to encourage more signups. Attention was directed towards the save function, which lead to the signup flow for anyone not signed it. Impact on signups was measured.

In this experiment, listing descriptions were shortened dynamically using exposable "more" links. This way, the variation showed shorter listings and therefore more listings per screens. Impact on listing clicks (progression) along with membership starts was measured.

This experiment added the simple verb ("play") before the "factory video" label to encourage more video plays. Impact on progression / clicks was measured.

This experiment attempted to increase the number of leads on a lead-funnel. As the first step, users were being asked to upload a file. The control showed the file types that were allowed, whereas the variation changed the copy to show a number of benefits for taking that action. The text-based benefits included the: receiving feedback, prices and lead times.

In this experiment, at the bottom of a search results screen, a membership join button was added along with 3 encouraging reasons. The experiment measured membership funnel starts, as well as paid membership transactions (sales).

In this experiment, structured data tags were displayed on a listing page to help potential buyers make better decisions. The additional information about the listed companies included: annual revenue, employee count, and year of establishment.

In this experiment, the B version condensed the company listings. This was done by showing less of the description and introducing a "more" and "less" dynamic links that would expand and collapse the description.

In this variation, a number of social proof references were added to a signup modal.

In 2012 a Microsoft employee working on Bing had an idea about changing the way the search engine displayed ad headlines. Developing it wouldn’t require much effort—just a few days of an engineer’s time—but it was one of hundreds of ideas proposed, and the program managers deemed it a low priority. So it languished for more than six months, until an engineer, who saw that the cost of writing the code for it would be small, launched a simple online controlled experiment—an A/B test—to assess its impact. Within hours the new headline variation was producing abnormally high revenue, triggering a “too good to be true” alert.

HBR, September–October 2017 Issue, https://hbr.org/2017/09/the-surprising-power-of-online-experiments

Note: This experiment was a solid success and replicated multiple times over a period of months. It worked at Bing and had a profound influence. The only reason why we atributed a 0.25 point (a "Maybe") was because we don't have the exact sample size and conversion data.