All Latest 614 A/B Tests

In this experiment, we tested a visible "Shipping Frequency" (A) option against a progressively displayed one (B) that would only appear after someone first chose a duration option. Thus in variation B, the buy box component would initially appear with fewer fields and smaller. The experiment measured initial progression and actual sales. 

Note on the data: the experiment was run a little shorter than usual, as one of the variations triggered a stop rule to protect losses (so the effect might be somewhat inflated from a lower power).

In this experiment, an extra step was prepended at the beginning of a multiple step signup modal flow. The signup modal would appear on listing pages after requests to contact a listed company. The idea was to prime users with benefits of signing up in order to increase their motivation to do so. The experiment measured the impact on the initial progression (to the step with the email form).

In this experiment, a search input field (to look for companies) along with most popular links (also company names) were displayed on the homepage of a leading contract cancellation service. The control (A) version instead had a button that sent users to a next page where the same selection could be made - only later. The measurable success criteria were the number of paid cancellations - a few steps down the funnel.

In this experiment a long form (A) was replaced with a progressive form interaction (B). Most of the form fields would appear in a grey-disabled style, until the prerequioste fields were first filled out.

In this experiment, a password field was removed on a contract cancellation form (Volders).

In the control version, users were required to enter their email address and a password. If the email address was associated with an existing account, then the password was used to authenticate the user (and validated). When users entered a new email address, then the password field was used to create a new account. 

In the variation, the password field was removed, as the authentication happened after the conversion itself using other backend mechanisms.

In this experiment, a simple pulldown menu (for US state selection) was replaced with all state options shown as selectable buttons. The states were also abbreviated.

In this experiment, a long contract cancellation landing page (control) was broken down into 4 steps with 1 final summary step (variation).

In this experiment on a contract cancellation funnel, one field was removed - a secondary contract ID. The control and variation both had a primary "customer ID" with which to identify and cancel someone's contract with.

In this experiment, a fully visible coupon field (A) was made less visible by turning it into a default collaped link (B). Clicking on the link caused the coupon field to appear.

In this experiment, the variation exposed 6 of the options from the pulldown menu as tabs.

In this variation an optional field label was added.

In this followup experiment, field labels without and with a marked asterisk were tested.

The original had no fields marked as required. The variant had all fields marked as required with an asterisk (and a reference note).

In this experiment, field labels without and with a marked asterisk were tested.

This test explored a progressive disclosure interaction in variation B. When a user started typing text into the first field, two more fields expanded into view.

In this experiment a single screen signup process was broken into 2 separate steps: account creation & details.

In this experiment, the old flow had a login / register / guest checkout as a second step. The variant B flow removed this step and treated everyone as a guest (and later offered to login / register). In the B version guests had a voluntary registration on the thank you page and in a confirmation email. 

This experiment reduced the search form by removing the distance field.