17 Tests By Craig Kistler

Tests

Test #652 on Kay.com by Craig KistlerCraig Kistler May 29, 2026 Desktop Mobile X.X% Sales

Craig Tested Pattern #26: Cart Reminder And Recently Viewed On Kay.com

Pattern #26: Cart Reminder And Recently Viewed - Variant A
Pattern #26: Cart Reminder And Recently Viewed - Variant B

In this experiment, returning homepage users were shown their recently viewed items in a carousel (in the upper part of the homepage). Users who viewed at least 1 product were triggered into the experiment. Impact on adds to cart and sales was measured.

Total Sample: 561,787 • Statistical Power at 5% MDE: 44.5%

Test #641 on Kay.com by Craig KistlerCraig Kistler Mar 26, 2026 Desktop Mobile X.X% Sales

Craig Tested Pattern #130: Less Or More Visible Offer Pages On Kay.com

Pattern #130: Less Or More Visible Offer Pages - Variant A
Pattern #130: Less Or More Visible Offer Pages - Variant B

A global flyout drawer was added that consolidated sale promotions into one centralized location, replacing the customary placement of promotions throughout the site.

Total Sample: 2,206,246 • Statistical Power at 5% MDE: 99.8%

Test #630 on Kay.com by Craig KistlerCraig Kistler Jan 27, 2026 Desktop Mobile X.X% Sales

Craig Tested Pattern #21: What It's Worth On Kay.com

Pattern #21: What It's Worth - Variant A
Pattern #21: What It's Worth - Variant B

In this experiment, a pricing table (an upsell of product protection plan coverages) was tested against a more explicit yes/no toggle for including the plan. Impact was measured on adds to cart, orders, and upsell rate, using an 80/20 traffic split.

Total Sample: 1,052,531 • Statistical Power at 5% MDE: 93.7%

Test #626 on Jared.com by Craig KistlerCraig Kistler Dec 26, 2025 Desktop Mobile X.X% Sales

Craig Tested Pattern #118: Category Images On Jared.com

Pattern #118: Category Images - Variant A
Pattern #118: Category Images - Variant B

In this experiment, two types of UI filters for metal color choices were shown at the top of product listing pages. One variant only used category labels, while the other variant combined images with labels to reinforce the categories. Impact on filter usage, adds to cart and sales were measured.

Total Sample: 34,191 • Statistical Power at 5% MDE: 7%

Test #621 on Kay.com by Craig KistlerCraig Kistler Nov 26, 2025 Desktop Mobile X.X% Sales

Craig Tested Pattern #36: Fewer Or More Results On Kay.com

Pattern #36: Fewer Or More Results - Variant A
Pattern #36: Fewer Or More Results - Variant B

In this experiment, a horizontally scrolling set of products was replaced with an expanded and more visible grid of suggested products - enabling more discovery. 

Total Sample: 428,454 • Statistical Power at 5% MDE: 8.6%

Test #614 on Kay.com by Craig KistlerCraig Kistler Oct 23, 2025 Mobile X.X% Sales

Craig Tested Pattern #26: Cart Reminder And Recently Viewed On Kay.com

Pattern #26: Cart Reminder And Recently Viewed - Variant A
Pattern #26: Cart Reminder And Recently Viewed - Variant B

In this experiment, returning users to a homepage would be shown recently viewed items. Impact on adds to cart and sales was measured. The experiment was triggered to returning users.

Total Sample: 122,483 • Statistical Power at 5% MDE: 58.1%

Test #603 on Kay.com by Craig KistlerCraig Kistler Jul 30, 2025 Desktop Mobile X.X% Sales

Craig Tested Pattern #55: Conversational Filters On Kay.com

Pattern #55: Conversational Filters - Variant A
Pattern #55: Conversational Filters - Variant B

In this experiment, product pages (variant) asked users if they were interested to see holiday gifts with two buttons. Upon clicking "yes", the UI expanded to make another choice in order to see gifts for: Her, Him or Kids. Clicking any of these three would send users to dedicated listing pages with more product recommendations. Impact on sales was measured.

Total Sample: 3,080,311 • Statistical Power at 5% MDE: 99.9%

Test #585 on Jared.com by Craig KistlerCraig Kistler Apr 10, 2025 Mobile Desktop X.X% Sales

Craig Tested Pattern #137: Visible Filters On Jared.com

Pattern #137: Visible Filters - Variant A
Pattern #137: Visible Filters - Variant B

In this experiment, UI filters with metal color options were surfaced at the top of product listing pages (for Jared - an online jewelry retailer). Impact on filter usage, adds to cart and sales were measured.

Total Sample: 33,984 • Statistical Power at 5% MDE: 7.6%

Test #579 on Jared.com by Craig KistlerCraig Kistler Feb 23, 2025 Mobile X.X% Sales

Craig Tested Pattern #21: What It's Worth On Jared.com

Pattern #21: What It's Worth - Variant A
Pattern #21: What It's Worth - Variant B

In this experiment, typical dollar values of typical coverages for an extended protection plan were shown. This ran on a product page of a jewelery site. Impact on on adds to cart, sales and uptake of the upsell was measured. Traffic split was 25/75.

Total Sample: 2,285,675 • Statistical Power at 5% MDE: 21.2%

Test #566 on Banter.com by Craig KistlerCraig Kistler Dec 11, 2024 Desktop X.X% Revenue

Craig Tested Pattern #66: Complementary Upsell On Banter.com

Pattern #66: Complementary Upsell - Variant A
Pattern #66: Complementary Upsell - Variant B

In this experiment, a modal based message was shown to encourage extra products being added as complementary upsells. In the control, the promotion text appeared at the bottom as red text ("Buy one, get up to two 50% Off"). Whereas in the variation, specific products were shown on the modal (post add-to-cart). Impact on adds-to-cart, sales and average revenue was measured.

Sample size too small for 80% power

Test #549 on Kay.com by Craig KistlerCraig Kistler Aug 13, 2024 Desktop X.X% Sales

Craig Tested Pattern #138: Visible Payment Options On Kay.com

Pattern #138: Visible Payment Options - Variant A
Pattern #138: Visible Payment Options - Variant B

In this experiment, an inline panel was shown as a product tile. The panel informed customers about a variety of payment options (with detailed information being further presented after a button click). Impact on sales was measured.

Total Sample: 342,123 • Statistical Power at 5% MDE: 22.7%

Test #545 on Banter.com by Craig KistlerCraig Kistler Jul 29, 2024 Desktop X.X% Revenue

Craig Tested Pattern #66: Complementary Upsell On Banter.com

Pattern #66: Complementary Upsell - Variant A
Pattern #66: Complementary Upsell - Variant B

In this experiment, a modal based message was shown to encourage extra products being added as complementary upsells. In the control, the promotion text appeared at the bottom as red text ("Buy one, get up to two 50% Off"). Whereas in the variation, specific products were shown on the modal (post add-to-cart). Impact on adds-to-cart, sales and average revenue was measured.

Sample size too small for 80% power

Test #529 on Jared.com by Craig KistlerCraig Kistler Apr 29, 2024 Mobile Desktop X.X% Sales

Craig Tested Pattern #55: Conversational Filters On Jared.com

Pattern #55: Conversational Filters - Variant A
Pattern #55: Conversational Filters - Variant B

In this experiment, conversational filters were tested at the top of some listing pages. Instead of showing one set of product filters, customers were shown three sets of product questions. After selecting each answer, product results would narrow and update further down on the page. Impact on adds to cart and sales were measured.

Total Sample: 189,872 • Statistical Power at 5% MDE: 7%

Test #524 on Jared.com by Craig KistlerCraig Kistler Mar 26, 2024 Mobile X.X% Revenue

Craig Tested Pattern #79: Product Highlights On Jared.com

Pattern #79: Product Highlights - Variant A
Pattern #79: Product Highlights - Variant B

In this experiment, additional (discounted) products were shown at the top of category listing pages with a link to see more such products ("View All Price Drops"). Impact on overall sales was measured.

Test #509 on Jared.com by Craig KistlerCraig Kistler Dec 18, 2023 Desktop X.X% Sales

Craig Tested Pattern #66: Complementary Upsell On Jared.com

Pattern #66: Complementary Upsell - Variant A
Pattern #66: Complementary Upsell - Variant B

In this experiment, a protection plan was launched as a modal for customers that clicked add-to-cart without choosing the upsell. Impact on adds-to-cart and sales was measured.

Total Sample: 407,785 • Statistical Power at 5% MDE: 23.2%

Test #500 on Kayoutlet.com by Craig KistlerCraig Kistler Oct 25, 2023 Mobile X.X% Sales

Craig Tested Pattern #7: Social Counts On Kayoutlet.com

Pattern #7: Social Counts - Variant A
Pattern #7: Social Counts - Variant B

This was a replication attempt similar to experiment 497. A social proof message was added below add to cart buttons on product detail pages. Impact on adds to cart and transactions was measured.

Total Sample: 106,467 • Statistical Power at 5% MDE: 19.1%

Test #497 on Jared.com by Craig KistlerCraig Kistler Oct 16, 2023 Mobile X.X% Sales

Craig Tested Pattern #7: Social Counts On Jared.com

Pattern #7: Social Counts - Variant A
Pattern #7: Social Counts - Variant B

In this experiment, a social proof message was added below add to cart buttons on product detail pages. The copy read "Hurry, X people have viewed this product". Impact on adds to cart and transactions was measured.

Total Sample: 451,850 • Statistical Power at 5% MDE: 44.3%