Pattern #94: Visible Search

Pattern #94  Tested 4 timesFirst tested by Kimberly Cheung Recently tested by Jakub Linowski on May 23, 2024

Based on 4 Tests, Members See How Likely Version B Wins Or Loses And By How Much

LOSSES
-5
-4
-3
-2
-1
FLAT
+1
+2
+3
+4
+5
WINS

Measured by the sum of negative and positive tests.

A B
Visible Search (Variant A) Visible Search (Variant B)

Expected Median Effects Of B

X.X%

Progression

(2 tests)

-

Leads

X.X%

Signups

(1 tests)

-

Engagement

X.X%

Sales

(3 tests)

-

Revenue

-

Retention

-

Referrals

X.X%

ANY PRIMARY

(4 tests)

All 4 tests meta-analyzed: +X.X% combined effect (p-val X.XXXX)

86.9% of 90% cumulative power target at 2% MDE from 4 tests Help replicate this with an A/B test

90%
↓
5.4% 4.7% 84.2% 7.8%
Home & Landing 2 tests: X%
Global 2 tests: X%

Tested on

Tests

Replaced

Confounded

Test # 533 May 23, 2024 Test link

Find Out How It Did With 70,933 Visitors

  • Measured by visits any product page

  • Measured by completed sales

  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
Zoom In
Visible Search - Variant A A
Visible Search - Variant B B
Global
Statistical Power by Minimum Detectable Effect
3%0.5%
4.6%1%
7.9%2%
27.4%5%
76.7%10%
80%10.4%
97.9%15%
99.9%20%

In this experiment, the presence of a search bar was tested against no search bar altogether. The control had search, and in the variation it was removed. The variation also exposed items from within the "Need Help?" menu, making "My account", "Contact Us" and "FAQ" more prominent.

(Here the AB test is inverted / flipped to match the pattern). 

Get Access To See The Test Results

Added

Isolated

Test # 432 on Snocks.com by Melina HessMelina Hess Sep 29, 2022 Test link

Find Out How It Did With 1,175,864 Visitors

  • Measured by completed sales

  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
Zoom In
Visible Search - Variant A A
Visible Search - Variant B B
Global
Statistical Power by Minimum Detectable Effect
11%0.5%
31.8%1%
80%1.9%
84.2%2%
99.9%5%
99.9%10%
99.9%15%
99.9%20%

In this experiment, search functionality was added in the variation, globally on mobile. Impact on sales was measured

Get Access To See The Test Results

Replaced

Isolated

Test # 305 on Volders.de by Michal FiechMichal Fiech Jun 30, 2020 Test link

Find Out How It Did With 10,350 Visitors

  • Measured by visits to a company cancellation page

  • Measured by completed cancellations

  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
Zoom In
Visible Search - Variant A A
Visible Search - Variant B B
Home & Landing
Statistical Power by Minimum Detectable Effect
3%0.5%
3.5%1%
4.7%2%
10.6%5%
29%10%
55%15%
78.5%20%
80%20.4%

In this experiment, a search input field (to look for companies) along with most popular links (also company names) were displayed on the homepage of a leading contract cancellation service. The control (A) version instead had a button that sent users to a next page where the same selection could be made - only later. The measurable success criteria were the number of paid cancellations - a few steps down the funnel.

Get Access To See The Test Results

Added

Isolated

Test # 218 on Yummly.com by Kimberly CheungKimberly Cheung Jan 14, 2019 Test link

Find Out How It Did With 20,122 Visitors

  • Measured by successful signups

  • XX.X% (XXX successes out of XXX,XXX visitors)
  • XX.X% (XXX successes out of XXX,XXX visitors)
Zoom In
Visible Search - Variant A A
Visible Search - Variant B B
Home & Landing
Statistical Power by Minimum Detectable Effect
3%0.5%
3.7%1%
5.4%2%
13.8%5%
40.5%10%
72.3%15%
80%16.5%
92%20%

Hypothesis: Anonymous users can't use global search while on the home promo page. We believe that if we show a global search bar to anonymous users, it presents a higher converting funnel (guided search) and will increase our sign-up rates significantly.

Control (A): Logged out users don't see global search bar.

Variant (B): Logged out users see global search bar. After searching for a keyword, the signup funnel starts with a more personalized reason to continue the signup process.

Get Access To See The Test Results

For each pattern, we measure three key data points derived from related tests:

REPEATABILITY - this is a measure of how often a given pattern has generated a positive or negative effect. The higher this number, the more likely the pattern will continue to repeat.

SHALLOW MEDIAN - this is a median effect measured with low intent actions such as initiating the first step of a lengthier process

DEEP MEDIAN - this is derived from the highest intent metrics that we have for a given test such as fully completed signups or sales.