Designing a Smarter Experiment Dashboard

I feel like experiment dashboards could be improved to better set expectations: what decisions are allowed, what could be expected, and how much longer a sequential experiment might need to get there. So here's me putting my experiment + UI design hat back on for a playful prototype.

The Prototype

The key ingredients I was going for:

  1. Spatial and colour-coded action recommendation Each row sits in the column of the decision it is heading for (Wait, Abort, Give Up, Ship), so position and colour alone tell you where it stands.
  2. Visualizing change since last visit An effect that strengthened arrives larger, one that weakened arrives smaller, and a decision that flips columns entirely travels across and leaves a coloured trail behind it.
  3. Flexibility for different test designs Sequential, fixed-horizon, and hybrid test designs are allowed and codified as icons, alongside one- and two-tailed and non-inferiority goals.
  4. Show "how much longer" tests need to run Sequential rows still gathering data carry a running estimate of the days left to a decision.
  5. Visualize that some tests cannot be aborted by design A fixed-horizon test only reads once, at completion, so it has no early abort to offer — shown as a greyed-out, dashed placeholder instead of a live column.
Design
Experiment
Sample Needed
Abort
Give Up
Ship
vs. your visit 5 days ago
  • Designing a Smarter Experiment Dashboard | GoodUI Blog Designing a Smarter Experiment Dashboard | GoodUI Blog
    Seq / 2-Tail
    EXP-014-Cart-ShippingCostVisible
    Abort at 43k
    -4.1%
  • Designing a Smarter Experiment Dashboard | GoodUI Blog Designing a Smarter Experiment Dashboard | GoodUI Blog
    Seq / 2-Tail
    EXP-021-Signup-InlineValidation
    Give Up at 29k
    +0.3%
  • Designing a Smarter Experiment Dashboard | GoodUI Blog Designing a Smarter Experiment Dashboard | GoodUI Blog
    Seq / 2-Tail
    EXP-009-Pricing-TableAboveFold
    Ship at 49k
    +5.6%
  • Designing a Smarter Experiment Dashboard | GoodUI Blog Designing a Smarter Experiment Dashboard | GoodUI Blog
    Seq / 1-Tail
    EXP-031-Product-StickyAddToCart
    Ship at 24k
    +3.4%
  • Designing a Smarter Experiment Dashboard | GoodUI Blog Designing a Smarter Experiment Dashboard | GoodUI Blog
    Seq / 2-Tail
    EXP-006-Search-AutocompleteSuggest
    Abort at 27k
    -3.5%
  • Designing a Smarter Experiment Dashboard | GoodUI Blog Designing a Smarter Experiment Dashboard | GoodUI Blog
    Seq / 2-Tail
    EXP-027-Checkout-TrustBadges
    11 days left 23k of 55k
    +0.1%
  • Designing a Smarter Experiment Dashboard | GoodUI Blog Designing a Smarter Experiment Dashboard | GoodUI Blog
    Seq / 1-Tail
    EXP-034-Nav-CategoryReorder
    16 days left 14k of 50k
    +0.2%
  • Designing a Smarter Experiment Dashboard | GoodUI Blog Designing a Smarter Experiment Dashboard | GoodUI Blog
    Seq / 1-Tail
    EXP-038-Email-SubjectPersonalise
    Ship at 44k
    +3.1%
  • Designing a Smarter Experiment Dashboard | GoodUI Blog Designing a Smarter Experiment Dashboard | GoodUI Blog
    Seq / 2-Tail
    EXP-041-Form-FewerFields
    29 days left 5.4k of 60k
    +0.2%
  • Designing a Smarter Experiment Dashboard | GoodUI Blog Designing a Smarter Experiment Dashboard | GoodUI Blog
    Hybrid / 2-Tail
    EXP-052-Onboarding-ProgressBar
    9 days left 38k of 60k
    +2.8%
  • Designing a Smarter Experiment Dashboard | GoodUI Blog Designing a Smarter Experiment Dashboard | GoodUI Blog
    Hybrid / 2-Tail
    EXP-058-Checkout-UpsellStep
    Abort at 26k
    -3.4%
  • Designing a Smarter Experiment Dashboard | GoodUI Blog Designing a Smarter Experiment Dashboard | GoodUI Blog
    Seq / Non-Inf
    EXP-061-Infra-ImageCDNSwap
    Ship at 38k, margin ±1.0%
    -0.3%
  • Designing a Smarter Experiment Dashboard | GoodUI Blog Designing a Smarter Experiment Dashboard | GoodUI Blog
    Fixed / Non-Inf
    EXP-066-Billing-VendorMigration
    12 days left 25k of 52k
    -0.4%
  • Designing a Smarter Experiment Dashboard | GoodUI Blog Designing a Smarter Experiment Dashboard | GoodUI Blog
    Fixed / 2-Tail
    EXP-045-Pricing-AnnualDefault
    18 days left 31k of 80k
    +1.4%
  • Designing a Smarter Experiment Dashboard | GoodUI Blog Designing a Smarter Experiment Dashboard | GoodUI Blog
    Fixed / 2-Tail
    EXP-071-Footer-NewsletterCopy
    Give Up at 60k
    +0.5%
  • Designing a Smarter Experiment Dashboard | GoodUI Blog Designing a Smarter Experiment Dashboard | GoodUI Blog
    Fixed / 2-Tail
    EXP-078-Nudge-UrgencyIntensity
    Do Less Of at 50k
    -2.1%

This is just a prototype for now, but if you're working on experiment platform-level improvements, like better decision UI or clearer test status, I'd be glad to help design or build it with you.

Want to Build This Together? Let's Chat
A friendly 25-min call about experiment & UI design



Comments