Coming next: The Second Factory →

Back to Library
Metrics & Evidence⚠️ Pitfall

The Statistician Who Drowned (in a River with Average Depth 3 Feet)

The Problem

A statistician drowned crossing a river with an average depth of 3 feet. How? Parts of it were 12 feet deep. Averages hide the extremes. Your average customer spends $50—great! Except 90% spend $5 and 10% spend $500. Those are two completely different customers who need different strategies. But if you only look at the average, you optimize for nobody. Same in manufacturing: average defect rate of 2% sounds fine until you realize one production line is at 0.1% and another is at 15%. Averages smooth out the truth. They make you comfortable when you should be alarmed. They hide the gold (power users) and the disasters (broken processes).

The Principle

Dig into the distribution, not just the average. Look at medians, percentiles, outliers. Segment your data—what's the average for each customer type, region, or product line? In manufacturing, you wouldn't just track overall output—you'd monitor each machine, each shift, each operator. Same here. The magic happens when you stop asking "what's the average?" and start asking "what's the spread?" That's where you find opportunities to fix disasters and double down on winners. Less averaging, more slicing. That's how you see the truth.

🎯

3 Steps to Take Action

  1. 1

    Always Look at the Distribution: For any metric you track, pull the distribution. Not just the average - show min, max, median, 25th/75th percentiles. Make it a standard in every report.

  2. 2

    Segment Everything: Break down averages by customer type, region, product line, whatever matters. Build dashboards that show segments by default, not just totals.

  3. 3

    Ban "Average" Without Context: Institute a rule: Nobody can present an average without also showing the distribution or key segments. Force people to see the spread.

In Practice

An app had an "average session length" of 8 minutes. Product team celebrated—users are engaged! Then someone looked at the distribution. 60% of sessions were under 30 seconds (people opening the app by accident). 30% were 2-3 minutes (quick tasks). 10% were 45+ minutes (power users binging content). Three totally different behaviors masked by one average. They segmented: built a better onboarding flow for the 30-second folks (converted 40% to 2+ minutes), optimized quick tasks for the 2-3 minute users (task completion up 50%), and built binge-worthy features for power users (session length jumped to 90 minutes). Revenue tripled in 6 months. All because they stopped looking at averages and started looking at reality.

More on Metrics & Evidence

From the book

In the book this comes to a head in a board meeting, where growth theater meets its limit. The fix is honest metrics, not better slides.

This card is one of 57 drawn from The Auction Block by Brian Cariveau — the story of an analytics leader with 90 days to fix what the cards describe.

Each leadership pattern falls into a few pitfalls most often. Find your pattern.The Analytics Leadership Mirror: free, 5 minutes, no account