Coming next: The Second Factory →

Back to Library
Metrics & Evidence⚠️ Pitfall

Ice Cream Causes Drowning (When Numbers Lie)

The Problem

VP of Sales storms into your office: "I ran the numbers—our email campaigns drive revenue! Every time we send emails, sales go up. Send more emails!" Sounds great, right? Except you dig deeper and find out sales go up on Mondays. You send emails on Mondays. Sales would've gone up anyway. The emails did nothing. But leadership is already budgeting for a bigger email team. It's the classic trap: ice cream sales correlate with drowning deaths. Both happen in summer. One doesn't cause the other. But if you don't understand causation, you might ban ice cream to save lives. Misunderstanding correlation leads to terrible decisions, wasted budget, and looking like an idiot when it doesn't work.

The Principle

Always ask: "What else could explain this relationship?" Correlation is a clue, not proof. To establish causation, you need controlled experiments—A/B tests, holdout groups, something that isolates the variable. Think manufacturing: if quality improves after you change suppliers, is it the new supplier? Or is it because you also trained staff that week? You need to isolate variables to know what actually works. The magic happens when you stop jumping to conclusions and start designing experiments. Less guessing, more testing. That's how you turn data into decisions you can trust.

🎯

3 Steps to Take Action

  1. 1

    Ask "What Else?" Before You Act: Anytime you see correlation, write down 3 other possible explanations. Force yourself to consider alternatives before jumping to conclusions.

  2. 2

    Run A/B Tests on Big Bets: Before you invest in something based on correlation, test it. Holdout groups, controlled experiments, anything that isolates the variable.

  3. 3

    Build a "Causation Checklist": Create a simple checklist your team uses before acting on data: Did we isolate the variable? Did we test it? Do we have a plausible mechanism? No shortcuts.

In Practice

A retailer noticed ice cream sales correlated perfectly with drowning deaths in their beach town markets. Data team built a model: "Reduce ice cream promotions to reduce drowning risk." Marketing almost cut ice cream displays. Then someone asked: "What else happens when ice cream sales spike?" Summer. Hot weather. More people at the beach. They ran an A/B test: some stores cut ice cream displays, others didn't. Drowning rates? Identical. Ice cream sales in test stores? Down 30%. Revenue hit. Turns out the real cause was beach traffic, not ice cream. They reversed course, ran beach safety campaigns instead, and kept selling ice cream. Revenue recovered. Lives saved. That's the difference between correlation and causation.

More on Metrics & Evidence

From the book

In the book this comes to a head in a board meeting, where growth theater meets its limit. The fix is honest metrics, not better slides.

This card is one of 57 drawn from The Auction Block by Brian Cariveau — the story of an analytics leader with 90 days to fix what the cards describe.

Each leadership pattern falls into a few pitfalls most often. Find your pattern.The Analytics Leadership Mirror: free, 5 minutes, no account