The Rain Dance That Worked (When You Credit the Wrong Thing)
The Problem
Sales team starts a new outreach campaign in January. Revenue spikes in February. VP of Sales declares victory: "Our campaign worked! Do more of it!" Except revenue spikes every February—it's tax refund season. The campaign did nothing. But the team doubles down, wastes budget, and wonders why results don't repeat. It's the rain dance fallacy: you do a rain dance, it rains, you assume the dance worked. Ignore that it was already cloudy. Humans are wired to find patterns and assign credit, even when it's coincidence. In business, this leads to investing in things that don't work and ignoring things that do. It's expensive and dumb.
The Principle
Before you credit success to your action, ask: "What would have happened anyway?" Use control groups, historical baselines, and holdout tests to separate your impact from external factors. Think manufacturing: if defect rates drop after you change suppliers, is it the supplier? Or is it the new QA process you also implemented? Isolate variables. The magic happens when you know what actually drives results so you can do more of it and less of everything else. Less guessing, more isolating. That's how you spend budget wisely.
3 Steps to Take Action
- 1
Check Historical Baselines: Before crediting your action, look at what happened in prior years. If sales spike in December, check last December. Separate your impact from seasonality.
- 2
Use Control Groups: For big initiatives, test with a control group. Half get the treatment, half don't. Measure the difference. That's your true incremental impact.
- 3
Build an "Impact Checklist": Before celebrating success, ask: What would have happened anyway? Did we isolate our impact? Do we have a control? No shortcuts on causality.
In Practice
A retail chain launched a loyalty program in November. December sales were up 30%. CMO presented to the board: "Our loyalty program is a massive success!" Allocated $5M to expand it nationally. But someone checked: December sales were up 30% last year too—holiday season. They ran a test: half the stores got the loyalty program, half didn't. Results? Stores with loyalty: +31%. Stores without: +29%. The program added 2%, not 30%. Holiday season did the heavy lifting. They scaled back the rollout, optimized the program, and actually measured incremental lift. Next iteration added 8%. That was worth scaling. Saved $3M by not over-investing in something that barely worked. All because they isolated causality instead of celebrating coincidence.
