Legal Reads Everything (When Checking Every Answer Catches Less)
The Problem
After the first scare, your lawyers do the responsible thing. Every new kind of message an agent sends gets reviewed by Legal first. It feels like the safe thing to do. It becomes a queue. Thousands of items a week for a handful of people. Lawyers reviewing marketing emails at nine at night. A team waits six days to update a refund answer. A product launch waits for page approvals. And the people doing the checking get so tired that they stop trusting their own reviews. One of your best lawyers approves an email with the wrong state's warranty language in it. Checking everything feels safe. After a while, it catches less than good sampling would.
The Principle
Decide what safe enough looks like before you need it. Check by risk, not by habit. Quality engineers call checking everything hundred-percent inspection, and they know its weakness: inspectors get tired, and tired inspectors miss things. Not every output is equally risky, so don't check them all the same way. Build a tolerance table. Loose: marketing stories, event announcements, thank-you notes. Sample a few a week and watch the trend. Medium: order updates, delivery dates, refund answers. Sample every day with tight limits, and have a person check anything unusual. Tight: warranty terms, legal promises, anything to do with safety. A person checks every one, and the agent doesn't improvise a word. Let the charts earn the right to move things down a tier. That's how Legal goes from guarding the gate to designing the guardrails.
5 Steps to Take Action
- 1
Measure the queue
Items reviewed per week, people reviewing, and the oldest item waiting. Then ask the reviewers how much they trust their own reviews.
- 2
Sort every output into loose, medium and tight
A half-day workshop with Legal, the business owners and someone who knows quality. Expect Legal to put everything in tight at first.
- 3
Sample the loose and medium tiers
On a schedule, against a short checklist, with the scores on a chart.
- 4
Keep tight tight
A person checks every one, and the agent uses approved wording only.
- 5
Move things down with evidence
When an agent's chart has stayed inside its limits for months, it earns a looser tier. Not before.
💡 Inspired By
Chapter 15 of The Second Factory, "Tolerances." And the old lesson from quality engineering that hundred-percent inspection, done by tired people, catches less than good sampling.
This is what Analytics Pitfalls & Principles is about: people helping people.
In Practice
After a runaway weekend, ClassicMotors' general counsel, Harriet Cole, had Legal check every new type of message an agent sent customers: every policy answer, every promise. "Four thousand two hundred items a week," she told Nora, with her office door closed. "Five people. We're reviewing marketing emails at nine at night. Marisol's team has been waiting six days to update the refund skill. Dana can't launch the Chevelle parts until we approve the product pages. And here's the part that terrifies me. We're so tired now that I don't trust our own reviews." The week before, one of her best lawyers had approved an email with the wrong state's warranty language in it. Customer service caught it. Nora brought in the factory's quality engineer, who had spent twenty-two years making bolts for airplanes. She listened to all of it. "You're doing hundred-percent inspection," she said. "It feels safe, but it isn't." "These aren't bolts," Harriet said. "This is legal language going to customers." "The principle's the same. First question. Is every message equally risky?"
