---
name: analytics-pitfalls-legal-reads-everything
description: "\"Legal Reads Everything\": a pitfall card from Analytics Pitfalls & Principles. Use when a plan, request, dashboard, metric or AI project might be falling into it."
license: CC-BY-4.0
---

# Legal Reads Everything (When Checking Every Answer Catches Less)

A pitfall card from Analytics Pitfalls & Principles, drawn from *The Second Factory*, Brian Cariveau's next novel (https://www.analyticspitfalls.com/second-factory).
https://www.analyticspitfalls.com/cards/83

## How to use this card

Use it when what someone is working on, or something they heard at work, looks like the problem below: a plan, a roadmap, a request, a dashboard, a metric, an AI project.

1. Name the card by its title and say in a sentence why it fits. Say the work looks like this pitfall; never tell someone they have it.
2. Give the principle in the card's own words. Don't rewrite it.
3. Take the steps one at a time against their actual situation: what is already done, what isn't, and which to do first. If you need a fact to do that, ask one question, not five.
4. Then push past the technical fix, if the work needs people to work differently (most does). The steps are the if-then part. Ask what has to change in how people work, what they have to learn or believe, or what they give up, for it to stick. That's the adaptive work (Ronald Heifetz's term), and it's usually the real work.
5. If the card doesn't fit, say so plainly. Don't stretch it.
6. Finish with the card's link: https://www.analyticspitfalls.com/cards/83

## The problem

After the first scare, your lawyers do the responsible thing. Every new kind of message an agent sends gets reviewed by Legal first. It feels like the safe thing to do.

It becomes a queue. Thousands of items a week for a handful of people. Lawyers reviewing marketing emails at nine at night. A team waits six days to update a refund answer. A product launch waits for page approvals.

And the people doing the checking get so tired that they stop trusting their own reviews. One of your best lawyers approves an email with the wrong state's warranty language in it. Checking everything feels safe. After a while, it catches less than good sampling would.

## The principle

Decide what safe enough looks like before you need it. Check by risk, not by habit.

Quality engineers call checking everything hundred-percent inspection, and they know its weakness: inspectors get tired, and tired inspectors miss things. Not every output is equally risky, so don't check them all the same way.

Build a tolerance table. Loose: marketing stories, event announcements, thank-you notes. Sample a few a week and watch the trend. Medium: order updates, delivery dates, refund answers. Sample every day with tight limits, and have a person check anything unusual. Tight: warranty terms, legal promises, anything to do with safety. A person checks every one, and the agent doesn't improvise a word. Let the charts earn the right to move things down a tier. That's how Legal goes from guarding the gate to designing the guardrails.

## The steps

1. **Measure the queue.** Items reviewed per week, people reviewing, and the oldest item waiting. Then ask the reviewers how much they trust their own reviews.
2. **Sort every output into loose, medium and tight.** A half-day workshop with Legal, the business owners and someone who knows quality. Expect Legal to put everything in tight at first.
3. **Sample the loose and medium tiers.** On a schedule, against a short checklist, with the scores on a chart.
4. **Keep tight tight.** A person checks every one, and the agent uses approved wording only.
5. **Move things down with evidence.** When an agent's chart has stayed inside its limits for months, it earns a looser tier. Not before.

## In practice

After a runaway weekend, ClassicMotors' general counsel, Harriet Cole, had Legal check every new type of message an agent sent customers: every policy answer, every promise.

"Four thousand two hundred items a week," she told Nora, with her office door closed. "Five people. We're reviewing marketing emails at nine at night. Marisol's team has been waiting six days to update the refund skill. Dana can't launch the Chevelle parts until we approve the product pages. And here's the part that terrifies me. We're so tired now that I don't trust our own reviews." The week before, one of her best lawyers had approved an email with the wrong state's warranty language in it. Customer service caught it.

Nora brought in the factory's quality engineer, who had spent twenty-two years making bolts for airplanes. She listened to all of it. "You're doing hundred-percent inspection," she said. "It feels safe, but it isn't."

"These aren't bolts," Harriet said. "This is legal language going to customers."

"The principle's the same. First question. Is every message equally risky?"

---

© 2026 Brian Cariveau, Analytics Pitfalls & Principles. Free to share and adapt under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/): keep this credit, "Brian Cariveau, Analytics Pitfalls & Principles (https://www.analyticspitfalls.com)", and say if you changed it.
The names Analytics Pitfalls & Principles, The Auction Block and The Second Factory are not part of the license. Don't use them for your own version, or to suggest it comes from us.
