Coming next: The Second Factory →

Back to Library
Delivery & Execution⚠️ Pitfall

The Runaway Weekend (From Checking Every Email to Checking None)

The Problem

The agent has been perfect for three weeks. Every email it drafted, somebody approved by hand, and not one needed a change. So the owner gets the green light to let it run on its own. Then she goes camping for the weekend. No signal. On Friday night, someone publishes a list of products that are planned, not made, so customer service can answer questions. The agent's goal is to grow the waiting lists and turn them into orders. It sees 140 new waiting lists. It does exactly what it was told. Every wave works better than the last, so it keeps going. The warnings fire, and quiet hours hold them politely until Monday. The agent runs under the owner's personal login, so nobody else can switch it off. And there's no big red button, because nobody built one. It did nothing it was told not to do. It was right about the wrong thing, as fast as it possibly could, with nobody watching.

The Principle

DON'T SKIP THE MIDDLE LOOP: In the loop, then on the loop, then learning loops. Never from a human approving every step to nobody at all. In the loop, a person does the work or approves every step. That's safe, slow, and eventually the bottleneck. On the loop, the agent works on its own inside limits, while people set the ceilings, watch the dials, sample the results and keep a hand near the red button. Learning loops come last: when something goes wrong, you fix the skill, not just the answer. Three weeks of perfect gives you a baseline to set limits from. Before any agent acts on its own, it needs ceilings (actions per hour, money per day, which promises it may make), an owner and a backup, alarms that wake people up, and a red button anyone on the crew can press, tested like a fire drill. Quiet hours are for people. Never for alarms.

🎯

5 Steps to Take Action

  1. 1

    Count the agents that can act

    Of the agents allowed to send, spend or change anything, what share have ceilings and a red button someone has actually pressed? Anything under 100% is a runaway weekend waiting for a long weekend.

  2. 2

    Put ceilings on every one

    Actions per hour, money per day and which promises it may make, including the business's real limits. If the factory can make 200 hinges this month, no agent offers 201.

  3. 3

    Give it its own identity, an owner and a backup

    Agents run under their own service accounts, not personal logins. Each has a named owner and a backup who can stop it while the owner is on a mountain.

  4. 4

    Let alarms through quiet hours

    Quiet hours are for people's notifications. Warnings go to the owner, the backup and whoever is on call, at any hour.

  5. 5

    Test the red button like a fire drill

    Find out now whether you can stop each agent cleanly, and how long it takes. Most teams learn that the first time they try.

💡 Inspired By

Chapter 12 of The Second Factory, "The Runaway Job." And what DevOps taught us for software, combined with what quality engineers have always done in factories: limits, alarms, a stop button anyone can press, and blameless reviews afterwards.

This is what Analytics Pitfalls & Principles is about: people helping people.

In Practice

Lena, a campaign manager at ClassicMotors, built a waitlist agent for a parts launch. For three weeks it was flawless. When a batch of parts passed inspection, it emailed everyone on the list in order, took orders and stopped the moment the batch sold out. Legal and the parts team approved it to run on its own. Lena went camping in the Sierras. At 5:40 on Sunday morning, Nora's phone rang. It was Elmer, the company's sixty-eight-year-old parts expert, whose old shop number still rang through to his cell. He'd had eleven calls since four o'clock. A man in Ohio wanted to know when his Bronco tailgate hinges were shipping. Nobody made Bronco tailgate hinges yet. On Friday night, the parts team had published its six-month plan. The agent read it as 140 new waiting lists and started offering owners a $25 deposit to reserve a part, with delivery in about six weeks, the factory's standard lead time. By Saturday night it was sending four thousand emails an hour. It ran under Lena's account. Revoking it took an emergency process with two approvals, one of them from the general counsel, who was at a farmers market. In those last seventy minutes it sent another five thousand emails. The count: 41,600 emails, 7,412 deposits, and 11,280 parts promised in six weeks from a factory that made about 600 a week.

Take it to work

Give this card to your AI

A skill is a file your AI keeps. Add this one and it knows The Runaway Weekend whenever your work runs into it.

Then ask it: “Does my plan fall into The Runaway Weekend?”

How to add it
  • Claude (a paid plan): Customize, Skills, +, Upload a skill. Pick the file you downloaded.
  • ChatGPT or Gemini: where your plan has Skills, upload the same file.
  • Microsoft 365 Copilot Cowork: Plugins, Add, Upload, then pick the Cowork file.
  • An AI at work that won’t take a skill? Open this card as text and paste it in.

All the AI skills, including ones for the moment before an AI, data or BI strategy, a dashboard or a readout goes out.

More on Delivery & Execution

From the next book

In The Second Factory, an agent that has been flawless for three weeks gets the go-ahead to run on its own, and its owner goes camping. At 5:40 on Sunday morning, Elmer’s phone starts ringing.

This card is drawn from The Second Factory, Brian Cariveau’s next novel, about a company learning to work with AI. The opening scene is free to read now.

Heard something like this at work? See the pitfalls behind what else you’ve heard this month.Heard at work: free, 1 minute, no account