Your AI Didn't Break the Process. You Never Had One.

A head of engineering I coach wired an agent into on-call triage in July. Read the alert, read the runbook, open the ticket, page the owner. Reasonable scope. Week one, it paged the wrong team three times and closed two real incidents as noise. He opened our call with the verdict already written: the AI broke triage.

I asked to see the runbook. He sent a wiki page last edited two years ago. Then I asked to sit in the on-call channel for a day. By noon I had watched one staff engineer make eleven decisions the runbook did not contain. Billing alerts on Monday morning are the batch job, ignore them. The severity field on the auth service has been wrong since the migration, treat a P3 as a P1. Anything from the payments queue pages Priya, no matter what the routing table says.

The agent had followed the page to the letter. The humans had been running a different process for two years, and nobody had written that process down because nobody had needed to. That includes her. The person who carries the real rules in her head owes the page too, whether or not anyone asks for it. The staff engineer was the process.

I have my own version, so this is not a story about someone else’s shop. This site runs on scripts I wrote for the cover image, the concept tagging, and the deploy. The first time I let an agent run the whole publish flow end to end, it did every step on my checklist and shipped a cover with a stranger’s name where mine goes. The image model had been inventing bylines for months. I had been catching them by eye and never wrote the check down, because I did the check. My checklist was a story I told about my process. The agent ran the story.

The Agent Runs the Process You Have

A model wired into a workflow does the system you actually have. It cannot do the system you describe in the offsite deck. The gap between those two has always existed. What changed is that a human reads a bad rule, shrugs, does the right thing, and never files a ticket about it. An agent reads the bad rule and follows it forty times before lunch.

A process is a set of rules that produce the same decision no matter who runs them, including a written step that says a human decides this one. A workflow that only works while one specific person is in the chair is a dependency on her, and the title of this piece is aimed at exactly that case. Every team I work with has three processes: the one on the wiki, the one in the best engineer’s head, and the one that actually runs, which is the second one with the first one as decoration. Automation finds the distance between them and makes it loud in the first week.

This is why teams that bought seats and gave the model nothing real still feel safe. Access is the prior constraint. A model that can only see a PDF cannot embarrass anyone. The moment it can see the ticket queue and act, it starts showing you what your process document left out. That is the feature, even though it feels like a bug, because the finding lands on the person who signed off on the document.

Deciding what to automate is a different essay. This one starts after the decision, when the agent is live and the first wrong page goes out. What you do in that hour tells me whether the automation will survive the quarter.

The Four Dodges

I watch the same four moves every time a wired-up agent misfires. They all protect the same thing: the belief that the process was real.

Blame the model. “It hallucinated.” Sometimes. Check the runbook before you say it. In that triage shop, every one of the five bad calls in week one matched a line on the wiki page. Not one was invented. The model read the routing table and paged what the table said. Calling that a hallucination let the team skip the question of who wrote the table and why nobody had touched it since the migration.

Keep the hero patch. One person fixes the exceptions every morning. The dashboard looks green because she catches the misroutes before anyone upstairs sees them. Wire the agent in and her patches stop, because the agent now owns the queue. The misroutes become visible. The team blames the agent for problems the hero used to eat in silence. In the triage shop, that staff engineer had been correcting the runbook by hand, in her head, every shift, for two years. Nobody upstairs knew. The dashboard did not show it. Her quiet competence was the reason the document never got fixed.

Automate before you audit. Buying the tool takes an afternoon. Writing down the real process takes weeks and a lot of awkward conversations. So the tool goes in first, with the idea that it will “learn the process.” It will learn the document instead. The audit gets skipped because the audit would show who is holding the place together and which steps have been dead for a year, and those are political findings. The discomfort does not cancel the duty. The leader owes the audit no matter whose name it lands on. Wiring in the agent feels like progress while the document stays wrong.

Defend dead rules. Every runbook has steps nobody can explain. Restart the cache before you check the logs. Always CC the old platform alias. Wait ten minutes after the deploy before you trust the health check. Someone added each of these for a reason that is now gone. Humans skip them without thinking. The agent runs them every single time, and now the dead rule has a blast radius. When you find one, the honest move is deletion. The common move is a long thread about how the rule “might still matter.”

What you tolerate becomes training. A wrong runbook you tolerated for two years trained the team to work around it. The agent has no idea how to work around anything, so it complies with the wrong page every time.

What Actually Works

The head of eng wanted to turn the agent off and go back to the humans. I talked him out of it, on one condition: the agent could not take an action for three weeks. Here is what we did instead.

Run it in shadow mode first. The agent reads every alert, decides what it would do, and writes the decision to a log. It touches nothing. Every morning, someone compares the agent’s log to what the humans actually did overnight. Each difference is a diff, and you write them all down.

Sort every diff into one of three buckets. An undocumented rule: the human did something the page did not say, and they can explain why. Write it into the runbook. A dead rule: the page says to do something and the humans never do it. Delete it. A judgment call: the human decided based on context the page could never hold, like knowing the product team was mid-launch. Keep a person on that one and give the agent a clear handoff. Writing down that a human decides this one keeps it a process, which is different from the silent step that only works because one person happens to be on shift.

Feed it real data. The example ticket in the wiki is fiction. Give the agent last quarter’s actual alerts, with all the malformed ones and the duplicate pages and the alert that fires with an empty body every Tuesday. If the agent only works on the tidy example, it does not work.

Notice what is yours in all of this and what is not. You own the document, the shadow period, the diff log, and the deletion of dead rules. You do not own the model’s quality or the vendor’s roadmap, and nothing in this method depends on either. A worse model next quarter changes the diff count, not the work.

Keep the order fixed: structure, then visibility, then AI last. Get the real rules out of people’s heads and onto the page. Make the process observable so a diff is even possible. Only then let the agent act. Most teams run this backwards. They buy the agent, hope it makes the process visible, and never get to structure because the fire starts before they do.

Two weeks of shadow mode in that shop produced forty-one diffs. Twenty-six were undocumented rules, which the staff engineer dictated in one afternoon once someone finally asked. Nine were dead rules, deleted with no complaints. Six were judgment calls, which stayed with a human. The runbook went from a two-year-old page to a document that matched the work. Then the agent went live. Wrong pages dropped to zero in the first week, and the staff engineer got her mornings back. That is one shop and one quarter, so take it as a report, not a law.

Between week one and week four, the only thing that changed was the document.

Agents make every interrupt feel free is the cousin about what the tool costs your attention. This one is about what it reveals. Both land in the same place: the tool is neutral, and your process is the variable.

Look Before You Wire It In

If your best on-call person quit this Friday, would the agent still work Monday? If the honest answer is no, you have a hero with a robot assistant. Write down what she knows before she takes a vacation and the agent takes the shift alone.

Open the runbook for the workflow you automated most recently. Count the steps nobody on the team can explain. Each one is a rule the agent is following right now for a reason that may no longer exist.

Pick one workflow you want to automate next. Before it touches anything, run it beside the humans for ten days and log every disagreement. If you are unwilling to spend ten days finding out what your process actually is, you are not ready to let a model run it.

Then say the finding out loud to the team. Skip “the agent needs tuning” and say the real sentence: we found out the process lived in three people’s heads, and now it lives on the page. That sentence is the only version that makes the next automation easier. The page itself matters less than what it proves: an understanding of the work that does not leave the building when one person does. The page is how you check whether you have it.

Final Thoughts

The agent is a mirror. What you see in it is the process you actually have, with the shrugs and the patches removed. The uncomfortable part is finding out how much of “our process” was a few people being very good at their jobs and never mentioning it.

If the first wrong page makes you want to blame the model, open the runbook first. The model probably did what it said. The question worth your quarter is why the page was wrong for two years and who was quietly making it right.

If you want a community that treats automation as a way to see your own process clearly, MasteryLab is where engineering leaders practice fixing the document before they wire in the agent.

FAQ

Why does an AI agent seem to break a process that worked for years?

Because the process that worked for years was not the one on the wiki. It was the wiki plus a few people who quietly did the right thing when the page was wrong. The agent follows the page. It cannot see the shrug that a human adds when the page does not fit. Check the document before you blame the model. Most of the time it did exactly what the document said.

What is shadow mode for AI automation?

You let the agent run beside the humans for a set period, usually one to three weeks, with no power to act. It reads the same inputs and writes down what it would have done. Every day you compare its decisions to the human decisions. Each difference is a finding: an undocumented rule, a dead rule, or a judgment call. You fix the process from those findings before the agent gets to touch anything.

How do I find tribal knowledge before automating a workflow?

Ask the person who has been on call the longest to walk one real week of decisions out loud while you compare each one to the written procedure. Every place they deviate is a rule that lives in their head. Then run the agent in shadow mode against real tickets, not the example ticket in the wiki, and log every disagreement. Two weeks of diffs will show you most of what was never written down.

What should a manager do when the agent exposes that the process lived in one person’s head?

Thank that person in public, then get the rules out of their head and into the document before they take a vacation. Do not treat the finding as their failure. They kept the process alive without help. Treat it as your finding: the team had a hero, not a process. Fix the document, delete the dead steps, and keep a human on the judgment calls.