Start With the Model. Harness What You Cannot Afford to Guess.

Claude offered to put itself in my build last week. I had asked it to fix a formatting bug on this site, a field that sometimes had quotes around it and sometimes did not. The first plan it wrote was to add a model call to the build step. Every build, forever, to strip quotes.

I said no and asked for the function. It wrote the function and the tests in one pass, faster than I could have typed the first line, and the model has not been in that build since. The model was never the problem. The build is a harness, and stripping quotes is not a decision I need made fresh every morning.

That is the whole argument, and it is the opposite of the one most engineering teams are having right now. They are arguing model versus code one ticket at a time, slowly, with a lot of nervous energy about the bill. The faster teams stopped arguing. They start with the model, let it run in a loop until its own checks pass, and keep a short written list of the actions it cannot take on its own. That list, plus the environment the loop runs in, is the harness. The rest of this piece is about what goes in it and how short it can be.

What Stays in the Harness Is the Only Question Left

Michael Frendo, the CTO at New Relic, runs his engineering organization on one sentence I heard in an interview: if it can be solved deterministically, it should be, because the cost is lower and the answer can be promised. I have been turning that sentence over for a month, and I think it is half right, and the two halves are not equal.

The cost half is weak and getting weaker, and the promise half is the entire harness.

Deterministic means the same input produces the same output every time, and a test can prove it. A tax-rounding function is deterministic. A model call is not. It gives you a distribution of answers with a most likely one on top, and the shape of that distribution moves when the vendor ships a new version on a Tuesday. You cannot test a distribution. You can sample it on a schedule and notice when it moved, which is what an eval is, and that is a different thing from a promise.

So the routing question is not “can the model do it.” The answer to that is yes for almost anything that crosses a desk, and a question whose answer is always yes cannot help you decide. The routing question is “what do I have to promise about this result, and to whom.” A promise is understanding in a form another person can rely on, which is why it outranks speed on the few paths where the two conflict. Every path where the answer is “nothing, and nobody” belongs to the model by default, and every path where the answer is “this exact number, to an auditor, next April” gets a gate.

Smart model, dumb harness. The model does the deciding. The harness is the environment it decides in: the repo it can read, the tests and linters that tell it whether it was right, the skills it can load, the hooks and permissions that block the few actions it cannot take alone, and the stop condition that ends the loop. The harness stays thin on purpose so it can be tested and promised, which is what lets the model stay as powerful and as fast as the tool allows.

Three Things That Set the Length of the List

Risk sets the gates. Governance, in the plain sense I mean here, is the set of places an action can be stopped before it is real, and risk decides how many of those places a path gets. The software development lifecycle has gates for a reason. A change to a function goes through a pull request, a review, a test run, and a deploy, and every one of those is a place a bad change can get caught. A policy the agent reads at run time has none of that unless you build it. The change is live the moment someone saves the file. That is the trade, stated plainly, and it is a good trade on a lot of paths. The moderation rules for a discussion forum, written in plain language and read by the model next to every post, are a better fit for a policy the owner edits than for a ticket that waits for a sprint, and the rounding on the invoice is not. The higher the risk, the more places an action passes through before it is real, and on the highest-risk paths one of those places is a person.

Volatility favors the model. Adding two numbers is never going to change, so it is a function and you will never think about it again. “What counts as an acceptable post” changes every month as a community grows, and the person who knows the new answer is a moderator, not an engineer. Every one of those changes is a deploy if the logic is code and an edit to a policy file if the logic is something the model reads. The more often a rule changes, and the further the people who change it sit from the repo, the more the model earns the path. There is a counterweight, and I take it seriously: with agents, changing code got cheap too. The function that used to be a sprint is twenty minutes and a pull request now. So the remaining advantage of model-driven logic is narrower than “it is easier to change.” It is that non-engineers can change it, that some rules are too fuzzy to write down at all, and that a loop can update its own rules when the data moves, which I will get to. All three are real, and none is a reason to keep the model out.

Latency is the cost that still counts. The bill is a weak reason to avoid the model, and I want to say that clearly because it is the reason most teams reach for first. Models get smarter, which means fewer calls to get the same outcome. Then they get cheaper and faster without losing the smartness. Local inference is going to move both the price and the round trip in ways nobody is pricing yet. Token usage never goes down, and that is fine. Calls per outcome fall while what counts as an outcome grows, so the total rises and the per-outcome argument still holds. Hunting for the cheapest model is a waste of an engineer’s afternoon. What does not get cheaper on its own is a second-long call in a checkout flow that used to take a millisecond. That is a product decision, and it belongs in the harness conversation, not the budget one.

Put the three together and you get two axes that matter: how much you must promise, and how often the rule changes. Risk and stakes are how you size the first axis, not a third one. Stable and must be promised, like auth or tax rounding, is code, and it is code you let the model write once. Stable and low stakes, like stripping quotes, is code because it is free and nobody wants to think about it again. Changes often and low stakes, like the moderation policy, is the model, and the owner edits the policy. Changes often and must be promised is the interesting quadrant, and it is where most of the real product logic in a company lives. The model stays in the loop there. The policy it reads gets an eval that runs on a schedule and a gate before an edit takes effect. That is a harness too, one built around a policy instead of a function.

The Harness Is the Environment, Not a Wrapper

Here is where I was wrong for a long time, and where most teams still are. I pictured the harness as code I would write around a model call: parse the input, call the model, check the output, retry. That is a wrapper, and it is what you build when you think the model is a function you are calling.

The teams going fastest do not work that way. OpenAI’s Codex team shipped about a million lines of production code in five months with no human-written source, and the engineers described their job as designing the environment, not writing prompts. The harness was the repo itself: a short table-of-contents file for the agent, design docs and execution plans checked in as markdown, linters the agents generated to enforce the architecture, structural tests that fail when a dependency points the wrong way. Their sentence for it was that from the agent’s point of view, anything it cannot reach while it is running does not exist. When something broke, nobody rewrote a prompt. They asked what capability was missing and how to make it legible and enforceable.

Anthropic describes the same thing from the inside. The loop is gather context, take an action, verify the result, repeat until a stop condition. The human sets what done looks like, the turn cap, and the checks. The loop does the rest, and the more quantitative the checks, the better it verifies itself. Skills are folders of instructions and scripts the agent loads only when a task calls for them, and other agents and automations can call them too. Cursor ran hundreds of agents on one codebase for a week as planners, workers, and a judge that decided each cycle whether to keep going. A flat pile of agents coordinating with each other stalled, and the hierarchy scaled.

So the harness is the world the loop lives in, and that changes what the written list is. The list names which actions the loop cannot take on its own, enforced by a hook or a permission that blocks the tool call rather than by a prompt that asks nicely. Mine, for the software I build, is short: spend money, act as someone else, write to production, merge to main, and anything a customer can quote back to me from a contract. Everything else, the loop can have, and the loop usually does.

The concept-tagging pipeline on this site is the smallest version of this I run. It is a script the publish workflow calls, and it does a list lookup. When it finds a Greek term that is not on the list, the model drafts the definition, the pronunciation, and the etymology into a file for me to read. The lookup is the harness, and the reason is not cost. A list cannot be wrong about a word that is on the list, and the definitions are the fuzzy part where I want the smartest thing available doing the first draft. If the site were a billing system, the list would be longer. It is a blog. The list is one item, and the gate is that I read the file.

Whether the model can even see your data is the prior constraint, and whether your process was ever real is what the loop shows you in week one. This piece assumes you have both. The question left is how long the list is.

The Harness Rule in Practice

Say what done looks like, and how the loop can check it. That means a goal, a turn cap, and a verification the loop can run without you: the test suite, the lint rule, the eval set, the screenshot diff. A prompt that describes the work is none of those things. If the check is something only a human can judge, the loop cannot close, and you will end up babysitting it. Make the check quantitative and the loop closes on its own.

Put the rules where the loop can reach them. A short file at the root that tells the agent where things are. Design docs and decisions in the repo as markdown. Skills for anything you do more than twice, so the next agent loads the instructions and the scripts instead of rediscovering them. If a rule lives in someone’s head or in a wiki the agent cannot open, the rule does not exist.

When it fails, fix the environment, not the prompt. Ask what the loop could not see or could not check. Add the test. Add the lint rule with an error message that tells the agent what to do instead. Fix the tool description. Anthropic’s research team had an agent try a broken tool and rewrite its own description, and the next agents finished 40 percent faster. Every failure is a missing capability, and capabilities compound in a way prompt tweaks never will.

Gate actions, not thoughts. The model decides freely, and the harness decides what it is allowed to do about it. A hook that blocks the spend, a permission that denies the production write, a merge that needs a green check or a human. This is the whole list, and it is enforced at the tool call, where the action happens, because a rule the model has to remember is a suggestion and only a rule the harness enforces is a gate.

Let the loop run. Give it a budget and a stop condition and walk away. Planners break the work down, workers grind on one task each and push, a judge decides whether the next cycle starts. A loop that runs unattended overnight and leaves a green pull request in the morning is the unit of work now, and a team that still measures itself in tickets closed by hand is measuring the wrong thing.

Let the loop maintain itself. Background agents on a cadence that pay down the debt the day’s work left behind, with the golden rules encoded as lint so the cleanup is mechanical. Evals whose results feed back into the skills and the tool descriptions. A judge that reads the misses and opens a change to its own instructions. That is the self-referential loop, and it is where this is going: the system watches the data, sees the trend move, and updates its own rules and its own code. The one thing that does not move in that picture is the merge gate, and the merge gate is on the list.

Keep the harness thin, and get ready to throw it away. Every clever piece of control flow you write today is a bet that the next model will still need it. Most of those bets lose. The harness that survives a model upgrade is the one you can rip yesterday’s logic out of without breaking the loop.

Big pull requests from agents already broke one old rule of thumb. This is the rule that replaces it: the size of the diff matters less than whether the loop that produced it could check its own work.

What Changes When the List Exists

The arguments stop. Design reviews stop relitigating model versus code, because the answer is on the list or it is not, and if it is not, the loop has the path. They stop because someone wrote the list, and until someone does, every one of those arguments is on the person who could have written it and did not.

Speed goes up, because the default is the loop and the exceptions are written down. Teams that argue every ticket move at the speed of the argument.

The promises get specific. The things you can put in a contract are the things on the list, and the things on the list have a gate. The things off the list have evals and an owner, and everyone knows which is which.

On every path I have watched get a real environment instead of a wrapper, the model’s hit rate went up, and that part surprises people. When the loop can read the repo, run the tests, and load the skill, it is handed a sharper problem than a prompt could ever describe.

And the list starts to change shape as the tools do. The next step after single agents is groups, then graphs, then loops that watch the data, see the trend move, and update their own skills and their own code. When the rule changes, the system changes itself. Where I think this goes is that the harness becomes the list of what the loop cannot rewrite without a gate, and governance becomes an eval and a rollback instead of a pull request. That is a different article, and I am already writing it in my head. The list is how you get ready for it.

Final Thoughts

Start with the model. Let the loop run. Then build the environment it works in, and write down, in one place, the short list of actions it does not get to take on its own, and set the length of that list by what you have to promise, not by what the vendor charges.

Frendo is right that a promised answer needs a deterministic path and wrong that cost is the reason, and the difference matters, because teams that harness for cost build long lists and slow down while teams that harness for risk build short lists and ship. The list is the proof that someone understands the system well enough to say what it must never do alone, and everything not on it belongs to the loop.

If you want a community of engineers and leaders who build fast with agents and argue in public about what belongs in the harness, MasteryLab is where that argument happens every week.

FAQ

Should you use an AI model or regular code for a given task?

Start with the model. Give it the goal, a way to check its own work, and access to the repo, then let it run in a loop until the check passes. Then ask how much you have to promise about the result. Money, identity, production writes, and anything you signed a contract on get a gate: a hook or permission that blocks the action until a check or a person clears it. Everything else the loop can have. The size of the harness is set by risk, not by the vendor’s price.

What is “smart model, dumb harness”?

The model does the deciding. The harness is the environment it decides in: the repo it can read, the tests and linters that tell it whether it is right, the skills it can load, the hooks and permissions that block the few actions it cannot take alone, and the stop condition that ends the loop. The harness stays thin on purpose so it can be tested and promised, and so you can rip out yesterday’s clever logic when the next model makes it unnecessary.

When does a rule belong in something the agent loads instead of in code?

When the rule changes often and the cost of a wrong answer is low enough that an eval can catch it before a customer does. A moderation policy written in plain language that the model reads alongside each post, returning accept, reject, or send to a human and why, is a good example. Tax rounding is not. If the rule changes often and the stakes are high, keep the model in the loop but give the rule an eval that runs on a schedule and a gate before a change takes effect.

Is the cost of calling an AI model a good reason to use code instead?

Rarely. Models get smarter, which means fewer calls per outcome, and then they get cheaper and faster without losing the smartness. Local inference is going to change both cost and latency further. Hunting for the cheapest model is a waste of engineering time. The one cost that still counts is latency in a request path, where a second-long model call in a checkout flow is a product decision, not a budget one.