Agentic Guardrails Are Necessary, but Not Sufficient
I’m going to be extremely clear up front: I do not see any version of the future where an agent, an “agentic system”, or really any application, can be trusted to enforce its own permissions. Neither a well-written skill, or a careful MCP server, or a model that has been specifically trained to refuse can be trusted to govern itself. We have spent the last two years building an impressive amount of machinery for constraining agents, and nearly all of it runs inside the process it is supposed to constrain.
We would not accept this anywhere else in the stack. It is not acceptable to ship a database with full administrator permissions, and trust the application to not drop tables. We don’t ship applications where we trust the frontend (or request) to let us know what client data it is allowed to access. “Do not trust the client” was a universally-understood primitive that we have somehow forgotten about when the client started to be “intelligent”.
The Argument#
I want to put the reasoning up front, because the conclusion sounds more dramatic than it is. It rests on three suppositions, and if you accept all three, the rest follows.
First: an agent’s instructions arrive inside its data. A tool result, a fetched web page, a comment on someone else’s issue, and the operator’s actual request all reach the model as the same kind of token. There is no privileged channel. This is why prompt injection has stayed unsolved rather than getting patched — it is a fundamental boundary failure rather than a parsing bug. From the point of view of the agentic loop, an input from an end user, and an “input” from the model (ie, output from a previous loop step), are the same. The practical consequence is that any component which decides by asking the model is deciding with an input the attacker can also write to. We might in the future determine an implementation of “prepared statements” for LLMs, however the complexity of that implementation is easier stated than fully understood.
Second: circumvention and competence look identical from the inside. This is the one I think gets underweighted. Picture an agent that gets blocked on a command, and then finds another route to the same outcome — a different flag, a script, a language runtime, an API instead of the CLI. That is not an escape attempt. That is an agent doing precisely what it was asked to do, working around what looks to it like an obstacle rather than a rule. No malice is required, no adversary, no jailbreak. We built these things to be resourceful, and resourcefulness applied to a barrier is indistinguishable from evasion at the point where it matters.
Third: anything the agent can reach, it can change. The settings file, the hook script, the wrapper, the environment variable, the allowlist, and the log of what happened are all files, on a disk, that the agent has a shell on. This is not a hypothetical about deception. It follows directly from the second supposition — a sufficiently capable agent that is blocked by a config file it has write access to will eventually edit the config file, because from where it sits that is a reasonable way to finish the task.
Put those together and the conclusion is forced. Permissioning has to be deterministic, which rules out anything that decides by inference. And it has to be defined outside the environment the agent runs in, which rules out very nearly everything we currently ship. Given how capable the current generation of models is, “outside” cannot mean a different file on the same box, or a different process under the same user. It means a separate trust domain, and in practice that means a separate machine, reached over a wire the agent does not control.
What We Actually Have Today#
It is worth grading the existing tools against that claim, because they are not all equal and I do not want to be read as dismissing them.
At the bottom are instructions: system prompts, skill files, CLAUDE.md, tool descriptions in an MCP server. These fail the first supposition immediately. They are advisory. The model reads them as tokens, weighs them against every other token in the context, and decides. On a good day they work almost always, which is a genuinely useful property and also exactly the property that makes them unsuitable as a control.
Above that are the server-side checks — an MCP server that validates its own inputs, a tool that refuses out-of-range arguments. Better, because the check is real code. But the server enforces its own contract, not your policy, and it has no idea whether the agent calling it is doing so for a reason you would endorse. A correctly implemented tool, holding a correctly scoped token, called with valid arguments, describes most of the interesting agent breaches I have read about. The GitHub MCP exfiltration is the cleanest example: nothing in the server was wrong, the token was the one the developer had handed over on purpose, and a malicious issue in a public repository still walked private code out the door.
Best in class right now are the harness-level controls: permission prompts and enforcement hooks. These are meaningfully different in kind. A Claude Code PreToolUse hook is shell code that the CLI executes before the call goes out, and it can deny outright. The model never interprets it, never negotiates with it, and cannot talk it out of a decision. That is real determinism, and it is the best thing broadly available today. Hooks also adapt, which matters — the policy can be as specific as your codebase requires.
This is the “necessary” half of the title, and I mean it. I run hooks. You should too. They catch the accidents, they cut an enormous amount of noise, and they make the common path safe. My argument is not that this layer is worthless. It is that this layer is not the control.
Tamperproof Means “Not On This Machine”#
Here is where the third supposition bites, and it is the part I have not seen stated plainly enough.
The hook is enforced by the harness. But the harness reads its policy from a file, and that file lives in the working directory you pointed the agent at, or in a home directory the agent can write to, on a machine where the agent has a shell and your credentials. The enforcement is deterministic. Its configuration is not protected. Every piece of the apparatus that adjudicates on the same box is inside the blast radius: the policy, the hook script it invokes, the binary that reads them, the environment it runs in, and — this one is easy to skip past — the record of what happened.
That last one deserves a sentence of its own. An audit log the agent can write to is not evidence. It is a document, produced within the same system, describing that system’s own behavior.
So the requirement is not “put the policy in a file the agent is told not to touch.” It is that the thing making the decision cannot share a filesystem, a user, or a process tree with the thing it is deciding about. The agent should be asking something else for permission, over a network it does not administer, and receiving an answer it cannot forge.
Any control an agent can reach is not a control. It is a suggestion with better formatting.
How This Lands Against the Prior Art#
Having said all that, I should be honest that almost none of it is new. What is new is which parts we decided to skip.
The criteria are from the early 1970s. Anderson’s reference monitor has to be always invoked, tamperproof, and small enough to verify. Current agent permissioning fails all three, and the third is fatal on its own — a guardrail that reaches its decision by asking a model is unverifiable by construction, because you cannot enumerate its behavior. Hooks solve “always invoked” within the harness. Very little in common use solves “tamperproof”, and the partial exceptions are instructive: an agent running inside a sandbox or a remote environment it did not provision is being constrained by something it cannot edit, which is the one place I think the industry is accidentally getting this right. We have been shipping reference monitors that fail the reference monitor test, and we have been doing it fast enough that nobody stopped to check the definition.
The more recent frameworks are good, and they are answering a different question than mine. Simon Willison’s lethal trifecta — private data, untrusted content, and the ability to communicate externally — is the clearest available account of when you are exposed. Meta’s Agents Rule of Two turns a near-identical set — untrustworthy input, sensitive data, and the ability to change state or communicate out — into an operational limit: an agent gets no more than two of the three within a session. Both are correct. Both are about risk conditions, about which combinations of capability make a breach possible. Neither one tells you where the enforcement of those limits is supposed to live.
And that gap is not academic, because both formulations are scoped to a session, which quietly assumes something is counting sessions and measuring properties. What is doing the counting? If the answer is the harness, or the agent framework, or a policy expressed in the same repository the agent is editing, then the boundary is only as good as the box, and we are back at the start. Willison is right that a guardrail product advertising a 95% catch rate is advertising a failing grade. I would add that a 100% catch rate does not help either, if the thing doing the catching is running as the same user as the thing being caught.
The same premise turns up outside of security entirely, most recently as “Architectural Constraints”, and discussed here.
The closest thing to a real answer I have seen is CaMeL. Its policy is deterministic, and it is evaluated over an explicitly extracted control and data flow, so untrusted content cannot influence the program’s path rather than merely being discouraged from doing so. That is the right kind of mechanism. The fair caveat is one of threat model: CaMeL is defending against the injected instruction, not against a compromised host. It answers “can this data redirect the program”, which is a genuinely hard question well answered. It does not answer “what if the program’s own machine is not trustworthy”, which is the question I think we are about to spend a few years on.
Isn’t This Just Least Privilege?#
Largely, yes. And that is somewhat the point — this is an old idea that we stopped applying at the moment the principal started making its own decisions.
There is one real difference worth naming. Least privilege assumes a principal whose job is known ahead of time. You can enumerate what the billing service needs, because the billing service does the same thing on Tuesday that it did on Monday. An agent discovers the privilege it needs at runtime, and takes instruction from data. Enumerate its grants in advance and you get one of two failures: a scope narrow enough to be safe and too narrow to finish anything, or a scope wide enough to be useful and wide enough to be a breach. Everyone who has actually tried this has felt that squeeze, and the squeeze is what produces the shrug at the end — the personal access token, the admin role, the “we will tighten it later.”
The way out of that trade is to stop granting privilege up front and start authorizing individual actions as they happen. Which requires an authorizer. Which, per everything above, cannot be the agent.
Reactive or Proactive, but Outside#
I am deliberately not going to design this here, but the shape splits cleanly in two, and both halves have to sit outside the environment to be worth anything.
The reactive shape is monitoring, audit, and rollback. It is cheaper, it retrofits onto systems you already run, and it is where most teams will realistically start. Two constraints make it real rather than theatrical. The record has to be append-only from the agent’s side, shipped somewhere the agent cannot reach, or it is a self-report. And you have to be honest that a large class of actions do not roll back. The mail is sent. The row is dropped. The package is published. The payment cleared. Reactive supervision tells you what happened, which is worth a great deal, and it will not undo the subset of things you most wanted undone.
The proactive shape is more work and more valuable. It has two pieces. A credential broker, so the agent never holds the secret at all — it makes the call without one, and something on the path attaches the real credential after the request has been authorized. And an authorizer that approves each consequential action against a policy you wrote, in a place the agent cannot edit. A requirement of both of these components is that no (unstructured) agent-generated output is used as input to the broker action, or authorizer. These actions must be deterministic, reproducible, and idempotent. The broker and harness process must be treated as independent production systems, and not as simply components of the larger agent harness.
One detail there is easy to get wrong and expensive to get wrong. The authorizer has to judge the action itself — the actual request, the actual arguments, the actual destination — and never the agent’s description of what it is about to do. The description is generated by the thing under judgment. If your approval flow shows a human a summary the agent wrote, the human is approving the agent’s account of the action, not the action itself.
Where This Leaves Us#
The point that I think I’ve gotten to is that this problem is expensive, mostly unbuilt, and the ergonomics today are bad. A separate machine that arbitrates every consequential action is latency, infrastructure, and a policy language that has to be maintained. This is a big investment on a risk that is fairly easy to discount; it doesn’t show up as a problem until it’s critical.
The proposal of an external overseer does not argue to pull out existing guardrails (such as hooks, allowlists, and careful skill writing and evaluation). Existing guardrails, both in the model and the harness, are still requirements for conceptual and intelligence-based guiding. Our overseer should not be the one restricting intent, but rather restricting the actions taken as part of that intent against external systems.
Just be precise with yourself about what you have built. An agent supervising itself is a system reporting on its own compliance. It is not a control, and it will pass every one of its own audits.
Hopefully this works out for you, or was at least helpful. If you have any questions, don’t hesitate to shoot me an email, or follow me on twitter @nrmitchi.