Dev.to · 9 min read

The permission you can't prompt away

The permission you can't prompt away

Enforcing agent safety in the tool layer, not the system prompt. Drawn from a personal agentic system ("Vantage") I built and run daily. Most agent "guardrails" are sentences. Somewhere in a system prompt there is a line like "Never delete files without asking" or "Do not send email on the user's behalf." The line is polite, it is well-intentioned, and it is not a control. It is a request addressed to a probabilistic text generator, evaluated anew on every turn, competing with everything else in the context window. The day it loses that competition is the day you find out your safety model was a suggestion. I want to argue a narrower and more useful position than "prompts are unreliable," which everyone already knows. The position is: the boundary between what an agent may suggest and what it may do should be enforced by the shape of the tools you give it — ideally by the tools you withhold — and that enforcement belongs in a service you can test without a model in the loop. A prompt is a request. A missing tool is a guarantee. I'll make the case with a system I actually run. The setup Vantage is a personal agentic system. The agent (Claude Code, today — the substrate is meant to be replaceable) connects over MCP to a Spring Boot service that owns everything durable: an append-only event log in MySQL, a set of read collectors, and a small number of write tools. The service is plain Java, tested with JUnit, with no LLM anywhere in its tests. That split is the whole design — the model is the controller, the service is the service layer — and the safety model falls directly out of it. The founding rule is three sentences long: Read, summarise and draft are autonomous. Send, spend, delete, or anything publicly visible requires explicit confirmation. This is enforced in the service, not in a prompt. The interesting word is the last one, and the interesting mistakes are in how you honour it. "Write" is the wrong category The first thing the tiering gets right is that it does not tier on read versus write. It tiers on effect in the world. That distinction matters because the obvious implementation — "reads are free, writes need approval" — immediately strangles the system. The agent's most useful autonomous act is to record things: file an idea, note an observation, draft a proposal. Those are writes. They are also completely safe, because they touch only the system's own metadata — nothing is sent, nothing is spent, nothing is deleted, nothing becomes visible to anyone but the user. We call that tier zero: a write whose blast radius is zero stays autonomous. So the question a new tool has to answer is not "does this write?" but "can the effect of this be seen or felt outside the system, and can it be undone?" Draft an email: tier zero. Send it: gated. The category that needs a human is irreversible or outward-facing effect, and conflating it with "write" is how you end up with an agent that needs permission to take a note and no permission to wire money. The gate that is a hole Here is the part I find most worth stealing. The system's one tool that speaks first — that produces output nobody asked for — lets the agent file a proposal: a suggested next action on a project, for the user to accept or reject. The agent can file one. It cannot accept one. Not because a prompt tells it not to, but because there is no tool that accepts a proposal. The accept path exists only in the user-facing console, behind a human click. The comment in the source is the entire security argument: /** * There is no accept_proposal and there must not be. Vantage suggests and Jason decides. * The service enforces the budget, but nothing except the absence of this tool stops an * agent accepting its own suggestion into Jason's declared next actions -- which would make * the accept/reject history a record of Vantage agreeing with itself, and that history is * the only feedback signal this feature has. */ Read that last clause carefully, because it is the general principle hiding inside a specific feature. An agent that can both propose and approve is not an assistant with oversight; it is an agent with a logbook it writes itself. The approval record becomes a record of the system agreeing with itself, which is worse than no record, because it looks like oversight. The only way to keep the human judgment real is to make it structurally impossible for the agent to forge — and the cheapest structural impossibility is a function that was never written. This generalises cleanly. Every time you are tempted to write "the agent should not do X on its own" into a prompt, ask whether X corresponds to a tool the agent is holding. If it does, you have not built a guardrail; you have built a label on a loaded tool. The fix is usually to split the tool: one that proposes X, one that commits X, and the second one lives somewhere the agent cannot reach. Bounding the human, not the agent The same proposal tool shows a second lesson, which is that even a perfectly safe action has a cost, and the cost is the human's attention. The first design was a per-run budget: an agent reviewing a project may file at most three proposals. It is the obvious cap and it does not work. Two runs produce six proposals; five runs produce fifteen; each run is individually within budget, and the review pane floods anyway, just more slowly. The flood you have to fear is the second review — the one nobody is watching for. The cap that works counts what is open, not what was filed per run: at most three undecided proposals may stand against any one project at a time. This looks like a cosmetic change and is a categorical one. A per-run cap bounds the generator's enthusiasm. An open-items cap bounds the human's backlog — the system cannot offer a fourth decision until you have made one of the first three. The limit is denominated in the scarce resource, which is not the agent's output but the user's willingness to keep reading it. A guardrail that caps the wrong quantity gives you the comfortable feeling of a limit and none of the protection. And because the limit lives in the service, the tool's own description can tell the model the reason — "every proposal costs a decision the user did not ask to make, and one that wasn't worth it trains him to dismiss the next one" — so the model files one strong proposal instead of three mediocre ones to fill the quota. The prompt-level instruction is still there. It is just no longer load-bearing. If the model ignores it, the service refuses the fourth proposal regardless. Belt from the service; suspenders from the prompt — never the other way around. Make the judgment measurable Two smaller decisions round out the pattern, both about keeping the human's judgment legible after the fact. Accepting is editing. When the user accepts a proposal, they can accept it as reworded. The system stores both the original text and what was actually accepted, so the gap between what the agent suggested and what the human was willing to endorse stays measurable. A system that overwrites the suggestion the instant it's accepted throws away its only signal about how good its suggestions were. Decisions are append-only. A decided proposal cannot be re-decided. If the answer changes, you file a new one; two dated rows disagreeing is the history. The approval log is a judgment record, and a judgment record you can quietly edit is not a record. This is the same instinct as the event log underneath the whole system: you append, you never rewrite, and you can therefore replay and audit the system's entire life at 3am when you need to know exactly what it did and when. What this costs, honestly This approach is not free, and it is not always right. It pushes work out of the prompt and into typed service code, which is slower to change — you ship a Java method and a migration, not a sentence. For a throwaway agent, that overhead is not worth it; a prompt-level "please don't" is a fine control for a toy. The structural approach earns its keep exactly when the cost of the agent being wrong is real and the system is going to live for a while — which is to say, in production, in a company, on anything that can send, spend, or be seen. It also demands that you actually enumerate your dangerous actions and give each one a home in the tiering. That is work, and it is the work — "which of our tools have outward-facing effect, and which of those is the agent currently holding?" is a question most teams shipping an agent have not written down the answer to. The value of the tool-layer approach is that it forces you to write it down, because a tool either exists or it doesn't. The one line to keep If you take a single thing from this: your agent's real permissions are the set of tools it can call, not the set of instructions you've written for it. Audit the first list. The second one is documentation of your intentions, and your intentions are not a security boundary. This is one of a set of write-ups drawn from real systems. If your team is putting agents into an environment where a wrong action has consequences — enterprise, regulated, on-prem — the same structural approach applies, and I do advisory work on exactly that: MCP tool design, tiered permission gating, and auditable agent runtimes in Java/Spring environments.

This is a summary aggregated from Dev.to. Read the complete article on the original site:

Read full article at Dev.to

More AI & Machine Learning News