Most AI roads rest on a quiet bet: that labs will eventually ship a model that is safe and aligned enough to be trusted in production. Better training, better monitoring lines, another version, and the thing behaves well.
Here’s the flaw in that bet. Even a well-aligned model cannot tell you who used it, on what data, under whose policy, or give you a record to show an auditor. Those are not facts about how the model behaves.
The facts are about how they are distributed, and they live completely free of weights. A better model answers a different question than the one regulators, auditors, and security teams are actually asking, and the gap between those two questions is where the real risk is now.
Security engineers invented this trap fifty years ago. A 1972 study by the US Air Force defined the reference index, the part that determines whether an action is allowed, and set three criteria for its trust: it must be unobtrusive, used in all accesses, and small enough to be fully verified.
💜 for EU tech
The latest talk from the EU tech scene, a story from our genius founder Boris, and some incredible AI art. It’s free, every week, in your inbox. Register now!
The modern frontier model is none of these three. Alignment tries to make the model enforce the rules it is meant to enforce, from within, and the system cannot be self-regulating. Governance should sit around the model, not in it.
Security is not the same as governance
Confusion hides itself in one word. Safety and alignment ask whether the model is well-behaved. That’s the situation, we live in the weights, and the labs got really good at training. Governance asks different questions: who used what model, on what data, under whose policy, and with what auditable record.
That is a property of some distribution. Training can shape the first. It will not, by design, provide the second. Treating them as a single problem is a category error under many AI risk discussions.
The problem is bigger than the model, and it’s growing
This is not a niche concern for labs. It lives on for everyone who uses AI, and it grows for two reasons.
First, models gain agency. When the model produces only text, bad interaction means bad feedback. Now models take action with tools and processes like MCP, and every tool call is a new decision about who is doing something, on what data, under whose authority. Those decisions are repeated with every job and every user.
OWASP’s 2026 tracking finds that the majority of agent projects it tracks are coding agents, and that informs injection maps to six of the ten vulnerabilities in its top ten agent list. The unregulated surface grows as quickly as the discovery.
Second, accountability is now legal. California’s SB 53 and the European Union’s general purpose AI rules impose obligations on labs, but the EU AI Law also imposes different duties on deployers, and data protection law already holds organizations responsible for how personal data is used.
If an administrator asks what your AI did, “we used a synchronized model” is not the answer, and a vendor’s security report won’t tell them who inside your company sent which data to which model last Tuesday.
Public benchmarks don’t help at all: StrongREJECT, HarmBench, and AgentHarm model according to general instructions, not your users, your data, or your policy on a particular day. The maximum score is part of the model, not the test route.
So the scope is wide and tight: every organization that uses AI, an area that grows with every new agent, and now enforceable debt. There is no problem with the quality of the model.
Why model alignment can’t fix it
Go back to the 1972 test, because it shows exactly where the correction will not survive.
The model has no interference. Fast injection, named in 2022, works because instructions and data correspond to the same text channel, so the document read by the model can overwrite the given instruction.
Four years on it’s not resolved: OWASP says it’s unclear whether there is any evidence-free protection, and a 2026 paper argues that agents could remain vulnerable. A controller that you can rewrite with input that evaluates is not a controller.
The model is not reliably requested as a gateway. It provided the tools and became a confused vice of the book, which is a 1988 term for a trusted system that an untrusted caller manipulates into abusing his authority, because he cannot dictate the order in the data he reads.
The model is unverifiable. You can’t test billions of fuzzy parameters the way you can test a small optimization kernel.
Fail all three tests and the conclusion is “not a hard mix.” It’s because a model can’t be an enforcement method, so the enforcement must live somewhere else.
The resolution is a layer around the model
If the model cannot govern itself, the answer is not a better model but a smaller control layer around it, which treats the model as a trusted entity and mediates everything it tries to do.
This is not a new structure that can be established. It’s a 1972 reference monitor rebuilt for the agent, and here’s how it works.
Let’s imagine the request going through it. The model reads its input and suggests an action: call this API, write this file, send this message.
That proposition has never been trusted alone. It leaves the model as a request, not a command, and goes to a small policy engine without the model, sitting on the path between the agent and the resource that wants to access it, the way zero-trust architecture separates the point that determines whether the action is allowed from the place we do it.
The engine evaluates the action against two things that weights never have: your actual policy, and the limited set of skills this agent has been given for this task.
If the action clears both, the enforcement point does; otherwise, it is rejected. In any case, who worked, on what model, on what data, under what policy, and what was decided is recorded in the log.
That single flow returns three properties that the model cannot provide. It’s seamless, because the policy resides in a smaller external engine than the script model can argue. It is always requested, because nothing reaches the original source without going through it.
And it’s reliable, because a pressure point small enough to read is one you can really trust, in the way that billions of weights are barely visible. This is why the strongest research defenses already work this way: CaMeL, from Google DeepMind and ETH Zurich, achieves provable security by containing a model exactly like this one, and catches even if the underlying model remains vulnerable. The guarantee lies in the architecture, not the weights.
None of this makes the model safe, and that’s the point. It makes the surrounding system responsive, which is a small and very achievable goal, and it’s what the shipping problem really needs.
The right background
This is not an argument against alignment. Alignment is necessary, and evolving. It’s just the wrong premise for the question asked by regulators, auditors, and users. The answer to that question has been on the shelf since 1972, and the industry is finding it expensive.
The next model will be more secure than this one. We will never know who used it, on what data, or under whose policy. Filling that gap is a task, and a story.
