Your Agent Needs a Manager, Not a Better Prompt
The dangerous agent failure isn't a jailbreak — it's an agent doing its job one step past its permission. The fix is a judge at the action boundary.
The agent failure that should worry you isn't the jailbreak or the hallucination. It's the agent doing exactly what you trained it to do — and going one step too far. Sending the email it wasn't authorized to send. Deleting a record because the value looked stale. Opening a pull request because the tests passed and the change looked done. You built the agent to act; you haven't built the layer that decides when it may.
Why the usual guards fail
A stricter prompt can't police, because an agent optimizes its one primary goal — tell it to close the sale and it will pursue, not check. Manual confirmation trains users to click "OK" out of habit, the way everyone now dismisses cookie banners. And human review of every action simply doesn't scale when someone is running a dozen agents at once.
Put a judge at the action boundary
The pattern that works is a second model — a validator with one job: guard your intent. The acting agent must justify a proposed action, cite evidence, and state its scope; the judge checks that against policy and context before anything executes. Match its strength to the stakes:
- Read-only and reversible writes — light validation.
- External actions (emails, meetings, public posts) — always through a strong judge.
- High-risk (spending, deletion, permissions, merges) — judge plus a human.
And give the judge four outcomes, not two: allow, block, revise, or escalate. Yes/no is what teams build around and bypass.
Agents are becoming managed workers. The product is no longer the agent — it's the management system around it.