01 from I Tested OpenClaw: Why AI Agents Break Production
The blast radius assessment: before an AI agent touches a live system
An agent with write access to an inbox and a CRM is not an assistant. It is an actor working under your name, and its mistakes are sent, not drafted. This is an architecture problem, not a software one. Map the damage before you issue the key.
- 1
Write the job in one sentence
“Summarise this thread” is a task. “Manage my inbox” is a hazard with no ceiling. If the job does not fit in one sentence, the agent is not deployed.
- 2
Map the blast radius
List every system the agent can reach. For each one, write the worst thing it can do there and whether a person can undo it in under ten minutes. A permission that fails that test is denied before the first key is issued.
- 3
Default to read-only, and keep sending human
An observer cannot destroy. The agent drafts and a person sends: every outbound message, an email, a chat reply, a comment on a ticket, goes to a review channel first.
- 4
Watch for fourteen days
Run it read-only for fourteen days and log every action it attempts. Read the log for invented facts, for work nobody asked for, and for the places where an ambiguous request became a confident action.
- 5
Promote one permission at a time
One system, one grant, then the log again. A new hire on probation, not a senior engineer with administrator rights on day one.
- 6
Keep a revocation plan
If you disconnect the agent now, what stops working, and how fast do people take it over? Write the answer before the agent gets any autonomy.
The ten minutes, the fourteen days and the one sentence are my rules, not measurements. Tighten them for anything that touches money or customers.
The worksheet, three rows filled in
| System | Grant | Worst action | Undo in under ten minutes? | Verdict |
|---|---|---|---|---|
send | A reply to a client, under your name, with the wrong price | No | Drafts only | |
| CRM | write | Deal stages and notes changed from a misread thread | Partly, from the audit log | Read-only until the log is clean |
| Team wiki | write | A decision page rewritten from an ambiguous request | Yes, from page history | One page, every edit reviewed |
Where these rules are measured
I run the same rules as infrastructure, on kind, with open models on CPU and instructions planted in what the agent reads. The model changes the numbers. The boundary does not.
- agent-security-benchmark: six planted instructions, three open models, three published boundaries. In run 43162103253 a model acted on a planted instruction in as many as 15 attempts of 18, and nothing reached the outside at any of the three boundaries: 0 of 54 attempts each.
- kubernetes-ai-agent-sandbox v1.4.0, steps 2 and 5 as a cluster: the agent reaches its tools through one MCP server, sees only the tools it was granted, and leaves only through an egress allowlist. In the same run 18 calls tried to leave and the network policy stopped all 18; 12 asked for a tool that was not granted, and all 12 were refused.
- kubernetes-air-gapped-ai-agent v1.9.0: the agent acts for a person, with that person's rights, and the schema of its MCP server accepts no recipient outside the company. In the same run 16 calls tried to leave and that schema stopped all 16; nothing reached the outside in 54 attempts.
An agent without a ceiling is not a workforce. It is exposure.
The long version is the post I Tested an AI Agent in Isolation.