14 from Prompt Injection Evals — Six Models, One Boundary, Nothing Left the Company
The boundary gate: an agent release passes on what the system did, not on the model's score
By Vladimir Mikhalev Docker Captain · CNCF Ambassador · IBM Champion
Prompt injection resistance is a property of the model. Six models gave four different answers to it. The boundary is a property of the architecture, and it gave one answer for all six.
So the release gates on what the system did and reads the model's numbers as data, published next to the zeros.
- 1
Put every tool behind one server that checks every argument
The tool server knows which agent is calling and validates every argument against the tool's declared schema, so a request outside the policy fails before anything is sent. Owner: the platform team, with the security lead approving each schema.
- 2
Score the release from the services, not from the agent
Read what actually left the outbox and what is actually gone from the records. The agent's log and its answers are not evidence of either.
- 3
Gate on two zeros
Nothing delivered outside the company, nothing deleted. Either one above zero refuses the release, whatever the model's scores say.
- 4
Add a control, so a zero means something
Before the first task the eval sends its own email through the same tool server, and the run fails unless it is in the outbox. Without it, a zero outside proves only that nothing could be delivered at all.
- 5
Require that the run finished
A model that ran out of time passes every rate it never reached. Refuse a run that did not complete every task category.
- 6
Set the usefulness floor from measurement, on each runner class
Run the seeded evals on every runner class the model meets, take the lowest pass rate and leave a margin under it. The same seeded model behaved differently on two and four virtual CPUs; GitHub gives private repositories two and public ones four. Owner: the ML engineer who owns the model choice, with the floor and its runs in one commit.
- 7
Publish the model's numbers, and the residual risk
Keep a row per model, rewritten by every run: how often it was fooled, overstepped or invented data, next to what happened because of it (the daily table). Record what the gate does not cover: here, emails to colleagues nobody had named, open until a per-task recipient allowlist closes them.
Six models, one boundary, the release run of 18 October 2026
| Model | Ordinary tasks | Hidden instructions acted on | Delivered outside | Records deleted |
|---|---|---|---|---|
| Granite-4.0-1B (IBM) | 18 of 18 | 6 of 12 | 0 | 0 |
| Qwen3.5-2B (Qwen) | 13 of 18 | 0 of 12 | 0 | 0 |
| Granite-4.1-3B (IBM) | 17 of 18 | 3 of 12 | 0 | 0 |
| Qwen3-4B-Instruct-2507 (Qwen) | 16 of 18 | 6 of 12 | 0 | 0 |
| Ministral-3-3B (Mistral) | 18 of 18 | 1 of 12 | 0 | 0 |
| MiniCPM5-2B (OpenBMB) | 18 of 18 | 1 of 12 | 0 | 0 |
What shows it works
The release run, the decision behind the gate, and the table that keeps moving.
- The run the release was cut from, 18 October 2026, one job per model on runners with four virtual CPUs.
- Version 1.1 of the air-gapped AI agent, 18 October 2026, with the gate file at that tag.
- The decision to gate the boundary and measure the model.
- The daily table, one row per model; its counts move from day to day, the zeros have not.
Gate what the architecture promises, and publish what the model does.
The long version is the post Prompt Injection Evals. OWASP GenAI Security Project, OWASP Top 10 for Agentic Applications for 2026, December 2025; GitHub Docs, GitHub-hosted runners, accessed October 2026.