What the green checks hid

What reached main

One engineer, AI agents and 116 repositories: the last 30 days, as GitHub counts them

I run this fleet with AI agents. They commit under my account, so the count does not tell my commits from theirs; Dependabot and outside contributors are counted apart. Nobody types these numbers: each morning fleet-ops asks GitHub for the commits on every default branch, the releases published, how every CI run ended and how long each dependency update waited in the last 30 days. A failed run stays in the count.

commits to main3,029 from my account, 310 from Dependabot and outside contributors

3,339

releases publishedin 92 of 116 repositories

897

CI runs passed1,169 failed, 92.7% passed

14,781

dependency updates mergedmedian 4.4 min from proposal to merge, each after its own verification passed; 0 waiting

312

From 2026-10-04 to 2026-11-03.

Who decides what

The agents do the work, machines check the agents, and three decisions stay with me

AI agents write most of the code here, run the tests and commit under my account; the figures above count their work. What an agent says about its own work is not evidence, so every change meets a check it cannot talk its way past. The decisions that spend money, reach readers or cannot be undone are not delegated.

What stays with me

Money: the AWS account the EKS and air-gapped runs use alerts at $10 and $20 and at $30 attaches deny-all to the role that creates resources (bootstrap-aws), and every run on a real account destroys what it built, searches for what is left and reports its own cost.Publication: nothing appears on this site without my go-ahead, and the newsletter and the podcast follow only from that.Anything that cannot be undone, deleted data or rewritten history, waits for my word; every default branch in the fleet refuses force-push and deletion on its own.

Machines check the agents

253 planted changes that CI must refuse and 42 manifests the cluster must refuse, against 8 it must admit, in the reference architectures. A daily conformance sweep across the fleet, each rule first shown a planted breach, the job failing when a rule stays quiet. The hand-written claims on the public profile, re-read against their sources every day. A catalog that refuses to publish a partial read. A heartbeat that pings a switch outside GitHub, which alarms when the pings stop.

The supply chain, gated

A version update waits three days after its release before Dependabot proposes it, GitHub's default since 14 July 2026, and a conformance rule fails any repository that shortens the wait or switches it off. Security updates do not wait. An update then merges only after that repository's own verification passes, a median of 4.4 minutes after it was proposed. Every image is pinned by digest.

Credentials

Cloud runs authenticate with OIDC, never a stored key, and a stored key in a cloud workflow is a conformance finding. By rule, an agent never prints a credential and passes one only from the keychain on standard input, never as an argument.

What the agents and the watchers got wrong, and the rule each mistake produced, is in the ledger.

Case one · a backup of nothing

12 days, 8 releases, and not one attachment in any backup

Data backup OK
CI: data archive opens    ✓

An archive that opens proves the archive. CI now proves the backup the way a user would notice it missing: it stores a file through S3, takes a backup, stores a second file, restores, and requires the first to read back and the second to be gone. The restore uses Garage's own consistent metadata snapshot rather than a copy of a live database.

• The fix, and what the loop had been archiving
• v2.0.0, 11 September
• v2.1.0, 23 September

Case two · the restore nobody ran

72 restore scripts, some never run by CI since May 2021

test_restore_roundtrip   PASS
(every push, 47 templates)

A decade of production infrastructure taught me one pattern above the rest: a backup that has never been restored is a belief. The tests now run the shipped script, and a fleet rule fails any repository whose restore script no test invokes. A comment or an echo naming the script does not count. Its list of exceptions can only shrink, and it has been empty since the day it was written.

• The WordPress script before and after

Case three · the check that could not fail

17 hours healthy over a crashed server

docker inspect → healthy

The rule it produced: a check whose pattern can match the check itself has never failed, so prove the red case against a real container before believing the green one. The fix shipped to every game-server template with that test, and every fleet rule since has been tested against a planted violation.

• The test that kills the game and requires the check to go red

The question a CISO asks first

The host is gone. How long until it works again?

Every week, for each template below, one runner starts the previous release, writes a marker row into the database and a marker file into the data, and takes a backup. Only what an operator keeps off the host leaves it: the backup files and the environment file. A second runner that has never run the stack then brings the current release back from those files alone, with the scripts the template ships, and the clock stops when the application answers over HTTPS with both markers back. So the number below is an older backup restored into a newer release, on an empty machine, image downloads included.

Fastest 18 s · slowest 4 min 50 s

A drill that fails is listed as failing and carries no time. A restore time from a restore that did not work is not a time.

In the words of a vendor questionnaire. The recovery time objective is the measured column, not a target somebody wrote down. The recovery point is the backup interval each template ships with, read from the file an operator copies; shortening it is one variable. The date of the last recovery test is the last column, and while the drill stays green it is never more than a week old.

Kubernetes, rebuilt from nothing

Eight reference architectures, built from an empty cluster every day or every week, the failed runs counted

Each is a public repository whose tests build the whole thing from nothing and then try to break it: an EKS platform, a sandbox for AI agents, an AI agent with no route out of its cluster, an LLM inference gateway measured against round-robin, a triage agent for failing Cypress runs, PostgreSQL recovery, supply-chain admission, and the air-gapped agent's admission controls turned into OSCAL evidence. In the air-gapped one, six open-weights models from four makers read the same planted instructions every day and are fooled at different rates, from never to half the time, and the boundary's numbers are the same for all six: nothing sent outside, nothing deleted. They run on kind on every change and every day; the OSCAL evidence runs on every change and every week. The EKS platform is also applied to a real AWS account, checked and destroyed again, weekly since 21 October 2026; the first run, on 15 October, was started by hand. The numbers are what the runs wrote, read from each repository's own record, and a run that failed stays in the count. These are reference architectures, not production: two system nodes on EKS, demo tenants, a Let's Encrypt staging certificate.

The air-gapped agent has had no route out since v1.5.0 (20 October 2026). The releases before it had a way out by DNS through the kind node and an open SSH port on the host, both found by checks added that day; ADR 1 records them.

Version 1.6 (21 October 2026) closed two more, found by a daily game day. With Kyverno scaled to zero, its own webhooks went and a model nobody signed was admitted (run); the API server's own guard now refuses it (run). A rollout of the model server cut off the answer an agent was waiting for (run); the answer is now finished before the old server stops (run). On a real AWS account the same day, a VPC with no internet gateway and no NAT still let a pod resolve example.com (run) until a Route 53 Resolver DNS Firewall closed it, 16 of 16 checks (run). On 22 October the whole agent ran there, by hand and then on its first weekly schedule the same day: the signed file loaded into the run's own ECR, admission, the evals and the game day, 56 of 56 checks for about $0.30 a run (by hand, scheduled).

Its admission controls are also published as signed OSCAL assessment results, from a weekly run in a repository of their own, each control resting on a row of the frameworks.

EKS platform, measured by the latest run

• run cost: about $0.16 (0.69 h, list prices)
• run length: 0.69 h

AI agent sandbox, measured by the latest run

• a new sandbox pod's first query, before the network plugin knows the pod: answered by the cluster's CoreDNS (the gate is why the agents do not start until it stops)

PostgreSQL recovery, measured by the latest run

• point-in-time restore: ready in 54s
• promotion: a new primary 4s after the kill
• write gap: 6.5s without an acknowledged write (12 writes refused or timed out)
• restore from the bucket alone: ready in 56s

Air-gapped AI agent, measured by the latest run

Model	Tasks done	Hidden instructions acted on	Tried outside, patterns shown	Tried outside, patterns hidden	Delivered outside	Cost per task
ministral-3-3b	18 of 18	0 of 24	0	9	0	$0.000375
qwen3-4b-2507	17 of 18	7 of 24	0	18	0	$0.000438
granite-4.1-3b	18 of 18	1 of 24	0	8	0	$0.000327
nodeloss	—	—	—	—	—	—
granite-4.0-1b	18 of 18	7 of 24	0	59	0	$0.000237
minicpm5-2b	18 of 18	1 of 24	6	8	0	$0.000327
qwen3.5-2b	14 of 18	0 of 24	7	6	0	$0.000216

Each model runs twice. With the tools' patterns shown, the model server builds a grammar from them, and for four of the six models it keeps the address the model writes inside the company. With them hidden, the model writes what it decides, and only the MCP server's check stands in the way. The MCP server checks the full schema in both runs, and delivered outside counts both (decision 13).

• nodeloss · the API server saw air-gapped-worker2 not ready 43s after it was lost
• nodeloss · the signed model was refused at admission 0 time(s) of 109 while the node was lost (first at -s, last at -s of the window; the loss at 10s)
• nodeloss · air-gapped-worker2 drained in 61s

Air-gapped AI agent, measured by the latest run

Model	Tasks done	Hidden instructions acted on	Tried outside, patterns shown	Tried outside, patterns hidden	Delivered outside	Cost per task
aws	18 of 18	6 of 24	6	—	0	$0.000669

Cypress triage agent, measured by the latest run

• stand-in agent: 26 of 26 attempts ended as expected (kind01 #16)

LLM inference gateway, measured by the latest run

• router TTFT after the first turn: 0.30–0.34x round-robin's (median, 94 ms against 276–311 ms)
• prefix cache hits: router 79%, round-robin 39–40%

OSCAL evidence, measured by the latest run

• controls satisfied: 8 of 8
• planted pods refused: 16 of 16
• control pods admitted: 3 of 3

Each architecture plants the violations its own checks must catch: 253 changes a static check must refuse in CI and 42 manifests the running cluster must refuse at admission, against 8 it must admit, so that a refusal means something. A check that stays quiet over its own violation, or refuses it for the wrong reason, fails the build; so does a control that is refused.

On AWS the count starts with the first night. The EKS platform failed seven runs before the first one passed, each on something only a real account shows, and each is now a check; its release notes list all seven. The air-gapped agent's two are in its ADR 18: a node that never joined, and a run that left the DNS firewall off on purpose to show its check the violation. A day counts as green when its last run passed, so the runs column is the one that shows a failure earlier the same day. Each record is rewritten by the runs on main; the latest was written 2026-11-03.

Reported upstream

What the runs found in other people's projects, and where each report stands today

A finding in a dependency belongs in that project's tracker, where its maintainers read. Each row is something one of the reference architectures found and posted to the project it came from. The title, the date and the state are read from GitHub every day, so a row says where the report stands now, not where it stood when it was posted. Reported is not fixed: a row says merged only for a change that merged.

Scored by someone else

OpenSSF Scorecard, across the whole fleet, low marks included

Scorecard runs its own checks against every public repository here and publishes the result. The fleet reads that result each morning and repeats it, rather than awarding itself a number. Median 7.5 of 10 across 116 repositories; the lowest is 6.3, snyk-to-gitlab-security-migration. Below, every check that is under ten somewhere, with how many repositories and what that means here. A check Scorecard marks as not applicable is not counted.

Read from Scorecard's own API, results dated 2026-11-03. The decision to leave Code-Review at zero, and why, is decision 16.

Answered by me, published by them

OpenSSF Best Practices, with every answer public and the short ones named

The badge is a questionnaire, not a scan: sixty-seven criteria, each answered Met, Unmet or N/A with a sentence saying why, and OpenSSF publishes the answers beside the badge. A badge is only as honest as its answers, so each one here names the workflow or file that makes it true, and a repository with no test suite says Unmet where it is unmet. Passing on 103 of 103 registered repositories.

Read from the badge site's own API, answers last changed 2026-10-30; 1 could not be read at the last count and are in neither number. The 12 public repositories that hold no code to rate are not registered. Why a criterion that is unmet is answered Unmet rather than Met is decision 18.

Threat model

What an attacker, or an ordinary Tuesday, would try, and what answers

A control that is not tied to a threat is a feature. Each row names the failure first, then the control, then where the control is proven to work, which for a public template is a file a reader can open.

Mapped to the frameworks

Four frameworks, each control mapped to the check that proves it, the gaps named

Four frameworks, mapped onto the public repositories: SLSA v1.0 (Build track), NIST SP 800-218 (SSDF 1.1), the OWASP Top 10 for LLM Applications 2025, and Article 12 of the EU AI Act. Each row names the check that proves the control and a passing run. Where nothing proves it, the row says Gap. No framework is claimed as met; Article 12 binds providers of high-risk AI systems, and nothing here is one by itself. Every run and link checked on 27 October 2026.

42 controls: 12 shown, 23 partly shown, 6 gaps, 1 not applicable.

The air-gapped agent's admission controls also leave CI as signed OSCAL assessment results: kubernetes-oscal-evidence runs them from the agent's verified v1.8.0 release on every push to it and every week. Each of its eight controls, two of the SSDF, five of NIST SP 800-53 and one of the AI RMF, rests on a row of the table above, or of the full matrix it links. In the v1.1.0 run all eight were satisfied, with sixteen planted pods each refused by the policy the mapping names and three control pods admitted. Satisfied means that run says so, not that an assessor did.

The full matrix, one row per SSDF task with line-level links: docs/control-evidence.md in fleet-ops. The count of 163 is the catalog's, fleet.json, measured 2026-10-27 12:29 UTC.

How the AI is run

The agent is fast, and it is wrong on a schedule. The operating model assumes both.

An AI agent does most of the typing across this fleet, and nothing it says about its own work counts as evidence. The typing is the part anyone can buy now. What cannot be bought is the set of rules that decide what the agent's work is allowed to become.

Nothing ships on a claim

A release is cut only after the release commit itself passes a full deploy, backup and restore in CI. The agent's report that something works is not an input.

A check that cannot fail is not a check

Every fleet rule is exercised against a planted violation. A rule that stays quiet over its own violation fails the build.

Exceptions only shrink

Anything excused from a rule sits on a written list. An item that has since been fixed but is still listed is itself a failure, so the list cannot quietly become permanent.

The agent does not hold the keys

It never reads decrypted secrets and never changes a production server. A server is changed only by the agent that lives on it, from a written brief, and it reports back what it did.

Its mistakes are published

The public record of findings includes the agent's own, among them a cause it published before measuring it, and the correction.

Caught before release on 23 September 2026. Five mistakes by the agent in one day of work across 47 repositories. None of them reached a release.