10 from AI Reviews That Read Nothing — Thirty-Four Verdicts in a Hundred
The read receipt: no AI verdict without the source it read
By Vladimir Mikhalev Docker Captain · CNCF Ambassador · IBM Champion
The model was the most reliable part of that pipeline. It read what it was handed and said what it could not read. Everything that failed happened before the prompt and after the response.
That is ordinary software, and it is tested like ordinary software: the lookup that picks the document, the envelope nobody opened, and the authority the verdict is given.
- 1
Prove the source before the prompt
A mapping from an image to a repository is accepted only when the version the fleet pins exists as a tag in that repository. Take the address from the image's own source label where there is one and from a table where there is not, and run the check on every address on a schedule. The label is the publisher's claim; the tag is the proof.
- 2
Make the lookup able to say no
Its test plants a repository that does not exist and requires “absent” back, not an error and not an answer. “This publisher keeps its notes elsewhere” is a third answer, kept in a short table with the reason beside each entry.
- 3
Name a missing source in a field, not in the last paragraph
No source, no verdict. A review that could not read the notes says so where the verdict would be.
- 4
Open the envelope before using the answer
Check how generation stopped, treat a response with no text blocks as a failure rather than an empty answer, and put the token counts in the error.
- 5
Hand the model facts, and let them win
The registry's image configurations sit beside the release notes, so a renamed user is read from the image, not guessed from a sentence.
- 6
Bound what the verdict can do
A verdict can slow a change down and never wave one through. Inside a major version a bump proceeds on the CI-gated path; DO NOT APPLY UNATTENDED reroutes it to a prepared branch for a person; a review that cannot run leaves the CI gate alone in charge. Owner: platform engineer.
The failure modes, by what the reviewer was holding
| What it was holding | What it looks like | What settles it |
|---|---|---|
| Nothing | A verdict on an upgrade nobody read | Prove the source before the prompt |
| The wrong project | Confident notes about somebody else's software | The pinned version must exist as a tag under that name |
| A blank | A clean report with nothing in it | Raise on zero text blocks, with the token counts |
| A description | NEEDS ATTENTION on a rename | Give the model the image configuration, and let it win |
| A third of the fleet | A smaller fleet, published as the whole | Read everything before writing anything |
| Another workflow's answer | Good changes reverted | Judge a change only by the workflow that verifies it; skipped is not red |
What shows it works
Three findings with the rules they produced, and the code that enforces them.
- Finding 124, found on 20 September 2026 (the post): 34 of 100 pinned images named a repository that does not exist; a mapping is now accepted only when the pinned version is a tag there.
- Finding 119: an empty answer published as a report,
in=44175 out=4000and no text block; the shared helper now raises on it. - Finding 39, 25 September 2026: twenty-three good refreshes reverted in one run because “skipped” was read as red.
- In fleet-ops, public, at the commit of 14 November 2026: the reviewer and its table of sources, the mapping check, and the test that requires it to say absent.
- Decision 5, 6 September 2026: Dependabot merges itself only after the repository's own verification is green on that commit.
- chatops-privilege-wall: the same rule for automation that may act. The half that listens cannot act.
A reviewer that cannot show what it read is a fluent guess.
The long version is the post AI Reviews That Read Nothing. Anthropic, Messages API reference (stop reasons); OCI Image Format Specification, Annotations (the standard source label); OWASP, Top 10 for LLM Applications 2025, Misinformation; ISMS.online on SOC 2 CC8.1 and ISO 27001 8.32.