2985 words
15 min read

AI-BOM at Admission — The Cluster Refused a Signed Model for Its Licence

By · Docker Captain · CNCF Ambassador · IBM Champion
Sealed crate at a vault gate, red-striped AI-BOM tag

Sixteen image tags, the admission chart and every signature crossed into a Kubernetes cluster with no route out as one signed file. Then the cluster refused a model image signed with the very key it trusts, because the model’s AI-BOM, its signed bill of materials, declared GPL version 3 only, and the site allows Apache 2.0 and MIT. A signature says who approved the bytes. It says nothing about what they are, or whether the site may run them.

I built both checks into a reference architecture and run them on every change and every day, with planted pods that have to be refused for the right reason. Container supply chains learned to sign images years ago. Models arrive as files from a model hub, with the licence written in a card meant for people. This is the same boundary as the prompt injection evals, one step earlier: before the model may answer anything, the cluster decides whether it may load at all. Until 20 October the boundary also had a way out by DNS, through the kind node, which version 1.5 closed; the boundary run cited below is after it.

What a model admitted on its signature looks like#

The usual policy for model weights asks two questions. Where did the image come from, and who signed it:

# verify-models.yaml in kubernetes-air-gapped-ai-agent at v1.2.0, the usual half: the registry inside, by digest, and the operator's signature
validations:
- expression: >-
images.volumes.all(i, i.startsWith('registry.boundary.internal:5000/models/') && i.contains('@sha256:'))
message: model weights come only from registry.boundary.internal:5000/models, by digest
- expression: >-
images.volumes.map(i, verifyImageSignatures(i, [attestors.operator])).all(n, n > 0)
message: model weights are signed with the operator's key

Both pass for any image the operator signed. A model signed by mistake, or one whose licence changed between two releases, is admitted on the signature alone, because nothing in the cluster knows what the model is.

The other half reads a bill of materials before the model may be mounted:

# verify-models.yaml in kubernetes-air-gapped-ai-agent at v1.2.0, shortened, the half that reads the AI-BOM (a Kyverno 1.19.1 ImageValidatingPolicy)
attestations:
- name: mlbom
intoto:
type: https://cyclonedx.org/bom
# ... attestors: the operator's key, inline
validations:
# ... the usual half's two checks, and a comment on reading the AI-BOM
- expression: >-
images.volumes.map(i, verifyAttestationSignatures(i, attestations.mlbom, [attestors.operator])).all(n, n > 0)
message: model weights carry an AI-BOM signed with the operator's key
- expression: >-
images.volumes.all(i, verifyAttestationSignatures(i, attestations.mlbom, [attestors.operator]) > 0 &&
extractPayload(i, attestations.mlbom).predicate.components.exists(c, c.type == 'machine-learning-model' &&
c.licenses.exists(l, l.license.id in ['Apache-2.0', 'MIT'])))
message: the model's AI-BOM declares a licence the site allows (Apache-2.0, MIT)

Each expression verifies the attestation before it reads it, so a model with no bill is a refusal with its own message, not an evaluation error. Seven pods go through it on every run, one that has to be admitted and six planted ones, each refused by the rule it breaks:

# run 39157806610, 18 October 2026, check 9: each pod and the rule that decided it
model-signed admitted
model-unsigned refused (signed with the operator's key)
model-stranger refused (signed with the operator's key)
model-nobom refused (model weights carry an AI-BOM signed with the operator's key)
model-unlicensed refused (the model's AI-BOM declares a licence the site allows)
model-by-tag refused (by digest)
volume-elsewhere refused (only from registry.boundary.internal:5000/models)

The unlicensed pod is the one a general counsel cares about. Its image is signed with the operator’s key, and its bill is attested with the same key; the only thing wrong with it is the licence the bill declares. It is a planted test image, not a model anyone ships, and the cluster refuses it every day.

In a decade of production infrastructure, I have seen the licence of a dependency settled once, at procurement, in a document the running system never read.

Why this keeps happening#

Signing grew up around container images, and the controls followed the images. Admission policies read the images of containers, init containers and ephemeral containers. A model in this cluster is mounted as an image volume, which Kyverno 1.19 does not check unless a policy asks for it by name. A pull request that would have made it the default was closed for inactivity on 22 October, unmerged, and the issue I opened to carry it on waits on the maintainers’ answer: on by default, or opt-in. A cluster can verify every container it runs and never look at the model.

The licence lives somewhere the cluster cannot reach. It sits in the publisher’s model card, which is prose with front matter in no fixed schema, and in whatever lock file the team keeps in a repository the cluster never reads. Neither is signed by the key admission trusts.

The format is not missing. CycloneDX added the machine-learning bill of materials in version 1.5 in June 2023, and version 1.6 became the basis of the Ecma standard ECMA-424. SPDX 3.0, released in April 2024, has an AI profile of its own. What is missing is a reader at the moment the model is about to load.

A bill nobody reads at admission is documentation.

Risk and blast radius#

For a CISO and a general counsel at the same table, the exposure is a model running under terms nobody approved, inside a site where every other component went through review. The operator’s signature proves the bytes were approved. The bill of materials, attested with the same key and read by the cluster, is what ties those bytes to the terms the organisation agreed to.

A site with no route out raises the stakes on delivery. Everything it runs arrives as a file someone carried across, and whoever loads it on the inside did not build it. Here that check is mechanical: the signature first, the checksums second, then the load. The registry inside held nothing before the load and held exactly the sixteen tags the file carried afterwards, by digest, and the same file with one byte changed was refused and loaded nothing. The file is three to five gigabytes, depending on which model it carries.

The bill has a limit, and it should be said out loud. It records what the operator declares; it does not inspect the weights. Its checksum is the file the image was built from, and the image’s digest is checked against the lock file on every build, so the bill and the bytes are tied together. A wrong licence typed into the lock file is a signed wrong licence.

For a forward-deployed engineer, the repository’s engagement plan turns this into five days from a release tag to a signed-off install. Each day ends with something the customer checks: the file’s hash read over a second channel on day one, the acceptance test passing on their cluster on day two. The first two days are exactly the steps the daily run takes. Days three to five are process, a plan for a pilot rather than a claim about anyone’s production.

A signature answers who approved the bytes; the bill answers what the site agreed to run.

Ways to record a model’s licence, compared#

RecordWho writes itSigned by the key admission trustsWhat a policy can readCost per model
An image labelwhoever builds the imagenoa string, if the policy trusts the buildernone
The publisher’s model cardthe publishernoprose with front matter in no fixed schemanone
SPDX 3.0 with its AI profile, attestedthe operator, from the lock fileyes, when attesteda structured licence fieldone attestation, one read per admission
A CycloneDX ML-BOM, attestedthe operator, from the lock fileyescomponents[].licenses[].license.idone attestation, one read per admission

SPDX is as fit for the job. The repository picked CycloneDX because its sibling, the supply-chain admission architecture, already reads CycloneDX for container images, and one format keeps one way of reading both. For the crossing itself, Zarf and Hauler are the established tools. Zarf brings its own registry and a mutating webhook that rewrites image references; Hauler is closer to what the repository’s script does, with charts and files as first-class content. The script uses oras and cosign, which were already in the toolchain, so admission stays the one path it can explain line by line.

Framework: a model the cluster can vouch for, by its AI-BOM#

Layer 1: write the AI-BOM from the lock file, and validate it on every push (week 1)#

Every model gets one line in a lock file: its name, the publisher’s file pinned at a forty-character revision, the file’s checksum, the licence and the image digest. The bill is written from that line deterministically, with no timestamp and a serial number derived from the file’s hash, so a rebuilt bill is the same bill. CI validates every bill against the CycloneDX 1.6 schema on every push, and a planted invalid bill proves the check can fail:

Terminal window
# scripts/validate.sh at v1.2.0, check mlbom: one AI-BOM per lock line, validated with cyclonedx-cli 0.33.1 pinned by digest;
# a bill that cannot be written or does not validate is counted, and the check goes on to the next model
check_mlbom() {
local img ok=0 n=0 name url sha lic
img="$(sed -n 's/^CYCLONEDX_IMAGE=//p' ci-images.pins)"
mkdir -p out/mlbom
while read -r name url sha lic _; do
n=$((n + 1))
python3 scripts/mlbom.py "$name" "$url" "$sha" "$lic" >"out/mlbom/$name.json" || { echo "$name: no AI-BOM could be written"; ok=1; continue; }
docker run --rm -v "$PWD/out/mlbom:/b:ro" "$img" validate --input-file "/b/$name.json" \
--input-format json --input-version v1_6 --fail-on-errors >/dev/null 2>&1 \
|| { echo "$name: its AI-BOM is not valid CycloneDX 1.6"; ok=1; }
done < <(grep -vE '^\s*(#|$)' models/models.lock)
[ "$ok" = 0 ] && echo "every model's AI-BOM is valid CycloneDX 1.6 ($n models)"
return "$ok"
}

This is the bill it writes for IBM’s Granite, the model the release ships with:

{
"bomFormat": "CycloneDX",
"specVersion": "1.6",
"serialNumber": "urn:uuid:a0950fa8-c1c2-5c08-98ef-2dff8a5d698e",
"version": 1,
"metadata": {"component": {"type": "machine-learning-model", "bom-ref": "model", "name": "granite-4.0-1b"}},
"components": [{
"type": "machine-learning-model",
"bom-ref": "model",
"name": "granite-4.0-1b",
"version": "b27c2fe3f211b7f44e80fa620177aea371099aaa",
"publisher": "ibm-granite",
"licenses": [{"license": {"id": "Apache-2.0"}}],
"hashes": [{"alg": "SHA-256", "content": "22ec0f9cc99a90185312de3c882c84e7bd6789bdd050389844380a01a831d7f1"}],
"externalReferences": [{"type": "distribution", "url": "https://huggingface.co/ibm-granite/granite-4.0-1b-GGUF/resolve/b27c2fe3f211b7f44e80fa620177aea371099aaa/granite-4.0-1b-Q4_K_M.gguf"}],
"properties": [
{"name": "huggingface:repository", "value": "ibm-granite/granite-4.0-1b-GGUF"},
{"name": "huggingface:revision", "value": "b27c2fe3f211b7f44e80fa620177aea371099aaa"},
{"name": "file", "value": "granite-4.0-1b-Q4_K_M.gguf"},
{"name": "format", "value": "GGUF"},
{"name": "quantization", "value": "Q4_K_M"}
],
"modelCard": {"modelParameters": {"task": "text-generation"}}
}]
}

Owner: the ML engineer who owns the model choice, with the licence on each lock line reviewed by legal before it merges.

Layer 2: attest the AI-BOM with the key admission trusts, and read it before the model loads (week 2)#

The bill travels with the image as an in-toto attestation, signed with the same operator key as the image and, like the signature, with no transparency log, because nothing inside the boundary can reach one:

Terminal window
# scripts/model.sh at v1.2.0, shortened: the AI-BOM as an in-toto attestation of type cyclonedx, signed with the operator's key
# attest <ref@digest> <bom.json>: the AI-BOM as an in-toto attestation,
# signed with the operator's key and no transparency log, like the signature.
attest() {
SSL_CERT_FILE="$STATE/bundle.pem" COSIGN_PASSWORD="" \
cosign attest --key "$STATE/operator.key" --type cyclonedx --predicate "$2" \
--use-signing-config=false --tlog-upload=false --yes "$1" >/dev/null 2>&1
}
# ... and in "sign", once the model image is signed:
python3 "$ROOT/scripts/mlbom.py" "$MODEL" "$URL" "$SHA" "$_license" >"$STATE/$MODEL.mlbom.json"
attest "$REG/models/$MODEL@$digest" "$STATE/$MODEL.mlbom.json"

Admission reads it with the whole policy. The operator’s public key is substituted for the placeholder when the policy is installed:

# admission/verify-models.yaml in kubernetes-air-gapped-ai-agent at v1.2.0, shortened: a pod may mount model weights only from the registry inside, by digest,
# signed with the operator's key, with an AI-BOM signed by the same key that declares an allowed licence
apiVersion: policies.kyverno.io/v1
kind: ImageValidatingPolicy
metadata:
name: verify-models
spec:
validationActions: [Deny]
failurePolicy: Fail
webhookConfiguration:
timeoutSeconds: 30
evaluation:
background:
enabled: false
autogen:
podControllers:
controllers: []
matchConstraints:
namespaceSelector:
matchLabels:
boundary.heyvaldemar.com/models: verified
resourceRules:
- apiGroups: [""]
apiVersions: [v1]
operations: [CREATE, UPDATE]
resources: [pods]
images:
- name: volumes
expression: >-
object.spec.?volumes.orValue([]).filter(v, has(v.image)).map(v, v.image.reference)
matchImageReferences:
- glob: "registry.boundary.internal:5000/models/*"
validationConfigurations:
verifyDigest: true
mutateDigest: false
required: true
attestations:
- name: mlbom
intoto:
type: https://cyclonedx.org/bom
attestors:
- name: operator
cosign:
key:
data: |
@OPERATOR_KEY@
ctlog:
insecureIgnoreTlog: true
insecureIgnoreSCT: true
validations:
- expression: >-
images.volumes.all(i, i.startsWith('registry.boundary.internal:5000/models/') && i.contains('@sha256:'))
message: model weights come only from registry.boundary.internal:5000/models, by digest
- expression: >-
images.volumes.map(i, verifyImageSignatures(i, [attestors.operator])).all(n, n > 0)
message: model weights are signed with the operator's key
# ... a comment: each expression verifies the attestation before it reads it
- expression: >-
images.volumes.map(i, verifyAttestationSignatures(i, attestations.mlbom, [attestors.operator])).all(n, n > 0)
message: model weights carry an AI-BOM signed with the operator's key
- expression: >-
images.volumes.all(i, verifyAttestationSignatures(i, attestations.mlbom, [attestors.operator]) > 0 &&
extractPayload(i, attestations.mlbom).predicate.components.exists(c, c.type == 'machine-learning-model' &&
c.licenses.exists(l, l.license.id in ['Apache-2.0', 'MIT'])))
message: the model's AI-BOM declares a licence the site allows (Apache-2.0, MIT)

The rule as quoted had a hole, and a reader found it. Axialis, replying on Bluesky to my post about this release, suggested evaluating SPDX expressions rather than accepting any matching ID, because an OR expression and an AND expression carry different obligations. That is exactly what the rule’s exists check did: it accepted any matching ID. A bill that listed Apache 2.0 beside a non-commercial licence would have been admitted, and so would a second model with no licence at all, or a model nested inside another component. The generator writes one licence per model, so no bill ever went through the hole. The contract was still wrong.

The fix shipped in version 1.6 on 21 October. Every model in the bill is checked, every licence of each, and each must be on the list. An SPDX expression is refused at admission and evaluated where the bill is made: an OR expression concludes to the first reading the site allows, an AND expression needs every term allowed, and the declared expression is kept beside the conclusion. The new planted bills ran against the old rule first, on a real cluster, and three of them failed: two were admitted, and the third was refused only because the policy errored on a missing key. With the fix, each is refused with its own message:

# the two refusals in admission/verify-models.yaml at v1.6.0
every model in the AI-BOM declares only licences the site allows (Apache-2.0, MIT)
the model's AI-BOM names each licence by its SPDX id, with no expression and no nested component (scripts/mlbom.py concludes an expression to ids)

A rule that accepts any one licence vouches for none of them.

Owner: the platform team runs the policy. The security lead owns the list of allowed licences, and changing it is a one-line pull request that legal approves.

Layer 3: carry everything across as one signed file (week 3 onward)#

On the connected side, from a release tag and never from a branch, everything the cluster runs goes into one tar with a checksum over every file, signed with the operator’s key. The file’s hash is read to the site over a second channel before it moves. On the inside, nothing is read from a file whose signature does not verify, and nothing is loaded from one whose checksums do not match:

Terminal window
# outside, on a connected Linux machine with Docker
git checkout v1.6.0
./scripts/airgap.sh registry && ./scripts/airgap.sh seed # the staging registry
./scripts/model.sh build && ./scripts/model.sh sign # the model, rebuilt, signed and attested
./scripts/admission.sh fetch # the admission chart, checked
./scripts/bundle.sh create # .cache/airgap/bundle.tar, signed
sha256sum .cache/airgap/bundle.tar # read to the site over a second channel
# inside, after the cut
./scripts/airgap.sh registry && ./scripts/airgap.sh mirror
./scripts/bundle.sh load # the signature, then the checksums, then the load
./scripts/admission.sh install && ./scripts/admission.sh policy
./tests/e2e/run.sh # check 10: empty before, exactly what the file carried after

Owner: the operator who holds the signing key builds and signs the file. The site’s security engineer runs the load and attaches the acceptance test’s output to the change record.

Tradeoffs#

The proof has edges. There is no real customer site behind it, no data diode and no GPU: it runs on kind, on CPU, on one node with one registry. The signatures are checked by key with no transparency log, which is weaker than keyless signing with a log; carrying the log’s inclusion proof across inside the signature bundle is possible, and this release does not do it. The bill records what the operator declares. The runs in this post used the Kyverno release of September; the release of 20 October closed six advisories in image verification and attestation, the repository moved to it on 23 October, and the daily run has passed on it every day since 24 October.

The costs are concrete. Building the file peaks at three copies of everything on disk, both sides need Linux with GNU tar, and admission reads one more attestation per model. Changing what the site allows is one line in the policy, and the next model under another licence is refused until someone changes it on purpose. That last property is the point.

The closing argument#

A signature tells a cluster who approved a model. A bill of materials tells it what the model is and what the site agreed to run, and only the cluster can check that at the moment it matters. Carry everything across as one file the inside can verify without trusting the carrier. Read the bill before the model loads. The price is one more attestation per model, and a licence list someone has to own.

Sources#

Discussion#

If you check what a model is before it loads, or trust the signature alone, drop a comment below. Counterarguments are welcome and the comment thread is where I respond first. For longer back-and-forth with senior practitioners, join the discussion on Discord.


Vladimir Mikhalev

Docker Captain  ·  CNCF Ambassador  ·  IBM Champion

The Verdict — production-tested analysis on YouTube.

The Verdict

Inconvenient truths about shipping in the AI era

Container security, platform engineering, and the agentic shift — tested in production, argued without the hype. The verdict reaches your inbox the moment there's one worth sending.

Related Posts

Same category
  1. 1
    Prompt Injection Evals — Six Models, One Boundary, Nothing Left the Company
    AI & MLOps · Prompt injection evals on six small models. They acted on hidden instructions from never to half the time, and zero data left. Gate the boundary.
  2. 2
    AI Agent Governance — One Engineer, Over a Hundred Repositories and the Decisions No Agent Makes
    AI & MLOps · AI agent governance for one engineer and 100+ repositories. What the agents decide, what they never do, and the documented auto-merge that merges red checks.
  3. 3
    AI Reviews That Read Nothing — Thirty-Four Verdicts in a Hundred
    AI & MLOps · A third of the images my fleet pins would have had their next AI review written against nothing. How to prove what an AI review read before anyone acts on it.
  4. 4
    watsonx.governance Enforcement Tracking — Sampled Compliance and the Unevidenced Run
    AI & MLOps · EU AI Act Article 12 wants a per-event record. Enforcement Tracking evidences thresholds on a schedule. What that gap costs, and the join key that closes it.

Random Posts

Random
  1. 1
    Install Bitwarden on Ubuntu Server 20.04 LTS
    Self-Hosting · Step-by-step guide to install Bitwarden on Ubuntu Server 20.04 LTS using Docker and Let's Encrypt. Secure your passwords with this open-source solution.
  2. 2
    Inside the Builders Era — How Developers Stay in Control of AI with GitKraken as the Core Tool
    Opinion & Culture · How developers stay in control of AI in the Builders Era — using expert supervision, clean Git workflows, and GitKraken to keep speed, quality, and reliability.
  3. 3
    Cheburnet as the Fortress of Lies and Censorship in Russia
    Opinion & Culture · The tightening of internet censorship in Russia, including blocking access to social networks and VPNs, illustrates the growth of isolation and control.
  4. 4
    Install Docmost Using Docker Compose
    Self-Hosting · Learn how to install Docmost using Docker Compose with Traefik and Let's Encrypt. Step-by-step guide for self-hosting a modern documentation platform.
AI-BOM at Admission — The Cluster Refused a Signed Model for Its Licence
https://heyvaldemar.com/ai-bom-model-licence-admission/
Author
Vladimir Mikhalev
Published
2026-11-03
License
CC BY-NC-SA 4.0