watsonx.governance Enforcement Tracking — Sampled Compliance and the Unevidenced Run

What sampled compliance looks like in a running agent#

Here is how most teams ship a containerized agent today.

1
# kubernetes 1.30 — claims-triage agent as most teams ship it
2
apiVersion: apps/v1
3
kind: Deployment
4
metadata:
5
  name: claims-triage-agent
6
spec:
7
  replicas: 3
8
  selector:
9
    matchLabels:
10
      app: claims-triage-agent
11
  template:
12
    metadata:
13
      labels:
14
        app: claims-triage-agent
15
    spec:
16
      containers:
17
        - name: agent
18
          image: registry.internal/claims-triage-agent:1.4.2
19
          env:
20
            - name: ORCHESTRATE_ENDPOINT
21
              valueFrom:
22
                secretKeyRef:
23
                  name: wxo-credentials
24
                  key: endpoint

Nothing on that manifest connects a single execution to a governance record. The agent runs, it writes to stdout, and the only durable statement about its behavior is whatever the evaluation schedule captured. Ask for run 4192 and you get a shrug and a log query against whatever retention the cluster happens to have.

The change is small and it is structural.

1
# kubernetes 1.30 — same agent, every execution addressable
2
apiVersion: apps/v1
3
kind: Deployment
4
metadata:
5
  name: claims-triage-agent
6
  labels:
7
    ai.use-case-id: uc-claims-triage-eu
8
    ai.policy-version: "2026.08.1"
9
spec:
10
  replicas: 3
11
  selector:
12
    matchLabels:
13
      app: claims-triage-agent
14
  template:
15
    metadata:
16
      labels:
17
        app: claims-triage-agent
18
        ai.use-case-id: uc-claims-triage-eu
19
        ai.policy-version: "2026.08.1"
20
    spec:
21
      containers:
22
        - name: agent
23
          image: registry.internal/claims-triage-agent:1.4.2
24
          env:
25
            - name: ORCHESTRATE_ENDPOINT
26
              valueFrom:
27
                secretKeyRef:
28
                  name: wxo-credentials
29
                  key: endpoint
30
            - name: OTEL_EXPORTER_OTLP_ENDPOINT
31
              value: http://otel-collector.observability:4317
32
            - name: OTEL_RESOURCE_ATTRIBUTES
33
              value: ai.use_case.id=uc-claims-triage-eu,ai.policy.version=2026.08.1

Two additions. The workload now carries the same use-case identifier the governance system uses, and every execution emits a span under that identifier with the policy version that was live at the time. You are not storing prompts or outputs. You are storing the fact that a run occurred, under which policy, touching which tools. That is the join key, and without it the two systems can never be reconciled at all.

Twenty years of production infrastructure and this pattern has arrived in three different decades of stack: snapshot intervals in banking, SNMP polling in telecom, and a change-management record sampling a pipeline that fired hundreds of times a day in a Series D environment. Each time the fix was the same and each time it looked like overhead until the first audit.

The interval is the record. Everything between two samples is a story.

Why this keeps happening#

The regulation asks for two different things and vendors ship the easier one first, because it is the one with a clean product surface.

Article 12 of the AI Act requires high-risk systems to automatically record events across the lifetime of the system, with three stated purposes: identifying risk situations under Article 79, feeding post-market monitoring under Article 72, and supporting deployer oversight under Article 26, paragraph 5. Article 19 sets the retention floor for providers at six months, longer where other Union or national law applies, and Article 26, paragraph 6 places a parallel six-month duty on deployers for the logs under their control. Article 18 puts technical documentation on a ten-year clock. None of those clauses describes a threshold check. They describe a per-event record with a retention obligation attached.

Enforcement Tracking, per IBM’s announcement of 11 August 2026, connects watsonx Orchestrate to watsonx.governance, associates agents with governance use cases and controls, retrieves evaluation metrics on a scheduled basis for production agents, evaluates agents still in development on demand, and writes pass or breach results against business-set thresholds directly into the governance record. Hallucination, helpfulness and toxicity are the named metrics. That is a control attestation pipeline, automated, and it removes the quarterly scramble of assembling evidence by hand.

The second thing is harder to productize because it is specific to your deployment. A vendor cannot know your correlation identifier, your policy versioning scheme, or which of your systems holds the execution ledger. So it stays where it has always stayed. The vendor shipped the evidence pipeline. The join key is still yours.

Risk and blast radius#

Direct exposure runs through retention and traceability rather than through the metrics themselves. If your agent falls inside Annex III, the Article 12 record has to exist and the Article 19 and Article 26, paragraph 6 floors mean six months of it has to still be there when someone asks. A threshold snapshot taken on a schedule does not satisfy a per-event recording obligation, and no amount of dashboard polish converts one into the other. Penalty tiers under the Act put transparency and high-risk breaches at up to 15 million euro or 3% of worldwide turnover, with prohibited practices at 35 million euro or 7%.

Read what the deferral actually deferred, because the headline flattens it. Annex III systems moved to 2 December 2027 and high-risk AI embedded in already-regulated products under Annex I moved to 2 August 2028. Article 50 transparency duties did not move at all; they took effect on 2 August 2026 as originally scheduled, alongside the prohibited-practices regime that has applied since February 2025 and the GPAI obligations running since August 2025. The Omnibus attached one new near-term date to them: systems already on the market before 2 August 2026 have until 2 December 2026 to carry machine-readable marking of AI-generated content. Sixteen months of relief on Annex III. Fourteen weeks on the marking obligation. And the stated reason for the delay was procedural rather than political: the CEN and CENELEC harmonised standards were not finished and several Member States had not designated their competent authorities. A deadline that moved because the referee was late is not a signal that the rules got softer.

Systemic exposure is the part the deferral headline hides. Systems already placed on the EU market before the new deadlines are grandfathered out of full high-risk compliance unless they are later substantially modified, and regulators have not yet defined that threshold. For an agent that reasons over changing context, calls tools dynamically, and gets its prompts and models updated on a normal release cadence, “substantially modified” is not a rhetorical question. It is an evidentiary one, and answering it requires a change record joined to an execution record. Sampling cannot produce that answer in either direction.

Before assuming Article 26, paragraph 6 is your ceiling, check which side of Article 25 you are standing on. A deployer who puts their own name or trademark on a high-risk system, or substantially modifies one already placed on the market, is treated as its provider. Assemble agents on a vendor platform, ship them under your own brand, and the provider obligations follow: the Article 12 recording capability and the Article 19 retention floor, not merely the deployer’s duty to keep whatever the platform hands you. The original provider is relieved of them at that moment and owes you cooperation instead. The platform did not transfer the obligation. It transferred the role.

Then the arithmetic. An agent invoked every five minutes executes 288 times a day, or 8,640 times over thirty days. Against a daily metrics synchronisation, that month closes with 30 evidence records covering 8,640 executions, or 0.35% of runs individually evidenced. Tighten the schedule to hourly and you get 720 records against 8,640 runs, which is 8.3%. The schedule is the right design for attesting a control. It is the wrong artifact for reconstructing an incident.

A deferral you cannot document is not a deferral.

Options compared#

Storage figures below are back-of-envelope from stated unit sizes, not measured: trace metadata at about 1.5 KB per span, full payload capture at about 1 MB per run average for a tool-calling agent, hash digest at about 150 bytes.

Most teams need rows one and two together. Row one without row two is a control with no event behind it.

Framework: reconciling the execution ledger with the evidence ledger#

1
# opentelemetry-sdk 1.x — one span per agent execution, carrying the governance join key
2
from opentelemetry import trace
3

4
tracer = trace.get_tracer("wxo.agent.claims_triage")
5

6
def run_agent(request, ai_use_case_id: str, policy_version: str):
7
    with tracer.start_as_current_span("agent.execution") as span:
8
        span.set_attribute("ai.use_case.id", ai_use_case_id)
9
        span.set_attribute("ai.policy.version", policy_version)
10
        span.set_attribute("ai.agent.id", "claims-triage")
11
        result = agent.invoke(request)
12
        span.set_attribute("ai.tools.invoked", ",".join(result.tool_calls))
13
        span.set_attribute("ai.outcome", result.status)
14
        return result

1
# terraform 1.x, aws provider 5.x — bound the agent's execution role, carry the governance join key
2
resource "aws_iam_role" "agent_execution" {
3
  name                 = "wxo-agent-claims-triage"
4
  permissions_boundary = aws_iam_policy.agent_boundary.arn
5
  assume_role_policy   = data.aws_iam_policy_document.agent_trust.json
6

7
  tags = {
8
    ai_use_case_id = var.ai_use_case_id # same key on every span and every evidence record
9
    policy_version = var.policy_version
10
  }
11
}
12

13
resource "aws_iam_policy" "agent_boundary" {
14
  name   = "wxo-agent-boundary"
15
  policy = data.aws_iam_policy_document.agent_boundary.json
16
}
17

18
data "aws_iam_policy_document" "agent_trust" {
19
  statement {
20
    effect  = "Allow"
21
    actions = ["sts:AssumeRoleWithWebIdentity"]
22

23
    principals {
24
      type        = "Federated"
25
      identifiers = [var.oidc_provider_arn]
26
    }
27

28
    condition {
29
      test     = "StringEquals"
30
      variable = "${var.oidc_provider_host}:sub"
31
      values   = ["system:serviceaccount:default:claims-triage-agent"]
32
    }
33
  }
34
}
35

36
variable "ai_use_case_id" { type = string }
37
variable "policy_version" { type = string }
38
variable "oidc_provider_arn" { type = string }
39
variable "oidc_provider_host" { type = string }
40

41
data "aws_iam_policy_document" "agent_boundary" {
42
  statement {
43
    effect    = "Allow"
44
    actions   = ["s3:GetObject"]
45
    resources = ["arn:aws:s3:::claims-reference/*"]
46
  }
47

48
  statement {
49
    effect    = "Deny"
50
    actions   = ["iam:*", "sts:AssumeRole"]
51
    resources = ["*"]
52
  }
53
}

1
# sentinel policy — refuse a plan that provisions an agent role with no governance use case
2
import "tfplan/v2" as tfplan
3

4
required_tag = "ai_use_case_id"
5

6
agent_roles = filter tfplan.resource_changes as _, rc {
7
  rc.type is "aws_iam_role" and
8
    rc.mode is "managed" and
9
    (rc.change.actions contains "create" or rc.change.actions contains "update")
10
}
11

12
main = rule {
13
  all agent_roles as _, rc {
14
    rc.change.after.tags[required_tag] else "" is not ""
15
  }
16
}

1
# opa 1.x / conftest — block agent deploys that cannot be joined to a governance use case
2
package agent.evidence
3

4
deny contains msg if {
5
  input.kind == "Deployment"
6
  not input.metadata.labels["ai.use-case-id"]
7
  msg := "missing ai.use-case-id label: runs cannot be joined to governance evidence"
8
}
9

10
deny contains msg if {
11
  input.kind == "Deployment"
12
  some c in input.spec.template.spec.containers
13
  not otlp_configured(c)
14
  msg := sprintf("container %q has no OTEL_EXPORTER_OTLP_ENDPOINT: executions will not be recorded", [c.name])
15
}
16

17
otlp_configured(c) if {
18
  some e in c.env
19
  e.name == "OTEL_EXPORTER_OTLP_ENDPOINT"
20
}

1
# prometheus 2.x — alert when agent runs have no matching governance evidence record
2
groups:
3
  - name: agent-evidence-coverage
4
    interval: 5m
5
    rules:
6
      - record: agent:runs_unevidenced:ratio1h
7
        expr: |
8
          1 - (
9
            sum by (ai_use_case_id) (increase(agent_runs_evidenced_total[1h]))
10
            /
11
            clamp_min(sum by (ai_use_case_id) (increase(agent_runs_total[1h])), 1)
12
          )
13

14
      - alert: AgentEvidenceCoverageBelowFloor
15
        expr: agent:runs_unevidenced:ratio1h > 0.05
16
        for: 15m
17
        labels:
18
          severity: page
19
          control: eu-ai-act-art-12
20
        annotations:
21
          summary: Agent runs without a matching governance evidence record
22
          description: >
23
            {{ $value | humanizePercentage }} of runs for use case
24
            {{ $labels.ai_use_case_id }} have no evidence record in the last hour.

Tradeoffs#

Adoption friction is real and it is mostly week one. Instrumenting spans across every agent path takes a sprint, the OTLP collector and its retention tier need a budget line, and someone has to own a versioning scheme for policies that was previously implicit. At roughly 1.5 KB per span you are looking at about 1.5 GB per million executions before compression, which is a rounding error next to any observability bill you already pay.

Ignoring it costs asymmetrically. The evidence you did not collect cannot be reconstructed after the request arrives, and the request arrives at the worst possible moment: a complaint, an incident, a due-diligence questionnaire, a market surveillance authority asking under Article 26, paragraph 6. The sixteen-month deferral makes this feel optional. It is the opposite. Sixteen months is exactly how long you now have to accumulate the record that proves your system was not substantially modified.

The closing argument#

Adopt Enforcement Tracking for what it is. It automates the control attestation, it writes pass and breach evidence without a human assembling screenshots, and for teams currently maintaining governance evidence by hand it removes real work on day one. Hold on treating it as the audit trail, because a threshold snapshot and a per-event record are different artifacts with different retention clocks. The gap is not a defect in the release; it is the part that was always going to be specific to your deployment, and the join key was always going to be yours to define. The storage is cheap. The reconstruction is not.

Sources#

• IBM, From governance policies to governance proof with Enforcement Tracking for watsonx Orchestrate, August 11, 2026
• IBM Documentation, Configuring metrics synchronization for watsonx Orchestrate agents (evaluation schedule, threshold evidence), accessed August 2026
• Official Journal of the European Union, Regulation (EU) 2026/1744 (Digital Omnibus on AI), published July 24, 2026, in force July 27, 2026
• European Commission AI Act Service Desk, Article 19: Automatically generated logs (six-month retention floor), accessed August 2026
• White & Case, EU AI Omnibus enters into force, amending the AI Act (Annex III and Annex I deadlines, Article 50 unchanged), 2026
• Cooley, Digital AI Omnibus Delays Key Deadlines, Introduces New Rules (2 December 2026 marking deadline, Annex I and public-authority dates), 2026
• Gibson Dunn, EU AI Act Omnibus Agreement: Postponed High-Risk Deadlines and Other Key Changes (CEN/CENELEC standards, authority designation), May 2026

Discussion#

If you have wired agent executions to a governance record, or you have decided the sampled attestation is enough for your risk profile, drop a comment on the post. Counterarguments are welcome and the comment thread is where I respond first. For longer back-and-forth with senior practitioners, join the discussion on Discord.