Implementing an AI Agent Governance Framework for Production SRE

Before you start

Governance is a set of controls in front of agents that already act, so the first job is an inventory rather than a policy. List every agent that can change production: coding agents with deploy rights, remediation agents on the on-call rota, pipeline bots that promote builds, and anything reaching production through an MCP server. For each one, write down the identity it runs as and the tools it can call today. Most teams find at least one agent acting through a shared service account, and that single finding decides where to start.

Two references are worth reading before you design anything. The OWASP AI Agent Security Cheat Sheet sets out the threat classes (tool misuse, excessive agency, privilege compromise) that the controls below answer. The OWASP MCP Security Cheat Sheet covers the protocol most agents now use to reach tools, including authorization and tool-definition trust. What AI agent governance means maps these to NIST AI RMF, ISO/IEC 42001, the EU AI Act and SOC 2.

Use this readiness checklist to score each agent before it runs unattended:

Readiness check	What passing looks like	Where it usually fails
Identity	Each action is attributable to one agent instance, role and session	Shared service accounts used by several agents
Tool grants	Scoped to the task and time-limited	Standing roles broad enough for any task
Write path	Writes are held for a policy verdict or a person	Writes go straight through, logged afterwards
Predicted impact	Every high-impact action carries a blast radius computed before it runs	Impact estimated after the incident
Irreversible actions	Classified and gated on explicit approval	Schema changes treated like restarts
Signed record	Verdicts, including refusals, are signed outside the agent	The agent's own transcript is the record
Kill switch	Halts all agent activity without agent cooperation	A flag the agent itself reads

The steps

Establish an AI governance framework

An AI agent governance framework defines which agents may act in production, under which identity, with which tools, and who is accountable when they do. It is established when those answers are enforced by the system rather than written in a document. Start by making identity and scope real, because every later control depends on knowing who is acting.

1. Give every agent its own identity. Replace shared service accounts with a distinct identity per agent instance, bound to a session. An action attributed to "the automation user" cannot be governed or audited.

2. Classify actions by reversibility. Sort the tools each agent can call into reads, reversible writes (scale, restart, roll back application code) and irreversible writes (schema migrations, data deletion, credential rotation). The classification drives every policy that follows.

3. Assign an owner per agent. Name the team accountable for each agent's grants and for reviewing its refusals. An agent with no owner accumulates permissions and nobody notices.

Implement runtime policy enforcement

Runtime policy enforcement evaluates each agent action against policy at the moment the agent tries it, and can refuse it before anything executes. A rule that only fires in a weekly review is a report, not enforcement. The policy layer has to sit between the agent and the tool, so the agent receives a verdict rather than a result.

4. Put a gate in front of every tool call. Route agent actions through one enforcement point, whether that is an API gateway, an MCP proxy or a platform built for it. The gate returns one of four verdicts: approve, scope down, escalate to a person, or refuse.

5. Let reads through and hold writes. Reads rarely need a verdict. Writes do. Start with every write held for approval, then relax specific reversible actions to policy-only as their refusal history shows the policy is right.

6. Scope grants per task and let them expire. A grant covers the tools one task needs and ends when the task does. That is least privilege for agents, which differs from human IAM because the next task may have nothing in common with the last.

NOFire AI implements this gate against the Context and Control Model: policy is evaluated against a time-versioned map of production, so the same restart can be approved in an idle cluster and escalated when it touches the last healthy replicas of a payment service.

Analyze blast radius and audit agent activity

Blast radius analysis predicts which services, users and transactions an agent action could affect before it runs, so policy can refuse actions above a bound. Auditing records each verdict and its predicted impact where the agent cannot edit it. Together they answer the two questions an incident review asks: what could this action reach, and what did the agent actually do.

7. Attach a predicted blast radius to each write. Compute impact by walking the observed dependency graph outward from the target, as described in how blast radius is calculated, not from the target's own traffic share. Set a ceiling per action class and refuse anything above it.

8. Sign every verdict outside the agent. Record the policy that decided, the verdict, the tool call and arguments, the predicted impact, the identity and any approver, each chained to the previous record. Export it to the SIEM you already audit. NOFire AI signs each record outside the agent process, so the evidence is not a story the agent wrote.

9. Correlate verdicts with traces. Write the verdict onto the same OpenTelemetry span as the action, so an incident review sees permitted and refused actions in one timeline.

Verify it worked

Run three tests before calling the framework live.

Trigger a refusal on purpose. Ask an agent to perform a write above its blast-radius ceiling in a non-production environment. The action should not execute, and a signed refusal naming the agent identity and the deciding policy should appear in your SIEM.

Sample a window. Pick a two-hour window from last week and produce every agent action in it, with verdicts, from the audit record alone. What an auditor asks for explains why this sampling test is the one that matters.

Pull the kill switch in staging. Every agent should stop acting within the time you have stated, without any agent reading a flag.

Where it breaks

Agents acting through channels the gate does not see. A coding agent with a personal cloud credential bypasses every control above. The inventory in step one has to be repeated, because new paths appear every time someone connects a tool.

Blast radius from a stale graph. A predicted impact built from a declared dependency list undercounts, and undercounting is the dangerous direction. The prediction is only as good as the observed topology behind it.

Approval fatigue. Holding every write works for a week. After that, people approve without reading. Relax specific action classes deliberately, with refusal history as the evidence, before the humans relax all of them informally.

Supply chain is a separate control. Runtime enforcement bounds what a compromised agent can do. It does not verify that the model, tool definition or dependency that loaded is the one you expected. That needs signature verification upstream.