AI agent audit and compliance

AI agent audit and compliance is the problem of producing evidence about what autonomous agents did in production that survives someone else examining it. The frameworks that ask for this, and what each one asks, are covered in AI agent governance. This page is the other half: what an auditor actually does with the record, and what makes one hold up.

The gap between the two is wider than most teams expect. Nearly every organisation running agents has logs. Considerably fewer can produce evidence.

What an auditor actually asks for

They do not ask whether you log agent actions. They assume you do, and then they sample.

The characteristic SOC 2 Type II move is to pick dates from inside the observation period and ask for everything in those windows. Type I assesses whether a control is designed correctly on a single date. Type II tests whether it operated across a period, typically three to twelve months, by pulling samples from within it. A control that was configured correctly and silently stopped running in month four fails Type II, and the failure is discovered by sampling month four rather than by anyone noticing at the time.

That shapes what the record has to support. Not "here is a log", but "here is every agent action between 02:00 and 04:00 on 14 March, and here is why you can believe that is all of them".

ISO/IEC 42001 certification and EU AI Act supervision work the same way. The question is never whether the control exists on paper. It is whether you can demonstrate it operating, over a window the examiner picks rather than one you prepared.

What makes a record defensible

Four properties, and most audit failures are one of them missing rather than the record being absent.

Completeness. You have to be able to show nothing was omitted. An appended text log cannot demonstrate this: entries can be dropped, and the file looks identical either way. Sequence numbers and a hash chain, where each record commits to the previous one, turn a gap into something detectable rather than invisible.

Integrity. The record has to be resistant to editing after the fact, including by people inside your organisation. Signing at the point of the decision, append-only storage, and keys held outside the system being audited are the usual mechanisms. A log an administrator could have edited proves less than it appears to.

Attribution. An action recorded against "the agent" or a shared service account is not attributable. The record needs a specific agent instance, role and session, which is why identity binding is a governance control rather than a logging one. This is the property that fails most often, and it usually fails at design time rather than in operation.

Decision, not just action. The record has to show which policy evaluated the action and what it decided. An entry saying an agent restarted a service proves the restart happened. It does not prove anything was governing the restart, which is the actual question being asked.

Refusals matter here more than people expect. An audit trail containing only permitted actions cannot distinguish a well-governed system from one where the policy never fired. The refusals are the evidence the control was live.

How long to keep it, and the tension

EU AI Act Article 26(6) requires deployers of high-risk systems to retain the automatically generated logs for at least six months. That is a floor, not a target, and the Act itself defers to other Union and national law where that demands longer.

A SOC 2 Type II observation period often exceeds it. Sector rules in financial services routinely do. So the practical retention figure is the longest applicable obligation rather than the six months most summaries quote.

Pulling the other way is GDPR storage limitation, which says personal data should not be kept longer than necessary for its purpose. Where an agent audit record contains personal data, and many do because the action payload includes it, these two obligations are in genuine tension. Article 26(6) anticipates this by explicitly deferring to data protection law.

The usual resolution is to separate the decision record from the payload. Retain the verdict, the policy, the identity, the timestamp and the signature for the full compliance period, and minimise or redact the personal data inside the action arguments. That keeps the property an auditor is testing, which is that the control operated, without keeping data you have no lawful basis to hold.

Producing it on demand

The last requirement is unglamorous and defeats more teams than the others combined: you have to be able to actually retrieve it.

Evidence you can only assemble by exporting from three systems and joining them by hand is evidence you will not produce inside an audit window. The queries that have to be cheap are narrow: every action in a time range, every action by one agent identity, every refusal, and every action that touched one service.

Test this before an auditor does. Pick a random two-hour window from four months ago and try to produce the complete, signed set for it. Whatever breaks in that exercise is what would have broken under examination, and it is almost always retention that quietly expired or attribution that was never captured.

NOFire AI produces this record as a byproduct of enforcement rather than as a separate logging pipeline, signing each verdict at the point the action is permitted or refused. That is a design choice with an audit consequence: because the evidence is generated by the control itself, a missing record and a missing enforcement are the same event rather than two things that can drift apart.

Common failures

Logging the action and not the decision. The most common gap. It proves what happened and nothing about whether it was governed.

Only recording permitted actions. Without refusals there is no evidence the policy was live rather than passive.

Shared service accounts. Attribution fails at design time, and no amount of retention fixes it afterwards.

Retention set to the six-month floor. It is the minimum in one framework, not the answer across all of them.

Treating an audit as a document exercise. A written policy with no demonstrable operation is precisely what SOC 2 Type II is built to catch.