AI Governance for the Enterprise: A Practical Framework
Enterprise AI governance is the set of engineering controls that let an organisation run AI systems, and especially autonomous agents, with the rigour it applies to any production software: identity per agent, autonomy granted in tiers, scoped credentials, a complete audit trail, continuous evaluation, and machine-enforced policy.
Done well, governance is not a document beside the system, it is wiring inside it. The distinction is testable. A governance programme that lives in a policy PDF cannot answer which agent performed a given write, what it was permitted to do at that moment, who granted that permission, what data it read, or how fast it can be contained. An operational programme answers all five from artefacts the system produces on its own: an inventory that records each agent's tools, identity, and autonomy classification; an append-only ledger that records the principal chain and the policy verdict for every action; and a stop mechanism that works from outside the agent process. Those artefacts are also what a certification audit tests, because auditors do not ask whether a control exists, they ask for the mechanism that enforces it and a sample of evidence that it ran. Build the practice first and the policy layer describes something real.
What does AI governance actually control?
Governance answers questions about a running system, and each question maps to a mechanism rather than a policy statement.
- Identity per agent. Each agent authenticates as itself, ideally through platform attestation rather than a stored secret, and its identity string is versioned so a prompt or model change produces a new principal. Actions tie back to a specific agent build, not to a shared service account.
- Tiered autonomy. Actions are classified by reversibility and blast radius. Read-only work runs freely, reversible internal writes run under sampled review, and irreversible or high-stakes actions require a named human to approve the exact parameters. The tier is resolved in a tool gateway outside the model, because a limit stated in a system prompt is a suggestion.
- Scoped credentials. An agent receives the narrowest permission its current task requires, issued just in time and short-lived. For gated actions the gateway holds the credential and the agent process never does, so an agent that decides to call a blocked tool has nothing to call it with.
- Audit and observability. Triggering input, retrieved context, tool calls, policy decisions, approvals, and results are captured in a tamper-evident trail, with denied calls recorded alongside successful ones.
- Evaluation and policy as code. Behaviour is measured against golden and adversarial suites on every change to prompts, tools, retrieval, or model selection, and policy is evaluated by a decision point at call time rather than remembered by people.
These layers reinforce each other. Identity is what makes audit meaningful, scoped credentials cap the damage a misclassified tier would otherwise allow, and evaluation catches the drift policy cannot foresee.
When does AI governance start to matter?
Governance matters the moment an AI system can do something rather than merely say something. A chatbot that answers questions carries reputational risk. An agent that can query a database, send a message, change a configuration, or move money carries operational risk, and the blast radius of a mistake is bounded by what its tools can reach rather than by what it says.
That is why the tool list is the impact surface. An impact assessment for an agent has to cover action risk, not just output risk: enumerate every tool, state the worst plausible outcome of a bad invocation, classify how reversible it is and at what cost, identify who is affected, and record the mitigation per tool. That per-tool risk register does double duty, satisfying the assessment requirement and serving as the direct input to autonomy tier design. Prompt injection deserves explicit treatment in it, because an agent that reads untrusted content and holds tool access has a documented path from attacker text to attacker-influenced action.
Approval is the other forcing function. Security, legal, and risk functions do not wave production deployments through on trust, and the shift toward AI agents and governed AI agents makes the evidence bar concrete. Teams that treat governance as engineering ship faster in the end, because they arrive at the review with controls built rather than promised. The preparation is set out in how to get AI agents through enterprise security review, and the sequencing in a phased governance rollout for AI agents.
How do the frameworks map onto real controls?
Two frameworks dominate. ISO/IEC 42001, published in December 2023, defines an AI management system on the same pattern as ISO 27001, with Annex A controls organised under nine control objectives and a plan-do-check-act cycle. It is certifiable, which is why procurement teams increasingly ask for it. NIST's AI Risk Management Framework 1.0, published in January 2023, is voluntary and organised around four functions, Govern, Map, Measure, and Manage, extended for generative systems by the Generative AI Profile (NIST AI 600-1) in July 2024. Neither was written for systems that select and execute actions at runtime, which is where teams get stuck. The mapping below is an engineering translation, not audit advice.
| Control area | Framework hook | The artefact that satisfies it |
|---|---|---|
| Accountability and inventory | 42001 leadership and internal organisation; RMF Govern | Agent registry: purpose, owner, model versions, every tool, identity, autonomy tier |
| Risk and impact assessment | 42001 clause 6 and impact assessment; RMF Map | Per-tool risk register with reversibility and mitigation per tool |
| Lifecycle and change control | 42001 Annex A lifecycle; RMF Manage | Version-controlled prompts and tool grants, pinned model identifiers, eval gate in CI |
| Measurement and monitoring | 42001 clause 9; RMF Measure | Task success, groundedness, guardrail trigger rates, approval and override statistics |
| Logging and records | 42001 documented information; RMF Govern and Manage | Append-only session ledger with principal chain, policy verdicts, and approvals |
| Third parties and data | 42001 Annex A third-party and data objectives; RMF Map and Govern | Provider inventory with data-crossing and retention terms, classification-aware retrieval |
The approval statistics are underrated evidence. A record showing that humans reviewed every gated action and rejected some of them demonstrates the control operates, which is exactly what a certification audit tests. The full walkthrough is in mapping AI agent deployments to ISO/IEC 42001 and NIST AI RMF.
What are the hard problems in AI governance?
Prompt injection turns a trusted agent into a confused deputy, which is why credential scoping and human gates matter as much as input filtering: a probabilistic defence at the model layer needs a deterministic backstop at the identity layer. Non-human identities proliferate faster than quarterly access reviews catch them, so discovery, ownership attribution, rotation, and a joiner-mover-leaver lifecycle have to apply to machines the way they apply to people. Audit logs are themselves sensitive and need protection from tampering and from over-retention of payloads. Review load is a control problem in its own right, because an approval humans click through without reading manufactures false evidence. And governance must not calcify the system: friction that pushes teams toward ungoverned shortcuts is a governance failure, not a governance success.
Containment deserves a named design. A single full-shutdown switch is one nobody pulls in time, so build graded levels: pause intake, freeze writes while reads keep helping responders, revoke credentials at the identity provider so enforcement happens outside the agent runtime, and only then scale to zero. Every trigger is itself a ledger event, and the drill is rehearsed rather than assumed. The detail sits in audit trails for AI agents, alongside the identity architecture in workload identity for AI agents and the wider posture on cloud security, security, and zero trust for AI.
How Citadel helps
Citadel Cloud Management builds AI systems that are governable from the first commit. We design governed AI agents with per-agent identity, tiered autonomy, scoped credentials, and audit built in, and we work with your existing controls rather than around them so governance is native to the architecture instead of bolted on before launch. The identity plane, credential broker, tool gateway, and session ledger are platform services every agent consumes.
Start with an assessment
The Agent Assurance Assessment is a fixed-scope, fixed-price three-week engagement: an inventory and risk map of the agents already running in your environment, an evidence pack mapped to NIST AI RMF and ISO/IEC 42001, one governed agent live on a real workflow, and a prioritised rollout plan. The Enterprise AI Agent Blueprint is a free governance guide covering the same control set in written form. To discuss your environment, book a discovery call or see the enterprise consulting overview.