A Practical AI Governance Framework for Enterprise Teams

A control-by-control framework for governing AI agents in any industry: risk tiers, ownership, runtime enforcement, evidence, and a 30/60/90-day rollout. This is general guidance, not legal advice.

Most AI governance frameworks were written for models that advise. AI agents act. They call tools, move money, write to production systems, and chain dozens of steps without pausing for a human. A policy that assumed a person would read every model output before anything happened does not survive contact with an agent that books the flight, sends the email, and updates the record on its own.

This is a practical AI governance framework for enterprise teams, built for agents and usable across any industry. It covers the eight controls that matter, who owns each one, how to tier risk, how to enforce policy at runtime, and how to prove afterwards that governance actually ran. Everything here is cross-industry. None of it is legal advice.

What is an AI governance framework?

An AI governance framework is the set of policies, roles, controls, and evidence practices an organisation uses to decide what its AI systems may do, enforce those decisions while the system runs, and prove afterwards that the decisions held. For AI agents, it must govern actions, not just outputs.

A framework is not a mission statement about responsible AI. It is closer to an operating system: an inventory of what you run, a risk rating for each system, named owners, machine-enforceable policies, a way to stop an action in flight, and an audit trail that survives scrutiny. The test of a framework is not whether it reads well. It is whether it changes what an agent is allowed to do on a Tuesday afternoon in production.

Why older AI governance frameworks fall short for agents

Older frameworks assume an AI system produces an output that a human then reviews. Agents break that assumption. They hold delegated permissions, call external tools, keep memory across steps, take multi-step actions, and increasingly talk to other agents. Governance that inspects a final output governs almost none of this.

Six properties of agents defeat output-centric governance:

• Autonomy. An agent chooses its own next step. There is no single output to gate, only a sequence of decisions, any one of which can cause harm.

• Delegated permissions. An agent acts with credentials a human handed it. If it can read a database or call a payment API, so can anyone who redirects it.

• Tool access. The risk is not the text the model writes. It is the function it invokes: a delete, a transfer, an external request.

• Memory. State persists across steps and sessions. Poisoned memory changes behaviour long after the input that planted it.

• Multi-step action. Each step can look benign. The pattern across ten steps is the attack, and a single-operation check cannot see it.

• Agent-to-agent communication. One agent delegates to another, and authority and accountability blur across the handoff.

NIST pointed in the same direction when its Center for AI Standards and Innovation launched the AI Agent Standards Initiative on 17 February 2026, with a research pillar dedicated to agent authentication and identity infrastructure. The reasonable read is that identity and authorisation are becoming first-order requirements for agents, not afterthoughts. A framework built for advisory models has nowhere to put any of this.

The eight core components of an agent-ready framework

An agent-ready AI governance framework has eight components: inventory and ownership, risk classification, agent identity and authorisation, policy definition and approval, pre-deployment testing, runtime enforcement with human approvals, monitoring with incident response, and reversibility with audit evidence. The first four establish control before deployment; the last four hold it during and after.

The first four components are governance you set up once and revisit: an accurate inventory, a defensible risk rating, verifiable identity, and approved policy. The last four are governance that runs continuously: testing before release, enforcement during execution, monitoring for the unexpected, and evidence you can hand to an auditor. A framework missing any of the last four looks complete on paper and fails in production.

Roles and responsibilities: a RACI you can adapt

AI agent governance fails when everyone assumes someone else owns it. A workable split makes engineering responsible for building controls, security accountable for how those controls are designed, and legal, compliance, risk, and the business owner consulted or informed at defined points. The table below is a starting RACI you can adapt: R is responsible, A is accountable, C is consulted, I is informed.

One rule keeps a RACI honest: every high-risk activity has exactly one accountable owner, never two and never none. Where this table shows a shared cell, split it in your own version so a single named person carries the decision.

A risk-tiering model for agent activities

Not every agent needs the same controls. Tier agent activities by the blast radius of their actions: low risk covers read-only, reversible work; medium risk covers writes to non-critical systems that are reversible with effort; high risk covers actions that are irreversible, financial, or safety-relevant. Each tier inherits the controls below it and adds its own.

Tiering does two useful things. It stops teams from wrapping a low-risk drafting agent in heavyweight controls nobody maintains, and it forces the high-risk cases to earn their autonomy. An agent that can move money starts at the strictest tier and only relaxes as it builds a track record.

A control matrix: preventive, detective, corrective

Map every agent risk to three kinds of control: preventive controls stop it happening, detective controls notice it happening, and corrective controls undo or contain it. A framework with only preventive controls has no answer for the failure that slips through, and no evidence for the auditor who asks what happened next.

The irreversible-action row carries the sharpest lesson. Some agent actions cannot be undone, so the corrective column is empty on purpose. For those actions, the preventive control, a human approving before execution, is the only control that exists, and the framework must make that approval non-optional.

The agent-specific governance layer

The agent-specific layer supplies what a general governance policy cannot: cryptographic identity for each agent, least-privilege tool scoping, a runtime decision on every operation, a kill switch, rollback, and signed evidence of what happened. This is where governance stops being a document and becomes enforcement. OpenBox is one platform built around this layer, and its mechanics make the abstract controls concrete.

Identity you can verify

Give each agent a verifiable identity separate from a shared API key. In the OpenBox model, each agent receives a Decentralized Identifier of the form did:aip:<uuidv5> and an Ed25519 signing key that you hold, while OpenBox keeps only the public half to verify signed requests. The API key authenticates the call; the signing key proves which agent produced the payload, so a leaked API key alone does not let an attacker act as the agent.

A decision on every operation

Every agent operation should resolve to one runtime verdict. When an operation is evaluated, OpenBox returns one of four governance decisions, applied in the precedence order HALT, then BLOCK, then REQUIRE_APPROVAL, then ALLOW. ALLOW lets the operation proceed. REQUIRE_APPROVAL pauses it and routes it to a human reviewer. BLOCK denies that single operation while the session continues. HALT terminates the entire agent session. If any policy returns HALT, the session ends regardless of the other decisions.

Least privilege and pattern detection

Two mechanisms decide those verdicts, and they are not the same thing. Policies are stateless checks, authored in OPA/Rego, that evaluate a single operation against allowed patterns and permissions. Behavioral Rules are stateful and watch multi-step sequences, escalating to BLOCK, REQUIRE_APPROVAL, or HALT when a threat pattern emerges across steps. Keeping the per-operation check and the sequence detector distinct is what lets you catch both a single forbidden call and a slow, multi-step attack.

Guardrails, kill switch, and trust scoring

Guardrails run as a pre- and post-processing pipeline on every operation, catching personal data, harmful content, toxicity, and banned words before they reach a tool or a user. The kill switch is the HALT decision: when a critical pattern appears, the session stops immediately. To decide how much autonomy each agent earns, OpenBox computes a Trust Score from 0 to 100 as (Risk Profile Score × 40%) + (Behavioral × 35%) + (Alignment × 25%), mapping to five tiers from Tier 1 (Trusted, 90 to 100) down to Untrusted (0 to 24). Lower-trust agents receive stricter decisions.

Evidence that survives an audit

Reversibility and accountability depend on evidence nobody can quietly rewrite. OpenBox hashes each governance event with SHA-256, combines those hashes into a Merkle tree using sorted-pair hashing, and signs the tree’s root, by default with ECDSA NIST P-256 through AWS KMS, producing one proof certificate per session that records the Merkle root, the signature, and the event count. The result is a tamper-evident record: an auditor can verify that the event timeline was not altered after the fact. OpenBox arranges all of this as a five-phase Trust Lifecycle: Assess, Authorize, Monitor, Verify, and Adapt.

Mapping the framework to standards and regulation

Several references now shape agent governance: the EU AI Act’s Article 50 transparency duties, the NIST AI Risk Management Framework, NIST’s AI Agent Standards Initiative, the OWASP Top 10 for Agentic Applications, and ISO/IEC 42001. None mandates a specific product, and together they describe what good governance looks like. This section maps them to the eight components and is not legal advice.

EU AI Act Article 50: transparency is now live

The EU AI Act’s Article 50 transparency obligations became applicable across the EU on 2 August 2026. They are not limited to systems classified as high-risk; they attach to specific categories of AI systems that interact with people or generate content. Per European Commission guidance, Article 50(1) requires providers of systems that interact directly with people, which the Commission names as including chatbots, AI agents, and avatars, to ensure users are informed they are interacting with AI. Article 50(2) requires providers to mark AI-generated content in a machine-readable, detectable format. Article 50(3) requires deployers of emotion recognition and biometric categorisation systems to inform the people exposed to them. Article 50(4) requires deployers of generative AI systems to clearly label AI-generated or manipulated text published to inform the public on matters of public interest, unless the text has undergone human review or editorial control.

The European Commission sets one narrow grace period: only for the Article 50(2) marking obligation, only for systems placed on the market before 2 August 2026, and only until 2 December 2026. The AI Omnibus, which entered into force on 27 July 2026, moved the Act’s high-risk obligations for Annex III systems to 2 December 2027, but it did not change Article 50. Enforcement sits with national market surveillance authorities, and fines can reach 15 million euros or 3% of total worldwide annual turnover.

The reasonable read for a deployer running agents is that two framework components carry the load here. Runtime enforcement produces the disclosure at the first interaction, and audit evidence proves the disclosure happened. A transparency duty you cannot demonstrate is a duty you cannot defend.

NIST AI RMF and the AI Agent Standards Initiative

The NIST AI Risk Management Framework (AI RMF 1.0, NIST AI 100-1, published in 2023) organises risk work into four functions: Govern, Map, Measure, and Manage, with Govern cutting across the other three. Map is your inventory and risk classification, Measure is testing and monitoring, Manage is incident response and treatment, and Govern is ownership and policy. NIST’s AI Agent Standards Initiative, launched on 17 February 2026, adds an agent-specific direction of travel across three pillars: industry-led standards, community-led open protocols, and research into agent authentication and identity infrastructure, with an NCCoE concept paper applying identity and authorisation standards to enterprise agent use cases.

OWASP Top 10 for Agentic Applications and ISO/IEC 42001

The OWASP GenAI Security Project’s Agentic Security Initiative published the OWASP Top 10 for Agentic Applications for 2026, a peer-reviewed taxonomy of ten agent-specific risk categories labelled ASI01 to ASI10, covering systems that plan, use tools, persist memory, and coordinate with other agents. Use it to scope threat modelling and to populate the preventive column of your control matrix. ISO/IEC 42001:2023, published in December 2023, is the first certifiable international standard for an AI management system; it specifies requirements for establishing, implementing, maintaining, and continually improving that system. Where the NIST AI RMF tells you how to reason about risk, ISO/IEC 42001 gives you a management system you can be audited against.

Framework component to reference mapping (an interpretive mapping, not the standards’ own words):

A 30/60/90-day implementation plan

Sequence the rollout. In the first 30 days, inventory every agent and assign owners. In days 31 to 60, classify risk and design controls and policies. In days 61 to 90, turn on runtime enforcement, monitoring, and signed evidence, then measure. Inventory comes first and enforcement last, because you cannot govern what you have not found.

Days 1 to 30: discover and own

• List every AI agent in production and in development, including shadow deployments built outside the central team.

• Record what each agent can do, which tools and credentials it holds, and which systems it touches.

• Assign one accountable owner per agent, and stand up the RACI so no activity is ownerless.

Days 31 to 60: classify and design

• Assign each agent a risk tier from the model above, based on the blast radius of its actions.

• Draft policies per tier, decide where human approval is mandatory, and route them for sign-off.

• Design the control matrix for your top risks, and provision verifiable identity for high-tier agents.

Days 61 to 90: enforce and measure

• Switch on runtime enforcement so every operation resolves to a decision, with high-risk actions gated by approval.

• Enable monitoring and anomaly alerts, and rehearse the incident-response path end to end.

• Turn on signed evidence, then report coverage: agents inventoried, tiered, and under enforcement.

Putting the framework to work

An AI governance framework earns its keep when it changes agent behaviour in production, not when it is filed. Everything above is designed to be lifted directly: the eight components as a checklist, the RACI and risk tiers as starting tables, the control matrix as a threat-modelling grid, and the 30/60/90-day plan as a rollout you can begin this quarter. Adapt the tables to your own risk tolerance and industry; the structure holds across sectors.

The hardest part is not writing the policy. It is turning the policy into controls that run and evidence that stands up. That is the gap between a governance document and governed agents, and it is where a runtime platform such as OpenBox turns each requirement into an enforced decision and a signed record.

Frequently asked questions

What is an AI governance framework?

An AI governance framework is the set of policies, roles, controls, and evidence practices an organisation uses to decide what its AI systems may do, enforce those decisions at runtime, and prove afterwards that they held. For AI agents, it governs actions such as tool calls, not just model outputs.

How is agentic AI governance different from traditional AI governance?

Traditional AI governance reviews an output before a human acts on it. Agentic AI governance must govern the agent’s own actions: delegated permissions, tool calls, persistent memory, and multi-step sequences that unfold without a human in the loop. The unit of control shifts from the output to the operation.

What are the core AI governance controls for agents?

The core controls are agent inventory and ownership, risk classification, cryptographic identity with least-privilege authorisation, policy definition and approval, pre-deployment testing, runtime enforcement with human approvals, monitoring with incident response, and reversibility backed by signed audit evidence. Preventive, detective, and corrective controls should each be present.

Does the EU AI Act require transparency for AI agents?

The EU AI Act’s Article 50 transparency obligations apply from 2 August 2026 and, per European Commission guidance, cover systems that interact directly with people, including chatbots and AI agents. Providers must ensure users are informed they are interacting with AI. This is general information, not legal advice.

How do NIST and ISO/IEC 42001 fit an AI governance framework?

The NIST AI Risk Management Framework organises risk work into four functions, Govern, Map, Measure, and Manage, that map cleanly onto the framework’s components. ISO/IEC 42001:2023 adds a certifiable AI management system you can be audited against. One guides reasoning; the other provides an auditable structure.

Where should an enterprise start with an AI governance framework?

Start with inventory and ownership: you cannot govern agents you have not catalogued. In the first 30 days, list every agent and name an owner. Then classify risk and design controls, and only then switch on runtime enforcement, monitoring, and signed evidence. Inventory first, enforcement last.