How to Conduct an AI Risk Assessment

A structured method to score an AI system’s autonomy, data access, impact and controls, then turn the result into authorisation, monitoring and human-approval decisions. Written for teams deploying models and autonomous agents.

A model that writes a wrong sentence is a content problem. An agent that reads a customer record, calls an external API and moves money is an action problem. The second kind can cause harm before a human sees the output, which is why the review checklist that governed traditional software no longer covers what these systems can do.

An AI risk assessment is a structured evaluation of an AI system’s purpose, autonomy, data access, permissions, potential impact and failure modes. It assigns a risk level, identifies the controls the system requires, and records who owns the risk that remains. This guide gives you a repeatable process, a scoring matrix, a control mapping, two worked examples and a copyable template.

What Is an AI Risk Assessment?

An AI risk assessment evaluates what an AI system can do, what it can reach, what happens when it fails, and which controls reduce that exposure to a level the organisation will accept. It produces three outputs: a risk level, a required control set, and a named owner for the residual risk.

It is not the same as three things it is often confused with.

• An AI audit checks, after the fact, whether the controls that were promised are in place and working. The assessment determines the risks and the controls that should apply; audits then evaluate whether those controls are implemented and operating as intended.

• A security test such as red teaming or penetration testing probes for specific technical weaknesses. It feeds the assessment; it does not replace the wider judgement about impact, permissions and accountability.

• A governance framework is the operating model above every assessment. The NIST AI Risk Management Framework and ISO/IEC 42001 set the broader approach to managing AI risk; an assessment applies those principles to a specific system. ISO/IEC 23894 covers AI risk management and ISO/IEC 42005 covers AI system impact assessment.

For agents, add a second question. A model assessment asks what the system might output. An agent assessment adds what the system can do with that output. When an AI system can call tools, hold credentials, or take multi-step actions, assess the actions as well as the model, because the action is where irreversible harm occurs.

When Should You Perform an AI Risk Assessment?

Assess before an AI system reaches production, and again whenever something material changes. Risk is not a one-time gate; it moves as the system’s tools, data, autonomy and user base change. A fixed review cadence catches the drift that event-based triggers miss.

• Before procurement or deployment, as the gate that decides whether the system ships at all.

• When models, tools, permissions or data sources change, because each change can alter what the system can reach.

• When the agent’s autonomy or user population expands, since wider reach raises impact.

• After an incident, a policy violation or material behavioural drift.

• On a recurring schedule set by risk tier, with higher-risk systems reviewed more often.

OpenBox documents similar re-assessment triggers for its Assess phase: a change in agent capabilities such as new data sources or APIs, a shift to a more critical business role, changed compliance requirements, and significant incidents (OpenBox, docs.openbox.ai). The pattern is consistent across sources: reassess on capability change, on context change, and after failure.

The AI Risk Assessment Process

The process moves from context to decision in nine steps, and each step produces evidence the next step depends on.

1. Define the use case and the accountable owner. Name the business purpose and the single person who owns the outcome and the residual risk. An assessment with no named owner has no one to accept what it finds.

2. Map models, data, tools, APIs, memory and downstream systems. List every model and provider, every data category the system reads or writes, every tool and integration it can call, and every system its actions reach. You cannot assess reach you have not written down.

3. Classify autonomy and decision authority. State whether the system suggests, acts with approval, or acts on its own, and for which operations. Autonomy is the multiplier on every other risk.

4. Identify affected people, business processes and regulated outcomes. Record who is affected when the system is right and when it is wrong, and whether any output touches a regulated decision such as hiring or credit.

5. Identify security, privacy, safety, operational and compliance risks. For each capability, write the specific ways it can fail or be abused, not a generic list of AI harms.

6. Score likelihood and impact. Rate how likely each risk is and how severe the impact would be, using the matrix below.

7. Select preventive, detective and corrective controls. Choose controls that stop the harm, controls that surface it, and controls that recover from it. Map each control to the risk it addresses.

8. Approve, restrict, redesign or reject the deployment. Decide explicitly: ship as designed, ship with tighter controls, send back for redesign, or do not deploy. Record the decision and its basis.

9. Record residual risk and reassessment triggers. State the risk that remains after controls, name who accepts it, and list the events and the date that will trigger the next assessment.

Agent-Specific Risk Factors

When a system can act rather than only generate text, failure becomes physical: money moves, records change, messages send. Agent risk therefore centres on what the agent can reach and how hard its actions are to reverse, not only on what it might say.

The OWASP Gen AI Security Project runs an Agentic Security Initiative for exactly these systems, and its Top 10 for Agentic Applications 2026 catalogues the most critical risks facing autonomous and agentic AI (OWASP). The factors below draw on that class of risk.

• Tool and API access: every tool is a new capability and a new attack surface.

• The ability to spend money, send messages, alter records or execute code: rank these by how hard each is to undo.

• Identity, credentials and delegated permissions: an over-privileged agent inherits every door its credentials can open.

• Memory, retrieval and data exfiltration: what the agent stores and retrieves can leak through its later actions.

• Multi-step and multi-agent behaviour: a chain of individually safe steps can combine into an unsafe outcome.

• Goal drift and reward hacking: an agent optimising a proxy can follow the letter of its instruction and miss the intent.

• Reversibility and blast radius: assess how far a single wrong action spreads and whether it can be rolled back.

• Human-approval latency and bypass paths: a slow approval gate invites workarounds, and a bypassed gate is not a control.

• Third-party skills, MCP servers and supply-chain dependencies: code and connectors you did not write extend the agent’s reach and its risk.

An AI Risk-Scoring Matrix You Can Calibrate

Score each risk on a 1 to 5 likelihood scale and a 1 to 5 impact scale, multiply for a base score of 1 to 25, then raise the tier for agent-specific modifiers. The matrix below is a starting point. Every organisation must calibrate the bands and modifiers to its own risk appetite.

Base risk score = likelihood × impact (1 to 25). Illustrative starting bands: Low 1 to 6, Moderate 7 to 12, High 13 to 19, Critical 20 to 25.

Then apply modifiers that raise the tier by one for each that is true: high autonomy (the system acts without a human), broad or standing privilege, access to sensitive or regulated data, and low reversibility (actions are hard to undo). Tier increases are capped at Critical.

This is a general method, not an industry constant. No single scoring formula fits every organisation, and a proprietary or vendor score should not be treated as a universal standard. Calibrate the bands, weights and modifiers with your risk, security and legal teams before you rely on the result.

From Risk Tier to Controls

The point of a score is the control decision it forces. Map each inherent-risk tier to a fixed baseline: how much autonomy is allowed, what authorisation is required, how often the system is monitored, when a human must approve, what evidence is kept, how incidents are handled, and how often the system is reassessed.

Tiers here are inherent-risk bands, not vendor trust tiers. Calibrate the thresholds to your organisation before use.

Worked Example 1: Internal Report-Generation Agent

An internal agent that reads a data warehouse and drafts a weekly report is low-to-moderate risk: read-only data, no external recipients, and output a human reviews before use. The residual risk is a plausible but wrong report, so the controls target data grounding and review, not blast radius.

Baseline controls: read-only credentials scoped to the reporting tables; a rule that the agent must query real data before it writes any file, so it cannot fabricate figures from the model alone; and a human review step before the report is circulated. OpenBox documents a Behavioral Rule of exactly this shape, where a file write is only permitted if a database query occurred first, and otherwise the session is halted (OpenBox, docs.openbox.ai).

Residual risk: an accurate-looking report built on stale or partial data. The named owner accepts it, with a monitoring control that samples outputs and a reassessment trigger if the agent gains write access or an external recipient.

Worked Example 2: Customer-Data or Transaction Agent

An agent that reads customer records and initiates payments could fall into a high-to-critical risk tier: sensitive data, write access and irreversible money movement. The controls must enforce least privilege, check each operation, gate high-value actions for human approval, and stop the session on a threat pattern.

Baseline controls: least-privilege credentials for each system; a per-operation policy check that inspects the specific tool and its arguments; an approval gate for payments above a threshold; and a halt condition for a multi-step threat pattern such as reading data and then attempting an external send without approval. In OpenBox, a stateless OPA/Rego policy can require approval for an invoice at or above a set amount, evaluated on each operation (OpenBox, docs.openbox.ai).

Evidence: each session is recorded as a tamper-evident, signed log so an auditor can later confirm what the agent did and under which decision. Residual risk: a wrongful payment slips through before the approval fires. The owner accepts it, with controls on approval latency and a reassessment trigger on any new payment method or raised limit.

How the Assessment Connects to Runtime Governance

An assessment only reduces risk if its findings become enforced controls. That means translating each finding into a guardrail, a policy or a behavioural rule, then letting the runtime return a governance decision on every operation and feeding what it observes into the next assessment.

OpenBox structures this as a Trust Lifecycle of five phases: Assess, Authorize, Monitor, Verify and Adapt (OpenBox, docs.openbox.ai). The Assess phase scores an agent’s inherent risk across three weighted categories: Base Security at 25 percent, AI-Specific at 45 percent, and Impact at 30 percent, producing a Risk Profile Score.

The Authorize phase turns findings into two distinct control types. Policies are stateless checks written in OPA/Rego that evaluate a single operation and return a decision. Behavioral Rules are stateful checks that track prior actions to detect multi-step patterns across a session (OpenBox, docs.openbox.ai). Keeping the two apart matters: one reasons about a single request, the other about a sequence.

At runtime, OpenBox returns one of four governance decisions. ALLOW lets the operation proceed. REQUIRE_APPROVAL pauses it for a human reviewer. BLOCK rejects the operation while the session continues. HALT terminates the entire session. Precedence runs HALT then BLOCK then REQUIRE_APPROVAL then ALLOW, so the strictest applicable decision wins (OpenBox, docs.openbox.ai).

For evidence, OpenBox hashes each session event with SHA-256, combines the hashes into a Merkle tree, and signs the root with ECDSA NIST P-256 through AWS KMS by default, producing a tamper-evident proof certificate per session (OpenBox, docs.openbox.ai). Monitoring and reassessment then close the loop: observed violations and drift lower the agent’s trust and can trigger a fresh assessment.

AI Risk Assessment Template

Copy the fields below into your assessment record. Fill every field before the authorisation decision; a blank field is an unassessed risk. Keep the completed record as the evidence that the assessment shaped the controls the system actually runs under.

Frequently Asked Questions

What is the difference between an AI risk assessment and an AI impact assessment?

An AI risk assessment scores the likelihood and severity of a system failing or being abused, and selects controls. An AI impact assessment focuses on how an AI system and its foreseeable applications may affect individuals, groups or society, including human-rights and societal impacts. They overlap: many higher-risk systems need both, and an impact assessment often feeds the wider risk assessment.

How often should an AI risk assessment be updated?

Update it before deployment, on any material change to models, tools, permissions or data, when autonomy or the user base expands, and after incidents or drift. Beyond event triggers, set a recurring cadence by risk tier, with higher-risk systems reviewed more frequently. Treat the last review date as a freshness signal, not a formality.

Who should approve an AI risk assessment?

The named accountable owner approves the residual risk, usually with sign-off from security, and from legal or compliance where regulated outcomes are involved. The person who builds the system should not be the only person who signs off on its risk. Record who approved, what they accepted, and when.

How do you assess the risk of an autonomous AI agent?

Assess the model and the actions separately. Map the agent’s tools, credentials and reachable systems, classify how much it can do without a human, and rank its possible actions by reversibility and blast radius. An agent that can move money or change records needs action-level controls, not only output filtering.

Can the NIST AI RMF be used as an assessment template?

The NIST AI Risk Management Framework, released in 2023 for voluntary use, provides four functions, GOVERN, MAP, MEASURE and MANAGE, that structure how you manage AI risk. It is a reference model, not a filled-in template. Use it to organise your process, then add the system-specific scoring and controls it deliberately leaves to you.

What evidence should be retained?

Retain the completed assessment, the scoring and its rationale, the control decisions, the named risk owner, and the approval record. In operation, retain the runtime decisions and a tamper-evident log of what the system actually did, so an auditor can confirm the assessment shaped the deployed controls. Set retention by policy and risk tier.

Turn the Assessment Into Enforced Controls

An AI risk assessment should be run by a named owner with security and compliance, should score autonomy, data access, impact and controls, and should output the authorisation, monitoring and human-approval rules the system runs under. Do that before deployment and on every material change, and the assessment stops being a document and becomes the control plane.

OpenBox is built to turn an assessment result into enforced controls: guardrails, OPA/Rego policies and Behavioral Rules, four runtime governance decisions, and tamper-evident evidence for each session.

To connect an assessment to runtime governance, start with the OpenBox Assess phase and its authorisation policies, and read how OpenBox produces verifiable governance evidence. For the wider operating model, see the guide to AI agent governance.

Last reviewed: 11 August 2026. Technically reviewed by Tahir Mahmood, CTO, OpenBox.