workflows
AI Agent Risk Management: A Guide for Compliance, Risk, and IT Leaders
How compliance, risk, and IT leaders at payment companies, fintechs, and banks manage AI agent risk: agent inventory, guardrails, audit trails, risk scoring, evals, and controls mapped to US, UK, EU, and global frameworks like NIST AI RMF, SR 26-2, NYDFS, the FCA Consumer Duty, the EU AI Act, DORA, and ISO/IEC 42001.
AI agents now touch money and customer data. They investigate payouts, answer disputes, review merchant onboarding files, and prepare responses to partner banks. The work moved from a person clicking through a dashboard to software acting with that person's access.
The questions arriving from partner banks, examiners, auditors, and boards are getting specific: which agents do you run, what can they touch, who approved what, and where is the record. Regulators are asking too. In July 2026 the European Central Bank asked the banks it supervises for action plans on AI-enabled cyber threats, due by October 31, and since January 2026 insurers can attach standard generative AI exclusions to US general liability policies. You can't govern what you can't see.
This is a guide to managing AI agent risk on Runtime, written for Chief Compliance Officers, Chief Risk Officers, and CIOs and CISOs at payment companies, fintechs, and banks. It covers what data Runtime keeps, how risk is scored, how evals catch drift, how an audit works, and how the controls map to the frameworks you already report against.
What it does. Runtime gives every agent an identity and an owner, enforces your rules on every action, records every run, scores each run's risk, re-checks agents when they change, and exports the evidence when someone asks for it.
What it is not. It does not replace your risk appetite, your policies, or your judgment. Runtime enforces and records the rules your team writes. Every material improvement comes from a human questioning a premise. Plan for that.
How Runtime manages agent risk
payout-investigator
Owner: Payments ops
dispute-responder
Owner: Support
kyb-reviewer
Owner: Compliance
Every agent has an owner, its own identity, and scoped access.
Illustrative
Your team sets the rules and reviews the evidence. Runtime keeps the inventory, enforces the guardrails, keeps the record, and scores every run.
The data Runtime keeps
| Data | What's in it | Who uses it |
|---|---|---|
| Agent inventory | Each agent's identity, owner, scoped API keys, the list of who can call it, and the systems it can reach | CIO, CISO, model risk |
| Run record | The request, each step, every query and tool call, the result, and the cost | Compliance, internal audit, the agent's owner |
| Approvals | Who approved which action, and when, in the dashboard or Slack | Compliance, partner bank, auditors |
| Risk scores | A score and a level (Low, Medium, High) for every run, with what drove it | CRO, risk committee |
| Change history | Changes to each agent's model, skills, and tools, with who approved them | Model risk, CISO |
| Eval results | The re-checks run after each change, kept with the record | Model risk, internal audit |
| Guardrail settings | Org-wide rules, egress allowlists, command allow and deny lists, approval gates | CISO, compliance |
All of it is searchable and exportable, ready when compliance, an auditor, or your partner bank asks.
How risk is scored
Every run is scored from its own record by what it touched, and whether a person approved the sensitive steps.
| Factor | Raises the score | Lowers the score |
|---|---|---|
| Money movement | The run proposed or executed a payment, refund, or payout | Read-only access to payment data |
| Customer data | The run read or wrote personal or account data | Data masked before the prompt |
| Production | The run changed a production system or deployed code | Sandbox or test systems only |
| Regulatory decisions | The run prepared a filing, a KYB decision, or a sanctions disposition | Internal summaries only |
| Human approval | Sensitive steps with no named approver | A person approved each sensitive step |
Each run gets a numeric score and a level of Low, Medium, or High. Scores sit next to each agent in the dashboard, so the risk committee sees where exposure sits without reading individual runs. A High run is a prompt to look, not a verdict. Your team decides what each level means for review and escalation.
Evals that catch drift
An agent that behaved well last month can behave differently today because something underneath it changed. Runtime re-checks agents whenever one of three things changes.
- The model. A new model version, or a switch from a frontier model to an open-weight one.
- The skills. The runbooks an agent follows. Agents learn from resolved cases and propose updates to their own skills; a person approves each change before it goes live.
- The tools. A new system connection, a wider permission, or a changed API.
Evals run the agent against cases your team has already resolved and compare the answers. Results are kept with the record, so you can show what changed, when, who approved it, and how the agent performed before and after.
How an audit works
- The request arrives. A partner bank, auditor, examiner, or insurer asks about an agent, a period, or a type of action.
- Pull the runs. Filter the record by agent and date range.
- Export the evidence packet. Inventory, runs with every step, approvals, risk scores, and the control mapping, in one export.
- Sample the approvals. Pick runs at random and show who approved each sensitive action.
- Show the change history. Every change to the agent's model, skills, and tools, with the eval results that came with it.
The record is the same one your team uses every day, so preparing for an audit is a filter and an export.
Mapping to frameworks
Runtime maps its controls to the frameworks your auditors, partner bank, and regulators already ask about, and exports the mapping with the evidence. The mapping supports your own assessment; it does not certify compliance. For the detail in each region, see the guides for the US, the UK, and the EU.
| Region | Framework | What it asks | Where it lives in Runtime |
|---|---|---|---|
| Global | NIST AI RMF (Govern, Map, Measure, Manage) | Accountability, context, measurement, and action on AI risk | Agent owners and RBAC; agent inventory; per-run risk scores and evals; approval gates and tiered guardrails |
| Global | ISO/IEC 42001 | An AI management system with defined roles, risk assessment, and monitoring | Org-wide rules, agent inventory, risk scores, review history |
| Global | SOC 2, PCI DSS | Security controls; protection of card data | Runtime is SOC 2 audited; card numbers masked before any prompt or log |
| US | SR 26-2 (OCC 2026-13), which replaced SR 11-7 in April 2026 | Model risk management sized to materiality. Generative and agentic AI are outside its scope, so banks govern agents under their own risk practices | Evals after every change, eval history, per-run scores as evidence for that program |
| US | Interagency third-party risk guidance | Sponsor banks oversee the fintechs and vendors they rely on | Exportable record of every run, approval, and access grant |
| US | GLBA Safeguards Rule, NYDFS Part 500 | Protect customer data; access controls, logging, and risk assessments | Scoped identity per agent, masking, isolated sandboxes, full run record |
| US | State AI laws (California ADMT rules, Colorado) | Risk assessments, notices, and transparency for automated decisions | Run record and risk scores as input to your assessments |
| UK | FCA Consumer Duty, SM&CR | Good customer outcomes; at firms under SM&CR, a named senior manager accountable for each area | Agent owners, approvals tied to named people, run record |
| UK | PRA SS1/23, UK GDPR Art. 22 | Model risk management; safeguards for automated decisions | Evals and change history; human approval before decisions with legal effect |
| EU | EU AI Act (Art. 9, 12, 13, 14, 15) | Risk management, logs, transparency, human oversight, and robustness for high-risk systems | Risk scores, full run record, visible skills and access, approval gates, evals and isolated sandboxes |
| EU | DORA, GDPR | ICT risk management and third-party oversight; data protection | Run in your own cloud account, isolated sandboxes, register of agents and access, masking |
EU AI Act obligations apply to systems classified as high-risk, and state AI laws apply to specific decisions. Your legal team decides which of your agents fall in scope.
Questions your partner bank will ask
| Question | Where the answer lives |
|---|---|
| Which AI agents do you run, and who owns each one? | Agent inventory |
| What can each agent access? | Scoped API keys, RBAC, and the systems listed per agent |
| Can an agent move money on its own? | Approval gates: money movement waits for a named person |
| Who approved this action? | Approvals in the run record |
| How do you protect customer data? | Card numbers, SSNs, and secrets masked before any prompt or log; isolated sandboxes; Runtime never trains on your data |
| Where do the agents run? | In isolated sandboxes, wiped at session end; in your own AWS, GCP, or Azure account, or self-hosted |
| What happens when you change the model? | Evals re-check the agent; results kept with the record |
| Can you show us every run last quarter? | Filter by date and export |
Rolling it out
Step 1: Write your risk appetite
This step is human. Write down which actions agents may never take, which need a person's approval, which data they may read, and who owns each agent. Keep it to a page. Runtime enforces it; it cannot write it for you.
Step 2: Inventory your agents
List every agent acting for your company today, including the ones running on a teammate's laptop. Give each one its own identity, an owner, and scoped access. Agents inherit the access of the person who starts the session, so a wide login becomes a wide agent.
List every agent in our workspace with its owner, the systems it can reach, and the API keys it uses. Flag any agent with write access to payments or production, and any agent without a named owner.
Step 3: Tier guardrails by risk
Match controls to what each agent touches.
| Tier | Example agents | Guardrails |
|---|---|---|
| Read-only | Reporting, internal summaries | Scoped read access, masked data, egress allowlist |
| Customer-facing | Dispute and support replies | Above, plus approval before anything goes to a customer |
| Money or production | Payout releases, deploys | Above, plus approval on every action and a named approver |
Step 4: Set evals
Pick 20 to 50 cases your team has already resolved for each agent, including the hard ones. These become the eval set that runs whenever the model, skills, or tools change.
Build an eval set for payout-investigator from last quarter's closed payout cases. Include the ones where analysts overruled the agent. Run it against the current version and summarize where the answers differ.
Step 5: Review weekly
Look at High runs, changed agents, and eval results once a week with the agent owners. Tighten rules where reviewers disagreed with the agent, then widen what it handles where it held up.
A forward-deployed AI engineer helps your team write the policy, build the first agents, and set the guardrails. The security page covers the controls in detail, and what is an agent harness explains why they matter.
What stays human
| Decision | Why |
|---|---|
| Risk appetite and policy | Accountability sits with your officers, not with software |
| Approving money movement | A named person answers for every release |
| Production deploys | Changes to live systems need an owner |
| Regulatory decisions and filings | Judgment on SARs, KYB decisions, and attestations stays with qualified staff |
| Approving skill changes | Agents propose updates; a person decides what they learn |
| Framework scope | Your legal team decides which agents count as high-risk |
The point
Agent risk is manageable once you can see it. Give every agent an owner and a limit, keep the record, score what it touched, and re-check it when it changes. Then the answer to every question from your partner bank is already on file.
Frequently asked questions
What is AI agent risk management?
AI agent risk management is how a company keeps track of the AI agents acting on its behalf: which agents exist, what each one can reach, what it actually did, how risky each run was, and who approved the sensitive steps. It covers inventory, guardrails, record-keeping, risk scoring, ongoing testing, and audits.
How is an AI agent different from a model for risk purposes?
A model produces an answer. An agent takes actions: it queries systems, calls tools, writes to tickets, and can request money movement or a production change. Model validation still matters, but agent risk also depends on what the agent is allowed to touch and whether a person approved each sensitive action.
How does Runtime score the risk of an agent run?
Every run is scored from its own record by what it touched: money movement, customer data, production systems, or a regulatory decision, and whether a person approved the sensitive steps. Each run gets a numeric score and a level of Low, Medium, or High.
Which AI compliance frameworks does Runtime map to?
Runtime maps its controls to the frameworks financial companies report against in the US (NIST AI RMF, SR 26-2, the interagency third-party risk guidance, GLBA, NYDFS Part 500, and state AI laws), the UK (FCA Consumer Duty, SM&CR, PRA SS1/23, UK GDPR), the EU (EU AI Act, DORA, GDPR), and globally (ISO/IEC 42001, PCI DSS, SOC 2). The mapping is exportable for auditors, partner banks, and regulators. Runtime is SOC 2 audited; it does not certify your compliance with any framework.
What do we show a partner bank or auditor?
An evidence packet pulled from the record: the agent inventory with owners and access, every run for the period with its steps and approvals, risk scores, the history of skill, model, and tool changes with eval results, and the control mapping. Everything is searchable and exportable.
Where should we start?
Write down your risk appetite for agents first: which actions are off limits, which need a person, and who owns each agent. Then inventory your agents, tier their guardrails by risk, set evals, and review the record weekly. A forward-deployed AI engineer from Runtime helps with the first agents and guardrails.
See the risk of every agent you run
Bring one agent or one policy. A forward-deployed AI engineer sets up the inventory, guardrails, and risk scoring with your team.