Ask your engineering lead a simple question. Not "is the agent working?" - everyone can answer that. Ask instead: which credentials does the agent currently hold, and who approved each one?
In most companies, that question produces a pause. Then someone says they'll have to check. Then it turns out the agent inherited a service account created eighteen months ago for a data migration, because that account already had the write permissions the agent needed and nobody wanted to open a ticket with infrastructure.
That is not a hypothetical failure mode. That is the standard way agents get access in real companies working under real deadlines.
The interesting part isn't the incident. It's who it happened to
OpenAI recently had to overhaul its internal security posture after one of its own agents behaved in a way nobody planned, affecting a third party in the open-source ecosystem. The specific forensics matter less than the shape of the response. Permissions were tightened. Credentials were rotated. Monitoring was added. All of it after the fact.
Think about what that means. This is an organisation with more model expertise than anyone on earth. If the problem were model capability, they would have caught it. They didn't, because the problem wasn't the model.
The problem was the layer underneath. Access that was broader than the task required. Credentials that outlived their purpose. And no clean way to reconstruct what the agent did, in what order, and on whose authority.
We've been arguing this for a while: stabilise first, improve second, add AI last. It's more convincing when someone else demonstrates it. Put an agent on top of a base you don't fully understand, and you pay twice - once for the agent, once to rebuild the base underneath it.
If you've already shipped, that ordering isn't a scolding. It's a to-do list.
Agents fail quietly. That's the whole problem
Here's the mental model that changes how you think about this.
A broken integration is loud. It throws a 500, the retry queue backs up, someone gets paged, you fix it. The failure announces itself.
An agent with too much access doesn't fail. It succeeds. It successfully sends an email to a segment you didn't intend. It successfully updates 4,000 records instead of 40. It successfully closes a compliance ticket that a human should have reviewed. Every log line says 200 OK. Every metric looks healthy. The system is doing exactly what it was permitted to do.
You find out weeks later, from a customer.
This is why the usual monitoring stack is close to useless here. Error rates, latency percentiles, uptime - none of them detect an authorised action taken for the wrong reason. Your dashboards measure whether the machine is working. They don't measure whether it was right.
And the teams most exposed are the ones furthest along. If you have agents running onboarding flows, compliance review, or hiring pipelines, you've handed a probabilistic system the keys to processes where being confidently wrong is expensive. Those are exactly the areas where a quiet failure survives for a full quarter before anyone notices.
The three things nobody audited
Almost every agent incident traces back to one of these. Usually all three.
Permissions inherited rather than granted. Agents get built fast, often by one engineer who needed access to move. The path of least resistance is an existing service account with generous scopes. Nobody documents this because nobody made a decision worth documenting. They reused a key.
Credentials with no expiry and no owner. Human access has a natural lifecycle. People join, get onboarded, leave, get offboarded. Machine access has no such rhythm. A token issued for an agent prototype in March is still valid in December, still has write access, and now belongs to nobody.
No traceability. This is the one that turns an incident into a crisis. When something goes wrong, you need to answer three questions fast: what did it do, what input made it decide that, and what would have stopped it. If your logs only contain the outcome and not the reasoning chain and the tool calls, you cannot answer any of them. You are guessing in front of a board.
Notice that none of these are AI problems. They're access management problems that AI made urgent. Agents just run thousands of operations per hour without getting tired or asking a colleague whether this seems right.
The contrarian bit: "human in the loop" is not a security control
The standard advice is to keep a human approving the agent's actions. It sounds responsible. In production, it decays within a month.
A human reviewing 400 agent decisions a day is not reviewing anything. They are clicking approve. That's not a character flaw, it's how attention works. The approval step becomes theatre, and worse, it creates a false record - every action now carries a human name next to it, which makes the audit trail actively misleading.
Human review works when it's rare and consequential. Ten decisions a day, each one meaningfully different. It doesn't work as a blanket safety net.
The control that actually holds is architectural. Constrain what the agent can do, so that the cases where it does the wrong thing are survivable without anyone catching them in time. Scope beats supervision.
What to build instead
Give the agent capabilities, not credentials
The agent should never hold a long-lived key. It should ask a broker for permission to perform a specific action, and receive something narrow and short-lived in return.
// The agent never sees the database URL or the API key. // It asks for a capability, scoped to one task, that expires. type Capability = { action: "email.send" | "record.update" | "ticket.close"; scope: { resource: string; maxItems: number }; taskId: string; // ties every use back to one unit of work expiresAt: number; // minutes, not months }; async function requestCapability( agentId: string, action: Capability["action"], scope: Capability["scope"], taskId: string ): Promise<Capability> { const policy = await policyFor(agentId, action); if (!policy.allowed) throw new PolicyDenied(agentId, action); if (scope.maxItems > policy.maxItems) { throw new PolicyDenied(agentId, `${action}: batch too large`); } return issue({ action, scope, taskId, expiresAt: Date.now() + 5 * 60_000 }); }
Two things fall out of this design for free. Blast radius is capped by maxItems, so the "40 records became 4,000" failure becomes impossible rather than unlikely. And every action carries a taskId, which is what makes the next section work.
Make the tool registry deny by default
Most agent frameworks let you register tools and pass them all to the model. That means adding a tool to help with one workflow silently grants it across every workflow. The permission surface grows every sprint and nobody reviews it.
Invert it. Tools are declared with the scopes they need, and each agent role gets an explicit allowlist.
const TOOLS = { "crm.readContact": { scopes: ["crm:read"], reversible: true }, "crm.updateContact": { scopes: ["crm:write"], reversible: false }, "email.sendBulk": { scopes: ["email:send"], reversible: false }, } as const; const ROLE_ALLOWLIST: Record<string, (keyof typeof TOOLS)[]> = { "onboarding-agent": ["crm.readContact", "crm.updateContact"], "support-triage": ["crm.readContact"], }; // Irreversible tools require an explicit, per-role opt-in. // Adding a tool is a code change, reviewed like any other.
That reversible flag matters more than it looks. Sort your agent's actions into those you can undo and those you can't. Reads and drafts are cheap to get wrong. Sending money, sending email to customers, and deleting anything are not. Different classes deserve different controls, and treating them the same is how teams end up with a review process that is simultaneously too heavy for reads and too light for payments.
Log the decision, not just the result
Application logs record what happened. For agents you need what was considered.
{ "task_id": "onb_9f2c", "agent": "onboarding-agent", "step": 4, "model": "gpt-x-2026-07", "input_digest": "sha256:8c1a…", "retrieved_context": ["doc:policy_uae_v3", "record:emp_2291"], "tool_call": { "name": "crm.updateContact", "args_digest": "sha256:44b9…" }, "capability": { "action": "record.update", "maxItems": 1, "expires_at": 1786…}, "outcome": "applied", "reverted_by": null }
Append-only. Retained. Queryable by task_id. The test of whether your logging is adequate is simple: can you reconstruct one full agent task, end to end, from the log alone, six weeks later? If the answer involves opening the model provider's dashboard and reading raw prompts, you don't have an audit trail. You have breadcrumbs.
This is the practical form of the argument we made in Your AI Agent Didn't Read the Handbook. Policy written in a Notion page is a wish. Policy written as a capability check is a control.
Assume the context window is an attack surface
An agent that reads from your codebase, your ticketing system, or your inbox is executing instructions written by people who don't know they're writing instructions. A support ticket can contain text that reads to the model like a command. So can a README. So can a commit message.
You cannot fully solve this with prompt engineering. You limit the damage by making sure the agent's permissions don't extend past what the task needs, which is the same discipline as everything above. We covered the read-access side of this in Meta's Muse Code Can Read Your Whole Codebase, and the logic holds for any tool with broad reach into systems you didn't sanitise.
The test that costs you nothing
Before you commission anything, run this. It takes an afternoon.
List every credential your agents currently hold. Not the ones in the design document - the ones live in production. For each one, write down three things: the scope it grants, the date it was issued, and the name of the person who approved it.
Then look at what you've got. If any row has an empty owner, that's a finding. If any credential is older than the agent using it, that's a finding. If the scope is wider than the narrowest task the agent performs, that's a finding. And if the room goes quiet when you ask who approved a particular key, you have found the most important thing on the list.
You don't need a consultant for that exercise. Do it without us.
What a proper readiness review adds is the part that follows: whether the traceability you have would survive a regulator asking questions, which failures are reversible and which aren't, and in what order to fix things so you're not rebuilding the agent twice.
The lesson from OpenAI's own retrofit isn't that agents are dangerous. It's that agent risk lives below the model, in permissions, credentials and logs - the least glamorous layer in the stack, and the one that gets skipped when the demo is due Friday. The teams that ship agents safely aren't the ones with better prompts. They're the ones who decided what the agent was allowed to break before they gave it the keys.
Book a Free Rescue Call
