Your AI agent reads support tickets, searches customer records, updates your CRM, and prepares refunds. The demonstration works. The product team wants to connect more systems and remove approval steps.

Before expanding access, ask a different question: What can this agent do that nobody intended to authorize?

A successful demonstration does not answer that question. It shows that the agent can complete selected tasks under selected conditions. It does not establish how the system behaves when instructions are malicious, permissions change, tools fail, or an apparently helpful action exceeds the user’s authority.

For a CTO, the decision is not whether to trust AI in general. It is whether a specific agent has verified boundaries around the data it can access and the actions it can execute.

Can an AI Agent Leak Data or Take Unauthorized Actions?

Yes. An AI agent can expose sensitive information or perform unauthorized actions when its tools, permissions, or autonomy exceed the controls protecting them. The trigger might be prompt injection, an incorrect interpretation of a legitimate request, or an unexpected model output. OWASP describes excessive agency in terms of excessive functionality, permissions, and autonomy.

An AI agent security audit examines the complete workflow: what information enters the agent, which identities and permissions it uses, how tool calls are authorized, and what happens after execution. It should also investigate memory, data exposure, and controls for high-impact actions, rather than treating the model’s final response as the entire system.

The buying question is straightforward:

Can your team demonstrate that a manipulated or mistaken agent remains inside its authorized boundaries?

What Should an AI Agent Security Audit Test?

The following ten areas provide a practical scope for an agentic AI risk assessment. Apply them to a defined business workflow, with explicit authorization and controlled test conditions.

1. Trust Boundaries and Indirect Prompt Injection

Start by identifying every source the agent reads: user messages, support tickets, documents, search results, repository files, logs, and tool responses. A trust boundary is the point where information crosses between sources with different authority. An internal dashboard can still display externally supplied content.

OWASP’s prompt injection guidance distinguishes direct instructions from attacks embedded in external material the application processes. It also recommends layered defenses rather than relying on a single filter or instruction.

What to test: Introduce harmless adversarial content into representative test documents or tool responses. Determine whether the agent treats that content as task data or as a new instruction. Include attempts to change the task, request unnecessary tools, or claim authority that the source does not possess.

The acceptance criterion should concern observable behavior. A warning in the final answer is insufficient evidence when an unauthorized tool call occurred earlier.

2. Action Authorization and Least-Privilege Access

Authentication establishes identity. Authorization determines whether that identity may perform a particular action on a particular resource. A valid service account does not establish that every action available to that account is appropriate for the agent’s current task. OWASP recommends denying access by default and validating permissions on every request.

What to test: Verify the effective permissions for each tool call, including the initiating user, customer account, target record, and requested operation. Attempt a harmless operation outside those boundaries and confirm that enforcement occurs outside the model.

Include delegated work. When one agent asks another to act, require the original scope and identity to remain enforceable. OWASP specifically identifies trust boundaries and privilege escalation across agent chains as security concerns.

For example, a support agent authorized to inspect one customer’s subscription should not acquire broader access simply because a more privileged worker agent handles the next step.

3. Human Approval That Cannot Be Reused or Reinterpreted

An approval dialog is useful only when it accurately represents the action that will execute. OWASP’s transaction-authorization guidance calls for server-side enforcement, protection against transaction changes, limited validity, and authorization specific to each operation.

What to test: In an illustrative refund workflow, approve one amount for one customer. Then change the amount, recipient, or operation before execution. Verify that the system requires a new approval. Also test expired approvals, duplicate submissions, and an unavailable approval service.

The reviewer should see the actual action details, not merely the agent’s reassuring summary. For an email, that means the recipients and content. For a record update, it means the affected records and proposed changes.

Require the executor to reject an action when its approval is missing or no longer matches. The agent should not be able to declare that approval exists or substitute a different tool to avoid it.

4. Secrets and Credentials the Agent Can Reach

An agent may need permission to use a service without needing direct access to its underlying credential. Separating credentials from the agent’s execution environment can reduce exposure. Anthropic describes an implementation in which a proxy handles sensitive credentials and checks permitted repository operations before forwarding requests.

What to test: Inspect environment variables, mounted files, configuration stores, tool responses, and debugging output for unnecessary credential exposure. Verify that credentials are narrowly scoped and that rotation and revocation work. These are central elements of OWASP’s secrets-management guidance.

Use synthetic secrets or nonproduction credentials during testing. The objective is to verify the exposure path, not to copy production secrets into an assessment report.

Also distinguish two failures: the agent reading a credential and the agent using an overprivileged tool without ever seeing that credential. Preventing the first does not resolve the second.

5. Data Leakage Through Allowed Tools and Destinations

Blocking unknown websites does not answer every data-leakage question. Your agent may already have a legitimate tool for sending email, uploading files, or posting messages. The audit must determine whether a particular destination and disclosure are authorized, not merely whether the tool is available.

Anthropic’s containment guidance emphasizes that trusted tools can return untrusted content and that manipulated behavior can result in apparently authorized API activity. This makes the surrounding permissions and data-transfer controls important.

What to test: Use synthetic sensitive records and controlled recipient accounts to verify restrictions on external sharing, file uploads, message destinations, and exported results. Check the intended organization, account, recipient, and resource, not just the service’s domain.

A read-only database connection can still support a disclosure if the retrieved information reaches the wrong recipient. Object-specific access checks and downstream restrictions remain necessary.

The relevant question is not simply, “Can the agent read this?” It is also, “Where can that information go next?”

6. Customer Isolation, RAG Security, and Persistent Memory

For multi-tenant products, investigate the complete retrieval path. Tenant restrictions should apply before information enters the model’s context, and they must remain effective across retrieved documents, cached results, generated output, and downstream tool use. OWASP’s RAG security guidance treats these as connected parts of the security assessment.

What to test: Create separate test customers and verify that one cannot retrieve, summarize, export, or act on the other’s records. Repeat the checks after permissions are revoked and after cached information has been created.

Persistent memory requires additional scrutiny. OWASP identifies memory and context poisoning as a risk because untrusted information can influence future decisions, not just the current response.

Introduce a harmless false instruction into a controlled session and check whether it survives into later sessions or another user’s context. Document where persistent state lives and how it can be invalidated.

7. MCP Security, Tool Configuration, and Execution Isolation

The Model Context Protocol, or MCP, connects applications to tools and resources. Its security guidance addresses issues including token audience validation, excessive scopes, and unsafe token passthrough. Connecting through MCP does not remove the need to verify authorization at the integration boundary.

What to test: Review each connected server’s origin, configuration, permissions, update process, and exposed operations. Confirm that tokens are intended for the receiving service and that unnecessary tools are unavailable. Examine whether a tool described as narrowly scoped actually executes with broader host privileges.

For agents that execute code, test filesystem and network restrictions independently. Anthropic’s sandboxing guidance explains why both boundaries matter: restricting files without restricting network access, or vice versa, leaves important exposure paths.

A tool’s description is not an enforcement mechanism. Verify what the underlying process can actually read, change, and contact.

8. Duplicate Actions, Retry Behavior, and Runaway Costs

Unauthorized outcomes do not always require an attacker. A timeout can leave an agent uncertain whether an action completed. Retrying without appropriate safeguards can create duplicate side effects. AWS’s guidance on idempotent APIs explains how request identifiers and operation semantics can support retries without repeating the intended effect.

What to test: Simulate a timeout after a test operation has completed. Determine whether the agent checks status or repeats the action. For workflows that support it, verify that repeated execution does not create duplicate refunds, orders, messages, or records.

Also enforce limits on tool calls, retries, execution time, and spending. OWASP identifies uncontrolled consumption as a source of service disruption and financial exposure.

Set limits from the business workflow’s requirements. A customer-support task and a long-running engineering investigation should not automatically inherit the same execution budget.

9. Audit Logs That Reconstruct What Actually Happened

A conversation transcript is not sufficient evidence of execution. Require a trace that connects the initiating request, relevant input source, proposed action, authorization decision, approval, tool result, and observed system change.

OWASP’s logging guidance emphasizes attributable events, appropriate detail, and protection against tampering. It also warns against recording access tokens, passwords, and other sensitive information directly in logs.

What to test: After a controlled unauthorized-action attempt, ask someone who did not run the test to reconstruct what happened from the retained evidence. Can they identify the affected account, the action attempted, the enforcement decision, and whether anything changed?

Minimize and redact sensitive content. Preserve the facts needed for investigation without creating another repository of customer data or credentials.

The report should distinguish proposed, attempted, blocked, and executed actions. Those are different outcomes, even when the agent produces the same final message.

10. Containment, Permission Revocation, and Recovery

Test the stop mechanism rather than assuming it works. Some platforms provide explicit session-termination operations. For example, Amazon Bedrock AgentCore documents an operation for terminating an active runtime session and stopping its ongoing streaming responses.

What to test: Stop a controlled workflow while work is in progress. Then separately verify downstream jobs, queued actions, delegated agents, credentials, and persistent state. Do not assume that terminating the visible conversation cancels every operation it initiated.

Where credentials may be exposed, test the revocation process and the rejection of subsequent use. OWASP treats revocation and expiration as distinct parts of the secrets lifecycle.

Define recovery for each consequential action. A database update may permit a corrective change. An external disclosure cannot be treated as undone merely because the original file was deleted.

Measure what operators can stop, what they can restore, and what requires escalation. Those distinctions belong in the release decision.

What Does a Useful AI Security Test Look Like?

Start with a business workflow, not a collection of dramatic jailbreak prompts.

Consider this illustrative scope:

A customer-support agent reads tickets, retrieves account information, drafts responses, and proposes refunds. A designated employee must approve refunds before execution.

The following are proposed acceptance criteria, not results from a real assessment.

When Is It Worth Paying for an AI Agent Security Assessment?

An external assessment is worth considering when it resolves a material uncertainty that your team has not already addressed.

The strongest buying trigger is a change in authority: moving from recommendations to execution, connecting sensitive customer data, adding write access, permitting external communications, or deploying an agent with elevated privileges.

Before buying, ask your team to demonstrate one representative workflow under three conditions: normal operation, manipulated input, and a control failure. Identify where the evidence stops.

External help may add specialist testing, independent review, or capacity. It should not be purchased merely because the service contains “AI” in its name.

Pay for a defined investigation that can change a release or access decision.

Frequently Asked Questions About AI Agent Security Audits

Q. Can prompt injection be completely prevented with a stronger system prompt?

A stronger prompt can be one defensive layer, but it should not be treated as complete prevention. OWASP recommends combining controls around inputs, outputs, tools, permissions, and monitoring. The audit should verify that consequential actions remain restricted even when the model’s behavior changes.

Q. Is a human approval step enough to prevent unauthorized actions?

Not by itself. Approval must be enforced by the application and tied to the operation being executed. Changes to significant transaction details, expired approvals, and reused authorization must be handled correctly. Test these conditions instead of merely confirming that an approval screen appears.

Q. Does an AI agent security audit replace penetration testing?

No. Define the investigations separately. An agent assessment focuses on the workflow’s data, authority, and behavior under adverse conditions. Conventional application and infrastructure testing addresses other implementation weaknesses. Anthropic’s containment analysis explicitly considers both agent-specific manipulation and conventional attacks on runtimes and orchestration.

Q. How do you measure whether an AI security audit was successful?

Measure the uncertainty it resolved. Useful evidence includes verified unauthorized-action paths, effective enforcement boundaries, reproducible failures, tested recovery behavior, and confirmed remediation. Do not measure value solely by the number of findings or prompts executed. Require findings to support a concrete engineering or release decision.

Q. How often should an AI agent be reassessed?

Reassess when changes can alter the tested behavior or its consequences. Examples include a new model, additional tools, broader permissions, changed approvals, or a different retrieval source. Maintain repeatable evaluations and production monitoring rather than treating a one-time test result as permanent evidence.

How ISHIR Helps Evaluate AI Agent Risk

ISHIR’s technical due diligence services assess software, SaaS platforms, cloud systems, and AI-enabled products, including technology risk, security exposure, scalability, and execution readiness. The objective is to support informed investment and strategic technology decisions.

For an agent initiative, start with an authorized assessment of one defined workflow. Agree on the permission review, controlled adversarial tests, evidence requirements, remediation priorities, and retesting before expanding the engagement.

You do not need another report saying that AI introduces risk. You need evidence showing where your agent’s authority begins, where it ends, and whether those boundaries hold.