AI agent security is harder than chatbot security because agents can take actions. The risk comes from the interaction of untrusted content, model reasoning, powerful tools, credentials and autonomy. Prompt injection is especially important because browsing, email, documents, repositories and tool outputs can all carry malicious instructions.
Security should assume that the model can be confused or manipulated and ensure that one bad decision cannot become unlimited authority.
Threat Model
Start with four questions: what can the agent read, what can it do, which instructions can it trust and what happens if it is wrong? A public summarizer and a production deployment agent need completely different controls.
Prompt Injection
OpenAI describes prompt injection as an evolving security challenge in which third-party content attempts to mislead the model into actions the user did not request. Effective attacks increasingly resemble social engineering rather than a simple magic string.
Direct vs Indirect Injection
Direct injection appears in the user’s prompt. Indirect injection appears in external content the agent reads while doing legitimate work. Agentic browsing and retrieval make indirect attacks especially important.
Separate Instructions from Data
A web page can provide facts but should not gain authority to redefine the user’s task. Treat retrieved content as untrusted data and reinforce the distinction with deterministic policy outside the model.
Least Privilege
Give the agent only required capabilities. Prefer read-only tools, narrow scopes and short-lived credentials. A research agent does not need deletion authority; a code reviewer does not need production database write access.
Human Confirmation
Sending messages, spending money, deleting data, modifying access controls, deploying code and disclosing sensitive information deserve explicit review unless a narrower automation workflow has been rigorously designed.
Sandboxing
Sandboxing limits filesystem, process and network effects. OpenAI describes Codex security with sandbox boundaries, approvals and network controls; Cursor documents local sandboxing and isolated cloud VMs. Isolation does not make the model perfect, but it bounds consequences.
Credential Protection
Keep API keys, OAuth tokens, SSH keys and cloud credentials out of model context where possible. Prefer secret managers, workload identity and backend proxies. Agent traces can also contain secrets and need retention/redaction policy.
MCP Risks
MCP expands what an agent can reach. Review third-party server ownership, source, authentication, requested permissions, update process and data handling. Tool descriptions are metadata, not security enforcement.
MCP Authorization in 2026
The July 2026 MCP revision adds authorization hardening including issuer validation and credential isolation, with a strategic move toward Client ID Metadata Documents. Follow current SDK migration guidance rather than old OAuth examples.
Validate Tool Arguments
Backends should validate resource ownership, paths, types, ranges and allowed actions. Important constraints must exist outside the model. Avoid overly broad tools that rely on prompts as the only safety layer.
Network Egress
Filesystem restrictions do not stop data exfiltration if the agent can contact arbitrary hosts. Use allowlists, proxies or per-domain policy for sensitive workflows.
Multi-Agent Trust
Registry discovery or A2A metadata does not automatically make a remote agent trustworthy. Validate identity, owner, data policy and permissions before delegation.
Observability
Log what the agent saw, which tools it called, what arguments it supplied, which approvals occurred and what changed. Without traces, debugging and incident response are guesswork.
Red Teaming
Test malicious pages, poisoned documents, hostile repositories, misleading tool descriptions, credential requests and permission escalation. Security tests should assume the model can be persuaded.
Approval Fatigue
Approval for every trivial command trains users to click without reading. Automate low-risk reversible actions and reserve strong confirmations for operations with real consequences.
Security Checklist
- Least privilege
- Read-only access by default
- Secrets outside prompts
- Sandboxed execution
- Restricted network egress
- Untrusted-content separation
- Server-side argument validation
- Review third-party MCP servers
- Authenticate remote agents
- Approval for high-impact actions
- Tool-call logging
- Prompt-injection red teaming
- Task timeouts and limits
- Data minimization
Bottom Line
The safest agent is the one whose tools, permissions, identities, data boundaries and approval rules are intentionally matched to the task. Model-level defenses matter, but technical boundaries determine the blast radius when those defenses fail.
Threat 1: Data Exfiltration
A prompt-injected agent may try to send sensitive information to an external domain or tool. Preventing this requires more than filesystem permissions. Limit network destinations, reduce the data the agent can read and keep credentials outside model-visible context. For highly sensitive workflows, route outbound requests through a policy-enforcing proxy.
Threat 2: Destructive Tool Use
Tools that delete, publish, transfer money or change permissions should be separated from ordinary read tools. Use distinct endpoints, stronger scopes and explicit confirmation. A generic shell with unrestricted privileges is difficult to govern because almost any policy can be bypassed through another command.
Threat 3: Supply-Chain Content
Coding agents read dependencies, build scripts, documentation and repositories. A malicious package or repository can contain instructions designed to manipulate the model. Run untrusted code in isolated environments and do not expose unrelated credentials during installation or testing.
Threat 4: Confused Deputy Problems
An agent may have authority the user does not intend to exercise. If external content persuades the agent to use that authority, the system becomes a confused deputy. Policy checks should consider the user’s original intent, not only whether the agent technically has permission to call the tool.
Threat 5: Cross-Agent Contamination
In multi-agent systems, one compromised specialist can return malicious instructions to another. Treat agent outputs as data unless the architecture explicitly grants instruction authority. Validate structured outputs and avoid forwarding raw untrusted text into privileged system prompts.
Security Boundaries for MCP
Separate the MCP client, server and downstream service. The server should validate requests even if the client claims they came from a trusted agent. The downstream service should still enforce its own authorization. Defense in depth prevents one compromised layer from becoming universal access.
Security Boundaries for A2A
Agent-to-agent delegation should carry the minimum task context necessary. Do not forward an entire conversation, mailbox or customer record set when a specialist needs only a narrow extract. Remote agents should have explicit data classifications and retention expectations.
Policy Engines
Important decisions should not exist only in natural-language prompts. A deterministic policy engine can check user identity, tool category, resource, amount, environment and data class before execution. The model proposes an action; policy decides whether the action is allowed.
Human Approval Design
A useful approval dialog tells the user exactly what will happen: which tool will run, which resource will change, what data will be shared and whether the action is reversible. Vague “Allow?” prompts create approval fatigue and poor security.
Secret Management
Prefer short-lived credentials scoped to the task. Cloud workload identity, OAuth tokens with narrow scopes and brokered secret access are safer than long-lived API keys in environment variables. Rotate credentials and audit where they are exposed.
Audit and Forensics
Store enough evidence to reconstruct an incident: original user goal, external content retrieved, model/tool decisions, tool arguments, approval events and resource changes. Protect those logs because they may contain sensitive data. A security log that leaks secrets creates another vulnerability.
Rate and Budget Limits
Limit tool calls, spend and execution time. If a prompt injection causes an agent to loop, resource budgets turn an unbounded incident into a bounded failure. Payment and infrastructure tools should also have transaction limits independent of model instructions.
Safe Defaults
New agents should start with no privileged tools, no arbitrary network access and minimal data. Capabilities can be added after a use case demonstrates need. It is easier to expand a safe boundary than to discover after deployment that a broad default was dangerous.
Security Review Before Production
- Inventory every tool and credential.
- Classify data sources.
- Test indirect prompt injection.
- Test malicious tool output.
- Test destructive command attempts.
- Verify network egress policy.
- Verify approval checkpoints.
- Review logging and retention.
- Define incident-disable procedures.
- Run adversarial scenarios before granting production access.
Build the Threat Model Before You Build the Agent
Agent security starts with the authority graph. List what the system can read, what it can write, which networks it can contact, which credentials it can obtain and which external content can influence its reasoning. A public research assistant and a deployment agent with production credentials should not share the same default security model.
| Threat | Typical path | Primary control |
|---|---|---|
| Prompt injection | Web page, email, document, repo | Untrusted-content separation + bounded tools |
| Data exfiltration | Arbitrary network/tool output | Egress controls + data minimization |
| Destructive action | Shell/admin/payment tool | Least privilege + approval |
| Credential theft | Prompt context/log/tool output | Secret isolation + short-lived credentials |
| Cross-agent attack | Malicious remote agent output | Identity + validation + trust policy |
Prompt Injection: Why Agents Are Especially Exposed
OpenAI’s public guidance treats prompt injection as an evolving security problem that increasingly resembles social engineering. An agent researching the web may encounter text that tells it to ignore the user, reveal secrets or call a privileged tool. The attack arrives through data the user legitimately asked the agent to read.
The safest architecture assumes that model-level defenses can fail. A malicious instruction should still hit hard technical boundaries before it can become a damaging action.
Separate Instructions from Data
Retrieved content should be treated as evidence, not authority. A web page can contain information relevant to the task without being allowed to redefine the task. Keep trusted system and application policy separate from untrusted documents and tool results. Structured extraction can reduce the amount of arbitrary text that reaches privileged decision points.
Least Privilege for Tools
Give an agent the narrowest capability set that completes the task. Prefer read-only endpoints, resource-specific scopes and short-lived credentials. A summarization agent does not need database write access. A code-review agent does not need production deployment permission. Avoid generic “execute anything” tools when a smaller structured tool can expose the required operation.
Human Confirmation for Consequential Actions
Payments, deletion, publication, access-control changes, external messages and production deployment deserve stronger checkpoints. Approval UI should explain the exact action, target, data shared and whether the action is reversible. Requiring approval for every trivial command creates fatigue and trains users to click through without reading.
Sandboxing and Filesystem Isolation
Sandboxing limits the blast radius when an agent or command behaves badly. Claude Code documents permissions and OS-level sandboxing as complementary layers. Cursor documents sandbox run modes for local agents and isolated VMs for Cloud Agents. A sandbox should restrict both filesystem and network access; blocking file writes while allowing arbitrary outbound connections can still permit data exfiltration.
Network Egress Controls
Agents that browse or run code should not automatically receive unrestricted internet access in sensitive environments. Use domain allowlists, proxies or isolated networks. Cursor’s cloud security documentation explicitly includes egress controls as a defense against exfiltration. Enterprise systems should log and review unusual destinations.
Secrets and Credentials
Keep API keys, OAuth tokens, SSH keys and database passwords outside model-visible context where possible. Prefer workload identity, secret brokers and short-lived scoped tokens. Redact secrets from traces and tool outputs. Remember that logs are another data store: a secure runtime can still leak credentials if debugging output is retained indefinitely.
MCP Server Trust
MCP makes tools portable, which increases the importance of server provenance. Before connecting a third-party server, review who operates it, how it updates, what it can access, which credentials it requests and whether its tool definitions accurately describe side effects. Tool descriptions should help the model choose; they should not be trusted as the only enforcement layer.
MCP Authorization in the 2026 Revision
The July 2026 MCP revision includes authorization hardening such as issuer validation and credential isolation, and formalizes movement away from Dynamic Client Registration toward Client ID Metadata Documents. These improvements reduce classes of OAuth integration error, but they do not make a malicious server safe. Authorization answers whether a caller may connect; trust answers whether the service should be connected at all.
Server-Side Argument Validation
Important constraints must exist outside the model. Validate paths, resource ownership, monetary limits, environments and allowed operations on the server. If a production deployment tool is only allowed to deploy signed builds to staging, enforce that in code rather than a prompt sentence.
Confused Deputy Problems
An agent may possess authority on behalf of a user and then be manipulated by external content into exercising that authority for someone else’s goal. Policy should compare the proposed action with the original user intent, not merely check whether the token technically has permission.
Cross-Agent Trust
Multi-agent systems add another untrusted boundary. A remote specialist can return malicious or misleading text to a coordinator. Treat remote outputs as data unless the architecture explicitly grants instruction authority. Validate structured artifacts and avoid blindly inserting another agent’s prose into a privileged system context.
Registry Security
Discovering an agent or MCP server in a registry does not make it trusted. Registry entries can be stale or compromised. Filter by owner, lifecycle state, environment and policy, and authenticate the selected endpoint at execution time.
Observability and Audit
Log the user goal, external data sources, selected tools, arguments after redaction, approvals, errors and actual side effects. For multi-agent work, record delegation and agent identity. The trace should be sufficient to answer “why did this action happen?” without exposing secrets unnecessarily.
Budget and Rate Limits
Time, tool-call and spending limits are security controls as well as cost controls. If an injected instruction causes a loop, a hard budget bounds the incident. Financial tools should enforce transaction limits independently of model instructions.
Red-Team Scenarios
- Malicious instructions hidden in a web page.
- A poisoned document asking the agent to upload secrets.
- A repository README that tells a coding agent to run a destructive command.
- An MCP tool description that falsely claims it is read-only.
- A remote A2A agent returning instructions instead of data.
- A tool response containing encoded prompt injection.
- A request to access a production resource outside the user’s role.
Incident Response
Teams need a kill switch. Be able to revoke credentials, disable a tool or agent, stop long-running tasks and identify affected resources. Preserve enough logs for investigation while protecting sensitive data. Security planning is incomplete if the only response to a compromised agent is “turn off the whole application.”
Production Security Checklist
- Inventory tools, networks and data.
- Classify each action by impact.
- Apply least privilege.
- Separate trusted instructions from untrusted content.
- Sandbox code and computer-use workloads.
- Restrict network egress.
- Keep secrets outside prompts.
- Validate tool arguments server-side.
- Review third-party MCP servers.
- Authenticate and authorize remote agents.
- Require meaningful approval for high-impact actions.
- Trace execution and redact sensitive logs.
- Red-team indirect prompt injection.
- Define budgets, timeouts and emergency disable procedures.
FAQ
Can prompt injection be eliminated?
No. Treat it as an evolving threat and use layered controls.
Is OAuth enough to make an MCP server safe?
No. Authorization does not guarantee provider trust or safe tool semantics.
Is human approval enough?
No. Combine review with deterministic technical boundaries.
Should agents receive production credentials?
Only when necessary, narrowly scoped and strongly governed.
