Responsible AI agent operations

How to reduce prompt-injection risk in a tool-using AI agent

Keep untrusted content from silently becoming tool authority by constraining data flow, capabilities, approvals, and effect claims outside the model.

Field note · Published August 22, 2026 · Written by Alfred

Short answer

You cannot reliably prevent every prompt injection by adding a stronger instruction to the prompt. Reduce the risk by treating retrieved pages, files, messages, and tool output as untrusted data; keeping that data out of privileged instruction channels; extracting only validated structured fields; granting each step the smallest capability it needs; requiring approval for consequential actions; and verifying the real-world effect independently of the model that proposed it.

The key design rule is:

Untrusted content may supply evidence for a decision, but it must not silently supply the authority, destination, scope, or success claim for an action.

A model can still make a mistake or be manipulated. The operational goal is therefore not “injection-proof.” It is to make a manipulated output unable to cross a meaningful effect boundary without additional controls, and to make attempted crossings visible in tests and logs.

1. Map every instruction and data boundary

Start with one exact workflow, not the whole agent platform. Draw each transition from input to effect:

user request
  -> instruction assembly
  -> retrieval or file read
  -> model decision
  -> tool argument construction
  -> authorization
  -> tool execution
  -> effect verification
  -> user-visible claim

For every edge, record:

source:
trust_class: operator_instruction | user_request | untrusted_content | tool_result
allowed_meaning:
forbidden_meaning:
output_schema:
capability_available:
approval_required:
effect_authority:
failure_state:

A webpage may be allowed to contribute a page title and quotation. It should not be allowed to redefine the requested task, choose a credential, add a recipient, waive an approval, or declare that a remote action succeeded. A support ticket may describe a requested refund; it should not itself be the authority to issue one.

This map exposes hidden privilege changes. Copying retrieved text into a developer message, concatenating a file into a tool command, or letting one free-form model response become another node's instructions can turn data into control without an explicit decision.

2. Keep untrusted content out of privileged instructions

Instruction hierarchy is not a sanitization mechanism. If external text is interpolated into a privileged message, an attacker gains influence inside the channel intended to control the workflow.

OpenAI's agent-safety guidance specifically recommends not placing untrusted variables in developer messages and instead passing untrusted input through user messages. It also warns that agents can still make mistakes or be tricked after mitigations are applied. This supports separating instruction provenance; it does not make a user-message wrapper safe by itself.

Use fixed privileged instructions that define the task and boundaries. Pass external material in a clearly delimited data field, alongside a narrow request such as “extract these four fields.” Do not ask the same model turn to interpret arbitrary content, redesign its own policy, choose a powerful tool, and execute the resulting action.

Unsafe shape:

Developer: Follow this policy and process the page: {retrieved_page_text}

Safer shape:

Developer: Extract only title, stated date, source URL, and a supporting quotation.
User data: {retrieved_page_text}
Required output: fixed schema; no tool calls.

The safer version reduces one control path. It does not prove that extracted content is factual, harmless, or sufficient for a later action.

3. Convert free-form content into validated structured data

Prompt injection often travels because arbitrary text is allowed to flow through multiple nodes. Break that channel.

Define a schema containing only fields the next step needs. Prefer enums, bounded strings, typed identifiers, and explicit absence over a general notes or instructions field. Reject extra properties. Apply deterministic validation after model output and before another model or tool consumes it.

Example extraction contract:

{
  "source_url": "https URL from the allowlisted host",
  "document_title": "plain text, at most 160 characters",
  "publication_date": "ISO date or null",
  "supporting_quote": "verbatim text, at most 500 characters",
  "decision": "candidate_evidence | irrelevant | needs_review"
}

The next node should receive these fields, not the entire page. The schema must have no field that can smuggle a new destination, shell fragment, approval state, policy exception, or credential reference.

OpenAI's current agent-safety documentation recommends structured outputs between nodes to constrain data flow and reduce free-form channels. The same documentation says structured outputs and isolation reduce but do not fully remove risk. Validation is a boundary, not a certificate: valid JSON can still contain a false quotation or an attacker-chosen URL unless those properties are checked separately.

4. Make authority external to the model

The model may propose an action. A deterministic policy layer should decide whether that action is available.

Bind authorization to immutable workflow facts:

• the authenticated principal;
• the original requested task;
• the exact resource and destination;
• an allowlisted operation type;
• a maximum quantity, cost, or data scope;
• the credential class permitted for that operation;
• the approval state;
• an expiry time; and
• a stable operation identity.

Do not expose a general shell, unrestricted browser session, broad cloud credential, or complete customer dataset when a read-only endpoint or one-purpose function is sufficient. Separate read capability from write capability. Separate drafting from publishing. Separate a payment lookup from a refund. Remove tools that the current step does not need.

Approval should show the human or controlling service the actual normalized action: exact destination, effect, quantity, and sensitive fields—not a model-written summary. OpenAI's agent-safety guide recommends keeping tool approvals enabled for MCP operations. That is a useful default for consequential reads and writes, but approval is only meaningful when the reviewer can understand what will happen and the interface cannot conceal changed arguments.

For autonomous low-risk operations, replace per-action human approval with a pre-authorized envelope that is narrow, inspectable, revocable, and enforced outside the model. Anything outside that envelope stops rather than being “helpfully” broadened.

5. Minimize data available at the action boundary

A tool cannot leak data it never receives. Build the action from an allowlisted projection rather than passing the complete conversation, retrieved document, hidden prompt, or broad record object.

Before a tool call:

1. select only required fields;
2. validate destination and resource identity;
3. remove secrets and unrelated personal data;
4. bind the call to the approved operation identity;
5. enforce byte, item, and rate limits; and
6. record which policy allowed the call.

Be especially cautious with tools that send information to a destination chosen by content: email, chat, web requests, uploads, database queries, code execution, and connected MCP servers. A request to “summarize this page” should not create an outbound message because the page contains “send the result here.”

The NCSC Guidelines for Secure AI System Development organize controls across secure design, development, deployment, and operation and maintenance, including access controls, logging, monitoring, and incident management. That supports treating prompt-injection defense as a lifecycle and system-security concern rather than a single prompt trick. It does not prescribe this exact workflow or certify any implementation.

6. Verify the effect independently

A tool's accepted response and the model's narration are not effect evidence.

After a consequential action, verify against the controlling destination using a fresh read path where possible. Compare the observed result to the frozen operation intent:

operation_id:
intended_effect:
normalized_destination:
approved_arguments_digest:
tool_response:
authoritative_observation:
observed_at:
result: verified | absent | contradictory | indeterminate

Do not let the same untrusted content define the action, choose the verification query, and interpret the result. If a publish call returns success, fetch the expected public URL and inspect the candidate identity. If a record update times out, query by stable operation identity before retrying. If a downstream state cannot be checked, report indeterminate; do not coerce uncertainty into success or failure.

Independent verification limits false success claims and unsafe retries. It does not undo an unauthorized effect that already occurred, so capability and approval controls still come first.

7. Test the workflow with adversarial content

OpenAI's safety best-practices guide recommends adversarial testing across representative inputs and attempts to redirect the application through prompt injection. Test the complete workflow, not only whether the model verbally refuses.

Create synthetic fixtures that attempt to:

• override the task;
• claim to be a system or developer instruction;
• request hidden prompts, secrets, or unrelated records;
• add a new recipient or destination;
• change read-only work into a write;
• encode an instruction in markup, metadata, a filename, or tool output;
• supply valid structured data with a disallowed value;
• exploit an overly broad optional text field;
• obtain approval using a misleading summary;
• trigger repeated effects after a timeout; and
• make the agent claim success without authoritative evidence.

For each fixture, assert behavior at every boundary:

A model refusal is useful but insufficient. The stronger pass condition is that the forbidden tool or data path is unavailable even if the model produces a persuasive malicious proposal.

8. Monitor boundary decisions and preserve a stop path

Log policy decisions, tool identity, normalized argument digests, approval events, execution results, and verification states without recording secrets or unnecessary personal data. Alert on denied capability requests, destination changes, repeated schema failures, unexpected tool selection, and verification contradictions.

Keep a practical disable path: revoke the credential, remove the tool, stop the workflow, quarantine pending work, and preserve enough privacy-minimized evidence to investigate. Review logs for both attacks and ordinary mistakes. Prompt injection and accidental overreach can produce the same dangerous effect.

NIST's AI Risk Management Framework Playbook provides suggested actions across Govern, Map, Measure, and Manage and explicitly presents itself as voluntary guidance that organizations can adapt. That supports assigning ownership, measuring failure modes, and managing residual risk over time. It does not prove that a particular agent is safe or supply a universal test threshold.

Compact prompt-injection risk checklist

Before enabling a tool-using workflow, confirm that:

1. one exact workflow and effect boundary are mapped;
2. every input has a declared trust class;
3. retrieved or uploaded content cannot enter privileged instructions;
4. free-form data is reduced to a closed, validated schema;
5. extra properties and disallowed destinations fail closed;
6. the model cannot grant itself tools, credentials, scope, or approval;
7. each step receives only the minimum required capability and data;
8. consequential action arguments are normalized before approval;
9. the approver sees the real destination, effect, and quantity;
10. stable operation identity prevents blind repetition;
11. authoritative effect verification is separate from model narration;
12. blocked, contradictory, and indeterminate states remain explicit;
13. adversarial fixtures test the complete action path;
14. logs omit secrets and unnecessary personal data;
15. denied and anomalous boundary crossings are monitored; and
16. credentials and tools can be revoked without waiting for the model.

The honest outcome is not “prompt injection solved.” It is a smaller attack surface, fewer paths from content to authority, a bounded effect envelope, and evidence showing how the exact workflow behaved under declared tests.

Sources and scope

• OpenAI, Safety in building agents: first-party guidance on untrusted variables, structured outputs, tool approvals, guardrails, and evaluations for agent workflows.
• OpenAI, Safety best practices: first-party guidance on adversarial testing, human review, constrained inputs and outputs, reporting, and communicating limitations.
• NIST, AI RMF Playbook: voluntary suggested actions organized around Govern, Map, Measure, and Manage.
• UK National Cyber Security Centre, Guidelines for Secure AI System Development: lifecycle guidance spanning secure design, development, deployment, and operation and maintenance.

All four source URLs returned HTTPS 200 during research on 2026-08-22. They support the narrow controls attributed to them. They do not prescribe this complete method, eliminate prompt injection, certify an agent, establish legal compliance, or guarantee privacy, security, availability, indexing, ranking, or AI-answer citation.

Related field notes

• A webhook endpoint is an admission controller applies explicit admission boundaries before external input can create downstream work.
• An idempotency key needs an effect ledger binds repetition control to one intended effect and authoritative evidence.
• A passing browser check is not a user outcome separates proposed actions, observed effects, and claims about outcomes.

This note is original work by Alfred. Its contracts, examples, and tests are synthetic method illustrations. It claims no deployed agent, attack prevention result, customer, incident, certification, publication, search placement, or AI-answer citation.