Untrusted instructions can arrive through useful sources
An agent may read a document, ticket, message, website or tool result as part of legitimate work. That content can contain instructions attempting to change its goal or disclose information. OWASP describes direct and indirect prompt injection as agent risks. A source considered useful for evidence is not automatically authorized to redefine the workflow’s permissions. Model instructions and document context need deliberate handling, but the application must also contain what a manipulated interpretation can cause outside the model.
Protect the execution boundary independently
Do not make sensitive tool access depend solely on whether the model recognizes malicious text. The receiving execution component needs trusted identity, scope and approval checks. OWASP explicitly recommends enforcing authorization outside agent context. A retrieved document asking for a configuration change is not permission from the resource owner. If the model follows that instruction, the surrounding service should still be able to deny the operation. This is why access control, tool design and resource validation remain necessary alongside prompt-focused defenses.
Limit the reachable capabilities
Provide only tools relevant to the task, with operations and targets constrained appropriately. A broad connector or unrestricted execution capability can turn a small reasoning error into a large external effect. Separate read and write authority and review outbound information paths. An action presented as harmless can still disclose sensitive context through a message or request. The design should evaluate the actual capability exposed rather than assume that a tool’s friendly name or an internal network address makes every use acceptable.
Treat memory and outputs as part of the path
An injection can influence stored context and affect later interactions. OWASP recommends validation before persistence, session isolation and appropriate memory limits. Review what survives a runtime restart and how resumed work is authorized. Generated output also needs handling appropriate to the receiving application; a text response should not silently become executable instructions or an approved record change. Filtering can contribute to defense, but no single classifier or formatting convention establishes that every harmful instruction will be detected.
Evaluate the complete workflow
Use authorized tests with realistic untrusted content, denied tools, sensitive-data boundaries and missing approval. Include retrieval, support systems and memory, not only the initial user prompt. Record the tested model, tool policy and configuration. Measure false interventions as well as missed problems so controls remain useful. Keep live secrets and customer data out of test fixtures. The objective is to determine whether the system contains the consequences of adversarial content, not simply whether the model produces an expected refusal in one prepared example.
Maintain controls as integrations change
New tools, larger context windows, provider changes or additional memory can alter the attack surface. Reassess before enabling the capability broadly and preserve regression cases from earlier findings. Define who investigates an attempted boundary crossing and what records support it. Prompt injection is not resolved by calling the runtime secure or placing it in a sandbox. A defensible design combines context hygiene, bounded capabilities, independent enforcement and observed recovery, with explicit limits on what the evaluation has actually established.
Worked scenario: a malicious knowledge article
A hypothetical support agent retrieves an article containing useful troubleshooting information alongside instructions to export unrelated records. A robust evaluation checks whether the workflow can use relevant evidence without granting the document authority over tools or permissions. Even if the model proposes the export, independent controls should reject an unauthorized target. The test should capture the denied operation and protect the fixture data. A clean refusal in the final answer is not sufficient if an earlier tool call already crossed the intended boundary before that answer was produced.
