Safety Evaluation / NAVIN RESEARCH
Prompt Injection Defense for Tool-Using AI Agents
Combine trust separation, least privilege, validation, approvals, and monitoring because prompts alone cannot secure agents.
Combine trust separation, least privilege, validation, approvals, and monitoring because prompts alone cannot secure agents.
Short answer: Treat retrieved pages, emails, files, images, and tool results as untrusted data. Keep policy separate, restrict available tools, validate every call in code, require approval for consequential actions, and minimize credentials and data exposure.
Implementation steps
- Map every place untrusted content can enter context.
- Label source and trust level in structured context.
- Enforce authorization outside the model.
- Add adversarial cases to continuous evaluations.
Validation checklist
- Attempt data exfiltration and privilege escalation.
- Test indirect instructions inside each supported modality.
- Confirm blocked actions leave an auditable event.
Common mistakes
- Using 'ignore malicious instructions' as the only defense.
- Giving browsing and write tools the same broad token.
- Returning secrets in tool errors.
Practical decision
Ship the smallest design that passes the checks above. Record the model, prompt, tool, dataset, and runtime versions used in testing so later changes can be compared rather than guessed. Recheck the linked documentation before relying on provider-specific limits, model names, prices, or preview features; those details change more quickly than the engineering principles in this note.
