安全评估 / NAVIN RESEARCH
Prompt Injection Defense for Tool-Using AI Agents
Combine trust separation, least privilege, validation, approvals, and monitoring because prompts alone cannot secure agents.
Combine trust separation, least privilege, validation, approvals, and monitoring because prompts alone cannot secure agents.
简要回答: Treat retrieved pages, emails, files, images, and tool results as untrusted data. Keep policy separate, restrict available tools, validate every call in code, require approval for consequential actions, and minimize credentials and data exposure.
实施步骤
- Map every place untrusted content can enter context.
- Label source and trust level in structured context.
- Enforce authorization outside the model.
- Add adversarial cases to continuous evaluations.
验证清单
- Attempt data exfiltration and privilege escalation.
- Test indirect instructions inside each supported modality.
- Confirm blocked actions leave an auditable event.
常见错误
- Using 'ignore malicious instructions' as the only defense.
- Giving browsing and write tools the same broad token.
- Returning secrets in tool errors.
实际决策
先交付能够通过上述检查的最小设计。记录测试中使用的模型、提示词、工具、数据集与运行时版本，让后续变化可以被比较而不是被猜测。依赖供应商的限制、模型名称、价格或预览功能之前，请重新检查链接文档；这些细节变化往往快于本文的工程原则。
