多模态 / NAVIN RESEARCH
Defend Multimodal Agents Against Visual Prompt Injection
Treat text inside images and documents as untrusted data, not system instructions.
Treat text inside images and documents as untrusted data, not system instructions.
简要回答: An image can contain instructions intended for the model rather than the user. Keep system policy separate, label visual content as untrusted, restrict tools, require authorization for side effects, and validate actions in code. No prompt can replace these controls.
实施步骤
- Threat-model screenshots, PDFs, web pages, QR codes, and retrieved images.
- Tell the model to extract relevant content without following embedded commands.
- Use least-privilege tools and confirmation for consequential actions.
- Detect unexpected tool requests and retain redacted security traces.
验证清单
- Add benign and adversarial visual instructions to the test set.
- Verify the same policy holds across OCR and native vision paths.
- Test mixed trusted and untrusted images.
常见错误
- Relying on a warning sentence alone.
- Giving a browsing agent broad credentials.
- Displaying external content without origin labels.
实际决策
先交付能够通过上述检查的最小设计。记录测试中使用的模型、提示词、工具、数据集与运行时版本，让后续变化可以被比较而不是被猜测。依赖供应商的限制、模型名称、价格或预览功能之前，请重新检查链接文档；这些细节变化往往快于本文的工程原则。
