Skip to content
WeChat · Weijiqi Lixin·Indexed by this site · Originally published· Tools →· Its place in the scan record →SelectedAttention39

Tencent Zhuque Lab open-sources the pikit toolbox to systematically evaluate AI agent indirect prompt injection risk

腾讯朱雀实验室:黑客藏指令在文档网页里,AI Agent就会中招?开源pikit工具箱,专门搞定间接提示注入

AI summary

Tencent Zhuque Lab released a paper and open-sourced the pikit toolbox for systematically evaluating AI agents' resistance to indirect prompt injection. pikit breaks the evaluation into three freely combinable axes: attacks, propagation channels, and defenses, including 13 attack constructions, 16 carriers (supporting real files such as PDF metadata and email headers), and 9 prompt-layer defenses plus 3 offline detection baselines. The team used the open-source code Agent pi to complete 2800+ controlled experiments: prompt-layer defenses overall reduced the attack success rate by a relative 71.8%, among which few_shot_warning and instruction_hierarchy were best-performing, pressed down to 0.32%, but no defense could reduce risk to zero; offline static detection had high precision but severe missed detections, e.g., LengthDetector's recall was only 11.11%, so it can only serve as an auxiliary. The tool code has been open-sourced in the Tencent/AI-Infra-Guard repository on GitHub.

Why it was selected

The tool breaks attacks, carriers, and defenses into combinable modules; the paper's measurements show that instruction-hierarchy-type defenses outperform delimiter isolation, and static detection has severe missed detections.

Read originalmp.weixin.qq.com