ObliInjection: Order-Oblivious Prompt Injection Attacks on LLM Agents with Multi-Source Data

The paper proposes ObliInjection, an order-oblivious prompt injection attack against LLM applications and Agents with multi-source inputs. It uses an order-oblivious loss to measure the likelihood that the LLM completes the attacker-specified task regardless of how clean segments and poisoned segments are ordered, and uses the orderGCG algorithm to minimize this loss and optimize the poisoned segments. The authors claim that on three datasets covering different application domains and twelve LLMs, the method remains highly effective even when only one of 6 to 100 segments of the input data is poisoned. Code and data are publicly available on GitHub.