A Framework for Formalizing LLM Agent Security

The paper proposes a framework that formalizes LLM agent safety from a contextual safety perspective, using four safety properties to uniformly characterize existing attacks and defenses. The paper points out that agent safety is contextual: whether the same action constitutes a violation depends on the source of the instruction, the goal being pursued, and whether the action serves that goal, whereas existing attack definitions often ignore this, causing defenses to fall into a trade-off between safety and usability. The four properties are task alignment (pursuing an authorized goal), action alignment (an individual action serves the goal), source authorization (executing commands from authenticated sources), and data isolation (information flow respects permission boundaries); the authors also introduce a set of oracle functions for verifying whether an agent violates these properties when executing a task. Under this framework, existing attacks such as indirect prompt injection, direct prompt injection, jailbreak, task drift, and memory poisoning are recast as violations of one or more properties, and defenses are recast as mechanisms that strengthen oracle functions or enforce property checks; the authors also discuss future research directions opened up by the framework.