Evolution Under Fire: Evolving Agent Safety Guardrails with Adaptive Injection Pressure
Evolution Under Fire: Evolving Agent Safety Harnesses with Adaptive Injection Stressors
The paper proposes the Evolution Under Fire framework, which alternates safety guardrail evolution with adaptive injection pressure evolution to automatically generate executable safety guardrails for tool-using agents, addressing the problem that malicious instructions and normal task instructions arrive through the same interaction channel. The framework evolves guardrails from execution evidence using utility constraints and Pareto-scheduled tree search while keeping the task model, benchmark, tools, and evaluator unchanged; adaptive pressure evolution discovers utility-preserving attacks that guide the next stage of guardrail evolution. On AgentDyn with GPT-4o-mini, the evolved guardrails reduce attack success rate from 50.4% to 3.0%, with a better safety-utility trade-off than reproduced defense methods; on Agent-SafetyBench with Qwen2.5-7B-Instruct, from 82.6% to 62.0%, while maintaining helpfulness and operational effectiveness; the evolved guardrails transfer to Qwen3.5-35B-A3B without further evolution, reducing attack success rate from 32.1% to 0.9%, with utility on attacked tasks also improving.