Skip to content

Evolution Under Fire: Evolving Agent Safety Guardrails with Adaptive Injection Pressure

Evolution Under Fire: Evolving Agent Safety Harnesses with Adaptive Injection Stressors

arXiv: 其他 AI 安全论文· arXiv:2611.22583· Yujun Wang, Yilun Liu, Jinhe Bi, Krishna Kanth Nakka, Xuebing Zhou, Hinrich Sch{ü}tze, Yunpu Ma·Indexed by this site · 2 days ago· Originally publishedDefenses· Its place in the scan record →
AI summary

The paper proposes the Evolution Under Fire framework, which alternates safety guardrail evolution with adaptive injection pressure evolution to automatically generate executable safety guardrails for tool-using agents, addressing the problem that malicious instructions and normal task instructions arrive through the same interaction channel. The framework evolves guardrails from execution evidence using utility constraints and Pareto-scheduled tree search while keeping the task model, benchmark, tools, and evaluator unchanged; adaptive pressure evolution discovers utility-preserving attacks that guide the next stage of guardrail evolution. On AgentDyn with GPT-4o-mini, the evolved guardrails reduce attack success rate from 50.4% to 3.0%, with a better safety-utility trade-off than reproduced defense methods; on Agent-SafetyBench with Qwen2.5-7B-Instruct, from 82.6% to 62.0%, while maintaining helpfulness and operational effectiveness; the evolved guardrails transfer to Qwen3.5-35B-A3B without further evolution, reducing attack success rate from 32.1% to 0.9%, with utility on attacked tasks also improving.

Read originalarxiv.org