arXiv survey reviews 100 LLM safety agent papers
Start here
A survey published on arXiv in August 2026, reviewing 100 papers published in 73 conferences and journals between January 2023 and March 2026, establishing a comparable taxonomy for LLM security agents from three dimensions: methods, applications, and evaluation. The study went through three stages—keyword search, forward and backward snowballing, and manual screening—converging from 108 candidates to 100 papers. The survey notes that single agents remain the mainstream, while multi-agent systems diverge into three topologies: collaborative, hierarchical, and peer-to-peer; applications cover 11 security task domains including vulnerability detection and penetration testing.
Generated by AI from reports · Updated 24 days ago
Related research
Papers whose work the reports mention
- The reports mention PentestAgentPentestAgent: Incorporating LLM Agents to Automated Penetration TestingA systematic review of 100 papers: methods, applications and evaluation of LLM security agents
AsiaCCS · 2024-11-07
- The reports mention PentestGPTPentestGPT: An LLM-empowered Automatic Penetration Testing ToolA systematic review of 100 papers: methods, applications and evaluation of LLM security agents
arXiv · 2023-08-13
Progress timeline
- WeChat · LLM Security ResearchA systematic review of 100 papers: methods, applications and evaluation of LLM security agents
A review published on arXiv in August 2026 sorted through 100 papers published between January 2023 and March 2026 in 73 conferences and journals, using the three dimensions of methods, applications and evaluation to build a comparable taxonomy for LLM security agents. The study went through three stages—keyword search, forward and backward snowballing, and manual screening—converging from 108 candidates to 100 papers. Four findings include: single agents remain mainstream, while multi-agent systems diverge into three topologies: collaborative, hierarchical and peer-to-peer; applications cover 11 security task domains including vulnerability detection and penetration testing; evaluation mostly stays at outcome metrics such as task success rate, while process and trajectory metrics such as hallucination rate and tool-call correctness are covered by only some papers. The review argues that in this field action capability is in place first, permission governance lags behind, and evaluation practice comes last, and recommends that enterprises define permission boundaries and require process trajectory auditing before deployment.