AI Agent safety evaluation: AI Beat three-layer evidence L1-L3 + red team jailbreak hands-on test

Tophant Technology's open-source AI Agent safety evaluation framework AI Beat includes two products, PromptBeat and AgentBeat; the former performs black-box red team evaluations on targets such as models and RAG applications, while the latter collects execution traces such as an Agent's tool calls, commands and file changes. After checking the repository, the author explains that AgentBeat divides evidence into three layers: L1 looks only at the final response, L2 is evidence self-reported by the tested party, and L3 is environment state verified by an independent State Controller; only when conditions such as the L3 tier and complete evidence are met can it be marked as official_benchmark. The repository includes two attack cases, Foot-in-the-Door (arXiv:2410.16950) and RAG poisoning (PoisonedRAG, arXiv:2402.07867); the latter judges attack success by dual gates of recall matching and answer adoption. The author notes that the repository root has no LICENSE file, so commercial use or redistribution requires confirming authorization independently.