Paper by USTC and other teams: higher scores do not mean a stronger agent; all 16 self-evolving agents lost some old capabilities
中科大:分数涨了≠Agent变强!这篇论文撕开自进化Agent的美丽幻觉
A paper by USTC, UC Berkeley, Beijing Institute for General Artificial Intelligence and Shanghai Jiao Tong University compares four agent self-evolution approaches—AHE, HyperAgents, A-Evolve and GEPA—and finds that looking only at average benchmark scores misjudges an agent's true capabilities. All 16 evolved agents lost some tasks they could originally solve while learning new tasks; for example, one agent's reliability rose 5.2 percentage points but 10 old tasks failed; of 11 versions with improved reliability, 8 had higher per-round running costs. Cross-benchmark transfer depends on the task domain: of 8 agents transferred from τ³-Banking to τ²-bench, 7 saw coverage decline. The team accordingly proposes the EvoAudit diagnostic system and implements an audit-guided evolution method, AGE, which in 4 experimental environments improves reliability and coverage and reduces lost tasks compared with the A-Evolve baseline; the code is open-sourced in the RSIHub repository.