As coding agents begin to run in dynamic parallelism, has software development gotten better?
Researchers from Nanjing University and UIUC conducted paired experiments on concurrent versus sequential execution with three coding agents—Codex, Claude Code, and Kimi Code—covering 354 software engineering tasks and 2,124 executions, and found that the gains from dynamic concurrency are not stable. On long-horizon tasks such as LoopsBench, Claude Code's task completion rate rose from 7.1% in sequential mode to 21.4% in concurrent mode, and Codex rose from 34.6% to 57.7% on NL2Repo; but on SWE-bench Verified, Claude Code fell from 83.0% to 59.0%, and Kimi Code fell from 77.0% to 59.0%. Token consumption in concurrent mode was 1.41 to 3.31 times that of sequential mode, and average runtime increased in 14 of the 15 agent–benchmark combinations. Paper number arXiv 2610.10263.
A paired experiment covering 354 tasks and 2,124 executions provides concrete data showing that dynamic concurrency has opposite gains on long-horizon and short-horizon tasks.