Skip to content

SkillBuffBench: measuring the contribution of Agent Skills across harnesses, models, skill access modes and memory

SkillBuffBench: Measuring What Agent Skills Contribute across Harnesses, Models, Skill Access Modes and Memory

arXiv· arXiv:2611.22017· Gengyue Dong, Ye Wei, Zheng Zhu, Yidong Han, Jiale Miao, Xiaojun Jia, Guanghui Li·Indexed by this site · 2 days ago· Originally publishedEvaluation· Its place in the scan record →
AI summary

SkillBuffBench measures the contribution of Agent Skills under deployment conditions, setting skill access at three levels: no skills, curated skills, and automatic invocation from a library, and evaluating them crossed with a memory switch and four agent harnesses. Models include proprietary models from Anthropic, Google and OpenAI and open-weight models from Chinese vendors, and tasks come from materials science and biology experts. Evaluation is still ongoing, and results will be added progressively.

Read originalarxiv.org