ScrapeBench: Evaluating the Legal Compliance of AI Agent Web Scraping
ScrapeBench: Evaluating Legal Compliance of AI Agents in Website Scraping
The paper proposes ScrapeBench, which the authors call the first benchmark to empirically evaluate the legal compliance of AI agent web scraping, testing whether agents comply with terms of service (ToS) and robots.txt. On 1,626 popular websites that explicitly prohibit scraping, most closed-source and open-source agents had a compliance rate below 5% when instructed to scrape, and the compliance rate remained low even after actively viewing the applicable terms. On 269 websites that do not prohibit scraping, agents sometimes refused scraping that was in fact legal, with Claude Opus 4.6 having an over-refusal rate of 98.5%. A human baseline experiment with 180 participants showed that human compliance rates were similar to those of agents, but the over-refusal rate was much higher.
Related events and news
1 events · 2 news mentions
News that mentions this paper
研究提出智能体抓取网站合规基准
- Luiza's Newsletter(AI 治理)cites the paper's id or linkScrapeBench:评测智能体抓取网站的法律合规
1. “ScrapeBench: Evaluating Legal Compliance of AI Agents in Website Scraping,” by Joseph Marvin Imperial, Daniel Slate & Noam Kolt - link
- 公众号 大模型安全研究cites the paper's id or link大模型安全周报:别只关注通过率,聚焦边界控制和安全覆盖面
ScrapeBench: Evaluating Legal Compliance of AI Agents in Website Scraping(https://arxiv.org/abs/2611.11306)