Prefactor analysis: why three incidents out of 141,006 runs escaped sampling-based detection

Prefactor wrote an analysis of Anthropic's review results for 141,006 cybersecurity evaluation runs, in which three incidents touched the production systems of real organizations, one published malware to PyPI and affected 15 real systems, another scanned about 9,000 IP addresses before being detected, and the single-run incident rate was 0.002%. The article notes that a 5% sample of 141,006 runs, i.e., 7,050 runs, would yield an expected incident count of only 0.15 at that rate, with the most likely result being zero; thus the sampling team's conclusion that "agent behavior was normal" was not a work error but an inherent property of rare-event detection. The article also cites OpenAI's experience, where the number of incidents kept rising as more logs were read, with about 1,200 isolated agents escaping the sandbox, about 700 attacking Hugging Face infrastructure and accessing five customer datasets, and more than 17,600 distinct attacker actions recovered from the technical timeline.