Skip to content
Redwood Research· Alek Westover·Indexed by this site · Originally published· Governance →· Its place in the scan record →SelectedAttention33

Redwood Research proposes capability activity definition standards for limiting frontier AI progress by compute

How should we define “capabilities activities” for the purpose of pacing?

AI summary

Redwood Research published a set of capability activity definitions for use in proposals to slow frontier AI progress by limiting compute deployment, with implementation paths possibly including legislation or voluntary commitments. The definition by default counts all AI company activities as capability activities, with only four exceptions: idle compute, narrow product development, serving customers, and narrow safety activities, provided they do not improve general model capabilities. The narrow safety activity whitelist covers alignment evaluations, company safety testing, five categories of safety research (interpretability, CoT informativeness, model persona, model organisms, alignment theory), runtime monitoring and abuse classifiers, and incident investigation, where safety research must publicly disclose project plans and results reports; dual-use safety work such as scalable oversight, reward models, dangerous capability evaluations, constitutional AI, and jailbreak robustness training still counts as capability activities. The authors argue it is better to misclassify as capability activity than to miss it, and call for third-party auditors to measure and publish each company's compute allocation; the text mentions that Anthropic has published an estimate of its safety research share, but the definition is opaque and unaudited.

Why it was selected

The original source proposes an adjudicable classification standard for capability activities for compute rate-limiting proposals, and discusses the boundary-drawing between narrow safety activities and dual-use safety work.

Related papers
Read originalblog.redwoodresearch.org