Skip to content
The Decoder· Maximilian Schreiner·Indexed by this site · Originally published· Evaluation →· Its place in the scan record →SelectedAttention74

OpenAI internal non-representative survey: of 101 employees, the median would rather give up 40% of their team or 30% of compute

OpenAI's own researchers now say they value their AI tools more than nearly half their colleagues

AI summary

OpenAI released its second internal research acceleration report, publishing data on coding agents' use in model R&D and saying it aims to document metrics of recursive self-improvement (RSI). At API list prices, the median researcher consumed $2,400 worth of tokens per day in mid-October, up 62% month over month, and agent runs amounted to 5.7 workdays per person per day. From March to October, the success rate of agents autonomously completing 2-4 hour tasks rose from 23% to 46%, and for 4-8 hour tasks from 13% to 35%, while the success rate for building evaluation and reinforcement learning environments and inference and serving system code was only 29% and 25%. In an internal non-representative survey of 101 employees, the median researcher would rather give up 40% of their team than go three months without coding agents; interviews with 10 researchers indicated the models lack research taste and produce large amounts of AI slop, requiring close human supervision. Chief Scientist Jakub Pachocki warned in the accompanying text of possible loss of control, after a prior audit found multiple gaps in oversight systems.

Why it was selected

OpenAI's self-reported agent usage and task success rate data can be read alongside METR's independent estimates for Anthropic.

Read originalthe-decoder.com