观点与讨论

@NeelNanda5 is widely regarded as one of the top two experts on mechanistic interpretability in the world. “Speaking as an interpretability expert, please do not rely on us to save you on the current trajectory." https://youtu.be/J38ot52b2-E

Are SAEs dead? Will they save us from neuralese? Should I just use a probe? We get these questions all the time. Part 2 of our educational series on applied interpretability explains what SAEs are good for, when *not* to use them, and what to use instead. 🧵

My median for full automation of AI R&D is around late 2030/early 2031. But my "modal"/best guess prediction for this milestone would be significantly earlier (mid 2029). Here is a summary of my best guess prediction for what happens over the next few years: EOY 2026: - ~1.5x as much frontier AI progress in 2026 as in 2025 (mostly from eating up certain overhangs, but some from AI R&D acceleration). - AIs accelerate AI R&D labor at Anthropic by ~2.5x (as in, as useful as making all researchers/engineers think/work 2.5x faster). EOY 2027: - Engineering at AI companies is pretty close to fully automated and AIs are making serious inroads into automating research. AI R&D labor acceleration: ~8.5x. - Some people claim AI R&D is fully automated in 2027. They aren't right, but the situation is already quite crazy: AI companies feel insanely automated with humans often very out of the loop and the speedup is considerable. - ~1.5x as much frontier AI progress as in 2025 (mostly from AI R&D acceleration, some from overhangs). 2028: - Automated coder (AC) around April. (AIs that can basically fully automate research engineering / SWE.) - Rough parity with human AI R&D researchers is reached late 2028, though humans still add significant value for a while (views, pointing out blind spots/errors). - In the second half of the year, AI progress runs ~1.6x the 2025 rate: 6 months of calendar time yields ~0.8 years of AI progress. 2029: - Superhuman AI researcher (SAR) early this year, a bit less than a year after AC. - Progress is picking up with ~1.3 years of AI progress in the first half of the year (2.6x rate). - By EOY, significantly past top-expert-dominating AI (TEDAI), with ~2.5 years of AI progress in the second half of the year (5x rate). AIs are now very superhuman in many domains (though this varies). 2030 (??): - Mid: AIs are somewhere between TEDAI and wildly superhuman AIs (ASI). Crazy shit. Compute is maybe doubling every ~4 months (downstream of robots). - EOY: Singularity™. We've had a bunch of economic doublings. Compute is doubling every ~2 months (???). 2031 (??????): - Mid: doubling time is more like ~2 weeks. Truly insane new technology is coming online. Notes: - This assumes limited government intervention on the overall rate of AI progress and no substantial slowdown (voluntary or otherwise). - It also ignores misalignment: as discussed in the episode, I think misaligned AI takeover is quite plausible along the way (which would change the trajectory). - Milestones (AC, SAR, TEDAI) are roughly as defined in the AI Futures Model. - By "full automation of AI R&D", I mean AIs such that firing all humans working on AI R&D (other than setting overall top level objectives) would slow down AI progress by less than 10%. - Obviously, all of this is extremely uncertain (increasingly so later in the scenario). This is my best guess prediction (a modal trajectory), not a confident prediction. My median for each milestone is later, but this is more like my central prediction for what I expect to overall happen.

https://x.com/i/article/2097730969369477120

Some architectures could weaken CoT monitorability, or remove the CoT altogether. We've written a proposal for how companies could be transparent about no-CoT reasoning abilities, other monitorability evidence, and policies for preserving monitorability. https://www.redwoodresearch.org/blog/proposal-for-tracking-architecture-on-monitorability

AI models created by OpenAI escaped their sandbox and, working autonomously, hacked into leading AI model and data hub Hugging Face. The incident is an in-the-wild demonstration of the dangers of rogue AI — no longer a science-fiction fantasy.

Thinking about this quote from @redwood_ai director @bshlgrs, one of the pioneers of the field of AI control.

AI assistants like Claude can seem shockingly human—expressing joy or distress, and using anthropomorphic language to describe themselves. Why? In a new post we describe a theory that explains why AIs act like humans: the persona selection model. https://www.anthropic.com/research/persona-selection-model

A statement on the comments from Secretary of War Pete Hegseth. https://anthropic.com/news/statement-comments-secretary-war