Harvard CS 2881 Lecture 4: Recursive Self-Improvement and AI Trajectories
Start here
The notes for Lecture 3 of Boaz Barak's Harvard CS 2881R "AI Alignment and Safety" course survey modern LLM training and inference, covering 9 pre-class readings including GPU performance, Transformer arithmetic, RLHF, policy gradients, and the SmolLM3 training recipe, and explain the mathematics of SFT and policy gradients as well as deliberative alignment safety training. The Lecture 4 class notes turn to recursive self-improvement and AI trajectories, given by Dwarkesh Patel and Daniel Kokotajlo, and, together with the pre-reading "AI 2027," propose that once AI can help improve successor systems, a feedback loop of accelerating progress will form; that scenario projects that automated AI R&D combined with recursive self-improvement could achieve ASI in 2027. The notes also point out that METR's time horizon benchmark has large uncertainty, and that benchmark scores may be affected by luck, loopholes, or test-taking strategies.
Generated by AI from reports · Updated 4 days ago
Attention and developments
Attention, hourly
Now 21, 0% in 24 hours·Peak 23 (Nov 9 03:00)
Related research
Cited in the reports
Latest developments
Progress timeline
- Boaz BarakHarvard CS 2881, Fall 2026, Lecture 4: recursive self-improvement and AI trajectories
Class notes for Lecture 4 of the Harvard CS 2881R "AI Alignment and Safety" course focus on recursive self-improvement and AI trajectories, with guest speakers Dwarkesh Patel and Daniel Kokotajlo. Drawing on the pre-reading material "AI 2027," the notes state that once AI can help improve successor systems it will form a feedback loop that accelerates progress, and that this scenario projects that automated AI R&D combined with recursive self-improvement could achieve ASI in 2027. The notes also point out that METR's time horizon benchmark has large uncertainty, and that benchmark scores may fail to reflect true capability due to luck, exploits or test-taking strategies.
- Boaz BarakBoaz Barak's CS 2881 course, Fall 2026, Lecture 3: Modern LLM Training and Inference
The notes for Lecture 3 of Boaz Barak's CS 2881 course survey modern LLM training and inference, covering 9 pre-class readings on topics including GPU performance, Transformer arithmetic, RLHF, policy gradients, and the SmolLM3 training recipe. The lecture explains why deep learning favors scalable simple methods, how GPUs shape the Transformer, and the mathematics of SFT and policy gradients along with deliberative alignment safety training. It concludes with experiments studying when natural language filler text gives models extra computation space.