Boaz Barak's CS 2881 course, Fall 2026, Lecture 3: Modern LLM Training and Inference

The notes for Lecture 3 of Boaz Barak's CS 2881 course survey modern LLM training and inference, covering 9 pre-class readings on topics including GPU performance, Transformer arithmetic, RLHF, policy gradients, and the SmolLM3 training recipe. The lecture explains why deep learning favors scalable simple methods, how GPUs shape the Transformer, and the mathematics of SFT and policy gradients along with deliberative alignment safety training. It concludes with experiments studying when natural language filler text gives models extra computation space.