What is a model extraction attack? How distillation attacks steal AI models through APIs

Model extraction (distillation) attacks let attackers query a deployed model only through a public API, then use input/output pairs to train a local model that replicates its functionality. In February 2026, Anthropic stated that three AI labs used about 24,000 fake accounts to extract more than 16 million queries from Claude and trained imitation models on them. In 2026, such attacks against Claude and Gemini also induced the models to output chain of thought step by step.