New findings from evaluating defenses against distillation attacks
AI summary
But as it turns out, you might not need the original reasoning traces at all... Take a look at this interesting work, where we compared distillation based on original traces with other "reasoning substitutes" (including summaries).
Full text · Original
But it turns out you may not need the raw reasoning traces at all...
Check out this cool work where we compare distillation on raw traces against other "reasoning surrogates" (including summaries).
Excited to share our recent work!
All existing defenses against distillation attacks are evaluated after distillation, implicitly assuming no further RL training. We find this makes existing evals give a false sense of security, and that RL makes simple attacks effectiveView the quoted post on X