Skip to content
arXiv· arXiv:2602.05164· Shoaib Ahmed Siddiqui, Eleni Triantafillou, David Krueger, Adrian Weller·Indexed by this site · Originally published

Position paper: Capability control should be an objective separate from alignment

Position: Capability Control Should be a Separate Goal From Alignment

AI summary

In a position paper published on arXiv, the researchers argue that capability control should be treated as an independent objective separate from alignment. The paper notes that alignment is typically context- and preference-driven, whereas capability control aims to impose hard operational limits on permitted model behavior, including under adversarial inducement. The authors divide capability control mechanisms covering the model lifecycle into three layers: data-based control of the training distribution, learning-based interventions at the weight or representation level, and system-based post-deployment guardrails on inputs, outputs, and actions. Because each layer used alone has its own failure modes, the paper advocates a defense-in-depth approach combining complementary controls across the full stack, and lists open challenges such as dual-use knowledge and compositional generalization.

Read originalarxiv.org