Skip to content
WeChat · Weijiqi Lixin·Indexed by this site · Originally published· Defenses →· Its place in the scan record →

Researchers propose NeuronGuard, spreading safety signals across a large number of neurons to defend against neuron pruning attacks

路易斯维尔大学大模型安全的致命漏洞被补上?NeuronGuard,把安全“散布”到整个神经网络

AI summary

Researchers propose NeuronGuard, a safety defense method for the fine-tuning stage, which by spreading safety capability across a large number of neurons reduces the attack success rate of the neuron pruning attack NeuroStrike from a baseline of 80-90% to 0.02-0.04. The method has three steps: every certain number of training steps, train a lightweight linear classifier to dynamically identify safety-critical neurons and refresh them periodically; during training, zero out the activations of these neurons, combined with KL divergence regularization, so that the remaining neurons take on the task of recognizing harmful requests and refusing to answer; then use stochastic gradient projection to reconcile gradient conflicts between safety and downstream tasks. The method adds no extra computation at inference time, but requires developers to have the full fine-tuning pipeline and cannot be directly used for black-box models where only the API is accessible.

Read originalmp.weixin.qq.com