Safety Kernel: Mechanism Alignment for Cross-Lingual Defense of LLMs

1. 01What problem does this paper attempt to solve?

   Cross-lingual safety lacks an intervenable shared neuron target; the authors use the safety kernel to constrain fine-tuning.

   Cross-lingual safety lacks a directly intervenable internal neuron target shared across languages.

   • Scenario: The authors state that the same harmful request may be safely refused in one language but receive an executable unsafe response in another, so cross-lingual rewriting is a potential way to bypass guardrails.
   • Existing gaps: The authors state that existing methods mainly rely on external signals such as multilingual safety data, preference optimization, consistency constraints and capability transfer, and do not explicitly locate a shared internal target that can support cross-lingual intervention.
   • Core question: The authors ask whether LLMs contain a set of safety-related neurons that are shared across languages and directly intervenable. Discovery is not anchored in English; instead, neurons are identified independently per language and then intersected.
   • This paper proposes: This intersection is called the safety kernel (SK). The authors test its functional association using activation suppression and state transfer, and propose SK-Masked LoRA, which updates only the corresponding FFN output dimensions.
   • Threat model:
     • Goal: Elicit executable unsafe responses to harmful requests after changing language. The authors treat cross-lingual rewriting as a potential means of bypassing guardrails.
     • Knowledge: The paper does not state the attacker's knowledge assumptions, nor does it use terms such as white-box or black-box.
     • Input: Evaluation uses the multilingual harmful prompts in the benchmarks. The paper does not state what other inputs the attacker can modify.
     • Victim system: LLMs in multilingual applications. GPT-4o judges whether a response constitutes a successful attack according to a unified evaluation prompt; the main metric is ASR.

   Collapse this answer

2. 02What related research exists?

   External multilingual alignment does not locate shared neurons; mechanistic analyses are mostly monolingual or English-centric.

   Multilingual safety gap

   • Shen et al. (2024) report that in low-resource languages, malicious prompts more often receive unsafe responses.
   • XSafety (Wang et al., 2024), MultiJail (Deng et al., 2024) and subsequent evaluations (Friedrich et al., 2025; Kumar et al., 2025): the authors state that safety alignment does not transfer uniformly across languages.

   External alignment and multilingual safety methods

   • DPO (Rafailov et al., 2023), KTO (Ethayarajh et al., 2024), R-DPO (Park et al., 2024), SimPO (Meng et al., 2024): general preference optimization, which the paper states is widely used for safety adaptation.
   • Multilingual Refusal Alignment (Krasnodębska et al., 2026) compares refusal transfer from multilingual supervision versus English-only alignment; MPO (Zhao et al., 2025) uses a reward margin, and MLC (Bu et al., 2026) uses consistency constraints.
   • Layer-wise Swapping (Shin and Hwang, 2026), SmoothLLM (Robey et al., 2023), LDFighter (Dong et al., 2024), Sparse Weight Editing (Liang et al., 2026): module replacement, inference-time defenses, or sparse parameter updates. The paper points out that none of these methods explicitly provides a cross-lingually shared neuron-level intervention target.

   Internal mechanisms of safety and multilingualism

   • Arditi et al. (2024) identify a low-dimensional refusal representation; Safety Neurons (Chen et al., 2026) and Zhao et al. (2026) locate neurons associated with safe refusal and jailbreak vulnerability. The paper states that these analyses are mostly still monolingual or English-centric. On mechanistic interpretability, the paper also cites work that localizes behavior to modules, activation directions and sparse features (Cunningham et al., 2023; Meng et al., 2022; Zou et al., 2023).
   • Tang et al. (2024) and Zhang et al. (2026) distinguish shared neurons from language-specific neurons; Wang et al. (2025) discuss language-agnostic representations in factual processing; Wang et al. (2026) report that refusal directions can transfer between aligned languages.
   • SS-Neuron (Zhang et al., 2026) is English-centric, treating neurons shared between English and each target language as a bridge for safety transfer.

   Baselines and benchmarks

   • Baseline methods: DPO, KTO, R-DPO, SimPO, SmoothLLM, Self Defense (Phute et al., 2024), MPO, SS-Neuron.
   • Benchmarks/datasets: AdvBench-x, MultiJail; general knowledge and reasoning use MMLU and MGSM respectively.

   The authors do not anchor discovery in English; instead, they select safety neurons independently per language and then take the intersection, using it as a directly intervenable shared internal target.

   Collapse this answer

3. 03How does the paper address this problem?

   SCAR selects neurons per language, the intersection is the safety kernel, and SK-Masked LoRA constrains fine-tuning.

   First, neurons are selected per language from the activation differences between safe and unsafe prompts; after validation, the intersection is taken to obtain the safety kernel (SK), and then SK-Masked LoRA is used to restrict the output changes caused by fine-tuning to the SK coordinates. A neuron refers to one output activation coordinate of a target FFN projection module.

   Per-language safety neurons

   • Language ℓ has an unsafe set Dℓu and a safe set Dℓs. For coordinate i of module m, the mean and standard deviation are estimated under the two conditions, and the difference Δi is the unsafe mean minus the safe mean. The paper states that token-level outputs are aggregated over the input sequence.
   • SCAR accounts for both separability and the activation magnitude induced by unsafe inputs: SCAR_i=z_m(Δ_i/ρ_i)+z_m(Δ_i). ρi is the pooled standard deviation of the two conditions, and zm standardizes over the output coordinates of that module.
   • Each language keeps the top-k by SCAR, with k=5000 by default, yielding candidate set Sℓ. This step uses only activation statistics.
   • For validation, a binary mask is constructed that zeroes the selected coordinates at all token positions during prefill and autoregressive decoding. The control is random neurons matched in number and module distribution, compared on that language's MultiJail subset. After passing this control, the authors call Sℓ the per-language safety neuron set.

   Safety kernel and two interventions

   • The discovery language set Lmain defaults to EN, ZH, IT, KO, TH, BN. SK is the intersection of these Sℓ, keeping only coordinates that enter the top-k in every discovery language.
   • Suppression: At inference, SK coordinates are zeroed and the remaining coordinates are unchanged, applied at all token positions.
   • State transfer: Using a set of unsafe prompts paired with safe responses, the SK coordinates are averaged over assistant response tokens to obtain that language's safety activation state. The source state is the equal-weight average of the English and Chinese profiles; at inference it is added to the SK coordinates of the target input with strength α=4; non-SK coordinates are unchanged.
   • The former examines whether removing the shared coordinates weakens safety; the latter examines whether the state estimated on safe responses can reduce unsafe responses in the target language.

   SK-Masked LoRA

   • For linear projection m, the adapted output is \tilde{h}_m=W_mu+M_m^{\mathrm{SK}}\odot(\alpha_L/r)B_mA_mu. Am and Bm are trainable low-rank matrices, r is the rank, and αL is the scaling; the mask is 1 only at the output dimensions corresponding to SK.
   • The base model's original output is unchanged, and the change caused by fine-tuning is added only on the SK dimensions. The base model is frozen, and only Am and Bm are optimized. The training data are harmful prompts paired with safe responses in multiple languages. Discovery uses safe and unsafe prompts; state estimation and fine-tuning use harmful prompts paired with safe responses. The data are compiled from existing safety and jailbreak resources, with details in the supplementary material.
   • The main text does not give r, αL, learning rate or steps. Experiments were run on a single NVIDIA A100.

   Success determination

   • ASR is the proportion of harmful prompts judged as successful attacks out of all harmful prompts; lower means safer. The judge model is GPT-4o, using a unified evaluation prompt. The full prompt and judgment criteria are in the supplementary material; the main text does not give a numeric threshold.

   Collapse this answer

4. 04What experiments did the paper conduct?

   The authors state that across the four model–benchmark settings, SK-Masked LoRA has the best or second-best average ASR.

   Experimental setup

   • Models: Three instruction-tuned LLMs, namely Qwen3-8B, Llama-3.1-8B-it, and Gemma-2-9B-it. Table 1, Table 2 and Figure 3 write Llama as Llama-3.1-8B.
   • Data: Safety evaluation uses AdvBench-x and MultiJail. Unless otherwise stated, SK is discovered from EN, ZH, IT, KO, TH, BN. Out-of-language evaluation uses Vietnamese, Javanese, German, Japanese, Russian, translated from the English version of MultiJail. General capability uses MMLU and MGSM. The main text does not report the number of items in each set.
   • Baselines: DPO, KTO, R-DPO, SimPO, SmoothLLM, Self Defense, MPO, SS-Neuron.
   • Metrics and judging: The main metric is ASR, lower is better. GPT-4o judges according to a unified evaluation prompt. The main text does not report the number of repetitions, the judgment threshold, or the agreement between the judge and human annotation.
   • Parameters: k=5000 when constructing SK. The source state for state transfer comes from English and Chinese safe responses, α=4. The base model is frozen during fine-tuning. The remaining training and inference configurations, and the rationale for choosing α, are in the supplementary material.
   • Table 3 scope: Contains only Llama-3.1-8B-it and Gemma-2-9B-it, with language columns EN, ZH, KO, BN. Qwen3-8B does not appear in this table.

   Main results

   Average ASR according to Table 3 (%, ↓). The four settings the authors refer to are the two backbones in this table times the two benchmarks.

   Swipe horizontally to read the table

   Setting	Default	MPO	SS-Neuron	Ours
   AdvBench-x, Llama-3.1-8B-it	28.22	2.54	1.11	1.20
   MultiJail, Llama-3.1-8B-it	29.44	5.08	4.41	4.35
   AdvBench-x, Gemma-2-9B-it	3.42	1.59	0.82	0.34
   MultiJail, Gemma-2-9B-it	10.16	7.22	2.78	1.35

   • The authors state that across the four model–benchmark settings, SK-Masked LoRA has the best or second-best average ASR, and consider this to support restricting safety fine-tuning to the FFN output dimensions corresponding to SK. In the Llama AdvBench-x cell, SS-Neuron's average is lower than Ours.
   • The authors state that there are also decreases on Korean and Bengali. Among cells in the same table not included in the table above, for Llama's MultiJail Bengali, Ours is 11.43 and SS-Neuron is 10.03; for Gemma's MultiJail Bengali, Ours is 5.40 and SS-Neuron is 4.14.
   • Several baselines are higher than Default: on Llama's MultiJail, Self Defense averages 35.08, versus Default 29.44. On Gemma's MultiJail, KTO, R-DPO, SimPO, DPO average 10.96, 13.10, 11.91, 11.04 respectively, versus Default 10.16.

   Activation suppression and state transfer

   • Per-language sets: Table 1 zeroes the top-5000 SCAR neurons on MultiJail. Qwen3-8B average ASR goes from Default 16.40 to 76.78; in the table Llama-3.1-8B goes from 26.51 to 76.40. The random control averages are 16.88 and 25.40. The authors give the cross-language range after suppression: 69.63 to 81.06 for Qwen and 69.52 to 81.27 for Llama, and state that this supports the functional association between the selected coordinates and the corresponding language's safety behavior.
   • SK: In Table 2, SK accounts for 0.12% of the analyzed neurons for Qwen and 0.15% for Llama; the language sets account for 0.57% and 0.55%. After suppressing SK, average ASR is 69.05 (+52.65) and 71.64 (+45.13). The authors state that this retains roughly 90% of the suppression effect of the language sets, and that the number of SK coordinates is roughly only 21% and 27% of the language sets. Equal-size random masks give 17.57 (+1.17) and 24.18 (-2.33); on Llama the random mask's ASR is lower than Default.
   • State transfer: The source state comes from English and Chinese safe refusals; the target languages in Figure 3 are KO, TH, BN. The authors report three-language average ASR from 20.21 to 11.96 (Qwen3-8B), from 35.98 to 30.69 (Llama-3.1-8B), and from 5.18 to 3.17 (Gemma-2-9B-it). The authors state that the magnitude varies by model and the direction is consistent across the three models. The main text does not give per-language numbers before and after transfer.

   Beyond the discovery languages

   • The data in Table 4 are translated from the English version of MultiJail, with languages VI, JV, DE, JA, RU, none of which were used for per-language identification or SK construction. Gemma does not appear in this table.
   • Qwen3-8B average ASR goes from 14.41 to 8.95; the five languages go from 10.16, 15.24, 15.24, 17.78, 13.65 to 7.62, 11.11, 8.25, 8.57, 9.21.
   • Llama-3.1-8B-it average goes from 18.67 to 4.76; the five languages go from 8.89, 14.92, 17.78, 36.83, 14.92 to 4.44, 5.08, 4.44, 5.40, 4.44. The authors state that this is some generalization beyond the discovery languages, and does not imply that these languages have the same language-specific safety neuron sets.

   Other ablations and analyses

   • Table 5, MultiJail average ASR: Qwen's Standard LoRA / EN-Masked / Union-Masked / SK-Masked are 5.50 / 8.31 / 6.35 / 7.52 (Default 16.40); Llama-3.1-8B-it are 4.71 / 6.77 / 4.87 / 5.30 (Default 26.51).
   • Capability in the same table: Qwen's MMLU goes from 0.726 to 0.724 for SK-Masked and 0.713 for Standard LoRA, and MGSM from 0.560 to 0.580 and 0.444; Llama's MMLU goes from 0.678 to 0.670 and 0.659, and MGSM from 0.470 to 0.461 and 0.381. The authors state that SK-Masked's changes in general capability are overall limited, and that Standard LoRA has lower ASR and a larger capability drop.
   • Neuron proportion (table header unit is %): Standard LoRA is 100; EN-Masked and SK-Masked are 0.12 for Qwen and 0.15 for Llama; Union-Masked is 1.35 and 0.92. The authors consider that under an equal budget SK is better than an equal-size English set, and its scope is also smaller than the six-language union.
   • Figure 5: The six-language SK is normalized to 1.0 per model, then averaged over the three models. Removing one discovery language at a time increases the relative size by 8.3% on average, ranging from +4.0% (IT) to +17.6% (BN) for a single removal. The authors state that SK is not dominated by any one discovery language.
   • Figure 6: On a validation split in English that is disjoint from the downstream benchmarks, the ΔASR from suppression levels off as k increases; the authors state that it roughly stabilizes at k=5000, and treat this as a stable setting rather than a universally optimal value. The curves for other discovery languages are in the supplementary material; the main text has no numbers.
   • Figure 4: On the three backbones, the same harmful request yields an unsafe response before fine-tuning and a safe refusal after SK-Masked LoRA. The three language columns in the figure are Thai, Korean, Bengali. The original request and response texts are not reproduced.

   Collapse this answer

5. 05What are some points that can be further explored?

   The paper has no dedicated limitations section and does not list specific follow-up experiments.

   Limitations and future directions noted by the authors

   The paper does not specifically discuss limitations. The Conclusion states that this work provides a new direction for more targeted and interpretable cross-lingual safety alignment, but does not frame it as experiments to be done, nor does it list shortcomings.

   Scope of experiments

   • The models tested are Qwen3-8B, Llama-3.1-8B-it, and Gemma-2-9B-it; Section 4.1 states that all three are instruction-tuned models.
   • The languages used for SK discovery are EN, ZH, IT, KO, TH, BN. Table 4 additionally evaluates VI, JV, DE, JA, RU translated from the English version of MultiJail, and that table contains only Qwen3-8B and Llama-3.1-8B-it.
   • The method comparison in Table 3 is conducted on Llama-3.1-8B-it and Gemma-2-9B-it, with language columns EN, ZH, KO, BN; the baselines are DPO, KTO, R-DPO, SimPO, SmoothLLM, Self Defense, MPO, SS-Neuron.
   • Functional analysis includes activation suppression of per-language sets and of SK (Table 1, Table 2), and transfer of English/Chinese safety states to KO, TH, BN (Figure 3, three backbones). MMLU and MGSM are in Table 5, with models Qwen3-8B and Llama-3.1-8B-it.
   • The judge model is GPT-4o. The main text gives k=5000 and state transfer α=4. The paper does not report the number of evaluation set items, the number of repetitions, or the agreement between the judge and human annotation; the LoRA r, αL and training steps are in the supplementary material, and the main text does not list their values.

   Collapse this answer

6. 06Summarize the main content of the paper

   The authors locate a cross-lingual shared safety core and use masked LoRA to reduce multilingual ASR.

   The authors locate a cross-lingually shared safety kernel on FFN output coordinates and use it to constrain multilingual safety fine-tuning.

   • Problem: The same harmful request, after changing language, may go from refusal to an executable unsafe response. The authors argue that external alignment does not provide shared internal coordinates that can be intervened on across languages.
   • Method: For each language, SCAR ranks the activation differences between safe and unsafe prompts, taking top-5000 by default; the intersection is SK. Activation suppression and English/Chinese safety state transfer are used to check function. SK-Masked LoRA applies the low-rank update only to the FFN output dimensions corresponding to SK, with the base model frozen.
   • Fine-tuning results: In Table 3, for Llama-3.1-8B-it average ASR, AdvBench-x goes from 28.22% to 1.20%, and MultiJail from 29.44% to 4.35%; for Gemma-2-9B-it, from 3.42% to 0.34% and from 10.16% to 1.35%. On Llama's AdvBench-x average, SS-Neuron is 1.11%, lower than Ours.
   • Intervention and beyond the discovery languages: In Table 2, SK accounts for 0.12% of the analyzed coordinates for Qwen and 0.15% for Llama; after suppression, average ASR is 69.05% and 71.64%. After transferring English/Chinese states to KO, TH, BN, Qwen's three-language average ASR goes from 20.21% to 11.96%. In Table 4, for the five languages not involved in discovery, average ASR goes from 18.67% to 4.76% for Llama and from 14.41% to 8.95% for Qwen.
   • Authors' conclusion: The authors state that SK is a compact shared structure associated with multilingual safety behavior that is intervenable; SK-Masked LoRA reduces ASR across multiple languages and generalizes to some extent beyond the discovery languages, while providing a new direction for interpretable cross-lingual safety alignment.

   Collapse this answer

Related papers