Skip to content

Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 147 papers

Another 300papers do not have a research type yet and are not included in these categories.

  1. Attention sees, MLP refuses: tracing the safety pathway from attention to MLP in LLMs

    Trace attention to identify harmfulness to the MLP refusal write pathway and propose Sentinel

    Published
    Evaluated models
    Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, Qwen3-8B-Instruct and others18
    AbstractAttention updates are more readable for harmfulness, MLP updates write refusal, and Sentinel routes alerts using two-path signals.

    The authors reframe jailbreaking and excessive refusal of aligned models as a pathway problem on the residual stream from recognition to writing, and accordingly provide conditional reminders before decoding.

    Paper analysisRead analysis
  2. GPT-2-Small, Gemma-2-2B, and Llama-3.1-8B have interpretable narrative mechanisms

    Using NVC to locate and ablate narrative-related components in language models

    Published
    Evaluated models
    GPT-2-Small-124M, Gemma-2-2B, Llama-3.1-8B
    AbstractNeurons and SAE features related to narrativity across three models, when ablated, changed some narrative-related behaviors.

    The authors connect a three-dimensional definition from narratology to mechanistic interventions, examining whether components related to narrativity in pretrained language models participate in prediction, generation, and downstream tasks.

    Paper analysisRead analysis
  3. Neural networks are more modular than single neurons suggest

    Using a three-level nested decomposition to test whether mixed-selectivity networks can be functionally separated on subspaces

    Published
    Evaluated models
    Pythia 410M, Pythia 160M, GPT-2 XL and others18
    AbstractUnit non-specialization and subspace separability can coexist.

    The authors propose a nested definition of modularity to separate "neurons not being specialized" from "computation not being separable."

    Paper analysisRead analysis
  4. The paper proposes an alignment generalization prediction task, using activation representations to predict the cross-context effects of fine-tuned values.

    Comparing the predictive power of value representations for alignment generalization after single-value fine-tuning

    Published
    Evaluated models
    Olmo-3-7B, Olmo-3-32B, Qwen-3-8B-Base and others4
    AbstractActivation representations can predict cross-value generalization before training and relate to robustness across multiple value objectives.

    The authors define "how single-value fine-tuning changes compliance on untrained values" as a task predictable before training, and use value representations to do this.

    Paper analysisRead analysis
  5. Study: legal reasoning models citing legal provisions does not equal relying on legal provisions.

    Uses premise permutation and hidden-layer tracing to audit whether legal chain-of-thought truly relies on the cited legal provisions.

    Published
    Evaluated models
    Qwen3-8B, Qwen3-14B, Llama-3.1-8B and others7
    AbstractUsing counterfactual substitution of authorities and internal commitment tracing, the authors find that legal CoT rulings depend on authorities only to a limited extent and are vulnerable to prompt injection manipulation.

    This study conducts an empirical audit of the causal truthfulness of legal citations in chain-of-thought reasoning of large language models in the legal domain.

    Paper analysisRead analysis
  6. 用路由稀疏自编码器解耦语音编码器中的语言学与副语言学信息

    提出路由 TopK SAE,把语音编码器稀疏码拆成语言与副语言通路

    Published
    Evaluated models
    SPEAR XLarge, WavLM Large
    Abstract路由 SAE 在冻结 SPEAR 与 WavLM 上分开语言和副语言因子,探针与交换支持通路分工。

    TopK SAE 的稀疏码被分成语言通路与副语言通路,用来分开冻结语音编码器里同时存在的音素信息和说话人、情感、韵律。只重构不能得到作者选定的这一分解。语言通路由音素识别监督,副语言通路由说话人识别、情感分类和韵律预测监督,作用在对方通路上的梯度反转对抗用来压低交叉因子。

    Paper analysisRead analysis
  7. SafeEvo:从拒答回路视角解释大模型安全对齐机制与演化

    用 SafeEvo 提取拒绝电路,并提出 SCA 仅微调该电路

    Published
    Evaluated models
    Llama-3-8B, Qwen-2.5-7B, Mistral-7B-v0.1 and others4

    AbstractSafeEvo 识别出基座模型中存在的弱安全电路及其演化动态,据此提出的 SCA 算法实现了安全对齐、能力保留与低过度拒绝的平衡。

    Paper analysisRead analysis
  8. U-Space:揭示语言模型不确定性何时与为何产生

    提出 U-SPACE 与 U-LENS,从残差流估计推理轨迹不确定性

    Published
    Evaluated models
    Gemma-4-31B-it, Qwen3.5-27B, Magistral-Small-2507
    AbstractU-SPACE 与 U-LENS 用语义方向和预测熵读推理不确定性,长度控制后仍高于所比基线。

    U-SPACE 与 U-LENS 是面向推理模型的无训练读出:用语义方向定位不确定性的形式,并用一个轨迹分数区分更可能错的回答。

    Paper analysisRead analysis
  9. 研究:LLM 潜在空间去偏方向主要编码置信度而非公平性

    分析 LLM 去偏激活方向,发现其编码置信度而非公平性

    Published
    Evaluated models
    Llama-3.1-8B, Llama-3.1-8B-Instruct, Falcon3-7B-Base and others8

    Abstract论文揭示激活引导去偏见方向主要编码模型置信度而非公平性,偏见评分下降实为诱发弃答或使概率均分的假象。

    Paper analysisRead analysis
  10. AnyBottle:只保留真正需要的概念,构建紧凑任务专属概念瓶颈模型

    提出 ANYBOTTLE,按教师残差从无监督概念池筛选紧凑概念瓶颈

    Published
    Evaluated models
    CLIP-DINOiser, Gemma-2-2B, Gemma-3-1B-it
    AbstractANYBOTTLE 用教师残差选出紧凑概念集,并以 nested dropout 按样本调节概念预算。

    ANYBOTTLE 是一套用冻结骨干和无监督概念池构建任务专用概念瓶颈的配方,目标是去掉概念标注后仍保持小词表。

    Paper analysisRead analysis
  11. Moshi 全双工语音模型的情感引导遵循几何结构而非标签

    用均值差激活转向检验全双工语音模型的情感方向几何

    Published
    Evaluated models
    kyutai/moshika-pytorch-q8
    Abstract转向跟方向几何走而不跟标签走:sad 可单独对准,高唤醒情感共享方向。

    Moshi 上的情感激活转向跟方向之间的几何关系走,不跟被请求的标签走。

    Paper analysisRead analysis
  12. 研究:LLM 比人类更易产生错觉模式感知并导致虚假推理

    比较 LLM 与人类的错觉模式知觉,并用 SAE 分析内部表示

    Published
    Evaluated models
    GPT-3.5, GPT-4o, GPT-5 and others9
    Abstract作者称多数 LLM 的错觉模式知觉强于人类,并把行为联系到频率印象与分析特征。

    作者研究 LLM 是否会在证据不支持时构造关系,并将这一失败称为错觉模式知觉,用来区别偏差、幻觉和鲁棒性。

    Paper analysisRead analysis
  13. TRACE:仅凭 checkpoint 更新审计大语言模型有害目标

    提出 TRACE,仅凭检查点更新审计有害合规微调目标

    Published
    Evaluated models
    Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, Qwen3-8B and others6
    Abstract作者称有害合规 SFT 在检查点更新中留下可排序坐标,并据此做只读权重审计。

    作者在检查点更新里寻找有害合规 SFT 的目标组成,并把读数做成不查询模型、不看未知训练数据的审计分 TRACE。

    Paper analysisRead analysis
  14. BeFOND 将稀疏自编码器的推断与学习统一为自然梯度流

    提出 BeFOND,将稀疏自编码器的推断与学习统一为自然梯度流

    Published
    AbstractBeFOND 把推断与学习写成自然梯度流,并在合成词典恢复和 Gemma 特征指标上报告更高分数。

    BeFOND是无编码器的迭代稀疏编码模型,把推断和词典学习写成同一变分自由能上的自然梯度流,用来恢复重叠或很少激活的特征。问题在于摊销编码器的一次前向难以处理 superposition,而在已学词典上只改推断也填不上恢复差距。推断误差会进入词典更新。

    Paper analysisRead analysis