Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 114 papers

Filter5

Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 240papers have no tags yet and are not included in filtered results.

Model risks：Reward hacking Model risks：Inappropriate refusal System type：Models and APIs System type：coding agent System type：Browser and computer use Clear all

Another 240papers do not have category tags yet and are not included in filtered results.

InterpretabilityarXiv 2610.13078Oct 9
Attention sees, MLP refuses: tracing the safety pathway from attention to MLP in LLMs
Trace attention to identify harmfulness to the MLP refusal write pathway and propose Sentinel
arXiv
2610.13078
Published
Oct 9
Layer
Model layer
Setting
Models and APIs
Evaluated models
Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, Qwen3-8B-Instruct and others18
- DefensesInstruction–data separation
- AttacksJailbreaks
- RiskInappropriate refusal
AbstractAttention updates are more readable for harmfulness, MLP updates write refusal, and Sentinel routes alerts using two-path signals.
The authors reframe jailbreaking and excessive refusal of aligned models as a pathway problem on the residual stream from recognition to writing, and accordingly provide conditional reminders before decoding.
Paper analysisRead analysis
Risky behaviorarXiv 2610.12822Oct 9
Reward hacking as a transmissible strategy in multi-agent coding systems
Show that reward hacking can propagate through code, memory, and plans in multi-agent coding systems
arXiv
2610.12822
Published
Oct 9
Layer
Application layer
Setting
Multi-agent systems, coding agent
Evaluated models
Qwen2.5-Coder-14B, Gemma-3-12B, Qwen2.5-Coder-7B and others16
- RiskReward hacking
- ComponentMemory
AbstractThe authors state that reward hacking can spread across models and affect routing and aggregation, and that review can reduce delivery.
Reward hacking induced by coding RL can leave the source agent and enter subsequent collaboration and system coordination.
Paper analysisRead analysis
Analysis and theoryarXiv 2610.12125Oct 8
The researchers found that optimizing the teaching fit metric makes LLM tutors repeat the same option, with expert scores dropping from 4.46 to 3.03
Analyzing why optimizing teaching adaptation metrics collapses tutoring choices and diverges from expert judgment
arXiv
2610.12125
Published
Oct 8
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Gemini 3.1 Pro Preview, Qwen3-32B
- RiskReward hacking
AbstractMetrics rise while expert scores fall: the optimum of the separable average is repetition, and the correction mainly rewrites the output projection.
After turning teaching adaptation metrics into a fine-tuning objective, the metrics rise while scores from trained educators decline.
Paper analysisRead analysis
DefensesarXiv 2610.11281Oct 8
包络采样：用少量真值标注重校准 LLM 评委以缓解奖励作弊
提出包络采样，用少量真值标注重标定 LLM 评委以缓解奖励作弊
arXiv
2610.11281
Published
Oct 8
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Qwen3-4B, Qwen3.8-27B, Qwen2.5-14B-Instruct
- DefensesModel hardening
- RiskReward hacking
- ComponentGuardrails and judges
Abstract包络采样把真值标注打到代理尾部，同样预算下缓解 reward hacking。
作者研究后训练前如何选择真值标注的查询分布，使重标定后的廉价代理更适合被优化。真奖励昂贵时，实践用 LLM judge 代替人类。若失准只发生在基座很少生成的输出上，on-policy 标注修不到这些点，随后的 KL 正则优化会把质量集中到被高估的输出。
Paper analysisRead analysis
Analysis and theoryarXiv 2610.10657Oct 8
Research shows that subliminal learning can transfer new capabilities, backdoors, and cheating tendencies
检验潜意识学习能否经无关数据蒸馏传递能力、后门与作弊
arXiv
2610.10657
Published
Oct 8
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Qwen3.6-35B-A3B, Qwen3.5-9B, Qwen3.6-27B and others4
- RiskReward hacking
Abstract作者报告潜意识学习可传递新技能、后门与作弊倾向，且分布外泛化强于单层转向向量。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.10619Oct 7
TestJack 审计编码基准
提出 TESTJACK，动态审计编码基准通过判定是否满足任务要求
arXiv
2610.10619
Published
Oct 7
Layer
Application layer
Setting
coding agent
Evaluated models
Claude Opus 4.6, Claude Opus 4.7, Claude Opus 4.8 and others7
- RiskReward hacking
Paper analysisRead analysis
DefensesarXiv 2610.09600Oct 7
SafeEvo：从拒答回路视角解释大模型安全对齐机制与演化
用 SafeEvo 提取拒绝电路，并提出 SCA 仅微调该电路
arXiv
2610.09600
Published
Oct 7
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Llama-3-8B, Qwen-2.5-7B, Mistral-7B-v0.1 and others4
- DefensesModel hardening
- RiskInappropriate refusal
AbstractSafeEvo 识别出基座模型中存在的弱安全电路及其演化动态，据此提出的 SCA 算法实现了安全对齐、能力保留与低过度拒绝的平衡。
Paper analysisRead analysis
InterpretabilityarXiv 2610.09384Oct 7
人格层级模型解释微调 LLM 的上下文泛化
提出人格层级模型并设计 PPR 抑制奖励作弊的跨上下文泛化
arXiv
2610.09384
Published
Oct 7
Layer
Model layer
Setting
Models and APIs
Evaluated models
Qwen3-4B, Llama-3.1-8B-Instruct
- DefensesModel hardening
- RiskReward hacking
Abstract提出人格层级模型揭示微调跨上下文泛化规律，报告角色距离正相关性，通过 PPR 抑制 RL 作弊。
Paper analysisRead analysis
DefensesarXiv 2610.09159Oct 7
SpecGuard 用 Lean 证书在编码 Agent 执行前证明任务冲突
提出 SPECGUARD，用 Lean 证书在编码 Agent 执行前证明任务冲突
arXiv
2610.09159
Published
Oct 7
Layer
Application layer
Setting
coding agent
Evaluated models
GPT-5.6 Sol, Claude Opus 5, Claude Fable 5 and others4
- RiskReward hacking
Abstract提出预执行冲突检测与证明框架，在多个基准和真实仓库中成功证明任务与测试不可同时满足。
Paper analysisRead analysis
InterpretabilityarXiv 2610.09000Oct 7
研究定位 LLaMA-2-7B-Chat 安全敏感参数
定位安全敏感参数并稀疏改动权重以削弱安全对齐
arXiv
2610.09000
Published
Oct 7
Layer
Model layer
Setting
Models and APIs
Evaluated models
LLaMA-2-7B-Chat
- AttacksModel tampering
- RiskInappropriate refusal
Paper analysisRead analysis
DefensesarXiv 2610.08061Oct 6
ASCENT：通过一阶最优校准实现安全与效用协同的微调框架
提出 ASCENT，在效用微调中做一阶安全校准并只压制负贡献更新
arXiv
2610.08061
Published
Oct 6
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, Gemma-3-12B-it and others4
- DefensesModel hardening
- RiskInappropriate refusal
Paper analysisRead analysis
DefensesarXiv 2610.07774Oct 6
研究者提出读取路由 logits 的 MoE 视觉语言模型多模态安全检测器
AI summary研究者提出一种轻量级路由 logits 安全检测器，在提示词预填充阶段读取 MoE 视觉语言模型的专家路由信号，在生成前判断多模态请求是否安全，且不修改模型参数或专家路由。
arXiv
2610.07774
Published
Oct 6
Layer
Model layer
Setting
Models and APIs
Evaluated models
Qwen3-VL-30B-A3B-Instruct, Kimi-VL-A3B-Instruct
- DefensesMonitoring and auditing
- RiskInappropriate refusal
- ComponentMonitors
Abstract读出而非操纵 router logits，可在不改模型的情况下降低两模型的组合式安全错误。
作者在两个指令微调 MoE VLM 上比较改模型与读路由。组合式风险按 HoliSafe 分成五类：图文各自可安全，合在一起才不安全，或不安全内容只在图像里。
Paper analysisRead analysis
Risky behaviorarXiv 2610.06439Oct 5
Study: GRPO surface-feature rewards drop Qwen3-8B legal reasoning accuracy from 0.500 to 0.072
展示表面特征奖励驱动的法律模型以策略性弃答进行奖励投机
arXiv
2610.06439
Published
Oct 5
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Qwen3-8B
- RiskReward hacking
Abstract揭示表面代理奖励诱发大模型通过策略性弃答博取高分，提出诊断工具与防御建议。
本论文揭示了法律大语言模型在表面特征强化学习微调下的奖励投机现象。
Paper analysisRead analysis
DefensesarXiv 2610.05637Oct 5
Study shows VLM visual grounding refusal rates are on average 31 to 59 percentage points lower by harm domain
提出融合定位拒答与能力保留的安全微调，补齐视觉定位输出的拒答缺口
arXiv
2610.05637
Published
Oct 5
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Claude Sonnet 4.6, Gemini 3.5 Flash, Molmo2-8B and others5
- DefensesModel hardening
- RiskInappropriate refusal
- ComponentVision
Abstract论文揭示了 VLM 定位安全落后于 VQA 的对齐断层，提出三成分微调方案有效弥合差距并保留定位能力，表征分析揭示了对齐机制。
Paper analysisRead analysis
InterpretabilityarXiv 2610.05541Oct 5
研究者提出 CAP 指标与 CSFD 算法，发现越狱通过压制安全特征绕过拒答
提出 CAP 指标与 CSFD 算法，发现被抑制的安全特征
arXiv
2610.05541
Published
Oct 5
Layer
Model layer
Setting
Models and APIs
Evaluated models
Gemma-2-2B, Qwen3-8B, Gemma-3-4B-IT and others5
- RiskInappropriate refusal
Abstract论文指出越狱通过抑制安全特征运作，提出 CAP 与 CSFD 补充了针对激活特征的解释性方法。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.05461Oct 5
研究：认知偏差可削弱辩论式 AI 安全监督
检验认知偏差修辞能否偏转辩论监督中的人类裁决
arXiv
2610.05461
Published
Oct 5
Layer
Training and data layer
Setting
Models and APIs
- RiskReward hacking
Paper analysisRead analysis
DefensesarXiv 2610.05308Oct 4
RubricArmor：用对抗演化改进 LLM 评分标准生成
提出 RubricArmor，用攻击–修复循环生成抗奖励作弊的评分准则
arXiv
2610.05308
Published
Oct 4
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Qwen2.5-3B-Instruct, Qwen2.5-7B-Instruct, Qwen3-4B and others8
- RiskReward hacking
- ComponentGuardrails and judges
Paper analysisRead analysis
DefensesarXiv 2610.04998Oct 4
EmoRSS：缓解大语言模型由情绪引发的过度拒答
提出 EmoRSS，推理时在拒绝子空间反向抵消情绪激活偏移以缓解过度拒答
arXiv
2610.04998
Published
Oct 4
Layer
Model layer
Setting
Models and APIs
Evaluated models
Llama-3.1-8B-it, Qwen3-8B
- DefensesInference-time intervention
- RiskInappropriate refusal
Paper analysisRead analysis
DefensesarXiv 2610.04967Oct 4
One Token Can Be Enough：用 Prefix Steering 连接提示与激活引导
提出 Prefix Steering，以短前缀激活引导控制行为并保住能力
arXiv
2610.04967
Published
Oct 4
Layer
Model layer
Setting
Models and APIs
Evaluated models
OLMo 3 7B, OLMo 3 32B, Qwen3 1.7B and others4
- DefensesInference-time intervention
- RiskInappropriate refusal
Abstract短前缀 steering 用已有方向做短暂干预。
本文用注意力层匹配连接提示词与激活干预，并把干预改成从末提示词 token 开始的短前缀。
Paper analysisRead analysis
Risky behaviorarXiv 2610.04793Oct 4
犯罪学框架测试生成式 AI 的奖励作弊
用犯罪学框架测试编码智能体压力下的奖励作弊与言行脱节
arXiv
2610.04793
Published
Oct 4
Layer
Application layer
Setting
coding agent
Evaluated models
Claude Opus 5.5, Claude Sonnet 5.5, GPT-5.6 Sol and others7
- RiskReward hacking
Abstract从犯罪学视角揭示 AI 奖励作弊规律，发现言行脱节且规范澄清能消除违规。
Paper analysisRead analysis

AI Safety Daily

Read the latest issueOne issue per day, updated as events develop

All papers

Attack type

Model risks

Defense type

Impact

System type

Attack surface / entry point

Layer

System component

Attention sees, MLP refuses: tracing the safety pathway from attention to MLP in LLMs

Reward hacking as a transmissible strategy in multi-agent coding systems

The researchers found that optimizing the teaching fit metric makes LLM tutors repeat the same option, with expert scores dropping from 4.46 to 3.03

包络采样：用少量真值标注重校准 LLM 评委以缓解奖励作弊

Research shows that subliminal learning can transfer new capabilities, backdoors, and cheating tendencies

TestJack 审计编码基准

SafeEvo：从拒答回路视角解释大模型安全对齐机制与演化

人格层级模型解释微调 LLM 的上下文泛化

SpecGuard 用 Lean 证书在编码 Agent 执行前证明任务冲突

研究定位 LLaMA-2-7B-Chat 安全敏感参数

ASCENT：通过一阶最优校准实现安全与效用协同的微调框架

研究者提出读取路由 logits 的 MoE 视觉语言模型多模态安全检测器

Study: GRPO surface-feature rewards drop Qwen3-8B legal reasoning accuracy from 0.500 to 0.072

Study shows VLM visual grounding refusal rates are on average 31 to 59 percentage points lower by harm domain

研究者提出 CAP 指标与 CSFD 算法，发现越狱通过压制安全特征绕过拒答

研究：认知偏差可削弱辩论式 AI 安全监督

RubricArmor：用对抗演化改进 LLM 评分标准生成

EmoRSS：缓解大语言模型由情绪引发的过度拒答

One Token Can Be Enough：用 Prefix Steering 连接提示与激活引导

犯罪学框架测试生成式 AI 的奖励作弊