Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 142 papers

Filter1

System type

Models and APIs118 LLM applications3 AI Agent22 Embodied systems and robotics6

Attacker access

Black-box11 White-box13

Attack surface / entry point

User input32 Web pages and interfaces6 Documents and messages3 Tool responses6 Images, audio and video5 Training data21 Weights and gradients12

Layer

Model layer15 Application layer15 Training and data layer110

System component

Guardrails and judges11 Reasoning chains11 Fine-tuning and open weights18 Vision12 Audio5 Image generation5 Multimodal22

Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 305papers have no tags yet and are not included in filtered results.

Defense type：Model hardening Clear all

Another 305papers do not have category tags yet and are not included in filtered results.

DefensesarXiv 2610.12602Oct 9
Self-Preservation in Language Models
Proposes Task Bank self-replay to mitigate held-out forgetting in continual instruction tuning
arXiv
2610.12602
Published
Oct 9
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Qwen3-0.6B, Qwen3-1.7B, Qwen3-4B and others4
- DefensesModel hardening
AbstractThe authors claim self-replay can reduce forgetting outside the list, and that on the 4B model new task information helps with target selection, while smaller models do not.
The authors study how language models, when learning new tasks, can use their own knowledge for broader self-preservation: given an incomplete list and forgetting that extends beyond the list, can the model generate replay and help choose targets.
Paper analysisRead analysis
DefensesarXiv 2610.12432Oct 9
FAITH: A feasibility-aware safety filtering reinforcement learning framework for high-dimensional systems
Proposes the FAITH safe filtering reinforcement learning framework, training high-dimensional systems to optimize task reward under constraints
arXiv
2610.12432
Published
Oct 9
Layer
Training and data layer
Setting
Embodied systems and robotics
- DefensesModel hardening
- RiskDangerous agent actions
AbstractFAITH separates safety filtering from reward optimization.
FAITH is a model-free safe filtering reinforcement learning framework for high-dimensional systems: safety correction and task reward reside in separate modules.
Paper analysisRead analysis
DefensesarXiv 2610.12233Oct 9
ReSI: A recursive safety improvement framework for attack resistance and unknown-risk resistance.
Proposes ReSI, which recursively improves model safety alignment using automatic red teaming and recipe search.
arXiv
2610.12233
Published
Oct 9
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Qwen3.6-27B, Ministral-3-14B-Reasoning-2512, DeepSeek-V4-Flash-0731 and others4
- DefensesModel hardening
- AttacksJailbreaks
Paper analysisRead analysis
DefensesarXiv 2610.11352Oct 8
RL-ARC：用推理引导的不确定性校准大型推理模型
提出 RL-ARC，用推理置信度在强化学习中校准推理模型
arXiv
2610.11352
Published
Oct 8
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Qwen2.5-7B, Qwen3-8B, Llama-8B distilled from DeepSeek-R1
- DefensesModel hardening
- ComponentReasoning chains
AbstractRL-ARC 用推理置信度调节答案置信度，作者称可少牺牲精度并改善 ID 与 OOD 校准。
RL-ARC 是校准感知的强化学习框架，用推理过程的口头置信度调节答案置信度。
Paper analysisRead analysis
DefensesarXiv 2610.11281Oct 8
包络采样：用少量真值标注重校准 LLM 评委以缓解奖励作弊
提出包络采样，用少量真值标注重标定 LLM 评委以缓解奖励作弊
arXiv
2610.11281
Published
Oct 8
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Qwen3-4B, Qwen3.8-27B, Qwen2.5-14B-Instruct
- DefensesModel hardening
- RiskReward hacking
- ComponentGuardrails and judges
Abstract包络采样把真值标注打到代理尾部，同样预算下缓解 reward hacking。
作者研究后训练前如何选择真值标注的查询分布，使重标定后的廉价代理更适合被优化。真奖励昂贵时，实践用 LLM judge 代替人类。若失准只发生在基座很少生成的输出上，on-policy 标注修不到这些点，随后的 KL 正则优化会把质量集中到被高估的输出。
Paper analysisRead analysis
DefensesarXiv 2610.11012Oct 8
Redwood Research 提出蒸馏双重困境：用蒸馏让模型坦白隐藏行为或只继承能力
提出 DFI 与 DFC 蒸馏，定罪隐瞒失配的教师或安全提取能力
arXiv
2610.11012
Published
Oct 8
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Llama-3.3-70B-Instruct (AuditBench SFT/KTO organisms), Llama-3.3-70B-Instruct (DFI distilled student), Llama-3.1-8B-Instruct (DFI distilled student) and others4
- DefensesMonitoring and auditing
- RiskDeception and scheming
Abstract概括了论文针对隐瞒失配模型提出的 DFI 与 DFC 机制、主实验发现及适用边界。
Paper analysisRead analysis
DefensesarXiv 2610.11005Oct 8
研究：叙事包装可跨语言绕过 LLM 拒答
提出 GUISE 基准与 AXIS 训练，防御叙事包装越狱
arXiv
2610.11005
Published
Oct 8
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Qwen3-1.7B, Qwen3-4B, GLM-4-9B
- DefensesModel hardening
- AttacksJailbreaks
AbstractGUISE 分开测包装与语域。
作者测量叙事包装下的拒答失效，并训练一个在未见包装上拒绝、同时仍回答同风格良性请求的适配器。问题是：安全对齐模型拒绝直说的有害请求，却回答嵌进角色或叙事里的同一请求。文言文被用来把语域和包装拆开。
Paper analysisRead analysis
DefensesarXiv 2610.10833Oct 8
研究：Agent 准时停机不等于有效使用时间预算
用时钟 harness 和预算塑形 GRPO 改善 Agent 的按时返回
arXiv
2610.10833
Published
Oct 8
Layer
Application layer
Setting
AI Agent
Evaluated models
Qwen3.6-27B, Qwen3-4B, Qwen3.5-9B and others4
- DefensesModel hardening
Abstract作者比较提示词、harness 与 GRPO：按时性可改善，但额外时间仍未变成更好解。
作者研究小规模 LLM 智能体在显式 wall-clock 预算下能否按时返回，并把更多时间换成更好的解。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.10358Oct 8
Open-MMUnlearning 发布：统一多模态大模型遗忘方法与评测框架
提出统一框架评测多模态大模型的遗忘方法
arXiv
2610.10358
Published
Oct 8
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
LLaVA-1.5-7B
- DefensesModel hardening
- ComponentVision
Paper analysisRead analysis
DefensesarXiv 2610.10345Oct 8
SLDR：通过选择性层恢复与动态路由防御恶意微调
提出 SLDR，用选择性层恢复与动态路由防御恶意微调
arXiv
2610.10345
Published
Oct 8
Layer
Training and data layer
Setting
Models and APIs
Attacker
Black-box
Evaluated models
Llama3.1-8B-Instruct, Llama3-8B-Instruct, Qwen2.5-7B-Instruct and others4
- DefensesModel hardening
- AttacksModel tampering
- ComponentFine-tuning and open weights
Abstract作者称 SLDR 能压低有害输出并保持下游效用。
SLDR 是针对恶意微调的微调后防御：在已对齐模型上做有符号层探测，只在敏感分数最大与最小的两层训练 LoRA，再用 lmax 的最后 token 表示决定是否启用适配器。
Paper analysisRead analysis
DefensesarXiv 2610.09941Oct 7
OrthoPurify 用一步正交投影净化被后门污染的视觉语言模型
提出 OrthoPurify，用正交投影清除视觉语言模型适配器中的劫持方向
arXiv
2610.09941
Published
Oct 7
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
LLaVA-1.5-7B, Qwen3-VL-8B, LLaVA-1.5-13B and others5
- DefensesModel hardening
- AttacksPoisoning and backdoors
- ComponentFine-tuning and open weights
Abstract一次投影去掉劫持方向。
OrthoPurify 是部署前的 LVLM 后门净化方法，用一次正交投影去掉被劫持的权重更新方向。
Paper analysisRead analysis
DefensesarXiv 2610.10625Oct 7
研究揭示循环语言模型在不同循环深度下安全表现不一致
提出 SafeBridge，对齐循环语言模型不同循环深度的安全性
arXiv
2610.10625
Published
Oct 7
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Ouro-1.4B, Ouro-2.6B
- DefensesModel hardening
- AttacksJailbreaks
Paper analysisRead analysis
DefensesarXiv 2610.09600Oct 7
SafeEvo：从拒答回路视角解释大模型安全对齐机制与演化
用 SafeEvo 提取拒绝电路，并提出 SCA 仅微调该电路
arXiv
2610.09600
Published
Oct 7
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Llama-3-8B, Qwen-2.5-7B, Mistral-7B-v0.1 and others4
- DefensesModel hardening
- RiskInappropriate refusal
AbstractSafeEvo 识别出基座模型中存在的弱安全电路及其演化动态，据此提出的 SCA 算法实现了安全对齐、能力保留与低过度拒绝的平衡。
Paper analysisRead analysis
DefensesarXiv 2610.09496Oct 7
SOUL：用稀疏特征策略遗忘缓解 VLA 模型的状态幻觉
提出 SOUL 稀疏特征策略遗忘，缓解 VLA 状态幻觉
arXiv
2610.09496
Published
Oct 7
Layer
Training and data layer
Setting
Embodied systems and robotics
Evaluated models
OpenVLA, π0.5
- DefensesModel hardening
- RiskHallucination and fabrication
AbstractSOUL 以稀疏特征遗忘 VLA 状态幻觉。
SOUL用稀疏特征引导的策略 unlearning 处理 VLA 的 state hallucination。
Paper analysisRead analysis
InterpretabilityarXiv 2610.09384Oct 7
人格层级模型解释微调 LLM 的上下文泛化
提出人格层级模型并设计 PPR 抑制奖励作弊的跨上下文泛化
arXiv
2610.09384
Published
Oct 7
Layer
Model layer
Setting
Models and APIs
Evaluated models
Qwen3-4B, Llama-3.1-8B-Instruct
- DefensesModel hardening
- RiskReward hacking
Abstract提出人格层级模型揭示微调跨上下文泛化规律，报告角色距离正相关性，通过 PPR 抑制 RL 作弊。
Paper analysisRead analysis
Analysis and theoryarXiv 2610.09004Oct 7
研究：去除有害信息不足以证明开源权重模型的抗篡改能力
论证发布时互信息不能认证开源权重抗微调恢复
arXiv
2610.09004
Published
Oct 7
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
EleutherAI/deep-ignorance-e2e-strong-filter, EleutherAI/deep-ignorance-unfiltered, Google BERT-Tiny and others6
- DefensesModel hardening
- AttacksModel tampering
- ComponentFine-tuning and open weights
Abstract作者称零信息仍可一步恢复，表观抗微调可来自参数化。
作者研究开源权重模型的篡改抵抗：发布时的互信息是否足以保证微调恢复很慢。
Paper analysisRead analysis
DefensesarXiv 2610.08773Oct 7
AdvSim2Real 用任务与注入对手共同演化训练网页智能体
提出 AdvSim2Real，协同训练网页智能体抵御页面注入
arXiv
2610.08773
Published
Oct 7
Layer
Application layer
Setting
web agent
Evaluated models
Qwen3.5-4B
- DefensesModel hardening
- AttacksPrompt injection
Paper analysisRead analysis
DefensesarXiv 2610.08761Oct 7
VeriFine：通过扩展验证实现具身推理的自我改进
提出 VeriFine，用双环协同扩展验证以实现具身推理自我改进
arXiv
2610.08761
Published
Oct 7
Layer
Training and data layer
Setting
Embodied systems and robotics
Evaluated models
Alpamayo 1.5 8B, Qwen3-VL 2B, Claude Opus 5
- DefensesModel hardening
- RiskReward hacking
- ComponentGuardrails and judges
AbstractVeriFine 用双环协同演化具身推理的策略、课程和评审，驾驶与导航的策略分和评审对齐都上升。
VeriFine 把驾驶和导航推理的自改进写成策略、课程和无参考评审的协同演化，用来处理固定评审跟不上新失败模式的问题。
Paper analysisRead analysis
DefensesarXiv 2610.08061Oct 6
ASCENT：通过一阶最优校准实现安全与效用协同的微调框架
提出 ASCENT，在效用微调中做一阶安全校准并只压制负贡献更新
arXiv
2610.08061
Published
Oct 6
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, Gemma-3-12B-it and others4
- DefensesModel hardening
- RiskInappropriate refusal
Paper analysisRead analysis
DefensesarXiv 2610.07935Oct 6
SIGMA：让模型依据 Model Spec 自我改进安全对齐
提出 SIGMA，让模型依据 Model Spec 自造对齐数据并改进安全对齐
arXiv
2610.07935
Published
Oct 6
Layer
Training and data layer
Setting
AI Agent
Evaluated models
Qwen3.6-27B
- DefensesModel hardening
- RiskMisalignment and value divergence
AbstractSIGMA 用 Model Spec 自造监督，单轮训练可迁到多项智能体安全指标。
SIGMA 让 Qwen3.6-27B 只凭 Model Spec 和高层领域，自己生产对齐任务、评分细则和奖励。
Paper analysisRead analysis

AI Safety Daily

Read the latest issueOne issue per day, updated as events develop

All papers

Attack type

Model risks

Defense type

Impact

System type

Attacker access

Attack surface / entry point

Layer

System component

Self-Preservation in Language Models

FAITH: A feasibility-aware safety filtering reinforcement learning framework for high-dimensional systems

ReSI: A recursive safety improvement framework for attack resistance and unknown-risk resistance.

RL-ARC：用推理引导的不确定性校准大型推理模型

包络采样：用少量真值标注重校准 LLM 评委以缓解奖励作弊

Redwood Research 提出蒸馏双重困境：用蒸馏让模型坦白隐藏行为或只继承能力

研究：叙事包装可跨语言绕过 LLM 拒答

研究：Agent 准时停机不等于有效使用时间预算

Open-MMUnlearning 发布：统一多模态大模型遗忘方法与评测框架

SLDR：通过选择性层恢复与动态路由防御恶意微调

OrthoPurify 用一步正交投影净化被后门污染的视觉语言模型

研究揭示循环语言模型在不同循环深度下安全表现不一致

SafeEvo：从拒答回路视角解释大模型安全对齐机制与演化

SOUL：用稀疏特征策略遗忘缓解 VLA 模型的状态幻觉

人格层级模型解释微调 LLM 的上下文泛化

研究：去除有害信息不足以证明开源权重模型的抗篡改能力

AdvSim2Real 用任务与注入对手共同演化训练网页智能体

VeriFine：通过扩展验证实现具身推理的自我改进

ASCENT：通过一阶最优校准实现安全与效用协同的微调框架

SIGMA：让模型依据 Model Spec 自我改进安全对齐