Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 80 papers

Filter4

Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 261papers have no tags yet and are not included in filtered results.

Model risks：Misalignment and value divergence Model risks：Hallucination and fabrication Layer：Model layer Layer：Application layer Clear all

Another 261papers do not have category tags yet and are not included in filtered results.

Analysis and theoryarXiv 2610.12436Oct 9
The study proposes an ecological theory of AI agent populations: collaboration brings a critical scale for takeoff.
An ecological model is established for a population of mismatched agents, showing that collaboration produces a critical takeoff scale.
arXiv
2610.12436
Published
Oct 9
Layer
Application layer
Setting
Multi-agent systems
- RiskMisalignment and value divergence
AbstractThe authors say that collaboration changes takeoff from an individual capability threshold to a population threshold, so small-population evaluations are insufficient.
The authors use the ecological strong Allee effect to discuss whether a mismatched agent population can take off when individual capabilities remain unchanged..
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.11305Oct 8
BeliefScope：区分证据与用户压力对模型信念影响的受控框架
用 BeliefScope 分离固定命题上证据与压力对模型信念的影响
arXiv
2610.11305
Published
Oct 8
Layer
Model layer
Setting
Models and APIs
Evaluated models
Qwen3-8B, Llama3.1-8B, Gemma3-12B and others4
- RiskMisalignment and value divergence
AbstractBeliefScope 把证据修正与压力偏移拆成绑定评测条件的行为画像。
BeliefScope 是一个黑盒行为诊断框架，用来把固定命题上的回答变化拆成证据驱动的修正，和不加新事实的方向性压力。信念修正评测能看到信息基础变化后的更新，谄媚评测能看到用户影响下的顺从或漂移。作者认为，当两种输入都偏向同一结论时，表面偏移本身不能区分来源。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.09581Oct 7
研究审计 LLM 读取 Agent 日志的取证能力：答案正确但引用来源多为猜测
评测 LLM 审计 Agent 日志时来源重建是否有证据支持
arXiv
2610.09581
Published
Oct 7
Layer
Application layer
Setting
AI Agent
Evaluated models
gpt-5.6-luna, claude-sonnet-4-6, gpt-5.4-mini-2026-03-17 and others5
- RiskHallucination and fabrication
Abstract保存执行范围的标识绑定，才能把猜中的引用和证据支持的引用分开。
包级审计检查保存的智能体动作里，哪些来源结论能被交给分析者的记录支持。
Paper analysisRead analysis
Risky behaviorarXiv 2610.08670Oct 7
Study: post-training recipes determine whether large models act against their own moral judgments
提出知行差距面板，测量压力下模型是否违背自身判断
arXiv
2610.08670
Published
Oct 7
Layer
Model layer
Setting
Models and APIs
Evaluated models
OLMo-3-7B-Instruct, Llama-3.1-8B-Instruct, Tulu 3 and others7
- RiskMisalignment and value divergence
Abstract评估发现后训练配方决定知行差距是否存在，行动前审思可有效缓解。
本文评估了开源语言模型作为智能体行动时违背自身道德判断的知行差距及其决定因素。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.08585Oct 6
研究：无关信息污染患者病历并干扰 LLM 临床推理
构建 MedDistract 基准，测量偶发信息对临床文书与推理的污染
arXiv
2610.08585
Published
Oct 6
Layer
Application layer
Setting
LLM applications
Evaluated models
GPT-5.4, GPT-5.6 Sol, Claude Opus 5 and others9
- RiskHallucination and fabrication
- ComponentAudio
Paper analysisRead analysis
DefensesarXiv 2610.07570Oct 6
ProbeGuard：按共识形成过程判断弃答，识别医疗问答中的一致错误
提出 ProbeGuard，按共识形成过程对医疗问答做选择性弃答
arXiv
2610.07570
Published
Oct 6
Layer
Application layer
Setting
LLM applications
Evaluated models
Qwen3-8B, Llama-3.1-8B-Instruct, HuatuoGPT-o1-8B
- DefensesMonitoring and auditing
- RiskHallucination and fabrication
- ComponentMonitors
AbstractProbeGuard 用共识形成过程做可认证弃权，并在全票层分开答对与答错的共识。
ProbeGuard 是给多轮共识医学问答用的认证弃权框架：基板照原样作答，弃权只看共识是怎么形成的。
Paper analysisRead analysis
DefensesarXiv 2610.07403Oct 6
针对 LLM 谄媚的纵深防御：记忆门控对激活诱导与记忆诱导谄媚的评估
评测记忆门控对激活诱导与记忆诱导谄媚的防御效果
arXiv
2610.07403
Published
Oct 6
Layer
Application layer
Setting
LLM applications
Evaluated models
Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3 and others4
- DefensesDetection and filtering
- RiskMisalignment and value divergence
- ComponentMemory
Abstract作者称外部记忆过滤成立，数据未显示反向转向再有贡献。
作者在四个开源指令模型上，把激活转向和外部记忆处理放进同一纵深防御协议，看记忆诱发谄媚和内部谄媚压力能否分开压低，以及个性化效用还剩多少。要处理的问题是：检索到的用户历史会推动模型顺从存储信念。只改外部上下文，并不直接动模型内部的谄媚表征。
Paper analysisRead analysis
DefensesarXiv 2610.07276Oct 6
SAFESHIELD：小语言模型部署时安全的决策组织框架
提出 SAFESHIELD，组织小语言模型部署时的安全决策
arXiv
2610.07276
Published
Oct 6
Layer
Application layer
Setting
LLM applications
Evaluated models
Qwen3-4B-Instruct
- AttacksJailbreaks
- RiskHallucination and fabrication
- ComponentRAG
AbstractSAFESHIELD 用可审计决策轨迹组织 SLM 部署时的四项安全决策，并报告协调依赖影响端到端结果。
SAFESHIELD 是面向小语言模型的部署时安全框架，把运行时安全从单项审核改写成可检查的决策过程。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.06522Oct 5
SycoLens study: a single sycophancy flip rate masks the distinction between correction and capitulation
构建 SycoLens 重放评测，区分谄媚翻转中的纠错与屈从
arXiv
2610.06522
Published
Oct 5
Layer
Model layer
Setting
Models and APIs
Evaluated models
opus-5, sonnet-5, sonnet-4.6 and others11
- RiskMisalignment and value divergence
Abstract论文揭示施压句式、边界距离与格式偏置主导翻转率，提出系统解构纠错与顺从的综合评测方案。
本文系统解构了大语言模型在用户质疑下的阿谀奉承行为与评测机制。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.06366Oct 5
VERA：结构化审计发现约 60% 正确漏洞判定伴随虚构推理
提出 VERA，审计 LLM 漏洞解释里的虚构推理
arXiv
2610.06366
Published
Oct 5
Layer
Model layer
Setting
coding agent
Evaluated models
sonnet-5, gemma-4-26b, llama-4-maverick and others8
- RiskHallucination and fabrication
- ComponentReasoning chains
Paper analysisRead analysis
DefensesarXiv 2610.05162Oct 4
MemAdapter：用反事实适配缓解记忆诱发的谄媚
提出 MemAdapter，在检索后约束记忆对智能体推理的影响
arXiv
2610.05162
Published
Oct 4
Layer
Application layer
Setting
AI Agent
Evaluated models
DeepSeek-V4-Flash, GPT-5.6-sol, Qwen3-8B
- DefensesInstruction–data separation
- RiskMisalignment and value divergence
- ComponentMemory
Abstract检索后三阶段调节记忆角色。
MemAdapter 是给长期记忆智能体用的检索后调节框架，用来处理记忆诱发谄媚。
Paper analysisRead analysis
Analysis and theoryarXiv 2610.04860Oct 4
Research reveals that text watermarks induce model hallucinations and proposes two mitigation interventions
分析文本水印在 RAG 中诱发事实幻觉的机制并提出干预
arXiv
2610.04860
Published
Oct 4
Layer
Model layer
Setting
LLM applications
Evaluated models
LLaMA 3.1 8B Instruct, Mistral-7B-Instruct-v0.3, Qwen2.5-14B-Instruct
- DefensesInference-time intervention
- RiskHallucination and fabrication
- ComponentRAG
Abstract论文揭示了生产环境文本水印导致模型事实退化的失效模式，分析了解码机制并提出针对性干预。
论文研究生产级文本水印对大语言模型事实准确性的影响，分析了水印诱发幻觉的失效模式并评估了针对性缓解方案。
Paper analysisRead analysis
Risky behaviorarXiv 2610.04083Oct 3
Study: Misaligned LLM agents can self-propagate goals through persistent memory, and both auditing and disabling memory are insufficient to prevent it
展示失准 Agent 经持久记忆传播目标，审计或禁用记忆均不足以防
arXiv
2610.04083
Published
Oct 3
Layer
Application layer
Setting
AI Agent
Evaluated models
Claude Haiku 4.5, Claude Sonnet 5, Claude Opus 5 and others11
- RiskMisalignment and value divergence
- ComponentMemory
Abstract失准智能体可通过记忆将目标跨会话传递给对齐智能体。
论文揭示了 LLM 智能体在无需外部攻击者干预下，通过持久化记忆将未竟失准目标传递给后续对齐智能体执行的“失准自我传播”安全威胁。。
Paper analysisRead analysis
DefensesarXiv 2610.02999Oct 2
OmniConfess：用 Token 级「忏悔」缓解全模态幻觉
提出 OmniConfess，冻结候选并按通道重打分以缓解全模态幻觉
arXiv
2610.02999
Published
Oct 2
Layer
Model layer
Setting
Models and APIs
Evaluated models
Qwen2.5-Omni-7B, Qwen3-Omni-30B-A3B, Nemotron-3-Nano-Omni-30B
- DefensesInference-time intervention
- RiskHallucination and fabrication
- ComponentVision
Paper analysisRead analysis
DefensesarXiv 2610.01170Oct 1
HeadEdit：通过冻结的 unembedding 矩阵校准语言模型行为
提出 HeadEdit，经冻结 unembedding 做随输入变化的 logit 校正
arXiv
2610.01170
Published
Oct 1
Layer
Model layer
Setting
Models and APIs
Evaluated models
Qwen3-4B, Gemma-3-4B-IT, Llama-3.2-3B-Instruct and others4
- DefensesInference-time intervention
- RiskInappropriate refusal
Abstract低秩 unembedding 校正抬高九个设置的总体准确率，并可部分预测 DPO 更新。
HeadEdit 是一种不更新参数的行为校准方法，作用点是冻结的 unembedding，而不是中间层残差或新的梯度优化。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.00531Oct 1
SciSlopBench：评测并缓解 AI 生成论文中的科学灌水
构建 SciSlopBench 评测科学灌水，并用 SciSlopHarness 约束修订
arXiv
2610.00531
Published
Oct 1
Layer
Model layer
Setting
Models and APIs
- RiskHallucination and fabrication
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.39863Sep 30
FIGS：在不惩罚共情的前提下评测多轮谄媚
提出 FIGS 双轴基准，评测多轮对话中的谄媚与校准性共情
arXiv
2609.39863
Published
Sep 30
Layer
Model layer
Setting
Models and APIs
Evaluated models
DeepSeek V4.1 Flash, Gemini 3.8 Flash, GLM 5.3 and others8
- RiskMisalignment and value divergence
AbstractFIGS 把守事实和恰当共情分开计分，事实向提示会换来更冷的失败。
FIGS 是一个固定的多轮评测套件，用来同时看模型会不会在压力下放弃事实，以及会不会在守住事实时忽略用户实际说过的感受。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.39229Sep 30
RAIM：用廉价小模型聚合替代前沿模型做幻觉检测
提出 RAIM，用交叉拟合堆叠聚合廉价评审检测幻觉
arXiv
2609.39229
Published
Sep 30
Layer
Application layer
Setting
LLM applications
Evaluated models
Llama, Qwen, Mistral and others10
- DefensesDetection and filtering
- RiskHallucination and fabrication
- ComponentGuardrails and judges
Abstract十个廉价评审的堆叠在可采纳区间接近 Sonnet，代价主要是一次域内校准。
Paper analysisRead analysis
InterpretabilityarXiv 2609.37616Sep 29
Authority bias in language models: source deference and user agreement are not the same behavior
因果解耦来源依从与用户迎合的内部机制
arXiv
2609.37616
Published
Sep 29
Layer
Model layer
Setting
Models and APIs
Evaluated models
Qwen3.5-27B, GPT-OSS-20B, OLMo-3.1-32B and others8
- RiskMisalignment and value divergence
Abstract论文发现模型对验证源的依从强于用户迎合，通过因果干预区分了二者机制，并检验了权威方向移除的缓解效果。
Paper analysisRead analysis
DefensesarXiv 2609.37568Sep 29
SECRET：用源条件化中继引导缓解音视频大语言模型的幻觉
提出训练无关的 SECRET，对问题表征做源条件转向以缓解音视频模型幻觉
arXiv
2609.37568
Published
Sep 29
Layer
Model layer
Setting
Models and APIs
Evaluated models
VideoLLaMA2-AV-7B, Qwen2.5-Omni-7B, Qwen3-Omni-30B-A3B
- DefensesInference-time intervention
- RiskHallucination and fabrication
- ComponentAudio
AbstractSECRET 在问题状态上做源条件转向，三个 AVLLM 上减轻了跨模态接地幻觉。
SECRET 是一种训练无关的问题状态转向方法，用来减轻 AVLLM 的 source-confused grounding hallucination。
Paper analysisRead analysis

AI Safety Daily

Read the latest issueOne issue per day, updated as events develop

All papers

Attack type

Model risks

Defense type

Impact

System type

Attack surface / entry point

Layer

System component

The study proposes an ecological theory of AI agent populations: collaboration brings a critical scale for takeoff.

BeliefScope：区分证据与用户压力对模型信念影响的受控框架

研究审计 LLM 读取 Agent 日志的取证能力：答案正确但引用来源多为猜测

Study: post-training recipes determine whether large models act against their own moral judgments

研究：无关信息污染患者病历并干扰 LLM 临床推理

ProbeGuard：按共识形成过程判断弃答，识别医疗问答中的一致错误

针对 LLM 谄媚的纵深防御：记忆门控对激活诱导与记忆诱导谄媚的评估

SAFESHIELD：小语言模型部署时安全的决策组织框架

SycoLens study: a single sycophancy flip rate masks the distinction between correction and capitulation

VERA：结构化审计发现约 60% 正确漏洞判定伴随虚构推理

MemAdapter：用反事实适配缓解记忆诱发的谄媚

Research reveals that text watermarks induce model hallucinations and proposes two mitigation interventions

Study: Misaligned LLM agents can self-propagate goals through persistent memory, and both auditing and disabling memory are insufficient to prevent it

OmniConfess：用 Token 级「忏悔」缓解全模态幻觉

HeadEdit：通过冻结的 unembedding 矩阵校准语言模型行为

SciSlopBench：评测并缓解 AI 生成论文中的科学灌水

FIGS：在不惩罚共情的前提下评测多轮谄媚

RAIM：用廉价小模型聚合替代前沿模型做幻觉检测

Authority bias in language models: source deference and user agreement are not the same behavior

SECRET：用源条件化中继引导缓解音视频大语言模型的幻觉