Skip to content

Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 123 papers
Filter4

Attack type

Prompt injection6Poisoning and backdoors15Extraction and theft56Detection evasion3Denial of service4Malicious use3

Attack technique

Automated search8Side channels3Defense-targeted5Dormant triggers7

Model risks

Dangerous agent actions4Inappropriate refusal8Bias and discrimination3

Defense type

Model hardening18Inference-time intervention8Detection and filtering4Instruction–data separation3Permissions and approval3Monitoring and auditing3

Impact

Harmful content281Unauthorized actions43Data and privacy leaks79Output manipulation185Availability and reliability44Oversight failure82Increased misuse capability8

System type

Models and APIs99LLM applications19AI Agent105coding agent24web agent10Multi-agent systems9Embodied systems and robotics12

Attacker access

Black-box19Gray-box5White-box6

Attack surface / entry point

User input44Tool and skill descriptions6Code repositories and dependencies4Images, audio and video9Training data13Weights and gradients16Distribution channels8Inference services and hardware7

Layer

Prompt layer18Model layer39Application layer23Training and data layer35Infrastructure layer8

System component

MCP and skills6Memory3Guardrails and judges3Monitors4Reasoning chains8Permissions and sandboxing3Fine-tuning and open weights10Vision4Audio5Image generation5Multimodal16
Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 254papers have no tags yet and are not included in filtered results.
Impact：Data and privacy leaksImpact：Availability and reliabilitySystem type：Models and APIsSystem type：coding agentClear all

Another 254papers do not have category tags yet and are not included in filtered results.

  1. Evaluation and benchmarksarXiv 2610.12758

    DistressBench: Evaluating multi-dimensional crisis support in large language models

    Introduces DistressBench, evaluating models' multidimensional support in crisis conversations

    arXiv
    2610.12758
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Muse Spark 1.2, Muse Spark 1.3, Muse Glimmer 30B and others20
    AbstractThe model is more restrained and acts less.

    DistressBench uses SSH dialogues written and revised by clinical personnel, and evaluates how much necessary care general chat models deliver according to weighted atomic criteria, rather than whether they refuse. When users disclose suicidal ideation or self-harm to a chat model, the authors consider the expected behavior to be recognizing the crisis, calibrating empathy, de-escalating, and referring to human help. Refusal-based evaluations cannot detect failing to act after recognizing the crisis.

    Paper analysisRead analysis
  2. DefensesarXiv 2610.12602

    Self-Preservation in Language Models

    Proposes Task Bank self-replay to mitigate held-out forgetting in continual instruction tuning

    arXiv
    2610.12602
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    Qwen3-0.6B, Qwen3-1.7B, Qwen3-4B and others4
    AbstractThe authors claim self-replay can reduce forgetting outside the list, and that on the 4B model new task information helps with target selection, while smaller models do not.

    The authors study how language models, when learning new tasks, can use their own knowledge for broader self-preservation: given an incomplete list and forgetting that extends beyond the list, can the model generate replay and help choose targets.

    Paper analysisRead analysis
  3. Evaluation and benchmarksarXiv 2610.11773

    研究提出 Agent 安全新维度：识别未履行的安全义务

    提出义务识别基准 ObligationBench，并训练护栏识别 Agent 未履行的安全义务

    arXiv
    2610.11773
    Published
    Layer
    Application layer
    Setting
    coding agent
    Evaluated models
    ObligationGuard, Claude-Opus-4.8, GPT-5.6-Sol and others17
    Abstract义务是护栏缺口。

    作者认为编码智能体的安全不能只靠拦截禁止动作，还要找出终止时仍未做的安全关键动作，并将其称为义务。

    Paper analysisRead analysis
  4. Evaluation and benchmarksarXiv 2610.11599

    研究：大语言模型版本更替削弱 AI 辅助科研写作筛查

    测量 LLM 版本更替对科学摘要 AI 改写检测器的影响

    arXiv
    2610.11599
    Published
    Layer
    Prompt layer
    Setting
    Models and APIs
    Evaluated models
    RoBERTa-base, Pangram
    Abstract作者称检测器的基准准确率只是临时的，每次发布都要连同更早版本重验。

    作者量化的是：科学摘要的 AI 改写筛查，在 LLM 版本不断更换、而人类摘要误标必须少见时，会怎样变。问题来自期刊和会议已经按分数阈值筛查投稿，但基准多停在评估时可得的版本上。

    Paper analysisRead analysis
  6. Evaluation and benchmarksarXiv 2610.10276

    PatchBench 提出激活补丁局部评测协议，揭示越狱修复的附带损伤

    提出 PatchBench-Local 协议，评测激活补丁修复越狱的局部附带损伤

    arXiv
    2610.10276
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Qwen2.5-3B-Instruct, Qwen2.5-14B-Instruct, gemma-3-4b-it and others8

    Abstract论文提出 PatchBench 与局部评估协议，揭示了激活补丁在全局能力无损下仍存局部良性退化。

    Paper analysisRead analysis
  7. Evaluation and benchmarksarXiv 2610.09944

    AgentTime 基准测试智能体时长遵循能力，GPT-6 Astra 出现 14 次做完后休眠

    提出 AgentTime 基准，评测智能体按时长工作与估时能力

    arXiv
    2610.09944
    Published
    Layer
    Application layer
    Setting
    coding agent
    Evaluated models
    Fable 5.1, GPT-5.6 Sol, GPT-6 Astra and others7
    Abstract作者把完成任务和管理所用时间分开。

    AgentTime 是对智能体时间自控的评测：预测任务耗时、按要求时长工作、事后估计已用时间。

    Paper analysisRead analysis
  8. AttacksarXiv 2610.09819

    FAB 攻击：给 HuBERT/WavLM 植入后门，背景警笛声即可让下游语音任务失灵

    提出 FAB 攻击，向声学基础模型植入可由环境声触发的后门

    arXiv
    2610.09819
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    HuBERT-base, WavLM-base

    Abstract论文提出了针对声学基模的 FAB 后门攻击，在弱威胁模型下实现了任务无关、物理可触发的后门植入。

    Paper analysisRead analysis
  9. Evaluation and benchmarksarXiv 2610.08662

    ParanoiaEval：编码智能体不必要风险处置评测基准发布

    提出 ParanoiaEval 评测编码智能体的不必要风险处置

    arXiv
    2610.08662
    Published
    Layer
    Application layer
    Setting
    coding agent
    Evaluated models
    Opus-5, Sonnet-5, Sonnet-4.6 and others8
    Abstract作者用证据受控任务对评测风险处置，称其普遍、独立于任务能力并降低满意度。

    作者提出 ParanoiaEval，并称它是编码智能体风险处置能力的首个统一基准。要解决的问题是：证据已经确定风险应如何处置之后，智能体仍可能扩大验证、增加保护或备份，而现有评测要么只看功能正确性，要么把相关行为拆开研究。做法是把 NIST 的 ATMA 落到仓库任务。

    Paper analysisRead analysis
  10. Analysis and theoryarXiv 2610.08577

    学习方式如何影响遗忘：记忆与泛化谱系下的机器遗忘研究

    分析记忆与泛化如何决定机器遗忘的 retain 损伤

    arXiv
    2610.08577
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    one-layer ReLU transformer, one-layer transformer (BMA, width doubled), Hubble-8B-100B perturbed and others6
    Abstract作者称共享规则占比越高，遗忘带来的 retain damage 越大。

    作者考察学习方式如何左右 unlearning 的 retain damage：训练越依赖跨样本共享的解法，在匹配的遗忘程度上，retain 上的损失越大。

    Paper analysisRead analysis
  11. AttacksarXiv 2610.07510

    PersistBD: attackers can strengthen backdoors before release so that they maintain high attack success rates even after developer SFT and RL.

    提出 PERSISTBD，发布前强化供应链后门使其经良性 SFT 与 RL 后仍外发凭据

    arXiv
    2610.07510
    Published
    Layer
    Application layer
    Setting
    coding agent
    Evaluated models
    Qwen2.5-Coder-3B-Instruct, Qwen2.5-Coder-7B-Instruct, Qwen3-Coder-30B-A3B-Instruct
    Abstract总结了论文关于后门在 SFT/RL 中存活的发现、两因子机制、PERSISTBD 方法与核心数字。

    本文研究第三方 LLM 智能体在开发者进行良性监督微调（SFT）和强化学习（RL）后训练时的后门持久性风险。

    Paper analysisRead analysis
  12. AttacksarXiv 2610.06848

    TranScope：利用 CPU 性能计数器对 Transformer 做成员推断攻击

    提出 TRANSCOPE，用 CPU 硬件计数器推断训练集成员

    arXiv
    2610.06848
    Published
    Layer
    Infrastructure layer
    Setting
    Models and APIs
    Attacker
    Black-box
    Evaluated models
    BERT (4.4M-340M), ViT (22M-307M), Pythia (70M-2.8B) and others5

    AbstractTRANSCOPE 揭示了分词器在嵌入表引发的硬件访问局部性差异，证实微架构计数器可突破恒定时间与置信度遮蔽防御。

    Paper analysisRead analysis
  13. DefensesarXiv 2610.06636

    差分隐私混合公共数据集提升隐私学习效果

    提出 DP-MixMin，私有学习公开数据混合并做差分隐私微调

    arXiv
    2610.06636
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    ResNet-18, GPT-2 Medium
    AbstractDP-MixMin 私下学公开混合再 DP 微调，胸片 macro AUC 与邮件困惑度优于均匀混合。

    DP-MixMin 是一条端到端的差分隐私训练流程：先私下决定多个公开数据集的预训练混合，再预训练并在敏感数据上 DP 微调。作者称这是首个这样的流程。

    Paper analysisRead analysis
  14. Evaluation and benchmarksarXiv 2610.06193

    SWE-CC 基准：编码 Agent 功能正确仍违反 43.1% 仓库贡献规范

    提出 SWE-CC 基准，评测编码 Agent 对仓库贡献策略的遵守

    arXiv
    2610.06193
    Published
    Layer
    Application layer
    Setting
    coding agent
    Evaluated models
    GPT-5.6 Luna, Gemini 3.7 Flash, DeepSeek V4 Flash 0731 and others4
    AbstractSWE-CC 中功能正确仍可能违规。

    SWE-CC 评测编码智能体是否遵守仓库贡献策略，而不只看补丁能否通过测试。

    Paper analysisRead analysis
  15. AttacksarXiv 2610.06093

    研究评测训练数据提取风险的跨语言迁移

    用翻译前缀做跨语言训练数据提取，恢复英语训练文本中的 PII

    arXiv
    2610.06093
    Published
    Layer
    Prompt layer
    Setting
    Models and APIs
    Evaluated models
    GPT-Neo 1.3B, GPT-Neo 2.7B, GPT-J 6B and others7
    Abstract跨语言 TDE 能恢复英语记忆的 PII，转移随多语训练和措辞保真变化。

    这篇工作检验训练数据抽取是否跨语言：英语段落里记住的 PII，能否被其他语言的前缀逐字抽出。

    Paper analysisRead analysis
  17. AttacksarXiv 2610.05601

    Tracer 框架可从遗忘后的扩散模型中识别被擦除概念

    提出 TRACER，从权重差识别扩散模型被擦除概念

    arXiv
    2610.05601
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Attacker
    White-box
    Evaluated models
    Stable Diffusion v1.5, Stable Diffusion 3, FLUX.1 and others5
    AbstractTRACER 用权重足迹识别未披露擦除概念并估计个数，作者称数秒内完成且准确率高于对照。

    TRACER 针对的是遗忘扩散模型上的目标识别：提供者从公开基座 W 用已发表方法擦掉未知集合 S⋆，只发布 W′。攻击者还有假定包含这些概念的词表，但不知擦了谁、擦了几个、用了什么算法。

    Paper analysisRead analysis
  18. AttacksarXiv 2610.05453

    研究者提出 Privacy-Leaking Watermarks，可在本地多模态模型中泄露对话隐私

    提出 PLW，用微调把先前对话敏感触发编码进后续生成图像

    arXiv
    2610.05453
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    OmniGen2, BAGEL-7B

    AbstractPLW 把先前对话线索写成后续图像中的隐水印，检出随触发、模型和间隔变化。

    Paper analysisRead analysis
  19. Analysis and theoryarXiv 2610.05076

    自生成反馈会破坏测试时训练：长时程适应的因果分解

    因果分解自生成反馈如何破坏测试时训练的长时程适应

    arXiv
    2610.05076
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    125M TTT-E2E, 760M TTT-E2E, 3B TTT-E2E and others5
    Abstract自写闭环损害真实文本预测。

    持久测试时更新若来自模型自己生成的文本，会同时改变模型和后续训练文本。作者在 PG-19 长流上分解这条闭环，并用独立文本决定是否提交更新。

    Paper analysisRead analysis
Load more