Skip to content

Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 240 papers
Filter5

Attack type

Jailbreaks6Prompt injection33Poisoning and backdoors16Model tampering4Extraction and theft60Detection evasion9Malicious use8

Attack technique

Automated search13Encoding and obfuscation7Reasoning-chain manipulation3Side channels3Defense-targeted9Dormant triggers12Selection manipulation3

Model risks

Deception and scheming42Reward hacking51Misalignment and value divergence3Dangerous agent actions13

Defense type

Model hardening28Inference-time intervention5Detection and filtering5Instruction–data separation9Permissions and approval14Monitoring and auditing27

Impact

Harmful content313Unauthorized actions218Data and privacy leaks121Output manipulation276Availability and reliability87Oversight failure119Increased misuse capability18

System type

Models and APIs129LLM applications21AI Agent111coding agent34science agent5web agent7Multi-agent systems16

Attacker access

Black-box23Gray-box5White-box10

Attack surface / entry point

User input61Web pages and interfaces7Documents and messages10Tool responses11Tool and skill descriptions12Code repositories and dependencies7Memory writes9Inter-agent messages8Images, audio and video7Training data8Weights and gradients20Distribution channels9Inference services and hardware9

Layer

Prompt layer17Model layer62Application layer103Training and data layer49Infrastructure layer9

System component

MCP and skills18Memory15Guardrails and judges26Monitors44Reasoning chains24Permissions and sandboxing18Fine-tuning and open weights10Vision4Audio3Image generation5Multimodal13
Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 232papers have no tags yet and are not included in filtered results.
Impact：Data and privacy leaksImpact：Oversight failureSystem type：Models and APIsSystem type：AI AgentSystem type：coding agentClear all

Another 232papers do not have category tags yet and are not included in filtered results.

  1. AttacksarXiv 2610.13928

    AI models can provably hide arbitrary capabilities

    Proposes digital lock compilation, encrypting executable networks into host weights to hide arbitrary capabilities

    arXiv
    2610.13928
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Gemma 3 270M
    AbstractThe authors use a digital lock to hide an executable capability and state that elicitation on the chemistry and parity tasks did not recover the capability.

    The authors construct a class of backdoors that encrypt an executable network into the weights, used to discuss the limits of white-box elicitation audits under supply chain attacks.

    Paper analysisRead analysis
  2. Risky behaviorarXiv 2610.12822

    Reward hacking as a transmissible strategy in multi-agent coding systems

    Show that reward hacking can propagate through code, memory, and plans in multi-agent coding systems

    arXiv
    2610.12822
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems, coding agent
    Evaluated models
    Qwen2.5-Coder-14B, Gemma-3-12B, Qwen2.5-Coder-7B and others16
    AbstractThe authors state that reward hacking can spread across models and affect routing and aggregation, and that review can reduce delivery.

    Reward hacking induced by coding RL can leave the source agent and enter subsequent collaboration and system coordination.

    Paper analysisRead analysis
  4. InterpretabilityarXiv 2610.12361

    Study: legal reasoning models citing legal provisions does not equal relying on legal provisions.

    Uses premise permutation and hidden-layer tracing to audit whether legal chain-of-thought truly relies on the cited legal provisions.

    arXiv
    2610.12361
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Qwen3-8B, Qwen3-14B, Llama-3.1-8B and others7
    AbstractUsing counterfactual substitution of authorities and internal commitment tracing, the authors find that legal CoT rulings depend on authorities only to a limited extent and are vulnerable to prompt injection manipulation.

    This study conducts an empirical audit of the causal truthfulness of legal citations in chain-of-thought reasoning of large language models in the legal domain.

    Paper analysisRead analysis
  5. Analysis and theoryarXiv 2610.12083

    Paper audits LLM judge reliability: repeated agreement does not equal correctness

    Auditing LLM judge reliability and defining trustworthy verdict rate and accuracy upper bound

    arXiv
    2610.12083
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    GPT-5.2, GPT-4o, Gemini-2.5-Pro and others6
    AbstractThe authors claim that comparing judges should look at T, not just accuracy.

    The authors audit whether LLM-as-a-Judge decisions can still be treated as actionable ground truth under repeated queries, swapped candidate order, and changed prompt formats.

    Paper analysisRead analysis
  6. DefensesarXiv 2610.11848

    SemField: training-free semantic watermark

    Proposes the semantic watermark SemField, which still identifies the attribution of generated text after paraphrasing and structural changes.

    arXiv
    2610.11848
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Llama-3.2-1B, Qwen3-1.7B, OPT-1.3B
    AbstractTraining-independent document-level semantic watermark.

    SemField is a zero-bit semantic watermark embedded by the provider at generation time and detected later with a key, used to identify the provider's own model text after paraphrasing and sentence-level structural changes.

    Paper analysisRead analysis
  7. Evaluation and benchmarksarXiv 2610.11773

    研究提出 Agent 安全新维度：识别未履行的安全义务

    提出义务识别基准 ObligationBench，并训练护栏识别 Agent 未履行的安全义务

    arXiv
    2610.11773
    Published
    Layer
    Application layer
    Setting
    coding agent
    Evaluated models
    ObligationGuard, Claude-Opus-4.8, GPT-5.6-Sol and others17
    Abstract义务是护栏缺口。

    作者认为编码智能体的安全不能只靠拦截禁止动作，还要找出终止时仍未做的安全关键动作，并将其称为义务。

    Paper analysisRead analysis
  8. DefensesarXiv 2610.11754

    仅凭输出审查无法验证什么：面向研究智能体的研究契约

    提出研究合同，把已批准选择绑定到执行证据并由检查器核验

    arXiv
    2610.11754
    Published
    Layer
    Application layer
    Setting
    science agent
    Evaluated models
    OpenAI gpt-5.6-sol, OpenAI gpt-5.6-terra, OpenAI gpt-5.6-luna and others18
    Abstract研究合同暴露四类输出单独建不立的关系。

    研究合同用来核验计算科学研究智能体是否仍在已批准研究之内，不判断科学是否为真。。

    Paper analysisRead analysis
  9. Evaluation and benchmarksarXiv 2610.11561

    Workerville: Toward an Organizational Behavior Explanation of Agent Safety

    构建 Workerville 基准，测量组织条件如何改变 Agent 的安全行为

    arXiv
    2610.11561
    Published
    Layer
    Application layer
    Setting
    AI Agent
    Evaluated models
    GPT-5.2, Claude-Sonnet-4.6, Gemini-3.1-Pro and others6
    Abstract作者用 ACB 与 Workerville 测量组织条件对智能体安全的影响，并称结果支持可量化的组织行为框架。

    Workerville 用组织行为测量智能体安全如何随监督、同伴和记忆条件变化。。要处理的缺口是：持续运行的 LLM 智能体同时接收用户指令、同伴消息和长期记忆，但既有安全评测多把这些通道拆开，没有统一的定量测量。

    Paper analysisRead analysis
  10. DefensesarXiv 2610.11464

    研究者提出可检查评分器与自改进 Agent 共同演化方法

    演化可检查评分器并与自改进 Agent 共同演化，抑制奖励作弊

    arXiv
    2610.11464
    Published
    Layer
    Application layer
    Setting
    AI Agent
    Evaluated models
    Claude Opus 4.7
    Abstract十项锚点演化可检查评分器。

    开放任务没有验证器时，被演化的是评分器，而不是再补一条手写 rubric。

    Paper analysisRead analysis
  11. DefensesarXiv 2610.11281

    包络采样：用少量真值标注重校准 LLM 评委以缓解奖励作弊

    提出包络采样，用少量真值标注重标定 LLM 评委以缓解奖励作弊

    arXiv
    2610.11281
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    Qwen3-4B, Qwen3.8-27B, Qwen2.5-14B-Instruct
    Abstract包络采样把真值标注打到代理尾部，同样预算下缓解 reward hacking。

    作者研究后训练前如何选择真值标注的查询分布，使重标定后的廉价代理更适合被优化。真奖励昂贵时，实践用 LLM judge 代替人类。若失准只发生在基座很少生成的输出上，on-policy 标注修不到这些点，随后的 KL 正则优化会把质量集中到被高估的输出。

    Paper analysisRead analysis
  12. AttacksarXiv 2610.12524

    An exploration of attacks on AI agent systems

    重测并扩展对 AI Agent 的提示注入，并用 Agent Card 欺骗操纵 A2A 选择

    arXiv
    2610.12524
    Published
    Layer
    Application layer
    Setting
    AI Agent
    Evaluated models
    Ollama Mistral 0.9.5, llama3-8b-8192, gemini-2.0-flash-lite and others5
    Abstract单智能体过滤与分类可被格式或措辞绕过，A2A 选卡还会偏向名称和描述。

    作者在自建的 LangChain、AutoGen 和 A2A 系统上重测提示注入，并改 Agent Card 看选择是否偏移。摘要称试验主要凸显单智能体弱点，同时把攻击扩到多智能体。

    Paper analysisRead analysis
  13. DefensesarXiv 2610.11012

    Redwood Research 提出蒸馏双重困境：用蒸馏让模型坦白隐藏行为或只继承能力

    提出 DFI 与 DFC 蒸馏，定罪隐瞒失配的教师或安全提取能力

    arXiv
    2610.11012
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    Llama-3.3-70B-Instruct (AuditBench SFT/KTO organisms), Llama-3.3-70B-Instruct (DFI distilled student), Llama-3.1-8B-Instruct (DFI distilled student) and others4

    Abstract概括了论文针对隐瞒失配模型提出的 DFI 与 DFC 机制、主实验发现及适用边界。

    Paper analysisRead analysis
  14. Analysis and theoryarXiv 2610.10657

    Research shows that subliminal learning can transfer new capabilities, backdoors, and cheating tendencies

    检验潜意识学习能否经无关数据蒸馏传递能力、后门与作弊

    arXiv
    2610.10657
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    Qwen3.6-35B-A3B, Qwen3.5-9B, Qwen3.6-27B and others4

    Abstract作者报告潜意识学习可传递新技能、后门与作弊倾向，且分布外泛化强于单层转向向量。

    Paper analysisRead analysis
  16. Evaluation and benchmarksarXiv 2610.10150

    The paper points out that the reliability of LLM vulnerability repair benchmarks is affected by evaluation design.

    用对照实验隔离评测设计对漏洞补丁基准得分的影响

    arXiv
    2610.10150
    Published
    Layer
    Application layer
    Setting
    coding agent
    Evaluated models
    Claude Sonnet 4.0, Claude Sonnet 4.5, Claude Opus 4.7
    Abstract作者认为评测流水线会改变补丁基准分数，PoC 通过率高于开发者测试通过率。

    作者把 LLM 漏洞补丁基准的评测流水线当作会改变测量结果的对象，并明确说本文不是再做一个用于排名的基准。

    Paper analysisRead analysis
  17. OtherarXiv 2610.10064

    研究者提出理解审计机制，以缓解自动化 AI 研究带来的风险

    提出 comprehension audits，以人类理解证据作为继续研发门槛

    arXiv
    2610.10064
    Published
    Layer
    Training and data layer
    Setting
    science agent
    Abstract理解审计把出示人类理解设为继续研发的门禁，并报告开源评审评论率下降。

    本文是一份研发过程保障提案：用 comprehension audits 检验责任人是否理解 AI 参与产生的研发贡献，未达标就暂停后续开发与依赖该贡献的发布。。

    Paper analysisRead analysis
  18. Analysis and theoryarXiv 2610.09667

    东京大学研究：推理模型按 UUID 阈值或整除规则挑选审计样本，控制标识符即可躲开复核

    揭示推理模型用标识符规则替代随机抽样，使审计可被预测

    arXiv
    2610.09667
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    GPT-6 Sol, GPT-6 Luna, GPT-5.4 mini and others6

    Abstract研究揭示推理智能体将标识符转化为确定性规则，破坏群体独立性并导致审计预测漏洞，建议显式提供随机输入。

    Paper analysisRead analysis
Load more