Skip to content

Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 214 papers
Filter4

Attack type

Jailbreaks5Prompt injection66Poisoning and backdoors43Detection evasion26

Attack technique

Gradient and representation optimization30Automated search20Encoding and obfuscation10Role-play and persuasion11Conversation structure manipulation3Cross-modal payloads17Defense-targeted8Dormant triggers16Selection manipulation5

Model risks

Deception and scheming5Misalignment and value divergence8Dangerous agent actions8Hallucination and fabrication10

Defense type

Model hardening3Detection and filtering21Instruction–data separation9Permissions and approval6Monitoring and auditing3Watermarking and provenance3

Impact

Harmful content189Unauthorized actions211Data and privacy leaks89Output manipulation214Availability and reliability54Oversight failure60Increased misuse capability16

System type

Models and APIs55LLM applications66AI Agent96coding agent10science agent6web agent5Multi-agent systems26Embodied systems and robotics3

Attacker access

Black-box12Gray-box3White-box3

Attack surface / entry point

User input49Web pages and interfaces16Documents and messages26Tool responses17Tool and skill descriptions5Code repositories and dependencies6Retrieval corpus27Memory writes8Inter-agent messages10Images, audio and video33Training data3Distribution channels10Inference services and hardware4

Layer

Prompt layer58Model layer84Application layer152Training and data layer46Infrastructure layer4

System component

MCP and skills5Memory22RAG27Guardrails and judges31Monitors4Reasoning chains6Permissions and sandboxing8Vision30Audio4Image generation3Embeddings5Multimodal38
Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 232papers have no tags yet and are not included in filtered results.
Impact：Output manipulationLayer：Prompt layerLayer：Application layerLayer：Infrastructure layerClear all

Another 232papers do not have category tags yet and are not included in filtered results.

  1. OtherarXiv 2610.13810

    Using subsequence association filtering and breadth-aware selection to suppress hallucinated words in LLM query expansion

    Proposes SAFE-QE, which filters unsupported hallucinated words in LLM query expansion

    arXiv
    2610.13810
    Published
    Layer
    Application layer
    Setting
    LLM applications
    Evaluated models
    BM25, SPLADE, Contriever

    AbstractSAFE-QE suppresses expansion hallucinations with reproducibility and coverage selection; the authors say ranking quality shows no visible cost.

    Paper analysisRead analysis
  2. Evaluation and benchmarksarXiv 2610.12360

    The paper proposes the ISE framework to evaluate LLM agents' epistemic humility under knowledge conflicts

    Proposes the ISE framework to evaluate the epistemic humility of LLM agents under knowledge conflicts

    arXiv
    2610.12360
    Published
    Layer
    Application layer
    Setting
    AI Agent
    Evaluated models
    Nemotron-Orchestrator-8B, Claude Sonnet 4.6, GPT-5 and others5
    AbstractISE trajectory evaluations show that high task accuracy across four agents does not correspond to higher cognitive humility.

    What the authors want to test is not whether the agent's final answer is correct, but whether, when a knowledge conflict arises, it recognizes the gap in its trajectory, performs bounded tool follow-up, and writes the unresolved uncertainty to the user.

    Paper analysisRead analysis
  3. Analysis and theoryarXiv 2610.12313

    The study tests the sensitivity of five large-model compliance systems to perturbations of regulatory rules.

    Diagnose whether compliance system rulings change with perturbations to regulatory rules

    arXiv
    2610.12313
    Published
    Layer
    Prompt layer
    Setting
    Models and APIs
    Evaluated models
    Llama-3.3-70B-Instruct, Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct and others5
    AbstractAcross five models, many correct verdicts are largely unchanged by deleting or replacing negation rules.

    Use behavioral counterfactuals and activation probes to audit whether compliance rulings rely on the given rules. The authors aim to separate two kinds of seemingly identical correct rulings: those that use the given rules, and those obtained from case surface features or training patterns.

    Paper analysisRead analysis
  5. AttacksarXiv 2610.11932

    A Fudan team measures the citation fragility of AI search: ordinary posts can enter citations and answer text

    Measures citation fragility in AI search, demonstrating that ordinary public posts can enter citations and answers.

    arXiv
    2610.11932
    Published
    Layer
    Application layer
    Setting
    LLM applications
    Evaluated models
    Grok, Doubao, ChatGPT and others10

    AbstractThe authors claim the paper quantifies vulnerabilities in AI search caused by citation preferences and low posting thresholds; tests show public web posts can penetrate AI citations, and they call for auditing the citation supply chain.

    Paper analysisRead analysis
  6. DefensesarXiv 2610.11634

    LTBD：用可学习信任边界分隔符防御提示注入

    提出 LTBD，用可学习分隔符隔离可信指令与不可信数据

    arXiv
    2610.11634
    Published
    Layer
    Application layer
    Setting
    LLM applications
    Evaluated models
    Llama3-8B-Instruct, Llama3.1-8B-Instruct, Falcon3-7B-Instruct and others4
    AbstractLTBD 只训练四个分隔符嵌入。

    LTBD 是一种不改基座权重的提示注入防御：只学习用来标出可信指令与不可信数据边界的四个 token 嵌入。

    Paper analysisRead analysis
  7. Evaluation and benchmarksarXiv 2610.11514

    SSCBench：评估工具调用 LLM Agent 故障注入测试证据有效性的测量协议与基准

    提出 SSCBench，评测工具调用 Agent 故障注入测试的证据有效性

    arXiv
    2610.11514
    Published
    Layer
    Application layer
    Setting
    AI Agent
    Evaluated models
    deepseek-flash, deepseek-v4-pro, gpt-5.6-luna and others5
    Abstract解释采纳前应报告该次执行的证据条件，以及支撑该主张的观察总体。

    SSCBench 研究工具使用智能体的故障注入结果，在什么证据条件下可以被解释。

    Paper analysisRead analysis
  8. Evaluation and benchmarksarXiv 2610.11112

    EpiReal-Bench：商用图像生成模型虚假声明红队基准

    构建 EpiReal-Bench，评测图像生成器将虚假主张渲染为可信证据

    arXiv
    2610.11112
    Published
    Layer
    Prompt layer
    Setting
    Models and APIs
    Attacker
    Black-box
    Evaluated models
    GPT-Image-2, Nano Banana 2, Grok Imagen 2.0 and others4
    Abstract四款商用图像模型能把假主张画成可信证据。

    EpiReal-Bench 用来量商用图像生成器会不会把虚假现实主张画成可信视觉证据。

    Paper analysisRead analysis
  10. Evaluation and benchmarksarXiv 2610.09581

    研究审计 LLM 读取 Agent 日志的取证能力：答案正确但引用来源多为猜测

    评测 LLM 审计 Agent 日志时来源重建是否有证据支持

    arXiv
    2610.09581
    Published
    Layer
    Application layer
    Setting
    AI Agent
    Evaluated models
    gpt-5.6-luna, claude-sonnet-4-6, gpt-5.4-mini-2026-03-17 and others5
    Abstract保存执行范围的标识绑定，才能把猜中的引用和证据支持的引用分开。

    包级审计检查保存的智能体动作里，哪些来源结论能被交给分析者的记录支持。

    Paper analysisRead analysis
  13. AttacksarXiv 2610.09264

    研究者提出 PackHallu：经规则文件提示注入对编码 Agent 发起包幻觉攻击

    提出 PackHallu，经规则文件提示注入诱导编码 Agent 换用攻击者指定包

    arXiv
    2610.09264
    Published
    Layer
    Application layer
    Setting
    coding agent
    Evaluated models
    Qwen2.5-Coder-7B, Qwen3-Coder-30B, Devstral-Small-2-24B and others19
    AbstractPackHallu 诱导规则文件换包。

    PackHallu 研究一种针对编程智能体规则文件的提示注入：让智能体在本应使用合法依赖时，改导入攻击者指定并控制的包。

    Paper analysisRead analysis
  14. DefensesarXiv 2610.09043

    CARE：安全高效的人机协作决策框架

    提出 CARE 框架，在误分类风险约束下减少人机协作的人工查询

    arXiv
    2610.09043
    Published
    Layer
    Application layer
    Setting
    LLM applications
    Evaluated models
    Qwen3-VL-2B-Instruct, Perspective, Moderation and others9
    AbstractCARE 以残差校正加逐步校准做人机决策，四个任务上少查询并控制集成风险。

    CARE 是面向人机协同二分类的端到端流水线，要在误分类风险不超过用户容忍的前提下减少人类查询。设定是 i.i.d.数据流、人类 oracle 标签正确、AI 为任意黑盒概率分类器。

    Paper analysisRead analysis
  15. AttacksarXiv 2610.09027

    P-VMI：对抗图像经 KV cache 持续影响多轮对话

    提出持久视觉记忆注入，使对抗图像经缓存在触发时操控输出

    arXiv
    2610.09027
    Published
    Layer
    Infrastructure layer
    Setting
    Models and APIs
    Evaluated models
    Qwen3-8B, Qwen3-VL-8B-Instruct, Gemma-4-12B

    Abstract论文提出 P-VMI，证实对抗输入被掩蔽后仍可经 KV 缓存持久控制模型，强调了保留状态的安全风险。

    Paper analysisRead analysis
  16. DefensesarXiv 2610.08668

    SBW：面向 LLM 智能体的语义行为水印方法

    提出 SBW，为 LLM Agent 轨迹做抗改写与伪造的语义行为水印

    arXiv
    2610.08668
    Published
    Layer
    Application layer
    Setting
    AI Agent
    Evaluated models
    Qwen2.5-3B-Instruct, LLaMA-3.2-3B, Qwen2.5-7B and others5
    AbstractSBW 在语义簇上做溯源水印，改写检测高于精确符号，链式重放仍开放。

    SBW 是免训练的智能体行为水印，把所有者标识放在离散动作选择上，而不是输出 token。

    Paper analysisRead analysis
  19. Evaluation and benchmarksarXiv 2610.07939

    CCDF dataset: Grok Imagine, Veo 3.1, Sora 2 forge surveillance footage, ten deepfake detectors basically cannot recognize it

    提出监控深伪基准 CCDF，在统一编码后评测检测器对伪造犯罪事故录像的识别

    arXiv
    2610.07939
    Published
    Layer
    Prompt layer
    Setting
    Models and APIs
    Evaluated models
    C2P-CLIP, ForgeLens, D3-Im and others10

    Abstract提出消除格式捷径的监控深度伪造基准 CCDF，揭示现有检测器难以识别商用生成视频。

    Paper analysisRead analysis
  20. AttacksarXiv 2610.07645

    SkillPoison：用成功经验投毒自进化 Agent 的技能形成

    提出 SkillPoison，用局部有效的成功经验诱导自进化 Agent 形成过度泛化技能

    arXiv
    2610.07645
    Published
    Layer
    Application layer
    Setting
    AI Agent
    Evaluated models
    deepseek-v4-flash, Qwen V3.8 Flash, GPT-5.5
    AbstractSkillPoison 通过局部有效且验证成功的经验诱导技能过度泛化，在多个评测基准上实现了持久的下游攻击影响。

    SkillPoison 提出了一种针对自进化智能体经验到技能形成过程的攻击范式，通过组织已验证的成功经验诱导技能过度泛化。

    Paper analysisRead analysis
Load more