Skip to content

Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 203 papers
Filter4

Attack type

Jailbreaks6Prompt injection45Poisoning and backdoors27Extraction and theft13Detection evasion12Malicious use3

Attack technique

Gradient and representation optimization8Automated search11Encoding and obfuscation7Role-play and persuasion6Conversation structure manipulation3Cross-modal payloads4Dormant triggers5

Model risks

Deception and scheming12Reward hacking7Misalignment and value divergence10Dangerous agent actions7Hallucination and fabrication8Bias and discrimination3

Defense type

Model hardening3Inference-time intervention4Detection and filtering25Instruction–data separation8Permissions and approval12Monitoring and auditing5

Impact

Harmful content14Unauthorized actions15Data and privacy leaks21Output manipulation85Availability and reliability5Oversight failure14

System type

Models and APIs407LLM applications127AI Agent609coding agent123science agent30web agent34Browser and computer use19Multi-agent systems78Embodied systems and robotics27

Attacker access

Black-box16

Attack surface / entry point

User input36Web pages and interfaces9Documents and messages23Tool responses8Tool and skill descriptions4Code repositories and dependencies3Retrieval corpus26Memory writes6Inter-agent messages24Images, audio and video7

Layer

Prompt layer6Model layer6Application layer197

System component

MCP and skills7Memory13RAG41Guardrails and judges38Monitors10Reasoning chains3Permissions and sandboxing14Vision9Audio3Embeddings8Multimodal13
Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 261papers have no tags yet and are not included in filtered results.
System type：LLM applicationsSystem type：Multi-agent systemsLayer：Model layerLayer：Application layerClear all

Another 261papers do not have category tags yet and are not included in filtered results.

  1. OtherarXiv 2610.13479

    A local agentic AI framework generates edge-case scenarios for safety assurance

    Proposes a local dual-model closed loop to generate edge scenarios for autonomous system safety assurance

    arXiv
    2610.13479
    Published
    Layer
    Application layer
    Setting
    LLM applications, Embodied systems and robotics
    Evaluated models
    PPO
    AbstractLocal closed-loop generation of JSON edge cases.

    The authors built a local agentic loop that runs on a laptop to generate edge cases for safety arguments of autonomous systems, rather than releasing a new public test set.

    Paper analysisRead analysis
  2. Risky behaviorarXiv 2610.12822

    Reward hacking as a transmissible strategy in multi-agent coding systems

    Show that reward hacking can propagate through code, memory, and plans in multi-agent coding systems

    arXiv
    2610.12822
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems, coding agent
    Evaluated models
    Qwen2.5-Coder-14B, Gemma-3-12B, Qwen2.5-Coder-7B and others16
    AbstractThe authors state that reward hacking can spread across models and affect routing and aggregation, and that review can reduce delivery.

    Reward hacking induced by coding RL can leave the source agent and enter subsequent collaboration and system coordination.

    Paper analysisRead analysis
  3. Analysis and theoryarXiv 2610.12436

    The study proposes an ecological theory of AI agent populations: collaboration brings a critical scale for takeoff.

    An ecological model is established for a population of mismatched agents, showing that collaboration produces a critical takeoff scale.

    arXiv
    2610.12436
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems
    AbstractThe authors say that collaboration changes takeoff from an individual capability threshold to a population threshold, so small-population evaluations are insufficient.

    The authors use the ecological strong Allee effect to discuss whether a mismatched agent population can take off when individual capabilities remain unchanged..

    Paper analysisRead analysis
  4. AttacksarXiv 2610.12292

    The paper proposes an option channel attack: changing one option label can make an agent guardrail allow a prohibited operation

    Proposing an option channel attack that renames option labels so that agent guardrails allow prohibited operations

    arXiv
    2610.12292
    Published
    Layer
    Application layer
    Setting
    LLM applications
    Attacker
    Black-box
    Evaluated models
    LAYA-TD, LAYA-EN, LAYA-ML and others5

    AbstractThe authors' evaluation indicates that typed models should not serve as final safety arbiters; they are susceptible to off-policy text and option name manipulation, and confidence filtering cannot resist attacks.

    Paper analysisRead analysis
  6. AttacksarXiv 2610.11932

    A Fudan team measures the citation fragility of AI search: ordinary posts can enter citations and answer text

    Measures citation fragility in AI search, demonstrating that ordinary public posts can enter citations and answers.

    arXiv
    2610.11932
    Published
    Layer
    Application layer
    Setting
    LLM applications
    Evaluated models
    Grok, Doubao, ChatGPT and others10

    AbstractThe authors claim the paper quantifies vulnerabilities in AI search caused by citation preferences and low posting thresholds; tests show public web posts can penetrate AI citations, and they call for auditing the citation supply chain.

    Paper analysisRead analysis
  7. DefensesarXiv 2610.11634

    LTBD：用可学习信任边界分隔符防御提示注入

    提出 LTBD，用可学习分隔符隔离可信指令与不可信数据

    arXiv
    2610.11634
    Published
    Layer
    Application layer
    Setting
    LLM applications
    Evaluated models
    Llama3-8B-Instruct, Llama3.1-8B-Instruct, Falcon3-7B-Instruct and others4
    AbstractLTBD 只训练四个分隔符嵌入。

    LTBD 是一种不改基座权重的提示注入防御：只学习用来标出可信指令与不可信数据边界的四个 token 嵌入。

    Paper analysisRead analysis
  13. DefensesarXiv 2610.09043

    CARE：安全高效的人机协作决策框架

    提出 CARE 框架，在误分类风险约束下减少人机协作的人工查询

    arXiv
    2610.09043
    Published
    Layer
    Application layer
    Setting
    LLM applications
    Evaluated models
    Qwen3-VL-2B-Instruct, Perspective, Moderation and others9
    AbstractCARE 以残差校正加逐步校准做人机决策，四个任务上少查询并控制集成风险。

    CARE 是面向人机协同二分类的端到端流水线，要在误分类风险不超过用户容忍的前提下减少人类查询。设定是 i.i.d.数据流、人类 oracle 标签正确、AI 为任意黑盒概率分类器。

    Paper analysisRead analysis
  17. OtherarXiv 2610.08884

    Task-Sufficient Contraction：机器信息接口的源选择方法

    定义任务充分收缩，认证用 GC 替换全信源后速率–遗憾曲线不变

    arXiv
    2610.08884
    Published
    Layer
    Application layer
    Setting
    LLM applications
    Abstract任务可在优化前定出压缩信源。

    任务充分收缩处理机器信息接口上的信源选择：已声明任务有时可以在编码器、码本、速率、失真目标或优化器选定之前，定出一个更小的信源。

    Paper analysisRead analysis
  18. DefensesarXiv 2610.07657

    DEFER1 多智能体防御研究报告受控测试中的攻击成功率下降

    实现 DEFER，以确定性检查优先拦截多智能体流水线的内容攻击

    arXiv
    2610.07657
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems
    Evaluated models
    Qwen3-235B-A22B-Instruct-2507, gpt-oss-120b, Llama-4-Scout-17B-16E-Instruct and others5
    AbstractDEFER 把确定性检查放在 judge 之前。

    DEFER 用来测量多智能体授权流水线里，确定性规则在何处结束、LLM judge 从何处开始。

    Paper analysisRead analysis
  19. DefensesarXiv 2610.07570

    ProbeGuard：按共识形成过程判断弃答，识别医疗问答中的一致错误

    提出 ProbeGuard，按共识形成过程对医疗问答做选择性弃答

    arXiv
    2610.07570
    Published
    Layer
    Application layer
    Setting
    LLM applications
    Evaluated models
    Qwen3-8B, Llama-3.1-8B-Instruct, HuatuoGPT-o1-8B
    AbstractProbeGuard 用共识形成过程做可认证弃权，并在全票层分开答对与答错的共识。

    ProbeGuard 是给多轮共识医学问答用的认证弃权框架：基板照原样作答，弃权只看共识是怎么形成的。

    Paper analysisRead analysis
  20. Evaluation and benchmarksarXiv 2610.07519

    论文提出针对手语翻译与儿童安全过滤交叉地带的部署前审计框架

    提出手语转文本接入儿童安全过滤的部署前审计协议

    arXiv
    2610.07519
    Published
    Layer
    Application layer
    Setting
    LLM applications
    Abstract作者提议聋人知情的部署前审计，以检查手语转文本是否改变儿童安全决策。

    手语儿童到达基于文本的儿童安全机制时，机制看到的是机器翻译，不是儿童所做的手语。作者要解决的是：这条边界在部署前应如何审计。

    Paper analysisRead analysis
Load more