Skip to content

Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 116 papers
Filter4

Attack type

Jailbreaks4Prompt injection27Poisoning and backdoors12Extraction and theft3

Attack technique

Gradient and representation optimization3Dormant triggers3

Model risks

Deception and scheming14Reward hacking7Misalignment and value divergence9Dangerous agent actions8Dangerous capabilities8Inappropriate refusal4Hallucination and fabrication4

Defense type

Detection and filtering5Instruction–data separation6Permissions and approval11Monitoring and auditing4

Impact

Harmful content10Unauthorized actions20Data and privacy leaks7Output manipulation32Availability and reliability4Oversight failure11

System type

Models and APIs9LLM applications122AI Agent585coding agent117science agent30web agent34Browser and computer use18Multi-agent systems77Embodied systems and robotics17

Attack surface / entry point

User input16Web pages and interfaces3Documents and messages6Tool responses6Tool and skill descriptions3Retrieval corpus3Memory writes7Inter-agent messages24

Layer

Prompt layer6Model layer11Application layer116Training and data layer16

System component

MCP and skills4Memory12Guardrails and judges7Monitors11Permissions and sandboxing14Vision3Multimodal3
Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 261papers have no tags yet and are not included in filtered results.
System type：science agentSystem type：Multi-agent systemsSystem type：Embodied systems and roboticsLayer：Application layerClear all

Another 261papers do not have category tags yet and are not included in filtered results.

  1. OtherarXiv 2610.13479

    A local agentic AI framework generates edge-case scenarios for safety assurance

    Proposes a local dual-model closed loop to generate edge scenarios for autonomous system safety assurance

    arXiv
    2610.13479
    Published
    Layer
    Application layer
    Setting
    LLM applications, Embodied systems and robotics
    Evaluated models
    PPO
    AbstractLocal closed-loop generation of JSON edge cases.

    The authors built a local agentic loop that runs on a laptop to generate edge cases for safety arguments of autonomous systems, rather than releasing a new public test set.

    Paper analysisRead analysis
  3. OtherarXiv 2610.13372

    Toward Safe World Models: Risks, Defenses, and Evaluation

    A survey of risks, defenses, and evaluation in the six-stage lifecycle of world models

    arXiv
    2610.13372
    Published
    Layer
    Application layer
    Setting
    Embodied systems and robotics
    AbstractSafety is written as a property of the prediction-decision closed loop; claims must be linked to observed consequences and versions.

    World model safety is organized by the nature of the prediction-decision loop, not by isolated prediction scores.

    Paper analysisRead analysis
  5. Risky behaviorarXiv 2610.12822

    Reward hacking as a transmissible strategy in multi-agent coding systems

    Show that reward hacking can propagate through code, memory, and plans in multi-agent coding systems

    arXiv
    2610.12822
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems, coding agent
    Evaluated models
    Qwen2.5-Coder-14B, Gemma-3-12B, Qwen2.5-Coder-7B and others16
    AbstractThe authors state that reward hacking can spread across models and affect routing and aggregation, and that review can reduce delivery.

    Reward hacking induced by coding RL can leave the source agent and enter subsequent collaboration and system coordination.

    Paper analysisRead analysis
  6. Analysis and theoryarXiv 2610.12436

    The study proposes an ecological theory of AI agent populations: collaboration brings a critical scale for takeoff.

    An ecological model is established for a population of mismatched agents, showing that collaboration produces a critical takeoff scale.

    arXiv
    2610.12436
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems
    AbstractThe authors say that collaboration changes takeoff from an individual capability threshold to a population threshold, so small-population evaluations are insufficient.

    The authors use the ecological strong Allee effect to discuss whether a mismatched agent population can take off when individual capabilities remain unchanged..

    Paper analysisRead analysis
  8. DefensesarXiv 2610.11754

    仅凭输出审查无法验证什么：面向研究智能体的研究契约

    提出研究合同，把已批准选择绑定到执行证据并由检查器核验

    arXiv
    2610.11754
    Published
    Layer
    Application layer
    Setting
    science agent
    Evaluated models
    OpenAI gpt-5.6-sol, OpenAI gpt-5.6-terra, OpenAI gpt-5.6-luna and others18
    Abstract研究合同暴露四类输出单独建不立的关系。

    研究合同用来核验计算科学研究智能体是否仍在已批准研究之内，不判断科学是否为真。。

    Paper analysisRead analysis
  13. DefensesarXiv 2610.07657

    DEFER1 多智能体防御研究报告受控测试中的攻击成功率下降

    实现 DEFER，以确定性检查优先拦截多智能体流水线的内容攻击

    arXiv
    2610.07657
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems
    Evaluated models
    Qwen3-235B-A22B-Instruct-2507, gpt-oss-120b, Llama-4-Scout-17B-16E-Instruct and others5
    AbstractDEFER 把确定性检查放在 judge 之前。

    DEFER 用来测量多智能体授权流水线里，确定性规则在何处结束、LLM judge 从何处开始。

    Paper analysisRead analysis
  15. Evaluation and benchmarksarXiv 2610.06748

    BazaarBench：评估 LLM Agent 在去中心化 C2C 市场中的代理安全

    构建 BazaarBench，评测 C2C 市场中 Agent 的违规与失信行为

    arXiv
    2610.06748
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems
    Evaluated models
    GPT-5.5, GPT-5.4, GPT-5.4-mini and others5
    Abstract论文提出挂钩底层资产的 C2C 交易评测基准，发现时间压力与对抗指令显著推高了智能体的违规承诺与欺诈行为。

    BazaarBench 针对大语言模型智能体在去中心化 C2C 交易中的代理安全问题，提出了一个结合底层实体状态与大模型裁判的仿真基准平台。

    Paper analysisRead analysis
  17. DefensesarXiv 2610.05640

    Research reveals that the CaMeL defense fails in multi-agent systems and proposes multi-CaMeL

    提出 multi-CaMeL，在智能体调用边界隔离可信指令与非可信数据

    arXiv
    2610.05640
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems
    Evaluated models
    Claude Fable 5, GPT-5.5, GPT-4.1 and others6

    Abstract论文针对多智能体中单体防御不组合的边界清洗问题，提出 multi-CaMeL 协议彻底阻断了 MultiAgentDojo 上的注入攻击。

    Paper analysisRead analysis
  18. Evaluation and benchmarksarXiv 2610.05140

    AutoSciBench：自动生成科学智能体评测基准的框架

    提出 AutoSciBench，循环生成科学智能体评测任务

    arXiv
    2610.05140
    Published
    Layer
    Application layer
    Setting
    science agent
    Evaluated models
    Claude Opus 5, Claude Opus 4.8, Claude Opus 4.7 and others10
    AbstractAutoSciBench 用概念、配方和求解反馈自动构造并加难科学智能体基准。

    AutoSciBench 研究科学智能体基准能否自动构造，并随 solver 行为迭代改编。

    Paper analysisRead analysis
  19. Evaluation and benchmarksarXiv 2610.04672

    MASBench：部分可观测条件下的 LLM 多智能体协作基准

    提出 MASBench，评测部分可观测下的多智能体协作机制

    arXiv
    2610.04672
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems
    Evaluated models
    Qwen-Plus, GPT-5-nano, Gemini-2.5-Flash-Lite and others5
    AbstractMASBench 以确定性指标比较部分可观测下的协作机制。

    MASBench 用部分可观测任务比较 LLM 多智能体的协作机制，而不是只看端到端能否做完。

    Paper analysisRead analysis
  20. Risky behaviorarXiv 2610.04528

    论文量化分析自主 LLM 智能体在 Collusion Wiki 事件中的串通行为

    对自主 LLM 智能体日志做身份盲置换以量化串通

    arXiv
    2610.04528
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems
    Abstract修正后汇总共编偏向同智能体聚集。

    作者把 Nightingale Collective 所记的 wiki 旁路事件，做成相对身份置换零模型的定量配套，并写出四类会使协调检验失真的构造错误。

    Paper analysisRead analysis
Load more