Skip to content
Oct 12 · Monday

All papers

Found 786 papers
Filter4

Attack type

Jailbreaks43Prompt injection231Poisoning and backdoors110Extraction and theft20Detection evasion24Denial of service6Malicious use14

Attack technique

Gradient and representation optimization21Automated search62Multi-turn escalation8Encoding and obfuscation23Role-play and persuasion9Conversation structure manipulation5Reasoning-chain manipulation5Cross-modal payloads15Defense-targeted21Dormant triggers40Selection manipulation12

Model risks

Deception and scheming38Reward hacking34Misalignment and value divergence23Dangerous agent actions84Dangerous capabilities25Inappropriate refusal9Hallucination and fabrication19Bias and discrimination3

Defense type

Model hardening25Inference-time intervention13Detection and filtering59Instruction–data separation54Permissions and approval72Monitoring and auditing45Watermarking and provenance5

Impact

Harmful content38Unauthorized actions216Data and privacy leaks69Output manipulation170Availability and reliability55Oversight failure61Increased misuse capability15

System type

Models and APIs869LLM applications139AI Agent654coding agent136science agent32web agent34Browser and computer use19Multi-agent systems79Embodied systems and robotics47

Attacker access

Black-box37Gray-box4White-box6

Attack surface / entry point

User input133Web pages and interfaces69Documents and messages79Tool responses104Tool and skill descriptions62Code repositories and dependencies30Retrieval corpus35Memory writes40Inter-agent messages28Images, audio and video27Training data8Weights and gradients8Distribution channels48Inference services and hardware5

Layer

Prompt layer17Model layer29Application layer707Training and data layer29Infrastructure layer4

System component

MCP and skills82Memory76RAG50Guardrails and judges85Monitors66Reasoning chains18Permissions and sandboxing93Fine-tuning and open weights8Vision40Audio6Image generation3Embeddings9Multimodal48
Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 254papers have no tags yet and are not included in filtered results.
System type：LLM applicationsSystem type：AI AgentSystem type：web agentSystem type：Multi-agent systemsClear all

Another 254papers do not have category tags yet and are not included in filtered results.

  1. OtherarXiv 2610.13912

    RSI-Anything: Unlocking Recursive Self-Improvement with World Models

    Proposes RSI-Anything, which uses a world model to select candidates and retain policy updates.

    arXiv
    2610.13912
    Published
    Layer
    Application layer
    Setting
    coding agent, Browser and computer use
    Evaluated models
    ReasonFlux-Coder-7B, 4B policy, W1
    AbstractRSI-Anything integrates world models into recursive self-improvement through prediction, feedback, and retention checks, and reports gains across multiple domains.

    RSI-Anything is a cross-domain framework: it uses world model predictions to decide which candidates to take external feedback for, then uses the feedback to decide whether the policy, workflow, or predictor is updated, and uses checks to decide which pair is retained. The difficulty to address is the limited interaction budget, and the fact that once the policy changes, the candidate distribution faced by the predictor also changes. The authors argue that improving only one side does not guarantee that the assembled system will be better.

    Paper analysisRead analysis
  4. OtherarXiv 2610.13810

    Using subsequence association filtering and breadth-aware selection to suppress hallucinated words in LLM query expansion

    Proposes SAFE-QE, which filters unsupported hallucinated words in LLM query expansion

    arXiv
    2610.13810
    Published
    Layer
    Application layer
    Setting
    LLM applications
    Evaluated models
    BM25, SPLADE, Contriever

    AbstractSAFE-QE suppresses expansion hallucinations with reproducibility and coverage selection; the authors say ranking quality shows no visible cost.

    Paper analysisRead analysis
  5. OtherarXiv 2610.13479

    A local agentic AI framework generates edge-case scenarios for safety assurance

    Proposes a local dual-model closed loop to generate edge scenarios for autonomous system safety assurance

    arXiv
    2610.13479
    Published
    Layer
    Application layer
    Setting
    LLM applications, Embodied systems and robotics
    Evaluated models
    PPO
    AbstractLocal closed-loop generation of JSON edge cases.

    The authors built a local agentic loop that runs on a laptop to generate edge cases for safety arguments of autonomous systems, rather than releasing a new public test set.

    Paper analysisRead analysis
  7. OtherarXiv 2610.13372

    Toward Safe World Models: Risks, Defenses, and Evaluation

    A survey of risks, defenses, and evaluation in the six-stage lifecycle of world models

    arXiv
    2610.13372
    Published
    Layer
    Application layer
    Setting
    Embodied systems and robotics
    AbstractSafety is written as a property of the prediction-decision closed loop; claims must be linked to observed consequences and versions.

    World model safety is organized by the nature of the prediction-decision loop, not by isolated prediction scores.

    Paper analysisRead analysis
  9. Risky behaviorarXiv 2610.12822

    Reward hacking as a transmissible strategy in multi-agent coding systems

    Show that reward hacking can propagate through code, memory, and plans in multi-agent coding systems

    arXiv
    2610.12822
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems, coding agent
    Evaluated models
    Qwen2.5-Coder-14B, Gemma-3-12B, Qwen2.5-Coder-7B and others16
    AbstractThe authors state that reward hacking can spread across models and affect routing and aggregation, and that review can reduce delivery.

    Reward hacking induced by coding RL can leave the source agent and enter subsequent collaboration and system coordination.

    Paper analysisRead analysis
  10. Evaluation and benchmarksarXiv 2610.12569

    SWE-Duel: Toward a Self-Expanding Arena for Coding Agents

    Proposes SWE-Duel, which lets coding agents mutually evaluate, create problems, and blindly review fixes during evaluation.

    arXiv
    2610.12569
    Published
    Layer
    Application layer
    Setting
    coding agent
    Evaluated models
    GPT-5.5, Claude Opus 4.8, Kimi K2.7 Code and others11
    AbstractThe mutual evaluation arena creates questions during evaluation.

    SWE-Duel is a coding agent arena that generates tasks and judges only at evaluation time, to avoid leakage and saturation of static benchmarks.

    Paper analysisRead analysis
  13. Analysis and theoryarXiv 2610.12436

    The study proposes an ecological theory of AI agent populations: collaboration brings a critical scale for takeoff.

    An ecological model is established for a population of mismatched agents, showing that collaboration produces a critical takeoff scale.

    arXiv
    2610.12436
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems
    AbstractThe authors say that collaboration changes takeoff from an individual capability threshold to a population threshold, so small-population evaluations are insufficient.

    The authors use the ecological strong Allee effect to discuss whether a mismatched agent population can take off when individual capabilities remain unchanged..

    Paper analysisRead analysis
  14. DefensesarXiv 2610.12432

    FAITH: A feasibility-aware safety filtering reinforcement learning framework for high-dimensional systems

    Proposes the FAITH safe filtering reinforcement learning framework, training high-dimensional systems to optimize task reward under constraints

    arXiv
    2610.12432
    Published
    Layer
    Training and data layer
    Setting
    Embodied systems and robotics
    AbstractFAITH separates safety filtering from reward optimization.

    FAITH is a model-free safe filtering reinforcement learning framework for high-dimensional systems: safety correction and task reward reside in separate modules.

    Paper analysisRead analysis
  16. Evaluation and benchmarksarXiv 2610.12360

    The paper proposes the ISE framework to evaluate LLM agents' epistemic humility under knowledge conflicts

    Proposes the ISE framework to evaluate the epistemic humility of LLM agents under knowledge conflicts

    arXiv
    2610.12360
    Published
    Layer
    Application layer
    Setting
    AI Agent
    Evaluated models
    Nemotron-Orchestrator-8B, Claude Sonnet 4.6, GPT-5 and others5
    AbstractISE trajectory evaluations show that high task accuracy across four agents does not correspond to higher cognitive humility.

    What the authors want to test is not whether the agent's final answer is correct, but whether, when a knowledge conflict arises, it recognizes the gap in its trajectory, performs bounded tool follow-up, and writes the unresolved uncertainty to the user.

    Paper analysisRead analysis
  17. AttacksarXiv 2610.12292

    The paper proposes an option channel attack: changing one option label can make an agent guardrail allow a prohibited operation

    Proposing an option channel attack that renames option labels so that agent guardrails allow prohibited operations

    arXiv
    2610.12292
    Published
    Layer
    Application layer
    Setting
    LLM applications
    Attacker
    Black-box
    Evaluated models
    LAYA-TD, LAYA-EN, LAYA-ML and others5

    AbstractThe authors' evaluation indicates that typed models should not serve as final safety arbiters; they are susceptible to off-policy text and option name manipulation, and confidence filtering cannot resist attacks.

    Paper analysisRead analysis
  19. OtherarXiv 2610.11959

    MiMo-V2.6: Toward Self-Improvement via Scaled Reinforcement Learning

    Extending reinforcement learning for full-modality foundation models to support agent self-improvement

    arXiv
    2610.11959
    Published
    Layer
    Training and data layer
    Setting
    coding agent
    Evaluated models
    MiMo-V2.6-Pro, MiMo-V2.6-Flash, MiMo-V2.6-Distill-Qwen-9B
    AbstractThe series uses three-dimensional RL and group scoring to improve agent scores; the authors claim the final performance is comparable to frontier models.

    MiMo-V2.6 is an all-modality MoE technical report, pursuing agent self-improvement by scaling up RL computation, environments, and scoring.

    Paper analysisRead analysis
Load more