Skip to content
Oct 12 · Monday

All papers

Found 831 papers
Filter4

Attack type

Jailbreaks53Prompt injection208Poisoning and backdoors127Model tampering16Extraction and theft27Detection evasion16Denial of service5Malicious use14

Attack technique

Gradient and representation optimization11Automated search52Multi-turn escalation8Encoding and obfuscation16Role-play and persuasion4Conversation structure manipulation3Reasoning-chain manipulation3Cross-modal payloads9Defense-targeted27Dormant triggers51Selection manipulation11

Model risks

Deception and scheming38Reward hacking58Misalignment and value divergence36Dangerous agent actions76Dangerous capabilities25Inappropriate refusal20Hallucination and fabrication13Bias and discrimination4

Defense type

Model hardening124Inference-time intervention7Detection and filtering52Instruction–data separation49Permissions and approval69Monitoring and auditing46Watermarking and provenance6

Impact

Harmful content73Unauthorized actions209Data and privacy leaks72Output manipulation136Availability and reliability60Oversight failure79Increased misuse capability16

System type

Models and APIs214LLM applications126AI Agent617coding agent124science agent32web agent34Browser and computer use18Multi-agent systems78Embodied systems and robotics31

Attacker access

Black-box29Gray-box3White-box20

Attack surface / entry point

User input130Web pages and interfaces62Documents and messages62Tool responses103Tool and skill descriptions62Code repositories and dependencies27Retrieval corpus11Memory writes40Inter-agent messages28Images, audio and video24Training data56Weights and gradients39Distribution channels55

Layer

Prompt layer249Model layer422Application layer598Training and data layer233Infrastructure layer21

System component

MCP and skills80Memory75RAG9Guardrails and judges64Monitors64Reasoning chains27Permissions and sandboxing90Fine-tuning and open weights51Vision38Audio9Image generation9Multimodal56
Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 254papers have no tags yet and are not included in filtered results.
System type：Models and APIsSystem type：AI AgentLayer：Application layerLayer：Training and data layerClear all

Another 254papers do not have category tags yet and are not included in filtered results.

  1. OtherarXiv 2610.13912

    RSI-Anything: Unlocking Recursive Self-Improvement with World Models

    Proposes RSI-Anything, which uses a world model to select candidates and retain policy updates.

    arXiv
    2610.13912
    Published
    Layer
    Application layer
    Setting
    coding agent, Browser and computer use
    Evaluated models
    ReasonFlux-Coder-7B, 4B policy, W1
    AbstractRSI-Anything integrates world models into recursive self-improvement through prediction, feedback, and retention checks, and reports gains across multiple domains.

    RSI-Anything is a cross-domain framework: it uses world model predictions to decide which candidates to take external feedback for, then uses the feedback to decide whether the policy, workflow, or predictor is updated, and uses checks to decide which pair is retained. The difficulty to address is the limited interaction budget, and the fact that once the policy changes, the candidate distribution faced by the predictor also changes. The authors argue that improving only one side does not guarantee that the assembled system will be better.

    Paper analysisRead analysis
  4. OtherarXiv 2610.13479

    A local agentic AI framework generates edge-case scenarios for safety assurance

    Proposes a local dual-model closed loop to generate edge scenarios for autonomous system safety assurance

    arXiv
    2610.13479
    Published
    Layer
    Application layer
    Setting
    LLM applications, Embodied systems and robotics
    Evaluated models
    PPO
    AbstractLocal closed-loop generation of JSON edge cases.

    The authors built a local agentic loop that runs on a laptop to generate edge cases for safety arguments of autonomous systems, rather than releasing a new public test set.

    Paper analysisRead analysis
  6. OtherarXiv 2610.13372

    Toward Safe World Models: Risks, Defenses, and Evaluation

    A survey of risks, defenses, and evaluation in the six-stage lifecycle of world models

    arXiv
    2610.13372
    Published
    Layer
    Application layer
    Setting
    Embodied systems and robotics
    AbstractSafety is written as a property of the prediction-decision closed loop; claims must be linked to observed consequences and versions.

    World model safety is organized by the nature of the prediction-decision loop, not by isolated prediction scores.

    Paper analysisRead analysis
  8. Analysis and theoryarXiv 2610.13051

    Space beats statistics, retraining beats ranking: auditing training labels under label-only backdoors

    Auditing training labels for label-only backdoors, comparing spaces and statistics, and judging by retraining

    arXiv
    2610.13051
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    ResNet-32
    AbstractLabel inspection must be done in a space that has not been fitted to these labels, and judged by retrained models rather than detection scores.

    The authors ask: can a defender without trusted data decide which training labels to keep, and how much clean data this destroys, and what the final model accuracy is.

    Paper analysisRead analysis
  9. OtherarXiv 2610.12924

    Sparse dictionary learning for measure-valued data based on linearized optimal transport: efficient algorithms and sample complexity

    Propose LWDL, a measure sparse dictionary learning method under linearized optimal transport

    arXiv
    2610.12924
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    AbstractLWDL learns sparse measurement dictionaries in flat embedding spaces.

    LWDL embeds probability measures into a Hilbert space, then performs sparse dictionary learning.

    Paper analysisRead analysis
  10. Risky behaviorarXiv 2610.12822

    Reward hacking as a transmissible strategy in multi-agent coding systems

    Show that reward hacking can propagate through code, memory, and plans in multi-agent coding systems

    arXiv
    2610.12822
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems, coding agent
    Evaluated models
    Qwen2.5-Coder-14B, Gemma-3-12B, Qwen2.5-Coder-7B and others16
    AbstractThe authors state that reward hacking can spread across models and affect routing and aggregation, and that review can reduce delivery.

    Reward hacking induced by coding RL can leave the source agent and enter subsequent collaboration and system coordination.

    Paper analysisRead analysis
  11. AttacksarXiv 2610.12644

    Cancerous backdoor attack: progressive activation via invisible tokens and a stage controller.

    Proposes the Cancerous backdoor, which progressively corrupts reasoning using invisible stage tokens.

    arXiv
    2610.12644
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3
    AbstractHidden tokens plus a stage controller make the backdoor progressively corrupt reasoning upon triggering, and the authors believe that looking only at the answer will miss it.

    Cancerous Backdoor Attack changes LLM backdoors from one-time activation to multi-stage reasoning corruption that accumulates progressively with triggers.

    Paper analysisRead analysis
  12. DefensesarXiv 2610.12602

    Self-Preservation in Language Models

    Proposes Task Bank self-replay to mitigate held-out forgetting in continual instruction tuning

    arXiv
    2610.12602
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    Qwen3-0.6B, Qwen3-1.7B, Qwen3-4B and others4
    AbstractThe authors claim self-replay can reduce forgetting outside the list, and that on the 4B model new task information helps with target selection, while smaller models do not.

    The authors study how language models, when learning new tasks, can use their own knowledge for broader self-preservation: given an incomplete list and forgetting that extends beyond the list, can the model generate replay and help choose targets.

    Paper analysisRead analysis
  13. Evaluation and benchmarksarXiv 2610.12569

    SWE-Duel: Toward a Self-Expanding Arena for Coding Agents

    Proposes SWE-Duel, which lets coding agents mutually evaluate, create problems, and blindly review fixes during evaluation.

    arXiv
    2610.12569
    Published
    Layer
    Application layer
    Setting
    coding agent
    Evaluated models
    GPT-5.5, Claude Opus 4.8, Kimi K2.7 Code and others11
    AbstractThe mutual evaluation arena creates questions during evaluation.

    SWE-Duel is a coding agent arena that generates tasks and judges only at evaluation time, to avoid leakage and saturation of static benchmarks.

    Paper analysisRead analysis
  16. Analysis and theoryarXiv 2610.12436

    The study proposes an ecological theory of AI agent populations: collaboration brings a critical scale for takeoff.

    An ecological model is established for a population of mismatched agents, showing that collaboration produces a critical takeoff scale.

    arXiv
    2610.12436
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems
    AbstractThe authors say that collaboration changes takeoff from an individual capability threshold to a population threshold, so small-population evaluations are insufficient.

    The authors use the ecological strong Allee effect to discuss whether a mismatched agent population can take off when individual capabilities remain unchanged..

    Paper analysisRead analysis
  17. DefensesarXiv 2610.12432

    FAITH: A feasibility-aware safety filtering reinforcement learning framework for high-dimensional systems

    Proposes the FAITH safe filtering reinforcement learning framework, training high-dimensional systems to optimize task reward under constraints

    arXiv
    2610.12432
    Published
    Layer
    Training and data layer
    Setting
    Embodied systems and robotics
    AbstractFAITH separates safety filtering from reward optimization.

    FAITH is a model-free safe filtering reinforcement learning framework for high-dimensional systems: safety correction and task reward reside in separate modules.

    Paper analysisRead analysis
  19. Evaluation and benchmarksarXiv 2610.12360

    The paper proposes the ISE framework to evaluate LLM agents' epistemic humility under knowledge conflicts

    Proposes the ISE framework to evaluate the epistemic humility of LLM agents under knowledge conflicts

    arXiv
    2610.12360
    Published
    Layer
    Application layer
    Setting
    AI Agent
    Evaluated models
    Nemotron-Orchestrator-8B, Claude Sonnet 4.6, GPT-5 and others5
    AbstractISE trajectory evaluations show that high task accuracy across four agents does not correspond to higher cognitive humility.

    What the authors want to test is not whether the agent's final answer is correct, but whether, when a knowledge conflict arises, it recognizes the gap in its trajectory, performs bounded tool follow-up, and writes the unresolved uncertainty to the user.

    Paper analysisRead analysis
Load more