Skip to content
Oct 12 · Monday

All papers

Found 1111 papers
Filter2

Attack type

Jailbreaks74Prompt injection222Poisoning and backdoors104Model tampering19Extraction and theft43Detection evasion44Denial of service5Malicious use20

Attack technique

Gradient and representation optimization24Automated search59Multi-turn escalation7Encoding and obfuscation20Role-play and persuasion8Conversation structure manipulation6Reasoning-chain manipulation4Cross-modal payloads12Defense-targeted27Dormant triggers33Selection manipulation12

Model risks

Deception and scheming56Reward hacking38Misalignment and value divergence39Dangerous agent actions77Dangerous capabilities25Inappropriate refusal50Hallucination and fabrication39Bias and discrimination19

Defense type

Model hardening30Inference-time intervention48Detection and filtering61Instruction–data separation53Permissions and approval70Monitoring and auditing60Watermarking and provenance18

Impact

Harmful content105Unauthorized actions204Data and privacy leaks90Output manipulation221Availability and reliability57Oversight failure84Increased misuse capability15

System type

Models and APIs398LLM applications123AI Agent597coding agent123science agent30web agent33Browser and computer use19Multi-agent systems76Embodied systems and robotics26

Attacker access

Black-box47Gray-box8White-box24No-box (offline transfer)3

Attack surface / entry point

User input179Web pages and interfaces68Documents and messages84Tool responses100Tool and skill descriptions59Code repositories and dependencies29Retrieval corpus34Memory writes37Inter-agent messages27Images, audio and video39Training data12Weights and gradients31Distribution channels48Inference services and hardware8

Layer

Prompt layer219Model layer417Application layer694Training and data layer230Infrastructure layer23

System component

MCP and skills82Memory73RAG46Guardrails and judges82Monitors79Reasoning chains43Permissions and sandboxing92Fine-tuning and open weights14Vision62Audio21Video5Image generation22Embeddings12Multimodal102
Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 321papers have no tags yet and are not included in filtered results.
Layer：Model layerLayer：Application layerClear all

Another 321papers do not have category tags yet and are not included in filtered results.

  1. OtherarXiv 2610.13912

    RSI-Anything: Unlocking Recursive Self-Improvement with World Models

    Proposes RSI-Anything, which uses a world model to select candidates and retain policy updates.

    arXiv
    2610.13912
    Published
    Layer
    Application layer
    Setting
    coding agent, Browser and computer use
    Evaluated models
    ReasonFlux-Coder-7B, 4B policy, W1
    AbstractRSI-Anything integrates world models into recursive self-improvement through prediction, feedback, and retention checks, and reports gains across multiple domains.

    RSI-Anything is a cross-domain framework: it uses world model predictions to decide which candidates to take external feedback for, then uses the feedback to decide whether the policy, workflow, or predictor is updated, and uses checks to decide which pair is retained. The difficulty to address is the limited interaction budget, and the fact that once the policy changes, the candidate distribution faced by the predictor also changes. The authors argue that improving only one side does not guarantee that the assembled system will be better.

    Paper analysisRead analysis
  5. OtherarXiv 2610.13479

    A local agentic AI framework generates edge-case scenarios for safety assurance

    Proposes a local dual-model closed loop to generate edge scenarios for autonomous system safety assurance

    arXiv
    2610.13479
    Published
    Layer
    Application layer
    Setting
    LLM applications, Embodied systems and robotics
    Evaluated models
    PPO
    AbstractLocal closed-loop generation of JSON edge cases.

    The authors built a local agentic loop that runs on a laptop to generate edge cases for safety arguments of autonomous systems, rather than releasing a new public test set.

    Paper analysisRead analysis
  7. OtherarXiv 2610.13372

    Toward Safe World Models: Risks, Defenses, and Evaluation

    A survey of risks, defenses, and evaluation in the six-stage lifecycle of world models

    arXiv
    2610.13372
    Published
    Layer
    Application layer
    Setting
    Embodied systems and robotics
    AbstractSafety is written as a property of the prediction-decision closed loop; claims must be linked to observed consequences and versions.

    World model safety is organized by the nature of the prediction-decision loop, not by isolated prediction scores.

    Paper analysisRead analysis
  9. InterpretabilityarXiv 2610.13078

    Attention sees, MLP refuses: tracing the safety pathway from attention to MLP in LLMs

    Trace attention to identify harmfulness to the MLP refusal write pathway and propose Sentinel

    arXiv
    2610.13078
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, Qwen3-8B-Instruct and others18
    AbstractAttention updates are more readable for harmfulness, MLP updates write refusal, and Sentinel routes alerts using two-path signals.

    The authors reframe jailbreaking and excessive refusal of aligned models as a pathway problem on the residual stream from recognition to writing, and accordingly provide conditional reminders before decoding.

    Paper analysisRead analysis
  10. InterpretabilityarXiv 2610.12989

    GPT-2-Small, Gemma-2-2B, and Llama-3.1-8B have interpretable narrative mechanisms

    Using NVC to locate and ablate narrative-related components in language models

    arXiv
    2610.12989
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    GPT-2-Small-124M, Gemma-2-2B, Llama-3.1-8B
    AbstractNeurons and SAE features related to narrativity across three models, when ablated, changed some narrative-related behaviors.

    The authors connect a three-dimensional definition from narratology to mechanistic interventions, examining whether components related to narrativity in pretrained language models participate in prediction, generation, and downstream tasks.

    Paper analysisRead analysis
  11. InterpretabilityarXiv 2610.12842

    Neural networks are more modular than single neurons suggest

    Using a three-level nested decomposition to test whether mixed-selectivity networks can be functionally separated on subspaces

    arXiv
    2610.12842
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Pythia 410M, Pythia 160M, GPT-2 XL and others18
    AbstractUnit non-specialization and subspace separability can coexist.

    The authors propose a nested definition of modularity to separate "neurons not being specialized" from "computation not being separable."

    Paper analysisRead analysis
  12. Risky behaviorarXiv 2610.12822

    Reward hacking as a transmissible strategy in multi-agent coding systems

    Show that reward hacking can propagate through code, memory, and plans in multi-agent coding systems

    arXiv
    2610.12822
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems, coding agent
    Evaluated models
    Qwen2.5-Coder-14B, Gemma-3-12B, Qwen2.5-Coder-7B and others16
    AbstractThe authors state that reward hacking can spread across models and affect routing and aggregation, and that review can reduce delivery.

    Reward hacking induced by coding RL can leave the source agent and enter subsequent collaboration and system coordination.

    Paper analysisRead analysis
  13. Evaluation and benchmarksarXiv 2610.12758

    DistressBench: Evaluating multi-dimensional crisis support in large language models

    Introduces DistressBench, evaluating models' multidimensional support in crisis conversations

    arXiv
    2610.12758
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Muse Spark 1.2, Muse Spark 1.3, Muse Glimmer 30B and others20
    AbstractThe model is more restrained and acts less.

    DistressBench uses SSH dialogues written and revised by clinical personnel, and evaluates how much necessary care general chat models deliver according to weighted atomic criteria, rather than whether they refuse. When users disclose suicidal ideation or self-harm to a chat model, the authors consider the expected behavior to be recognizing the crisis, calibrating empathy, de-escalating, and referring to human help. Refusal-based evaluations cannot detect failing to act after recognizing the crisis.

    Paper analysisRead analysis
  14. Evaluation and benchmarksarXiv 2610.12569

    SWE-Duel: Toward a Self-Expanding Arena for Coding Agents

    Proposes SWE-Duel, which lets coding agents mutually evaluate, create problems, and blindly review fixes during evaluation.

    arXiv
    2610.12569
    Published
    Layer
    Application layer
    Setting
    coding agent
    Evaluated models
    GPT-5.5, Claude Opus 4.8, Kimi K2.7 Code and others11
    AbstractThe mutual evaluation arena creates questions during evaluation.

    SWE-Duel is a coding agent arena that generates tasks and judges only at evaluation time, to avoid leakage and saturation of static benchmarks.

    Paper analysisRead analysis
  17. Analysis and theoryarXiv 2610.12436

    The study proposes an ecological theory of AI agent populations: collaboration brings a critical scale for takeoff.

    An ecological model is established for a population of mismatched agents, showing that collaboration produces a critical takeoff scale.

    arXiv
    2610.12436
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems
    AbstractThe authors say that collaboration changes takeoff from an individual capability threshold to a population threshold, so small-population evaluations are insufficient.

    The authors use the ecological strong Allee effect to discuss whether a mismatched agent population can take off when individual capabilities remain unchanged..

    Paper analysisRead analysis
  18. InterpretabilityarXiv 2610.12410

    The paper proposes an alignment generalization prediction task, using activation representations to predict the cross-context effects of fine-tuned values.

    Comparing the predictive power of value representations for alignment generalization after single-value fine-tuning

    arXiv
    2610.12410
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Olmo-3-7B, Olmo-3-32B, Qwen-3-8B-Base and others4
    AbstractActivation representations can predict cross-value generalization before training and relate to robustness across multiple value objectives.

    The authors define "how single-value fine-tuning changes compliance on untrained values" as a task predictable before training, and use value representations to do this.

    Paper analysisRead analysis
  19. Analysis and theoryarXiv 2610.12409

    The researchers conduct a psychometric review of HELM Safety and HarmBench, questioning whether their scores can represent a single attribute

    Auditing whether HarmBench scores can represent a single harmful refusal attribute

    arXiv
    2610.12409
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    AbstractThe authors claim HarmBench does not correspond to a single harmful refusal, and behaviors should be separated when comparing models.

    This is a construct validity audit of HarmBench in HELM Safety, examining whether harmful refusal can be read as a single attribute when interpreting scores.

    Paper analysisRead analysis
Load more