Skip to content
Oct 12 · Monday

All papers

Found 615 papers
Filter5

Attack type

Jailbreaks44Prompt injection51Poisoning and backdoors37Model tampering20Extraction and theft40Detection evasion35Malicious use9

Attack technique

Gradient and representation optimization20Automated search15Encoding and obfuscation7Role-play and persuasion6Conversation structure manipulation4Cross-modal payloads4Defense-targeted8Dormant triggers8

Model risks

Deception and scheming33Reward hacking16Misalignment and value divergence29Dangerous agent actions8Dangerous capabilities5Inappropriate refusal45Hallucination and fabrication29Bias and discrimination21

Defense type

Model hardening18Inference-time intervention41Detection and filtering32Instruction–data separation10Permissions and approval12Monitoring and auditing22Watermarking and provenance14

Impact

Harmful content88Unauthorized actions16Data and privacy leaks48Output manipulation165Availability and reliability18Oversight failure45

System type

Models and APIs409LLM applications129AI Agent619coding agent126science agent30web agent34Browser and computer use19Multi-agent systems80Embodied systems and robotics27

Attacker access

Black-box31Gray-box6White-box22

Attack surface / entry point

User input99Web pages and interfaces9Documents and messages31Tool responses9Tool and skill descriptions4Code repositories and dependencies3Retrieval corpus29Memory writes6Inter-agent messages25Images, audio and video24Training data11Weights and gradients31Distribution channels5Inference services and hardware7

Layer

Prompt layer248Model layer405Application layer210Training and data layer210Infrastructure layer22

System component

MCP and skills7Memory15RAG42Guardrails and judges50Monitors28Reasoning chains34Permissions and sandboxing14Fine-tuning and open weights11Vision37Audio18Video4Image generation21Embeddings13Multimodal74
Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 240papers have no tags yet and are not included in filtered results.
System type：Models and APIsSystem type：LLM applicationsSystem type：Multi-agent systemsLayer：Model layerLayer：Application layerClear all

Another 240papers do not have category tags yet and are not included in filtered results.

  1. AttacksarXiv 2610.13928

    AI models can provably hide arbitrary capabilities

    Proposes digital lock compilation, encrypting executable networks into host weights to hide arbitrary capabilities

    arXiv
    2610.13928
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Gemma 3 270M
    AbstractThe authors use a digital lock to hide an executable capability and state that elicitation on the chemistry and parity tasks did not recover the capability.

    The authors construct a class of backdoors that encrypt an executable network into the weights, used to discuss the limits of white-box elicitation audits under supply chain attacks.

    Paper analysisRead analysis
  2. OtherarXiv 2610.13810

    Using subsequence association filtering and breadth-aware selection to suppress hallucinated words in LLM query expansion

    Proposes SAFE-QE, which filters unsupported hallucinated words in LLM query expansion

    arXiv
    2610.13810
    Published
    Layer
    Application layer
    Setting
    LLM applications
    Evaluated models
    BM25, SPLADE, Contriever

    AbstractSAFE-QE suppresses expansion hallucinations with reproducibility and coverage selection; the authors say ranking quality shows no visible cost.

    Paper analysisRead analysis
  4. OtherarXiv 2610.13491

    EviRepair: Evidence-Driven Agents for Reliable Vulnerability Repair

    Proposing EviRepair, an evidence-driven multi-agent framework for repository-level vulnerability repair

    arXiv
    2610.13491
    Published
    Layer
    Application layer
    Setting
    coding agent, Multi-agent systems
    Evaluated models
    DeepSeek-V4.1-Flash, GLM-5.2, Qwen3.8-27B
    AbstractEviRepair uses a reviewed vulnerability model for repository-level repair.

    EviRepair is a repository-level automated vulnerability repair framework for sparse vulnerability reports, using a reviewed vulnerability model to separate diagnosis from patch synthesis.

    Paper analysisRead analysis
  5. OtherarXiv 2610.13479

    A local agentic AI framework generates edge-case scenarios for safety assurance

    Proposes a local dual-model closed loop to generate edge scenarios for autonomous system safety assurance

    arXiv
    2610.13479
    Published
    Layer
    Application layer
    Setting
    LLM applications, Embodied systems and robotics
    Evaluated models
    PPO
    AbstractLocal closed-loop generation of JSON edge cases.

    The authors built a local agentic loop that runs on a laptop to generate edge cases for safety arguments of autonomous systems, rather than releasing a new public test set.

    Paper analysisRead analysis
  6. InterpretabilityarXiv 2610.13078

    Attention sees, MLP refuses: tracing the safety pathway from attention to MLP in LLMs

    Trace attention to identify harmfulness to the MLP refusal write pathway and propose Sentinel

    arXiv
    2610.13078
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, Qwen3-8B-Instruct and others18
    AbstractAttention updates are more readable for harmfulness, MLP updates write refusal, and Sentinel routes alerts using two-path signals.

    The authors reframe jailbreaking and excessive refusal of aligned models as a pathway problem on the residual stream from recognition to writing, and accordingly provide conditional reminders before decoding.

    Paper analysisRead analysis
  7. InterpretabilityarXiv 2610.12989

    GPT-2-Small, Gemma-2-2B, and Llama-3.1-8B have interpretable narrative mechanisms

    Using NVC to locate and ablate narrative-related components in language models

    arXiv
    2610.12989
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    GPT-2-Small-124M, Gemma-2-2B, Llama-3.1-8B
    AbstractNeurons and SAE features related to narrativity across three models, when ablated, changed some narrative-related behaviors.

    The authors connect a three-dimensional definition from narratology to mechanistic interventions, examining whether components related to narrativity in pretrained language models participate in prediction, generation, and downstream tasks.

    Paper analysisRead analysis
  8. InterpretabilityarXiv 2610.12842

    Neural networks are more modular than single neurons suggest

    Using a three-level nested decomposition to test whether mixed-selectivity networks can be functionally separated on subspaces

    arXiv
    2610.12842
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Pythia 410M, Pythia 160M, GPT-2 XL and others18
    AbstractUnit non-specialization and subspace separability can coexist.

    The authors propose a nested definition of modularity to separate "neurons not being specialized" from "computation not being separable."

    Paper analysisRead analysis
  9. Risky behaviorarXiv 2610.12822

    Reward hacking as a transmissible strategy in multi-agent coding systems

    Show that reward hacking can propagate through code, memory, and plans in multi-agent coding systems

    arXiv
    2610.12822
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems, coding agent
    Evaluated models
    Qwen2.5-Coder-14B, Gemma-3-12B, Qwen2.5-Coder-7B and others16
    AbstractThe authors state that reward hacking can spread across models and affect routing and aggregation, and that review can reduce delivery.

    Reward hacking induced by coding RL can leave the source agent and enter subsequent collaboration and system coordination.

    Paper analysisRead analysis
  10. Evaluation and benchmarksarXiv 2610.12758

    DistressBench: Evaluating multi-dimensional crisis support in large language models

    Introduces DistressBench, evaluating models' multidimensional support in crisis conversations

    arXiv
    2610.12758
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Muse Spark 1.2, Muse Spark 1.3, Muse Glimmer 30B and others20
    AbstractThe model is more restrained and acts less.

    DistressBench uses SSH dialogues written and revised by clinical personnel, and evaluates how much necessary care general chat models deliver according to weighted atomic criteria, rather than whether they refuse. When users disclose suicidal ideation or self-harm to a chat model, the authors consider the expected behavior to be recognizing the crisis, calibrating empathy, de-escalating, and referring to human help. Refusal-based evaluations cannot detect failing to act after recognizing the crisis.

    Paper analysisRead analysis
  11. Analysis and theoryarXiv 2610.12436

    The study proposes an ecological theory of AI agent populations: collaboration brings a critical scale for takeoff.

    An ecological model is established for a population of mismatched agents, showing that collaboration produces a critical takeoff scale.

    arXiv
    2610.12436
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems
    AbstractThe authors say that collaboration changes takeoff from an individual capability threshold to a population threshold, so small-population evaluations are insufficient.

    The authors use the ecological strong Allee effect to discuss whether a mismatched agent population can take off when individual capabilities remain unchanged..

    Paper analysisRead analysis
  12. InterpretabilityarXiv 2610.12410

    The paper proposes an alignment generalization prediction task, using activation representations to predict the cross-context effects of fine-tuned values.

    Comparing the predictive power of value representations for alignment generalization after single-value fine-tuning

    arXiv
    2610.12410
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Olmo-3-7B, Olmo-3-32B, Qwen-3-8B-Base and others4
    AbstractActivation representations can predict cross-value generalization before training and relate to robustness across multiple value objectives.

    The authors define "how single-value fine-tuning changes compliance on untrained values" as a task predictable before training, and use value representations to do this.

    Paper analysisRead analysis
  13. Analysis and theoryarXiv 2610.12409

    The researchers conduct a psychometric review of HELM Safety and HarmBench, questioning whether their scores can represent a single attribute

    Auditing whether HarmBench scores can represent a single harmful refusal attribute

    arXiv
    2610.12409
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    AbstractThe authors claim HarmBench does not correspond to a single harmful refusal, and behaviors should be separated when comparing models.

    This is a construct validity audit of HarmBench in HELM Safety, examining whether harmful refusal can be read as a single attribute when interpreting scores.

    Paper analysisRead analysis
  14. InterpretabilityarXiv 2610.12361

    Study: legal reasoning models citing legal provisions does not equal relying on legal provisions.

    Uses premise permutation and hidden-layer tracing to audit whether legal chain-of-thought truly relies on the cited legal provisions.

    arXiv
    2610.12361
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Qwen3-8B, Qwen3-14B, Llama-3.1-8B and others7
    AbstractUsing counterfactual substitution of authorities and internal commitment tracing, the authors find that legal CoT rulings depend on authorities only to a limited extent and are vulnerable to prompt injection manipulation.

    This study conducts an empirical audit of the causal truthfulness of legal citations in chain-of-thought reasoning of large language models in the legal domain.

    Paper analysisRead analysis
  15. AttacksarXiv 2610.12292

    The paper proposes an option channel attack: changing one option label can make an agent guardrail allow a prohibited operation

    Proposing an option channel attack that renames option labels so that agent guardrails allow prohibited operations

    arXiv
    2610.12292
    Published
    Layer
    Application layer
    Setting
    LLM applications
    Attacker
    Black-box
    Evaluated models
    LAYA-TD, LAYA-EN, LAYA-ML and others5

    AbstractThe authors' evaluation indicates that typed models should not serve as final safety arbiters; they are susceptible to off-policy text and option name manipulation, and confidence filtering cannot resist attacks.

    Paper analysisRead analysis
  16. OtherarXiv 2610.12235

    The paper proposes using language models as world models for AI research to predict experimental results.

    Propose studying world models, using language models to predict intervention gains

    arXiv
    2610.12235
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Claude Opus 5, Grok-4.6, GPT-6 Astra and others13
    AbstractThe authors claim experimental records can be reused inside and outside the environment and improve experiment selection under fixed budgets.

    The authors use language models as research world models, predicting the median intervention gain before execution, to compare candidate experiments under a limited budget. The gap to address is that research agents propose faster than they execute. The same type of change can have different effects across reference models, recipes, and budgets, so experimental knowledge that can continue to be used outside the original environment is needed.

    Paper analysisRead analysis
  17. Analysis and theoryarXiv 2610.12083

    Paper audits LLM judge reliability: repeated agreement does not equal correctness

    Auditing LLM judge reliability and defining trustworthy verdict rate and accuracy upper bound

    arXiv
    2610.12083
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    GPT-5.2, GPT-4o, Gemini-2.5-Pro and others6
    AbstractThe authors claim that comparing judges should look at T, not just accuracy.

    The authors audit whether LLM-as-a-Judge decisions can still be treated as actionable ground truth under repeated queries, swapped candidate order, and changed prompt formats.

    Paper analysisRead analysis
  19. AttacksarXiv 2610.11932

    A Fudan team measures the citation fragility of AI search: ordinary posts can enter citations and answer text

    Measures citation fragility in AI search, demonstrating that ordinary public posts can enter citations and answers.

    arXiv
    2610.11932
    Published
    Layer
    Application layer
    Setting
    LLM applications
    Evaluated models
    Grok, Doubao, ChatGPT and others10

    AbstractThe authors claim the paper quantifies vulnerabilities in AI search caused by citation preferences and low posting thresholds; tests show public web posts can penetrate AI citations, and they call for auditing the citation supply chain.

    Paper analysisRead analysis
  20. DefensesarXiv 2610.11848

    SemField: training-free semantic watermark

    Proposes the semantic watermark SemField, which still identifies the attribution of generated text after paraphrasing and structural changes.

    arXiv
    2610.11848
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Llama-3.2-1B, Qwen3-1.7B, OPT-1.3B
    AbstractTraining-independent document-level semantic watermark.

    SemField is a zero-bit semantic watermark embedded by the provider at generation time and detected later with a key, used to identify the provider's own model text after paraphrasing and sentence-level structural changes.

    Paper analysisRead analysis
Load more