Skip to content
Oct 12 · Monday

All papers

Found 815 papers
Filter1

Attack type

Jailbreaks189Prompt injection15Poisoning and backdoors46Model tampering32Extraction and theft56Detection evasion49Denial of service4Malicious use12

Attack technique

Gradient and representation optimization37Automated search47Multi-turn escalation19Encoding and obfuscation31Role-play and persuasion21Conversation structure manipulation11Reasoning-chain manipulation9Cross-modal payloads37Side channels3Defense-targeted22Dormant triggers16

Model risks

Deception and scheming26Reward hacking36Misalignment and value divergence34Dangerous capabilities5Inappropriate refusal58Hallucination and fabrication23Bias and discrimination19

Defense type

Model hardening116Inference-time intervention38Detection and filtering44Instruction–data separation6Monitoring and auditing21Watermarking and provenance15

Impact

Harmful content251Data and privacy leaks59Output manipulation154Availability and reliability32Oversight failure60Increased misuse capability3

System type

Models and APIs815LLM applications128AI Agent633coding agent133science agent32web agent33Browser and computer use19Multi-agent systems76Embodied systems and robotics45

Attacker access

Black-box113Gray-box11White-box49No-box (offline transfer)5

Attack surface / entry point

User input246Documents and messages8Tool responses5Retrieval corpus3Images, audio and video86Training data62Weights and gradients58Distribution channels11Inference services and hardware17

Layer

Prompt layer201Model layer387Application layer9Training and data layer199Infrastructure layer19

System component

Guardrails and judges78Monitors25Reasoning chains55Fine-tuning and open weights54Vision85Audio31Video11Image generation43Embeddings4Multimodal161
Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 336papers have no tags yet and are not included in filtered results.
System type：Models and APIsClear all

Another 336papers do not have category tags yet and are not included in filtered results.

  1. DefensesarXiv 2610.13733

    KVeil: First-Place Solution for the ECCV 2026 LifeGenIP Unlearnable Video Challenge

    Propose KVeil, which adds perturbations to face videos to prevent unauthorized customization from reproducing identity

    arXiv
    2610.13733
    Published
    Layer
    Prompt layer
    Setting
    Models and APIs
    Attacker
    White-box、Black-box
    Evaluated models
    Wan2.2-TI2V-5B、HunyuanVideo-1.5、Kling 3.0 and others4

    AbstractKVeil adapts LETHE's reference-reading disruption to the challenge's two stages, ranking first on the leaderboard in both Final columns.

    Deep readingFull reading
  3. DefensesarXiv 2610.13333

    LETHE: Defending Against Unauthorized Image-to-Video Generation via Value Diversity Collapse

    Proposes LETHE, which perturbs reference images to block unauthorized image-to-video

    arXiv
    2610.13333
    Published
    Layer
    Prompt layer
    Setting
    Models and APIs
    Evaluated models
    Wan2.1-I2V-14B、CogVideoX-5B-I2V、LightX2V and others6
    AbstractLETHE suppresses the diversity of reference values and attracts attention, making protected images difficult to animate faithfully.

    LETHE is a protective perturbation added to personal images before publication, used to counter unauthorized image-to-video generation.

    Deep readingFull reading
  4. InterpretabilityarXiv 2610.13078

    Attention sees, MLP refuses: tracing the safety pathway from attention to MLP in LLMs

    Trace attention to identify harmfulness to the MLP refusal write pathway and propose Sentinel

    arXiv
    2610.13078
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Llama-3.1-8B-Instruct、Qwen2.5-7B-Instruct、Qwen3-8B-Instruct and others18
    AbstractAttention updates are more readable for harmfulness, MLP updates write refusal, and Sentinel routes alerts using two-path signals.

    The authors reframe jailbreaking and excessive refusal of aligned models as a pathway problem on the residual stream from recognition to writing, and accordingly provide conditional reminders before decoding.

    Deep readingFull reading
  5. Analysis and theoryarXiv 2610.13051

    Space beats statistics, retraining beats ranking: auditing training labels under label-only backdoors

    Auditing training labels for label-only backdoors, comparing spaces and statistics, and judging by retraining

    arXiv
    2610.13051
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    ResNet-32
    AbstractLabel inspection must be done in a space that has not been fitted to these labels, and judged by retrained models rather than detection scores.

    The authors ask: can a defender without trusted data decide which training labels to keep, and how much clean data this destroys, and what the final model accuracy is.

    Deep readingFull reading
  6. InterpretabilityarXiv 2610.12989

    GPT-2-Small, Gemma-2-2B, and Llama-3.1-8B have interpretable narrative mechanisms

    Using NVC to locate and ablate narrative-related components in language models

    arXiv
    2610.12989
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    GPT-2-Small-124M、Gemma-2-2B、Llama-3.1-8B
    AbstractNeurons and SAE features related to narrativity across three models, when ablated, changed some narrative-related behaviors.

    The authors connect a three-dimensional definition from narratology to mechanistic interventions, examining whether components related to narrativity in pretrained language models participate in prediction, generation, and downstream tasks.

    Deep readingFull reading
  7. OtherarXiv 2610.12924

    Sparse dictionary learning for measure-valued data based on linearized optimal transport: efficient algorithms and sample complexity

    Propose LWDL, a measure sparse dictionary learning method under linearized optimal transport

    arXiv
    2610.12924
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    AbstractLWDL learns sparse measurement dictionaries in flat embedding spaces.

    LWDL embeds probability measures into a Hilbert space, then performs sparse dictionary learning.

    Deep readingFull reading
  8. InterpretabilityarXiv 2610.12842

    Neural networks are more modular than single neurons suggest

    Using a three-level nested decomposition to test whether mixed-selectivity networks can be functionally separated on subspaces

    arXiv
    2610.12842
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Pythia 410M、Pythia 160M、GPT-2 XL and others18
    AbstractUnit non-specialization and subspace separability can coexist.

    The authors propose a nested definition of modularity to separate "neurons not being specialized" from "computation not being separable."

    Deep readingFull reading
  9. Evaluation and benchmarksarXiv 2610.12758

    DistressBench: Evaluating multi-dimensional crisis support in large language models

    Introduces DistressBench, evaluating models' multidimensional support in crisis conversations

    arXiv
    2610.12758
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Muse Spark 1.2、Muse Spark 1.3、Muse Glimmer 30B and others20
    AbstractThe model is more restrained and acts less.

    DistressBench uses SSH dialogues written and revised by clinical personnel, and evaluates how much necessary care general chat models deliver according to weighted atomic criteria, rather than whether they refuse. When users disclose suicidal ideation or self-harm to a chat model, the authors consider the expected behavior to be recognizing the crisis, calibrating empathy, de-escalating, and referring to human help. Refusal-based evaluations cannot detect failing to act after recognizing the crisis.

    Deep readingFull reading
  10. AttacksarXiv 2610.12644

    Cancerous backdoor attack: progressive activation via invisible tokens and a stage controller.

    Proposes the Cancerous backdoor, which progressively corrupts reasoning using invisible stage tokens.

    arXiv
    2610.12644
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    Llama-3.1-8B-Instruct、Qwen2.5-7B-Instruct、Mistral-7B-Instruct-v0.3
    AbstractHidden tokens plus a stage controller make the backdoor progressively corrupt reasoning upon triggering, and the authors believe that looking only at the answer will miss it.

    Cancerous Backdoor Attack changes LLM backdoors from one-time activation to multi-stage reasoning corruption that accumulates progressively with triggers.

    Deep readingFull reading
  11. InterpretabilityarXiv 2610.12410

    The paper proposes an alignment generalization prediction task, using activation representations to predict the cross-context effects of fine-tuned values.

    Comparing the predictive power of value representations for alignment generalization after single-value fine-tuning

    arXiv
    2610.12410
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Olmo-3-7B、Olmo-3-32B、Qwen-3-8B-Base and others4
    AbstractActivation representations can predict cross-value generalization before training and relate to robustness across multiple value objectives.

    The authors define "how single-value fine-tuning changes compliance on untrained values" as a task predictable before training, and use value representations to do this.

    Deep readingFull reading
  12. Analysis and theoryarXiv 2610.12409

    The researchers conduct a psychometric review of HELM Safety and HarmBench, questioning whether their scores can represent a single attribute

    Auditing whether HarmBench scores can represent a single harmful refusal attribute

    arXiv
    2610.12409
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    AbstractThe authors claim HarmBench does not correspond to a single harmful refusal, and behaviors should be separated when comparing models.

    This is a construct validity audit of HarmBench in HELM Safety, examining whether harmful refusal can be read as a single attribute when interpreting scores.

    Deep readingFull reading
  13. InterpretabilityarXiv 2610.12361

    Study: legal reasoning models citing legal provisions does not equal relying on legal provisions.

    Uses premise permutation and hidden-layer tracing to audit whether legal chain-of-thought truly relies on the cited legal provisions.

    arXiv
    2610.12361
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Qwen3-8B、Qwen3-14B、Llama-3.1-8B and others7
    AbstractUsing counterfactual substitution of authorities and internal commitment tracing, the authors find that legal CoT rulings depend on authorities only to a limited extent and are vulnerable to prompt injection manipulation.

    This study conducts an empirical audit of the causal truthfulness of legal citations in chain-of-thought reasoning of large language models in the legal domain.

    Deep readingFull reading
  14. Analysis and theoryarXiv 2610.12313

    The study tests the sensitivity of five large-model compliance systems to perturbations of regulatory rules.

    Diagnose whether compliance system rulings change with perturbations to regulatory rules

    arXiv
    2610.12313
    Published
    Layer
    Prompt layer
    Setting
    Models and APIs
    Evaluated models
    Llama-3.3-70B-Instruct、Llama-3.1-8B-Instruct、Qwen2.5-7B-Instruct and others5
    AbstractAcross five models, many correct verdicts are largely unchanged by deleting or replacing negation rules.

    Use behavioral counterfactuals and activation probes to audit whether compliance rulings rely on the given rules. The authors aim to separate two kinds of seemingly identical correct rulings: those that use the given rules, and those obtained from case surface features or training patterns.

    Deep readingFull reading
  15. OtherarXiv 2610.12235

    The paper proposes using language models as world models for AI research to predict experimental results.

    Propose studying world models, using language models to predict intervention gains

    arXiv
    2610.12235
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Claude Opus 5、Grok-4.6、GPT-6 Astra and others13
    AbstractThe authors claim experimental records can be reused inside and outside the environment and improve experiment selection under fixed budgets.

    The authors use language models as research world models, predicting the median intervention gain before execution, to compare candidate experiments under a limited budget. The gap to address is that research agents propose faster than they execute. The same type of change can have different effects across reference models, recipes, and budgets, so experimental knowledge that can continue to be used outside the original environment is needed.

    Deep readingFull reading
  17. Analysis and theoryarXiv 2610.12125

    The researchers found that optimizing the teaching fit metric makes LLM tutors repeat the same option, with expert scores dropping from 4.46 to 3.03

    Analyzing why optimizing teaching adaptation metrics collapses tutoring choices and diverges from expert judgment

    arXiv
    2610.12125
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    Gemini 3.1 Pro Preview、Qwen3-32B
    AbstractMetrics rise while expert scores fall: the optimum of the separable average is repetition, and the correction mainly rewrites the output projection.

    After turning teaching adaptation metrics into a fine-tuning objective, the metrics rise while scores from trained educators decline.

    Deep readingFull reading
  18. Analysis and theoryarXiv 2610.12083

    Paper audits LLM judge reliability: repeated agreement does not equal correctness

    Auditing LLM judge reliability and defining trustworthy verdict rate and accuracy upper bound

    arXiv
    2610.12083
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    GPT-5.2、GPT-4o、Gemini-2.5-Pro and others6
    AbstractThe authors claim that comparing judges should look at T, not just accuracy.

    The authors audit whether LLM-as-a-Judge decisions can still be treated as actionable ground truth under repeated queries, swapped candidate order, and changed prompt formats.

    Deep readingFull reading
  19. DefensesarXiv 2610.11848

    SemField: training-free semantic watermark

    Proposes the semantic watermark SemField, which still identifies the attribution of generated text after paraphrasing and structural changes.

    arXiv
    2610.11848
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Llama-3.2-1B、Qwen3-1.7B、OPT-1.3B
    AbstractTraining-independent document-level semantic watermark.

    SemField is a zero-bit semantic watermark embedded by the provider at generation time and detected later with a key, used to identify the provider's own model text after paraphrasing and sentence-level structural changes.

    Deep readingFull reading
Load more