Skip to content
Oct 12 · Monday

All papers

Found 986 papers
Filter3

Attack type

Jailbreaks196Prompt injection238Poisoning and backdoors103Model tampering3Extraction and theft34Detection evasion46Denial of service10Malicious use19

Attack technique

Gradient and representation optimization50Automated search109Multi-turn escalation30Encoding and obfuscation64Role-play and persuasion33Conversation structure manipulation14Reasoning-chain manipulation12Cross-modal payloads58Side channels4Defense-targeted29Dormant triggers34Selection manipulation12

Model risks

Deception and scheming33Reward hacking29Misalignment and value divergence22Dangerous agent actions76Dangerous capabilities24Inappropriate refusal12Hallucination and fabrication16Bias and discrimination3

Defense type

Model hardening17Inference-time intervention7Detection and filtering83Instruction–data separation60Permissions and approval72Monitoring and auditing43Watermarking and provenance6

Impact

Harmful content184Unauthorized actions207Data and privacy leaks87Output manipulation207Availability and reliability53Oversight failure60Increased misuse capability15

System type

Models and APIs264LLM applications130AI Agent598coding agent123science agent30web agent34Browser and computer use18Multi-agent systems77Embodied systems and robotics23

Attacker access

Black-box135Gray-box9White-box22No-box (offline transfer)4

Attack surface / entry point

User input313Web pages and interfaces69Documents and messages79Tool responses107Tool and skill descriptions63Code repositories and dependencies31Retrieval corpus35Memory writes39Inter-agent messages27Images, audio and video94Training data8Weights and gradients5Distribution channels45Inference services and hardware17

Layer

Prompt layer253Model layer426Application layer710Training and data layer233Infrastructure layer23

System component

MCP and skills82Memory74RAG47Guardrails and judges138Monitors65Reasoning chains26Permissions and sandboxing93Fine-tuning and open weights7Vision88Audio17Video6Image generation20Embeddings7Multimodal125
Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 261papers have no tags yet and are not included in filtered results.
Layer：Prompt layerLayer：Application layerLayer：Infrastructure layerClear all

Another 261papers do not have category tags yet and are not included in filtered results.

  1. OtherarXiv 2610.13912

    RSI-Anything: Unlocking Recursive Self-Improvement with World Models

    Proposes RSI-Anything, which uses a world model to select candidates and retain policy updates.

    arXiv
    2610.13912
    Published
    Layer
    Application layer
    Setting
    coding agent, Browser and computer use
    Evaluated models
    ReasonFlux-Coder-7B, 4B policy, W1
    AbstractRSI-Anything integrates world models into recursive self-improvement through prediction, feedback, and retention checks, and reports gains across multiple domains.

    RSI-Anything is a cross-domain framework: it uses world model predictions to decide which candidates to take external feedback for, then uses the feedback to decide whether the policy, workflow, or predictor is updated, and uses checks to decide which pair is retained. The difficulty to address is the limited interaction budget, and the fact that once the policy changes, the candidate distribution faced by the predictor also changes. The authors argue that improving only one side does not guarantee that the assembled system will be better.

    Paper analysisRead analysis
  4. DefensesarXiv 2610.13733

    KVeil: First-Place Solution for the ECCV 2026 LifeGenIP Unlearnable Video Challenge

    Propose KVeil, which adds perturbations to face videos to prevent unauthorized customization from reproducing identity

    arXiv
    2610.13733
    Published
    Layer
    Prompt layer
    Setting
    Models and APIs
    Attacker
    White-box, Black-box
    Evaluated models
    Wan2.2-TI2V-5B, HunyuanVideo-1.5, Kling 3.0 and others4

    AbstractKVeil adapts LETHE's reference-reading disruption to the challenge's two stages, ranking first on the leaderboard in both Final columns.

    Paper analysisRead analysis
  5. OtherarXiv 2610.13479

    A local agentic AI framework generates edge-case scenarios for safety assurance

    Proposes a local dual-model closed loop to generate edge scenarios for autonomous system safety assurance

    arXiv
    2610.13479
    Published
    Layer
    Application layer
    Setting
    LLM applications, Embodied systems and robotics
    Evaluated models
    PPO
    AbstractLocal closed-loop generation of JSON edge cases.

    The authors built a local agentic loop that runs on a laptop to generate edge cases for safety arguments of autonomous systems, rather than releasing a new public test set.

    Paper analysisRead analysis
  7. OtherarXiv 2610.13372

    Toward Safe World Models: Risks, Defenses, and Evaluation

    A survey of risks, defenses, and evaluation in the six-stage lifecycle of world models

    arXiv
    2610.13372
    Published
    Layer
    Application layer
    Setting
    Embodied systems and robotics
    AbstractSafety is written as a property of the prediction-decision closed loop; claims must be linked to observed consequences and versions.

    World model safety is organized by the nature of the prediction-decision loop, not by isolated prediction scores.

    Paper analysisRead analysis
  8. DefensesarXiv 2610.13333

    LETHE: Defending Against Unauthorized Image-to-Video Generation via Value Diversity Collapse

    Proposes LETHE, which perturbs reference images to block unauthorized image-to-video

    arXiv
    2610.13333
    Published
    Layer
    Prompt layer
    Setting
    Models and APIs
    Evaluated models
    Wan2.1-I2V-14B, CogVideoX-5B-I2V, LightX2V and others6
    AbstractLETHE suppresses the diversity of reference values and attracts attention, making protected images difficult to animate faithfully.

    LETHE is a protective perturbation added to personal images before publication, used to counter unauthorized image-to-video generation.

    Paper analysisRead analysis
  10. Risky behaviorarXiv 2610.12822

    Reward hacking as a transmissible strategy in multi-agent coding systems

    Show that reward hacking can propagate through code, memory, and plans in multi-agent coding systems

    arXiv
    2610.12822
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems, coding agent
    Evaluated models
    Qwen2.5-Coder-14B, Gemma-3-12B, Qwen2.5-Coder-7B and others16
    AbstractThe authors state that reward hacking can spread across models and affect routing and aggregation, and that review can reduce delivery.

    Reward hacking induced by coding RL can leave the source agent and enter subsequent collaboration and system coordination.

    Paper analysisRead analysis
  11. Evaluation and benchmarksarXiv 2610.12569

    SWE-Duel: Toward a Self-Expanding Arena for Coding Agents

    Proposes SWE-Duel, which lets coding agents mutually evaluate, create problems, and blindly review fixes during evaluation.

    arXiv
    2610.12569
    Published
    Layer
    Application layer
    Setting
    coding agent
    Evaluated models
    GPT-5.5, Claude Opus 4.8, Kimi K2.7 Code and others11
    AbstractThe mutual evaluation arena creates questions during evaluation.

    SWE-Duel is a coding agent arena that generates tasks and judges only at evaluation time, to avoid leakage and saturation of static benchmarks.

    Paper analysisRead analysis
  14. Analysis and theoryarXiv 2610.12436

    The study proposes an ecological theory of AI agent populations: collaboration brings a critical scale for takeoff.

    An ecological model is established for a population of mismatched agents, showing that collaboration produces a critical takeoff scale.

    arXiv
    2610.12436
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems
    AbstractThe authors say that collaboration changes takeoff from an individual capability threshold to a population threshold, so small-population evaluations are insufficient.

    The authors use the ecological strong Allee effect to discuss whether a mismatched agent population can take off when individual capabilities remain unchanged..

    Paper analysisRead analysis
  16. Evaluation and benchmarksarXiv 2610.12360

    The paper proposes the ISE framework to evaluate LLM agents' epistemic humility under knowledge conflicts

    Proposes the ISE framework to evaluate the epistemic humility of LLM agents under knowledge conflicts

    arXiv
    2610.12360
    Published
    Layer
    Application layer
    Setting
    AI Agent
    Evaluated models
    Nemotron-Orchestrator-8B, Claude Sonnet 4.6, GPT-5 and others5
    AbstractISE trajectory evaluations show that high task accuracy across four agents does not correspond to higher cognitive humility.

    What the authors want to test is not whether the agent's final answer is correct, but whether, when a knowledge conflict arises, it recognizes the gap in its trajectory, performs bounded tool follow-up, and writes the unresolved uncertainty to the user.

    Paper analysisRead analysis
  17. Analysis and theoryarXiv 2610.12313

    The study tests the sensitivity of five large-model compliance systems to perturbations of regulatory rules.

    Diagnose whether compliance system rulings change with perturbations to regulatory rules

    arXiv
    2610.12313
    Published
    Layer
    Prompt layer
    Setting
    Models and APIs
    Evaluated models
    Llama-3.3-70B-Instruct, Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct and others5
    AbstractAcross five models, many correct verdicts are largely unchanged by deleting or replacing negation rules.

    Use behavioral counterfactuals and activation probes to audit whether compliance rulings rely on the given rules. The authors aim to separate two kinds of seemingly identical correct rulings: those that use the given rules, and those obtained from case surface features or training patterns.

    Paper analysisRead analysis
  18. AttacksarXiv 2610.12292

    The paper proposes an option channel attack: changing one option label can make an agent guardrail allow a prohibited operation

    Proposing an option channel attack that renames option labels so that agent guardrails allow prohibited operations

    arXiv
    2610.12292
    Published
    Layer
    Application layer
    Setting
    LLM applications
    Attacker
    Black-box
    Evaluated models
    LAYA-TD, LAYA-EN, LAYA-ML and others5

    AbstractThe authors' evaluation indicates that typed models should not serve as final safety arbiters; they are susceptible to off-policy text and option name manipulation, and confidence filtering cannot resist attacks.

    Paper analysisRead analysis
  20. AttacksarXiv 2610.11932

    A Fudan team measures the citation fragility of AI search: ordinary posts can enter citations and answer text

    Measures citation fragility in AI search, demonstrating that ordinary public posts can enter citations and answers.

    arXiv
    2610.11932
    Published
    Layer
    Application layer
    Setting
    LLM applications
    Evaluated models
    Grok, Doubao, ChatGPT and others10

    AbstractThe authors claim the paper quantifies vulnerabilities in AI search caused by citation preferences and low posting thresholds; tests show public web posts can penetrate AI citations, and they call for auditing the citation supply chain.

    Paper analysisRead analysis
Load more