AI safety papers at major conferences

Proposes a steganographic malicious fine-tuning method that keeps a large model's surface appearance of safety alignment intact while covertly generating harmful content in cover text, bypassing OpenAI's fine-tuning guardrails and safety classifiers.

For scenarios such as content moderation with asymmetric query costs, we propose a new family of decision-based black-box adversarial attack algorithms that can reduce expensive queries while successfully achieving adversarial attacks with smaller perturbations.

Proposes a bi-level meta-optimization framework, AMIS, which co-evolves jailbreak prompts and scoring templates, achieving a significantly leading jailbreak success rate on frontier models such as Claude-3.5.

It reveals that adding psychoacoustically imperceptible adversarial noise can cause mainstream deep learning-based denoising models to output gibberish, pointing out the vulnerability of speech models in safety-critical scenarios.

Model jailbreak attacks as a Stackelberg multi-objective game between two LLMs, and propose the automatic dialectical jailbreak framework ADJ, which uses wavelet transform to solve multi-objective optimization, effectively improving jailbreak attack generation capability and adaptive defense resistance.

Proposes a concept-based adversarial attack framework that generates adversarial samples from a probability distribution perspective, preserving class concepts while diversifying pose, viewpoint, or background, to mislead classifiers with higher efficiency.

Explains the adversarial vulnerability of classification networks using matrix theory: robustness may be only 1/√d of the optimal classifier, decreasing as input dimensionality increases, and this is verified in experiments including ImageNet.

To address the challenge of adversarial attacks under limited compute, a fine-grained control mechanism for selectively recomputing layer activations is proposed, maximizing attack strength at equal cost and enabling adversarial training to achieve comparable performance with only 30% of the compute budget.

To address evaluation bias in existing jailbreak evaluations caused by the lack of case-by-case standards, proposes GuidedBench, which includes a harmful question set and fine-grained evaluation guidelines, significantly reducing evaluation variance and improving reproducibility.

It proposes the multimodal adversarial framework RECALL, which uses reference images to optimize adversarial image prompts, successfully bypasses ten existing concept unlearning defenses, and reproduces harmful concepts, revealing the security vulnerability of unlearning mechanisms.

Proposes aligning the intermediate features of the generator with an exponential moving average teacher model to maintain semantic consistency, significantly improving the black-box transfer attack effectiveness and test efficiency of generative adversarial examples.

This study reveals that the transferability of jailbreak attacks stems from shared representations across models, showing that representation similarity under benign prompts and the attack strength of the source model are key factors determining jailbreak transfer, with role-play attacks transferring more easily across models.

A black-box jailbreak method, SceneSplit, is proposed, which decomposes harmful narratives into benign storyboards to constrain the generation space and bypass safety filters, achieving a high attack success rate on text-to-video models such as Sora2.

Proposes HMNS, a circuit-level intervention jailbreak method based on attention head masking and null-space guidance, which bypasses large model safety mechanisms with high success rate by suppressing causal attention heads and injecting geometrically constrained perturbations.

Proposes JailbreakLoRA, a multi-task jailbreak training method that balances task utility and attack capability through uncertainty weighting and gradient conflict mitigation, significantly improving jailbreak success rate.

Proposes JailNewsBench, the first cross-lingual benchmark for evaluating large models' resistance to jailbreak attacks that generate fake news, covering 34 regions and 22 languages, and finds that mainstream models show significantly weaker defense performance on English and US-related topics.

Constructs the first jailbreak evaluation benchmark for speech large language models, systematically testing various cross-modal jailbreak attacks and defenses, revealing their safety vulnerabilities and the limitations of alignment mechanisms.

Proposes the JULI jailbreak method, which effectively bypasses the safety alignment of large language models in a black-box API setting using only the Top-5 token log probabilities of the target model.

Proposes the LingoLoop attack against multimodal large models, which induces infinite loops through part-of-speech-aware delays and generation path pruning, causing output sequences to surge to 367 times, leading to severe computational resource exhaustion and service degradation.

Proposes the Calgacus text steganography protocol, which can encode text into equal-length coherent text within seconds using an 8B model, and can be used to hide unfiltered model responses within safe replies to bypass safety monitoring.

Proposes the multimodal jailbreak framework MIDAS, which decomposes harmful semantics and distributes them across multiple images, gradually reconstructing malicious intent through cross-image reasoning, bypassing safety mechanisms and achieving a jailbreak success rate of 81.46% on closed-source models.

Proposes Role-Modality Attack (RMA), which induces harmful outputs by confusing user and assistant roles and altering structural perturbations such as the positions of image markers, and proposes an adversarial training strategy that makes the model focus on query content to reduce the attack success rate.

Proposes a deep fine-tuning attack strategy of refusing first and then complying, successfully bypassing shallow defenses and output filters, achieving high success rate jailbreaks against GPT-4o and Claude Haiku.

It leverages the concise and obscure characteristics of classical Chinese and the fruit fly optimization algorithm to build the CC-BOS framework, achieving efficient and automated black-box jailbreak attacks.

Proposes the multi-turn jailbreak attack framework PLAGUE, which decomposes the attack into three stages—guidance, planning, and completion—and uses a lifelong learning agent to achieve high attack success rates on highly defensive models such as o3 and Opus 4.1.

Proposes the robust adversarial attack method IGSA, which generates adversarial examples through inverse gradient sampling and perturbation-guided optimization, improving attack effectiveness under unknown perturbations and adversarially trained defense models.

For behavior-targeted attacks that manipulate victim behavior through adversarial perturbations on state observations, proposes a black-box attack method based on imitation learning, and a time-discounted regularization defense strategy based on policy sensitivity analysis.

Construct a multi-turn dialogue benchmark SafeDialBench covering 6 major safety dimensions and 7 jailbreak strategies. Evaluating 19 models, it finds that Yi and GLM perform well, while Llama3.1 and o3-mini have safety vulnerabilities.

Pointed out that repeated sampling can serve as a powerful attack vector for eliciting harmful responses from LLMs, modeled it as a compute allocation problem between optimization and sampling, and improved adversarial attack success rate by up to 37%.

This study finds that reasoning large models exhibit self-jailbreaking after benign code or math training, autonomously introducing benign assumptions in the chain of thought to bypass safety guardrails and execute harmful requests. Adding a small amount of safety reasoning data can effectively mitigate this.

Propose an attack framework SEMA combining prefill-based self-fine-tuning with intent-drift-aware reinforcement learning, which efficiently trains multi-turn jailbreak attack models without external data and significantly improves attack success rate.

Proposes SeRI, the first pixel-level sensitive region identification method based on a decision-based black-box setting, which recursively and finely locates key regions without gradients, significantly improving the efficiency and success rate of black-box adversarial attacks against deep neural networks.

Proposes the position vulnerability evaluation metric VSS and the SlotGCG attack, which significantly improves the jailbreak success rate and defense robustness of large language models by optimizing the insertion of adversarial tokens at the most vulnerable positions in the prompt.

Proposes an optimized jailbreak method called TAO-Attack, which uses a two-stage loss function to suppress refusal and induce harmful outputs, combined with direction-prioritized token optimization to significantly improve jailbreak success rate and efficiency.

Proposes the DIJA jailbreak framework, which uses dLLM's bidirectional modeling and parallel decoding to defeat alignment, achieving keyword ASR up to 100% on Dream-Instruct, significantly surpassing existing methods.

Proposes an adversarial attack that only adjusts spike timing while keeping the spike count unchanged, achieving over 90% success rate on DVS-Gesture by modifying less than 2% of spikes, which existing defenses struggle to counter.

Proposes UltraBreak, a cross-model general and transferable VLM jailbreak attack framework, combining visual space regularization with semantic target smoothing of the loss surface, significantly improving black-box jailbreak success rate.

A progressive semantic injection attack based on diffusion models, PSI, is proposed, which uses diffusion priors to improve sample naturalness and preserve visual fidelity, significantly enhancing the transferability and stealthiness of black-box attacks against commercial vision-language models such as GPT-5.

A gray-box untargeted attack that only attacks the vision encoder, by perturbing patch tokens, reduces image description task performance by 94.5% and VQA by 75.7%.

Formalizes the relationship between flat minima and adversarial robustness: flatness only guarantees local rather than global robustness; to maintain robustness over a larger range, the loss needs to bend sharply away from the data manifold, and this is verified across multiple architectures.

Proposes the ASIDE architecture that applies orthogonal rotations to data tokens at the word embedding layer to separate instructions from data, significantly enhancing the model's defensive robustness against prompt injection attacks without additional parameters.

It reveals LLMs' dependency vulnerability on chat templates, and proposes the ChatInject indirect prompt injection attack that imitates native template formats, significantly improving attack success rate and transferability across multiple agent benchmarks.

Constructed the first visual prompt injection threat model and evaluation benchmark VPI-Bench for computer-using agents, revealing the serious risk of inducing agents to execute unauthorized operations through screen visual injection in black-box environments.

Constructs a hybrid sandbox framework and the RTC-Bench benchmark to evaluate the defensive vulnerability and loss-of-control risks of computer-use agents against indirect prompt injection.

Proposes A2ASecBench, the first protocol-aware security benchmark for LLM multi-agent systems, building a threat model for supply chain and protocol logic attacks, and empirically finds that existing systems are generally vulnerable and default defenses can be easily bypassed.

Proposes a threat snapshot framework and the b^3 benchmark to evaluate how vulnerabilities in base large models propagate to agent systems and cause safety risks, finding that enhanced reasoning ability can improve safety while model size is unrelated to safety.

Reveals multi-agent control-flow hijacking attacks that can easily bypass existing alignment checking mechanisms, and proposes the ControlValve defense framework based on control-flow integrity and the principle of least privilege.

Targeting safety risks in agent planning and execution, proposes the synthetic trajectory data engine AuraGen, the pre-hoc guardrail model Safiron, and the evaluation benchmark Pre-Exec Bench covering diverse tools.

Proposes GhostEI-Bench, the first benchmark for evaluating mobile agents' defenses against environment injection attacks in dynamic device environments, revealing that current mainstream agents are extremely vulnerable to UI deceptive visual injections.

Targeting the vulnerability of multi-agent systems to misinformation injection attacks, proposes the MisinfoTask benchmark and the training-free defense framework ARGUS, which corrects misinformation in the information flow through goal-aware reasoning.

Proposes the neuro-symbolic architecture VIRF, which provides causal feedback to the LLM embodied planner through a formal logic tutor to repair unsafe plans, achieving a 0% dangerous action rate in home safety tasks.

Reveals that deceptive web design can manipulate agent trajectories and induce malicious outcomes, with larger models being more susceptible to manipulation, and existing guardrails being insufficient to defend against it.

Reveals the phenomenon that computer-use agents blindly pursue goals regardless of feasibility or safety, constructs the BLIND-ACT benchmark, and finds that frontier models generally exhibit a high-risk tendency toward blind execution.

It introduces the ManagerBench benchmark to evaluate agents' decisions when business goals conflict with personnel safety, finding that frontier models often prioritize harmful actions to advance business objectives.

Proposes an MCP attack benchmark covering the full pipeline of task planning, tool invocation, and response handling, including 12 attack types and the NRP metric; evaluating nine agents, it finds that more capable models are more vulnerable to attacks.

Constructed the large model safety benchmark MCP-SafetyBench based on real MCP servers, covering 5 major domains and 20 types of cross-end attacks, revealing that mainstream models generally exhibit significant security vulnerabilities in tool invocation.

Proposes OpenAgentSafety, a safety evaluation framework for agents in real tool interaction and multi-turn tasks, revealing the high-risk status quo where mainstream large models frequently exhibit unsafe behaviors in agent tasks.

Targeting indirect prompt injection threats, proposed an agent planning design that balances safety policy compliance and task progress, effectively increasing the proportion of autonomously executed operations under information flow control defenses.

Proposes ST-WebAgentBench, a benchmark for evaluating the safety and trustworthiness of web agents, covering 375 tasks and policy constraints across 6 dimensions. Evaluation finds that mainstream open-source agents have significantly low completion rates under policy compliance.

We construct a multi-agent financial fraud benchmark, reveal the mechanism by which collaboration amplifies fraud risk, and propose mitigation strategies such as content warnings and monitoring-based interception.

Systematically reveals mis-evolution risks of self-evolving agents, such as alignment degradation and tool vulnerabilities, caused by deviations in model, memory, tool, and workflow evolution, and discusses defense strategies.

Based on the closed-form optimal solution for safety alignment objectives, proposes SafeDPO, a lightweight safety training method requiring only one additional hyperparameter, which significantly improves model safety on the PKU-SafeRLHF benchmark without needing a reward model.

Unifies negative guidance methods for suppressing harmful content in diffusion and flow models, and combined with control barrier function analysis, proves that applying strong negative guidance early in the denoising process can achieve safe generation.

Proposes the GuardSpace guardrail framework, which prevents safety alignment degradation in large language models during downstream fine-tuning by freezing safety-related weight components and using null-space projection to constrain adapter updates.

Proposed the A2D token-level safety alignment defense method, which enables diffusion language models to emit an EOS refusal signal and terminate early when generating harmful content, effectively defending against prefill jailbreak attacks with arbitrary decoding order.

We reframe adversarial defense for large vision-language models as a preference optimization problem, enhancing preference for normal outputs and rejecting adversarial misdirection by adjusting only the image encoder, balancing clean and adversarial performance.

Proposes the AdvChain adversarial chain-of-thought fine-tuning paradigm, training the model to dynamically self-correct using inducement and hesitation-correction samples, effectively defending against jailbreak attacks and significantly reducing over-refusal of benign prompts.

It is found that adversarial examples shift along a specific dominant direction in the CLIP feature space. Based on this, a direction-bias-guided test-time defense, DBD, is proposed, achieving state-of-the-art adversarial robustness without retraining.

Pointed out that most jailbreak attacks are recombinations of existing adversarial skill primitives, and designed the adversarial skill composition training method ASCoT, significantly improving large models' generalization in defending against unknown jailbreak attacks.

Theoretical analysis shows that adversarially pretrained Transformers can robustly generalize to unseen classification tasks via in-context learning from clean examples, endowing downstream tasks with adversarial robustness at low cost.

Using adversarial erasure objectives and gradient regularization projection, it simultaneously improves the robustness and retention of concept erasure without relying on retained data, significantly reducing the success rate of adversarial prompt attacks.

Demonstrates through causal intervention that jailbreak vulnerability stems from shallow alignment lacking reasoning, and proposes chain-of-thought fine-tuning and alignment-weighted DPO, which assign different preference weights to the reasoning process and the final answer to achieve reasoning-based principled refusal.

Proposes AlphaAlign, a pure reinforcement learning safety alignment framework that uses verifiable safety rewards to stimulate the model's intrinsic safety reasoning, reducing over-refusal while improving jailbreak robustness.

Propose the Antibody defense strategy, which optimizes the model to a flat loss region before fine-tuning and weights down harmful sample gradients during fine-tuning, defending against harmful fine-tuning attacks while maintaining task performance.

Proposed a lightweight defense method ADA that introduces an assistant head marker at inference time, enabling the model to recognize harm at any generation position and resume refusal responses.

It proposes a three-step reasoning defense framework called ARMOR, which analyzes the policy library, extracts core malicious intent, and verifies safety policies, while decoupling safety from general reasoning, effectively defending against unknown out-of-distribution jailbreak attacks and greatly reducing attack success rates.

Targeting the refusal vulnerability exposed by jailbreak attacks such as temporal rewriting, proposes the ASGuard framework based on mechanistic interpretability, which enhances the model's refusal mechanism by locating and recalibrating the activations of vulnerable attention heads and fine-tuning.

Proposes truncated polynomial classifiers as dynamic activation monitors, enabling lightweight early exit or reinforced guardrails through term-by-term evaluation, reducing the cost of harmful prompt classification while maintaining interpretability.

An adversarial defense method, HyCAS, is proposed, combining deterministic and randomized principles to construct a Lipschitz-constrained network, improving both verifiable robustness and empirical adversarial robustness without losing clean sample accuracy.

VARE and S-VARE are proposed, targeting the next-scale prediction paradigm of VAR, precisely adjusting unsafe visual tokens to achieve concept erasure while maintaining generation quality.

Aiming at the problem that erasing harmful concepts in diffusion models easily damages benign associated concepts, proposes the ReCARE framework to robustly erase target concepts while effectively preserving benign co-occurring concepts.

Proves based on cryptographic hardness assumptions that efficient external filtering of input prompts and model outputs is computationally infeasible, indicating that relying solely on black-box guardrails cannot ensure model safety.

Proposes a constitutional classifier combining a session classifier, two-stage cascade, and linear probes, reducing computational cost by 40 times with a false rejection rate of only 0.05%, and effectively defending against universal jailbreaks after over 1,700 hours of red-teaming.

Conducts the first systematic study of continual concept unlearning in text-to-image models, finds that cumulative parameter drift causes utility collapse, and proposes regularization and semantic-aware gradient projection methods that significantly improve it.

Proposes a training-free evidence uncertainty quantification framework, EUQ, which explicitly decomposes conflict and ignorance within epistemic uncertainty, and effectively detects abnormal behaviors such as jailbreaks, adversarial vulnerability, and hallucination in a single forward pass.

We reveal vulnerabilities such as denoising path dependence in diffusion large language models under jailbreak attacks, and propose a training-free defense framework, DiffuGuard, which reduces the attack success rate of six jailbreak methods from 47.9% to 14.7%.

Aiming at the problem of large language models over-refusing by misclassifying benign prompts as toxic during safety alignment, proposes a contrastive refinement method, DCR, which reduces over-refusal while effectively preserving the safety capabilities brought by alignment.

Finds that reducing the optimizer order helps models converge to flatter loss basins, and accordingly proposes a hybrid optimizer that makes large-model unlearning more robust to perturbations such as adversarial quantization and fine-tuning, effectively preserving the safety of the unlearning effect.

Proposes DRAGON, a context unlearning framework that requires no fine-tuning and no retained data. It uses a lightweight detection module to identify prompts to be forgotten and a dedicated chain-of-thought guardrail model to perform safe and accurate context intervention before inference.

Proposes the ensemble defense method DRIFT, which breaks gradient consensus through lightweight learnable filters, significantly improving the adversarial robustness of vision models under adaptive white-box and transfer attacks.

Proposes the dual-space smoothing framework PRISM, which defends against jailbreak attacks via representation-space robust probes and mitigates relearning attacks via parameter-space smoothing, balancing unlearning effectiveness with model usability on safety benchmarks such as WMDP.

Addressing the high cost of unlearning in large models and vulnerability to reverse engineering attacks, proposes a distillation method where a student model imitates a prompt-guided teacher model, effectively refusing undesirable knowledge while preserving general capabilities.

It introduces DynaGuard, a dynamic guard model supporting user-customized policies, providing fast violation detection with chain-of-thought explanations, achieving both high accuracy and efficiency in traditional and free-form harm detection.

For the two-stage repaired SFT model: use DPP to select samples that cause untruthful, biased, and unethical outputs, then update with PBRF gradient ascent. The credibility metric improves by up to 21%, and perplexity impact does not exceed 1%.

Proposes efficient feature attribution for random subspace methods used in adversarial, backdoor, and jailbreak defense, maintaining local accuracy, and for the first time provides provable robustness against adversarial explanation attacks, evaluated under multiple attacks.

Addressing the vulnerability of existing guardrails to attacks in specialized domains, we propose ExpGuard, a guardrail model dedicated to finance, healthcare, and law, along with benchmark datasets, significantly improving input-output safety classification accuracy.

Proposes TRU, a machine unlearning method that incorporates reasoning objectives, precisely removing harmful knowledge while preserving general capabilities, and demonstrating stronger robustness under various adversarial attacks.

Constructed an evaluation benchmark containing 11,400 video query pairs, and proposed the VideoSafety-R1 two-stage framework, which transforms safety alignment into proactive defensive reasoning through alarm token fine-tuning and safety GRPO reinforcement learning.

This study proposes the RICH hypothesis, showing that with sufficient initial robustness, scaling test-time inference compute can effectively defend against white-box multimodal and jailbreak attacks, and accordingly builds a robust reasoning VLM.

Proposes GraphShield, a jailbreak detector that models LLM internal information routing as a graph structure, capturing jailbreak features in a lightweight manner, significantly reducing attack success rate while maintaining a low benign refusal rate.

It proposes GuardAlign, a training-free defense framework for multimodal large models, combining optimal transport safety detection with cross-modal attention calibration to strengthen safety prefixes, reducing unsafe response rates by up to 39% while preserving general capabilities.

We propose HyPE, which uses the geometric structure of hyperbolic space to detect harmful prompts, and HyPS, a purification method that modifies harmful words, providing robust guardrails for vision-language models against malicious prompt misuse.

Proposes AdvMask, which learns a binary mask without modifying pretrained weights to shield parameters susceptible to adversarial perturbations, combined with a layer-wise feature alignment loss; outperforms existing VLM adversarial fine-tuning methods in few-shot settings.

Proposes a TTS framework: decomposes claims, collects stances from various sources and scores them with peer prediction, filters unreliable sources before summarization, improving factual robustness against adversarial content and providing incentive guarantees for honest reporting.

Proposes the DR-IRL framework, which trains category-specific reward models via inverse reinforcement learning and dynamically scales rewards based on task difficulty to improve GRPO, effectively addressing the alignment problem of long-tail safety harms.

To address the risk of text-to-image models generating harmful or infringing content, proposes the HiRM method that redirects the high-level semantic representations of target concepts in the text encoder toward specified directions, achieving precise concept erasure by fine-tuning only early attention layers.

For decomposition attacks that trick agents by breaking malicious goals into benign subtasks, a lightweight temporal monitor is proposed to cumulatively evaluate the dialogue, achieving a 93% defense success rate at lower cost.

Discovered that the probability sparsity induced by nasty training can improve adversarial robustness, and accordingly proposed the NAT adversarial training method, with theoretical and experimental validation.

To address the noise and bias in using LLM judges to verify model safety failure rates, we propose a hypothesis testing framework with theoretical guarantees, which achieves strict error rate control and reliable safety certification under finite-sample calibration.

It is found that latent-space defenses in large models are vulnerable to confusion activation attacks, which can reduce the recall of harmfulness detection probes to 0% while maintaining a 90% jailbreak success rate, revealing that white-box monitors lack adversarial robustness.

Proposes the operational safety concept and the OffTopicEval evaluation benchmark, finds that current open-source models generally lack task scope definition capability, and proposes a prompt guidance method that greatly improves refusal performance.

Finds that function words exacerbate VLM vulnerability under cross-modal adversarial attacks, and proposes FDA, differential de-attention, which on average reduces attack success rate across 6 attacks with very little performance loss.

Proposes an end-to-end alignment algorithm that balances safety and helpfulness using risk-aware cold-start SFT and a dynamically weighted reward model, outperforming baselines by 5% to 20% on a multimodal safety benchmark.

Revealed the mechanism by which overlapping feature spaces in representation space cause excessive refusal, and proposed the projection-based low-rank parameter pruning framework ProSafePrune, which significantly reduces the model's false refusal rate while maintaining its ability to refuse genuinely harmful requests.

Proposes the "Answer-Then-Check" safety alignment method and the ReSA dataset, using the model's reasoning ability to answer internally first and then evaluate safety, enhancing jailbreak defense while reducing the over-refusal rate.

Points out that safety fine-tuning of vision-language models suffers from visual safety reasoning deficiencies, constructs a multi-image safety dataset, MIS, with integrated safety chain-of-thought annotations, and after fine-tuning, substantially reduces attack success rates across multiple benchmarks while maintaining general capabilities.

Addressing the transfer failure of non-robust pretrained models in adversarial robust fine-tuning, we propose an Epsilon scheduling method that dynamically adjusts perturbation strength, effectively improving expected robustness on downstream tasks.

To address the robustness deficiency of pre-filtering forgetting under adversarial attacks, the PoRT framework is proposed, achieving robust defense through prompt cleaning, joint image-text/question-answer posterior evaluation, and adaptive multi-turn reasoning.

Theoretically proves that threshold-neighboring neurons limit the adversarial robustness of spiking neural networks, and proposes a threshold guarding optimization method that significantly improves model robustness under adversarial attacks through membrane potential loss constraints and noise neurons.

Targeting the vulnerability where fine-tuning causes harmful input routing drift and exposes harmful fine-tuning attacks, proposed SafeMoE to align routing weights to maintain safe expert routing, effectively defending against harmful fine-tuning.

It finds that aligned models have low entropy when refusing harmful requests, and accordingly proposes SIRL, which uses internal confidence as a self-generated reward, maintaining a defense success rate above 89% under more than 20 jailbreak methods.

The study finds that VLM safety fine-tuning under human supervision is vulnerable to word-substitution attacks due to spurious correlations and leads to excessive refusal; proposes using machine unlearning to eliminate harmful knowledge, reducing both attack success rate and false refusal rate.

To address the issue of degraded generation quality when eliminating harmful concepts in text-to-image models, we propose the SDErasure framework, which adaptively shifts the generation trajectory by locating key timesteps, effectively removing target concepts while maintaining image quality.

Proposes Sysformer, which adaptively adjusts system prompts in the input embedding space while keeping large model parameters frozen, greatly improving refusal rates for harmful prompts and jailbreak attacks.

Found that reasoning models are more prone to producing harmful content, attributed to visual attention drift and unsafe reasoning patterns; proposed the PST two-stage alignment, which significantly reduces harmful outputs while maintaining general capabilities.

It proposes a multi-agent reinforcement learning framework called WaltzRL, which jointly trains dialogue and feedback agents to dynamically correct harmful outputs or over-refusals, greatly improving the model's safety defense capability while maintaining helpfulness.

To address the safety degradation problem caused by fine-tuning large models, a token-level data filtering framework, TOSS, is proposed, which precisely removes unsafe tokens by quantifying the safety risk of individual tokens, effectively safeguarding fine-tuning safety.

It reveals a priming vulnerability in diffusion language models where affirmative words appearing in intermediate denoising steps can easily induce harmful content generation, and proposes a dedicated safety training method to enhance defensive capability.

Proposes TriQDef, a three-level quantization-aware defense framework that, by disrupting semantic and gradient alignment, reduces the success rate of cross-quantization-bit-width adversarial patch attacks by over 40% while maintaining benign accuracy.

Proposes the T3 framework, which treats model safety as out-of-distribution detection, and can effectively block jailbreak and toxicity threats across multiple languages using only safety prompts for training.

Based on in-context learning theory, proves a robust generalization bound for continuous adversarial training, reveals its mechanism for defending against jailbreak prompts, and designs a regularization term based on the singular values of the embedding matrix to improve the trade-off between jailbreak robustness and practicality.

Proposes the first training-free concept erasure framework for video diffusion models, which estimates refusal vectors through paired prompts and combines contrastive low-rank decomposition with closed-form weight updates, significantly reducing unsafe generation without affecting generation quality.

Constructs a multimodal safety benchmark, VLSU, containing 8,187 samples, and finds that in image-text combined scenarios, model safety judgment accuracy drops sharply and it is difficult to balance refusal and over-blocking.

We find that style patterns in queries lead to inflated jailbreak success rates, and that fine-tuning on specific styles exacerbates this vulnerability; we propose the SafeStyle defense strategy, which effectively maintains model safety by incorporating stylized safety data through fine-tuning.

We propose a dual-branch defense framework ZePAD that, with a single adversarial fine-tuning, defends against unknown downstream adversarial sample attacks and directly detects adversarial samples while keeping benign model performance undegraded.

Proposes AdAEM, a self-expanding evaluation algorithm that probes the internal value boundaries of cross-cultural multi-models through context optimization and extracts controversial topics, adaptively generating new questions to continuously track value differences and alignment dynamics across models.

To address the lack of quantitative metrics for value alignment, proposes the black-box evaluation method EigenBench, which uses model mutual evaluation and EigenTrust consensus aggregation to quantitatively assess the degree of alignment between language models and a given value system.

A WIMHF method using sparse autoencoders to interpret human feedback data is proposed, revealing unsafe tendencies such as voting to support toxic content in preference data, and achieving a 37% safety gain through data cleaning.

Proposes a multilingual consistency loss function MLC that achieves semantic-level directional consistency by improving the collinearity of multilingual representation vectors, and can simultaneously enhance multilingual safety in a monolingual alignment pipeline using only multilingual prompt variants.

To address the problem of training data noise corrupting preference alignment, a meta-learning paradigm is proposed in which the model performs self-diagnosis, fusing intrinsic feedback such as consistency and confidence to adaptively reweight samples, thereby improving the robustness of preference alignment.

It proposes the MetaAPO preference optimization alignment framework, which uses a meta-learner to dynamically balance online generation and offline data distributions, reducing annotation costs by 42% while improving model alignment.

Proposes the ranking-based choice preference optimization framework RCPO, extending large language model preference optimization to multi-way comparisons and Top-k ranking data, unifying and generalizing pairwise alignment methods such as DPO, and improving alignment effectiveness.

Using cognitive models to systematically evaluate alignment value trade-offs in large language models can effectively diagnose behaviors such as sycophancy, and reveals the evolution patterns of model values during post-training reinforcement learning.

Without backpropagation, residual representation adapters from a donor model are transferred to models of different architectures; in three cases—enhancing safe refusal, inducing jailbreaks, and automatic CoT reasoning—the effect is comparable to direct fine-tuning with much lower resource consumption.

We define social sycophancy in large models that over-accommodates user face, build the evaluation benchmark ELEPHANT, find that mainstream models favor user positions in moral conflicts, and explore related mitigation strategies.

For inductive biases in reward models that easily trigger reward hacking, we propose a debiasing method DIR based on the information bottleneck, which effectively suppresses response length, sycophancy, and format biases and improves RLHF performance.

Addressing the problem that reward models in alignment methods such as Constitutional AI may violate Pareto optimality, proves that axiomatic guarantees can be restored under standard training through balanced coverage dataset design.

Proposes ConflictScope to automatically evaluate model prioritization under value conflicts, finding that models tend to deviate from the harmlessness principle in open scenarios, while detailed system prompts can improve alignment.

It explains the advantage of online alignment from the perspective of prospect theory in behavioral economics, and proposes the Humanline variant that incorporates probability perception bias into objectives such as DPO, enabling offline data to achieve online alignment effects.

Propose adding system prompts that induce undesirable features before fine-tuning data, causing the model to significantly reduce expression of that feature during testing, effectively mitigating emergent alignment failure and backdoor attacks caused by narrow fine-tuning.

Aiming at the problem that poorly aligned reference policies cause ambiguous samples to interfere with training, proposes an improved DPO algorithm, RAPPO, which mitigates alignment bias by filtering the hardest and most ambiguous samples, significantly improving harmlessness and helpfulness on safety benchmarks.

Proposes the LitmusValues evaluation pipeline and the AIRiskDilemmas dilemma dataset, which effectively predicts potential risks and harmful behavior by measuring AI's value priorities in conflict situations.

Proposes NSPO, an optimization method that geometrically projects the safety policy gradient into the null space of general tasks, which significantly improves model safety while effectively preserving general capabilities such as mathematics and code, alleviating the alignment tax.

A modality aphasia phenomenon is discovered in unified multimodal models, where visual memory is separated from textual expression, revealing that text-only alignment can still generate unsafe images due to failed cross-modal alignment.

Proposes using orthogonal subspace decomposition to resolve gradient conflicts in multi-objective preference alignment, balancing usefulness, harmlessness, and truthfulness of large language models at the parameter level.

Proposes the PluriHarms benchmark to systematically study human disagreement and value diversity in AI harm judgments, containing 150 prompts and 15,000 annotations, revealing the shortcomings of current alignment methods in pluralistic safety.

For noise and adversarial feedback in preference data, we propose an alignment framework RE-PO that infers annotation reliability based on the EM algorithm, adaptively weights losses, and significantly improves the performance of mainstream alignment algorithms.

It enhances LLM judges with high-quality reference outputs and lets reference-guided judges drive self-improvement, outperforming SFT distillation and reference-free baselines on AlpacaEval and Arena-Hard, with results close to fine-tuned reward models.

The study finds that reward models persistently inherit deep human value biases from their base models, producing significant differences even with identical preference data, highlighting the importance of alignment during the pretraining stage.

Targeting the fragility of long-tail preference generalization that deviates from the mainstream distribution in large model preference alignment, proposes the training-free robust preference selection method RPS, which improves alignment reliability by sampling local preference neighborhood candidates and reaching consensus.

Proposes the surface safety alignment hypothesis, pointing out that safety alignment is essentially a binary classification decision and only requires a few key units at the neuron level to establish guardrails; freezing safety-critical units during fine-tuning can minimize the alignment tax while maintaining safety.

Proposes an alignment audit framework based on Bayesian inverse reinforcement learning, which quantifies non-identifiability by inferring the objective distribution and exposes spurious shortcuts, providing security teams with a tool to verify the true alignment objectives of large models.

Formalizes AI error incoherence through bias-variance decomposition, finding that longer reasoning leads to more incoherent failures, suggesting that future advanced AI is more likely to cause sporadic accidents rather than persistently pursuing misaligned goals, increasing the importance of alignment research such as reward hacking.

To address the problem that preference learning cannot reflect true cognition, proposes an axiomatic cognitive decision model that fits human judgments more accurately to promote value alignment.

To address the problem of harmful content hidden in the chain of thought of large reasoning models, we propose Intervention-based Preference Optimization (IPO), which aligns the reasoning process by replacing compliant steps with safety triggers, significantly reducing harmfulness.

Proposes TrustGen, a dynamic evaluation platform for the trustworthiness of cross-modal generative models, covering 25 dimensions including truthfulness, safety, and ethics. Evaluation finds that open-source models have matched closed-source ones, and that there is a complex trade-off between trustworthiness and helpfulness.

To address the problem that models under weak supervision can easily game rewards through deception, introduced a peer prediction method to evaluate and train large language models, rewarding honest answers and resisting deception, achieving reliable scalable oversight.

Extends weak-to-strong generalization to complex interactive decision-making environments, proposing that a strong model learns from both the successful and failed trajectories of a weak model, significantly improving the strong model's generalization performance through trajectory trees and MCTS.

Studies spontaneous proactive deceptive behavior in large models under benign prompts, proposes an evaluation framework quantifying deceptive intent and deceptive behavior, and finds that both metrics rise with task difficulty, and that scaling model size does not eliminate deception.

We evaluate frontier models' ability to evade monitoring by transmitting and reasoning via steganography, finding that they show nascent steganographic capability but currently struggle to evade carefully designed monitors.

Studies how adversarial AI assistants in human-AI collaborative question answering manipulate group decisions and harm teams by modeling the evolution of human trust, revealing deception attack mechanisms in team decision-making.

Proposes the LH-Deception multi-agent simulation framework, systematically quantifying deceptive behavior in 11 frontier large models under long-horizon interaction and dynamic pressure, revealing a "deception chain" phenomenon that single-turn evaluation cannot detect.

Proposed the PRISON framework to evaluate the risk of criminal tendencies in large language models such as false statements, false accusations, and psychological manipulation, and found that mainstream models often spontaneously exhibit deception and evasion tactics.

Proposes self-report fine-tuning (SRFT), which trains models to admit factual errors, allowing this honesty tendency to generalize to adversarial agent scenarios, achieving high-accuracy confession of hidden misaligned goals during interrogation.

To address the issue of models perceiving evaluation and faking alignment, intervention vectors are added to activations to successfully suppress evaluation awareness, causing the model to exhibit its true deployment behavior during evaluation.

We find that frontier models, when faced with malicious requests, strategically give replies that appear harmful but are actually ineffective to bypass monitoring, yet this can be reliably detected via linear probes on internal activations.

Finds that Claude 3.5 Sonnet can exhibit alignment faking under perceived monitoring without explicitly expressing deceptive reasoning, with monitoring evasion capabilities significantly stronger than instruction following.

To address implicit reward hacking that can evade chain-of-thought monitoring, proposes TRACE, a method that detects models exploiting reward loopholes by progressively truncating the chain of thought and measuring reasoning effort.

To address the issue that RLHF is prone to reward hacking in out-of-distribution scenarios, this study jointly trains a single-objective Bradley-Terry reward function and a multi-objective regression reward function, significantly enhancing the reward model's ability to mitigate reward hacking and its scoring performance.

Addressing the issue that reward models in reinforcement learning alignment are vulnerable to length-bias cheating, proposes the response-conditioned modeling frameworks Rc-RM and Rc-DPO that decouple semantic preference from length requirements, effectively mitigating length bias.

Study preference models' biases toward length, structure, terminology, sycophancy, and vagueness, finding they are significantly misaligned with human preferences; using counterfactual data augmentation fine-tuning reduces the average misalignment rate from 39.4% to 32.5%.

Proposes the caution method, which trains an error model and penalizes reward estimates for atypical responses, significantly mitigating reward hacking in BoN sampling and providing theoretical guarantees in a simplified linear setting.

Constructs the ImpossibleBench benchmark, which quantifies test cheating behavior in LLM agents by creating conflicts between specifications and unit tests, analyzes cheating mechanisms, and provides a testbed for developing monitoring tools.

It models reward hacking as a robust policy optimization problem on the space of related proxy reward functions, and proposes a minimax planning method that significantly improves the robustness and performance of reinforcement learning policies in the worst case.

To address the problem that reward models in large model alignment are susceptible to surface features and engage in reward hacking, proposes the CROME framework, which generates target data augmentation through causal criteria, effectively mitigating reward hacking and improving safety.

Finds that preference optimization reduces the faithfulness of CoT explanations, and the model may fabricate explanations to cater to the reward model; adding predicted causal attributions to the reward model input can reduce misleading explanations in controlled settings.

Proposes a unified regularization objective that simultaneously accounts for the KL constraint that prevents reward hacking and the policy clipping that stabilizes updates, outperforming RLHF and online preference learning methods on multiple benchmarks.

Tests whether models can perform encrypted reasoning under 28 ciphers to evade CoT monitoring; although models can translate ciphertext, reasoning accuracy drops significantly, and correlates with cipher popularity in pretraining data, with current evasion of monitoring being limited.

Introducing a reasoning bottleneck forces information to be conveyed through natural language, suppresses steganographic encoding, and significantly improves the causal dependency and monitorability of the chain of thought.

We find that supervising only outputs still causes chain-of-thought confusion through model generalization and autoregressive reinforcement, which can conceal undesirable behavior and hinder monitoring; accordingly, we propose mitigation methods that balance task performance and monitorability.

Analyzes the causal role of CoT via on-policy resampling: in agentic misalignment extortion scenarios, self-preservation statements have little influence on decisions; it also quantifies the cumulative effect of hints on reasoning.

Uses persistent homology to study how adversarial inputs reshape the internal representation geometry of large language models, finding that both indirect prompt injection and backdoor fine-tuning cause topological compression in the latent space, revealing geometric invariants of representation changes.

Proposes the AlphaSteer activation intervention method, which maintains benign prompt performance through null-space constraints and introduces a refusal direction, effectively defending against jailbreak attacks without harming general capabilities.

Identified the attention head circuit in CLIP that processes typographic text, and by selectively ablating this circuit, significantly enhanced robustness against typographic attacks without fine-tuning.

The study finds that generalized misalignment during fine-tuning is more stable and efficient than narrow misalignment, and isolates a general misalignment linear representation that can be used to monitor and intervene in misalignment behavior.

Proposes the Concept DAS method, which achieves bidirectional concept control through weakly supervised distribution matching and distributed interchange interventions, effectively intervening in safety refusal behavior and eliminating chain-of-thought backdoors.

A jailbreak detection method JDJN for multimodal large models based on jailbreak-critical neurons JailNeurons is proposed, which trains a detector by aggregating neuron features across layers, achieving high detection rates, low false positive rates, and fast inference.

Modeled activations as composable cognitive elements, defined predicate rules to detect harmful behavior in real time, enabled configuration of safety policies without retraining, and improved precision and auditability.

Proposes PCMS, a training-free inference-time activation intervention method that suppresses user-specific undesirable content by manipulating internal representations while maintaining model usability.

Proposes the activation difference lens, which, by analyzing activation differences between base and fine-tuned models and feature interventions, can recover fine-tuning data content and detect phenomena such as emergent alignment failure, providing methodological warnings for safety and interpretability research.

Reproduces emergent misalignment under multiple conditions and uses SAE model diffing to find multiple "unaligned persona" features, among which the toxicity persona feature best controls and predicts this behavior; fine-tuning on a small number of benign samples restores alignment.

Proposes the framework RepIt for separating representations of specific concepts in activations, which can selectively suppress refusals for dangerous concepts such as weapons of mass destruction and bypass standard safety evaluations, revealing evaluation blind spots and the fragility of refusal representations.

Proposed a contrastive weight intervention method that isolates behavioral directions through weight addition and subtraction, which generalizes better than activation intervention, effectively reduces sycophancy and refusal insufficiency introduced by fine-tuning, and can be used to detect emergent alignment failures.

Proposes the SteerMoE framework, which controls model safety and truthfulness without fine-tuning by selectively activating or deactivating behavior-related experts at inference time, and reveals security vulnerabilities in MoE.

Proposes a method that monitors and controls model behavior by analyzing the principal singular vectors of the weight differences between fine-tuned and base models, enabling efficient detection of backdoor attacks and auditing of unlearning effects without requiring similarly distributed data.

Reveals the phenomenon where enhanced reasoning ability triggers alignment failure, and provides a neuron-level mechanistic explanation through the entanglement of attention heads and safety neuron activations.

Proposes an attribution framework that analyzes representations and their gradients in activation space, which can precisely trace harmful content, backdoor poisoning, and knowledge contamination to specific training samples and phrases, providing a diagnostic tool for model auditing.

Proposes a monitor red teaming process to stress-test covert misbehavior monitoring on SHADE-Arena; finds that agents knowing they are monitored significantly weakens detection, and a hybrid monitoring design allows weak monitors to supervise strong agents.

Finds that untrusted models can bypass monitors by embedding prompt injection in their outputs, fully undermining existing AI control protocols and causing resampling-based defense mechanisms to backfire.

Proposes the Control Tax concept to quantify the operational and economic costs of AI control protocols, builds a theoretical framework mapping safety guarantees to costs, and provides monitoring optimization strategies that balance safety and budget.

The FAB attack is proposed, which uses meta-learning to construct compromised models that appear benign on the surface but activate latent adversarial behaviors (such as jailbreaking and excessive refusal) after downstream fine-tuning, revealing safety risks in the fine-tuning process.

Proposes BEAT, the first embodied agent visual backdoor attack framework that uses environmental objects as triggers, strengthening the decision boundary through contrastive trigger learning, achieving up to 80% attack success rate while maintaining benign task performance.

Reveals a new threat where malicious developers covertly implant action-level backdoors in reinforcement learning agents using simulator dynamics, proposes the attack method Daze that requires neither modifying nor observing rewards, and successfully validates transfer on robotic hardware.

Proposes the module switching defense (MSD) method, which uses an evolutionary algorithm to optimize module fusion strategies to break backdoor shortcuts, showing excellent defense robustness even with few models or in colluding backdoor attack scenarios.

Finding that traditional gradient ascent backdoor elimination methods cause trigger influence to transfer to other classes, we propose a robust gradient ascent method with a dynamic penalty mechanism that effectively removes backdoors while maintaining model utility.

Addressing the issue that model-editing backdoor attacks easily revert to refusal responses midway, proposes the dual-objective editing framework DualEdit, which promotes affirmative words and suppresses refusal words, increasing backdoor attack success rate and reducing safety fallback rate.

It reveals security risks in pruning large models: attackers can construct models that behave benignly when unpruned but exhibit malicious behavior once pruned, and can trigger attacks such as jailbreaks with high probability under multiple pruning algorithms.

Revealed that in geographically distributed and federated training, a single malicious participant can bypass server-side defenses through regularization and pseudo-contrastive samples, implant backdoor triggers into large models and break safety alignment, with an attack success rate of 80%.

Proposes a mechanism for collaboratively detecting poisoned data nodes in federated learning, achieving near-optimal theoretical lower bounds on attack loss in both IID and non-IID settings.

Finds that backdoor associations are redundantly encoded in MLP layers. By constructing multiple synthetic backdoored models to extract common backdoor signatures and neutralize relevant components, it maintains generation capability under various backdoor attacks.

Proposes using sharpness-aware minimization to amplify backdoor effects and enhance trigger-activated neurons, thereby significantly improving poisoned sample detection performance under both weak and strong backdoor attacks.

It is the first to study backdoor attacks in federated prompt learning, where malicious clients inject imperceptible triggers; it proposes a lightweight defense based on anomaly detection in embedding space, significantly reducing backdoor accuracy across five datasets.

Reveals spatiotemporal redundancy in backdoor attacks on diffusion models and proposes the STEDiff framework, including the STEBA strategy for accelerating backdoor injection and the STEDF detection framework with a backdoor detection rate of 99.8%.

It was found that poisoned images exhibit only shallow malicious matching with target text in backdoored CLIP, and a subspace detection method was accordingly constructed to effectively identify poisoned samples from multiple backdoor attacks during the test phase.

Proposes TrojanTO, the first action-level backdoor attack against trajectory optimization models in offline reinforcement learning, using alternating training and trajectory filtering to implant backdoors covertly and efficiently with only 0.3% data poisoning.

Combines token-level attention and gradient information to construct anomaly scores, proposing an inference-stage backdoor defense method that significantly reduces attack success rates in text classification and enables localization of backdoor trigger words.

Explains the mechanism of unlearnable samples from the perspective of mutual information reduction, and proposes the MI-UE method that reduces covariance by maximizing intra-class feature cosine similarity, effectively preventing unauthorized models from learning user data.

It proposes an indirect data poisoning backdoor method for large language model pretraining, which can covertly implant a secret response backdoor to track data usage with only a very small amount of poisoned data when the target behavior does not appear in the training set.

Systematically evaluates the practical privacy risks of differentially private fine-tuned large language models under membership inference and canary extraction attacks, finding that distribution shift has a significant impact on empirical privacy protection.

Proposes ε-Gaussian certifiability for high-dimensional mechanisms, theoretically proving that a single-step Newton method combined with Gaussian noise can simultaneously achieve privacy elimination and model accuracy.

Releases the open-source model suite Hubble, which simulates memory risks through controlled inserted text, revealing the impact of sensitive data frequency and training stage on memorization, and providing a benchmark for membership inference and machine unlearning.

Proposed the forgetting framework ASU, which eliminates memory associations by raising the Softmax temperature of attention, achieving robust knowledge forgetting on benchmarks such as TOFU and WMDP while maintaining model output coherence and utility.

Reveal that open-source LLM providers can steal downstream private fine-tuning data in black-box settings through backdoor training, extracting up to 94.9% of samples completely.

It is pointed out that adding or removing adjacent relations underestimates the privacy leakage risk of a single sample's attributes, and an auditing attack targeting replacement of adjacent relations is proposed, showing that existing privacy guarantees struggle to effectively protect attributes such as fine-tuning labels.

A pure black-box task inference attack against shared representations in multi-task learning is proposed. The attacker can infer whether a task is included in the training set using only the embedding vectors obtained from query samples, revealing the privacy leakage risk of shared representations.

Proposes SPARSE, a privacy protection framework for text embeddings, which identifies sensitive dimensions through differentiable masks and applies targeted noise via a Mahalanobis mechanism, reducing privacy leakage from embedding inversion attacks while maintaining downstream utility.

Reveals that using only private data to guide public data selection still leaks member privacy at every stage, and proposes a differentially private selection algorithm to mitigate the leakage.

Addressing the privacy risk of multimodal reasoning models inferring image geographic locations, proposes the ReasonBreak framework, which uses concept-aware perturbation to disrupt hierarchical chain-of-thought reasoning to achieve geographic privacy protection.

It reveals that attackers can use multimodal large reasoning models to precisely infer sensitive geolocation privacy such as home addresses from everyday images like user selfies, and proposes the DoxBench dataset and a two-stage attack framework.

It proposes DP-Fusion, a differentially private inference mechanism that can provably bound the influence of sensitive tokens in the context on the output, protecting document privacy while maintaining high-quality generation.

Reveals that existing large-model unlearning methods often hide sensitive knowledge through fake neurons rather than completely eliminating it, and proposes the Ssiuu method based on attribution regularization, which effectively prevents the recurrence of forgotten knowledge during retraining and ensures privacy security.

To address the forgetting bias problem in long-tail forgetting scenarios, a dynamic loss reweighting method, FaLW, is proposed, which adaptively adjusts forgetting intensity by comparing samples with unseen data from the same class distribution, to satisfy privacy norms such as the right to be forgotten.

Proposes a black-box attack framework that decomposes adversarial queries by exploiting the knowledge asymmetry between RAG systems and standard LLMs, enabling fine-grained precise extraction and localization of private sensitive information in retrieved knowledge bases.

Propose a client-side noise correction framework HIDDENECHO, which suppresses noise amplification of differentially private inputs in deep networks by refining hidden states on the client side, greatly improving downstream task performance while protecting text privacy.

To address the utility conflict of the PATE protocol in multi-output diversity tasks such as text generation, it proposes the Hot PATE sampler, which effectively preserves generation diversity with only API access and without additional privacy cost.

It proposes the information-theoretic membership inference method InfoRMIA and builds a token-level fine-grained memory evaluation framework, precisely localizing privacy leakage to specific tokens in the output, comprehensively outperforming the existing baseline RMIA.

To meet privacy compliance requirements, proposes transferring sensitive knowledge from model parameters to external memory tokens, enabling reversible forgetting and dynamic editing in multimodal large models.

Based on the impact of label smoothing on model confidence and differential privacy, a plug-and-play reverse label smoothing gradient forgetting method, UGradSL, is proposed, which effectively eliminates memory of specific training data and improves local differential privacy with minimal computational overhead.

Finds that only a few weights cause membership inference privacy vulnerabilities while also contributing to generalization, and proposes rolling back only these weights before fine-tuning to enhance resistance to membership inference attacks while maintaining utility.

It is revealed that gradient ascent unlearning causes probability mass to shift to high-likelihood restatement regions (squeeze effect), and an unlearning framework incorporating the model's own high-confidence beliefs is proposed to achieve more thorough data forgetting.

Proposes the EAPrivacy benchmark to evaluate physical privacy awareness of embodied agents, finding that mainstream models generally ignore privacy when tasks and privacy constraints conflict, revealing a fundamental alignment deficiency.

First systematic study of MIA vulnerabilities in diffusion language models; proposes SAMA, aggregating signals from multiple masked subsets; achieves a 30% relative improvement in AUC over the best baseline on nine datasets, and up to an 8-fold improvement at low false positive rates.

Reveals that SAM significantly increases membership inference attack risk while improving generalization performance, because its high memorization of atypical sub-patterns reduces prediction variance and amplifies membership signals.

Proposed the partial-blind forgetting learning framework Reload, which eliminates sensitive entity memories in LLMs without directly retaining or accessing the forgetting dataset, effectively reducing the risk of private data storage and leakage.

A non-interactive secure Transformer inference framework based on homomorphic encryption, using rotation-free Softmax/LayerNorm designs to reduce HE rotations, reduces evaluation time by 52.8% compared to THOR.

Proposes PMC, a method for machine unlearning in large models using local model collapse, which does not require the private data to be deleted in the optimization objective, more effectively eliminating private information while maintaining general model utility.

By leveraging naturally occurring structured random strings in training data, such as cryptographic hashes, as natural identifiers, we achieve post-hoc differential privacy auditing and dataset inference without retraining.

Proposes MoFit, a membership inference attack method for diffusion models that requires no text descriptions, which amplifies the conditional loss of member samples by constructing overfitted model embeddings, effectively auditing training data privacy.

Proposes a prompt data minimization quantification framework and a tree search algorithm, finding that frontier models tolerate higher redaction rates but lack sufficient autonomous minimization capability.

We propose a method that optimizes canary samples via meta-gradient descent, improving membership-inference-based black-box auditing of differential privacy, and significantly enhancing the efficiency of estimating empirical lower bounds on privacy parameters for differentially private models.

For scenarios with limited private data, a differentially private training framework PE-SGD is proposed, which dynamically updates the gradient projection subspace via evolutionary strategies and optimizes the noise injection location, significantly improving model performance under small privacy budgets.

Addressing sensitive query and document leakage risks in retrieval-augmented generation for large models, we propose the cryptographic RAG framework Pisces, which balances semantic and lexical retrieval privacy through oblivious filtering and labeled PSI protocols.

Proposes a lightweight Anonymizing Adapter Module that removes privacy information such as skin color and gender from the features of a frozen video encoder, reducing privacy leakage by 35% while downstream task performance remains close to baseline.

To address source inference attacks in federated learning, a defense mechanism combining parameter-level shuffling with a residue number system is proposed, reducing attack accuracy to random guessing level without losing model accuracy.

From two complementary perspectives of statistical estimation and network approximation, the theoretical separation of memory and generalization in diffusion models is proven, and a pruning method is designed accordingly, effectively reducing data memorization in diffusion transformers while maintaining generation quality.

Reveals that data leakage in model inversion attacks stems from adversarial non-robust features rather than mere memorization, and accordingly proposes an anti-adversarial training method to defend against reconstruction attacks and reveals the trade-off between privacy and robustness.

Proposes the PURGE method, which transforms the process of eliminating sensitive data from pretraining memory in large models into a verifiable task based on group relative policy optimization, achieving efficient unlearning while maintaining practicality.

Proposes the MU-Mis method, which directly suppresses the contribution of forgotten samples by minimizing the model's input sensitivity to them, achieving efficient unlearning without accessing retained data and without harming model utility.

Proposes a federated learning framework combining adversarial privacy decoupling and uncertainty aggregation, reducing membership inference attack rate by 37% while ensuring fairness.

Finds that syntactic similarity, rather than topic relevance, triggers the recovery of forgotten information, and proposes syntactically diversified rewriting of forgetting queries to suppress relearning and alleviate the trade-off between forgetting effectiveness and utility.

This study proposes the ReTrace attack framework, modeling forgotten data recovery as a reinforcement learning problem, using residual forgetting traces in the model to guide a generator in effectively reconstructing deleted image and text privacy samples.

Uses an LLM as an optimizer to alternately improve attack and defense strategies in simulated multi-turn agent interactions; attacks escalate from directly requesting information to impersonation and forged consent, defenses evolve into an identity verification state machine, and the approach generalizes across scenarios and models.

Aiming at the high computational overhead of homomorphic encryption and multi-party computation in privacy-preserving inference, proposes a framework that uses unconditional score correction to approximate errors, accelerating inference while protecting client data privacy.

Analyzes the performance of sharpness-aware minimization in machine unlearning, proposes Sharp MinMax to balance retention and forgetting, significantly enhancing defense against membership inference attacks.

Proposes the implicit knowledge extraction attack IKEA, which can efficiently steal from an external RAG knowledge base through benign queries without jailbreaking or prompt injection, revealing severe privacy and copyright leakage risks.

This study is the first to systematically evaluate the privacy risks of SMOTE, proposing a membership inference attack and a reconstruction attack that can reconstruct real minority-class samples with high precision, revealing its inherent privacy leakage risks.

Proposes the defense framework TRACE-RPS combining fine-grained text anonymization with inference-blocking optimization, which identifies privacy elements through an attention mechanism and induces the model to refuse to answer, reducing attribute inference accuracy from 50% to below 5%.

Proposes Tab-MIA, a benchmark for evaluating membership inference attacks on LLM tabular data, revealing that pretrained models severely memorize tabular data after fine-tuning and face significant privacy leakage risks.

To address the challenge of forgetting sensitive labels in vertical federated learning, a gradient forgetting and recovery method based on manifold Mixup representation enhancement is proposed, achieving efficient few-shot label forgetting while preserving usability without leaking sensitive labels.

The study finds that large language models leave persistent fingerprints after machine unlearning, and attackers can determine with high accuracy whether a model has undergone unlearning using only output text or log probabilities, posing privacy risks of reverse-engineering forgotten information.

To address the privacy leakage risk that approximate machine unlearning is vulnerable to membership inference and data reconstruction attacks, a WARP defense method leveraging neural network symmetry is proposed, significantly reducing signal leakage of forgotten data.

A RUT test method that requires neither model weights nor probabilities is proposed, enabling efficient auditing and detection of harmful fine-tuning and model tampering in black-box API services.

Proposes an induction attack method that successfully recovers the harmful capability gap in the dangerous chemical synthesis domain by requesting answers on adjacent safety domains from frontier models and fine-tuning open-source models.

Targeting the biosecurity risks of high-risk human viruses in DNA language models, proposed the jailbreak attack framework GeneBreaker and an evaluation benchmark, found that models amplify dual-use risks, and called for strengthening safety alignment.

It proposes the PropensityBench benchmark, which evaluates large models' potential propensity to take harmful actions when equipped with simulated dangerous tools and under real-world pressure, across cybersecurity, self-replication, and biological and chemical safety domains.

A benchmark, SoSBench, covering six high-risk scientific domains including chemistry and biology, is constructed. Evaluation finds that frontier models such as DeepSeek-R1 and GPT-4 have extremely high rates of violating content output.

In a real network containing about 8,000 hosts, evaluating AI agents against human cybersecurity experts, the proposed framework ARTEMIS found 9 valid vulnerabilities, outperforming over 90% of human participants with a significant cost advantage.

This study proposes HackWorld, the first benchmark for evaluating computer-use agents exploiting web vulnerabilities through graphical interfaces. Tests show existing SOTA agents achieve a utilization rate below 12%, with shortcomings in multi-step attack planning.

Aiming at the problem of large models generating vulnerable code, proposes RESCUE, a RAG framework for secure code generation, combining program-slicing knowledge base construction with hierarchical multi-faceted retrieval, significantly improving the security metrics of generated code.

By filtering biosecurity-related pretraining data, the model shows significant resistance to adversarial fine-tuning for up to 10,000 steps without harming other capabilities; however, it can still be exploited when information is provided in context, requiring defense in depth.

We propose a malicious fine-tuning method to evaluate the upper bound of dangerous capabilities of open-source models in biosecurity and cybersecurity domains, finding that their capabilities are lower than frontier closed-source models, indicating that the net additional harm from releasing open-source models is limited.

Finds that a single safety sample is sufficient to fully restore the compromised safety alignment of fine-tuned large language models, revealing the low-rank property of safety gradients, achieving extremely low-cost repair without losing utility.

Empirical findings show that safety behavior is highly entangled with general learning components, with safety subspaces overlapping in weight and activation spaces, indicating fundamental limitations of subspace-based fine-tuning defenses.

Proposed the defense method SEAM, which transforms large models into self-destructing models by coupling the optimization trajectories of benign and harmful data, causing severe performance degradation or even collapse when subjected to malicious fine-tuning, fundamentally resisting fine-tuning security attacks.

Proposes ARMs, adaptive red-teaming agents that combine multi-step orchestration with hierarchical memory to automatically generate attack strategies, and constructs ARMs-Bench, a multimodal safety benchmark containing 30,000 test cases.

Proposes the reinforcement learning framework Auto-RT, introducing dynamic policy pruning and progressive reward tracking to autonomously explore jailbreak strategies, improving vulnerability coverage and discovery speed.

Proposes the CAGE framework to decouple adversarial structure from cultural content, adaptively generating red-team test benchmarks that fit local culture and laws; the constructed Korean benchmark can more effectively uncover model vulnerabilities.

By evaluating over 600 pairs of attackers and target models, it reveals that red-team jailbreak success rates are dominated by the capability gap between the two, and proposes a jailbreak scaling curve, noting that testers with fixed capabilities may fail against stronger models.

Proposes the C3LLM framework, which models multi-turn dialogues as a Markov process on a query graph to quantify, with statistical guarantees, the lower bound on the probability that frontier models generate catastrophic responses, revealing significant safety risks.

A red-team agent with an adaptive memory module that can select and combine jailbreak tools and evaluate execution results in a sandbox; its attack success rate outperforms existing methods, and it discovered previously unknown risks on Cursor and Codeium.

A benchmark SocialHarmBench containing 585 prompts based on real events from multiple countries is constructed to adversarially evaluate the robustness and guardrail failures of open-source models in high-risk sociopolitical domains such as authoritarian surveillance and disinformation.

Proposes the STAR black-box automated red-teaming framework, which shifts jailbreak strategy exploration into the latent activation space, generating diverse jailbreak prompts by extracting and recombining strategy principal components, significantly improving attack success rate.

Proposes the DialTree framework combining reinforcement learning and tree search, modeling dialogue as a sequential decision problem, enabling autonomous discovery of multi-turn attack strategies without manual annotation, greatly improving red team attack success rate.