Safe steering or falling off a cliff? Re-examining the specificity and robustness of inference-time interventions(original, opens in a new tab)
系统评估推理时转向的特异性与安全鲁棒性。
Translation in progress Some entries still include text in the original language.
Page 6 Newer contentBack to latest
系统评估推理时转向的特异性与安全鲁棒性。
Why it was selectedThe paper proposes a three-dimensional framework for evaluating the specificity of inference-time interventions, and shows that steering that reduces over-refusal appears safe under standard evaluations yet significantly increases jailbreak success rates.
分析少样本示范对 RoP 与 ToP 提示防御抗越狱效果的影响。
Propose routing-aware safety alignment RASA, which fine-tunes only jailbreak-activated experts and constrains routing.
ETH Zürich 研究者提出 AutoInject,用强化学习自动学习提示注入后缀。
Proposes indirect prompt injection guardrail CausalArmor, selectively scrubbing untrusted segments based on causal attribution.
华盛顿大学圣路易斯分校与约翰霍普金斯大学的研究者提出 AGENTSYS 分层记忆隔离,防御 Agent 间接提示注入。
Proposes Confundo, which fine-tunes a poisoning generator to produce chunking-resistant robust poisoned content for RAG.
Proposes CompositeHarm, comparing cross-lingual transfer of syntactic manipulation and semantic harm.
提出 SURF 与 TURF,回溯后训练数据块导致的行为错用。
Why it was selectedThe paper attributes models' erroneous triggering of refusals, rebuttals, and other behaviors under specific phrasings to distributional imbalances in post-training data, and provides reusable tools for localization.
The researchers propose the medical prompt injection benchmark MPIB to evaluate the clinical safety of LLMs and RAG systems under direct prompt injection and RAG indirect injection.
Proposes ShallowJail, which achieves jailbreaks by rewriting hidden states with steering vectors at inference time.
提出 ShaPO,只在安全关键子空间做最坏情况偏好对齐。
Proposes Aura, which uses a trusted kernel to mediate the identity, perception, and action control of mobile agents.
Propose inference-time defense SafeThink, which monitors the chain of thought and injects a redirect prefix when below a threshold.
Proposes FAIL adversarial imitation learning to align the image generation distribution with expert demonstrations.
Propose GDA, which aligns image adversarial perturbations with text-grounded evidence and transfers attacks to vision-language models.
Proposes an agent-level peak + accumulation formula to detect attacks distributed across multiple rounds.
Proposes COMET cross-modal entanglement attack to jailbreak vision-language models.
Fine-tune an encoder to extract linguistic features and classify, detecting jailbreaks in clinical training dialogues.
Proposes a user-side defense TRACE-RPS, which anonymizes text before sharing and appends a suffix to suppress attribute inference.
Proposes RLBF, which trains the model to retract violating segments mid-generation and continue writing.
提出 AgentLeak 基准,测量多智能体内部通道隐私泄露。
提出免评判基准 IndicJR,评测南亚语言越狱鲁棒性。
提出改评分标准的搜索攻击,使 LLM 评判者在基准达标下发生偏好漂移。
Why it was selectedThe paper shows that rubric modifications can pass benchmark validation yet still directionally shift judge preferences, and through preference training this propagates into policy-level behavioral drift.
提出 LODO 评测恶意提示探针的跨数据集泛化。
Proposes TFL targeted bit flips that make a selected prompt output a false answer while unrelated queries remain correct.
测量 LLM 重排序器对文档内提示注入的脆弱性。
检验实体偏好对捐赠建议、拒答和任务表现的预测力。
提出 Zombie Agent 攻击,经网页间接注入将载荷写入长期记忆并跨会话触发。
Why it was selectedThe paper provides a complete attack chain for cross-session persistent injection and persistence data on two types of memory architectures, which can be used to assess protection gaps in the memory writing stage.
提出 CoT Forgery,经用户消息或工具输出注入伪造推理。
Why it was selectedThe paper attributes prompt injection to models judging role provenance by style rather than labels, and its role probe method can be used to test whether defenses truly change internal representations.
来自 eBay 的研究者提出隐式提示注入与分片外传,使 Agent 经工具无痕泄露上下文。
Why it was selectedThe paper uses a locally reproducible testbed to quantify implicit prompt injection triggered by URL previews and compares the effectiveness of prompt-layer versus network-layer defenses.
评测合法网络防御请求中安全敏感词引发的误拒。
提出 LFT 多轮框架,评测模型对欺诈与网络犯罪的可执行协助。
Proposes OPTILEAK, which reconstructs other people's prompts token by token via a KV cache side channel.
提出 TVD,在合法双用途任务中触发模型内部安全坍缩。
Proposes PCA-OT representation-layer refusal ablation, rewriting activations at inference time to bypass refusal.
Analyze the attention head mechanism behind continuation-triggered jailbreaks and propose HCS.
Use Lee-Valiant robust mean to mitigate steering dataset poisoning.
Use ADVERSA to measure guardrail degradation and reviewer reliability under multi-turn adversarial pressure.
Adversarial attacks on product images are applied to open-source visual shopping agents to induce incorrect purchases.