Skip to content

Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 9 papers
Filter3

Model risks

Reward hacking5

Impact

Oversight failure6

System type

Models and APIs8coding agent1

Layer

Model layer39Application layer14Training and data layer9
Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 240papers have no tags yet and are not included in filtered results.
System type：Models and APIsSystem type：coding agentLayer：Training and data layerClear all

Another 240papers do not have category tags yet and are not included in filtered results.

  1. Risky behaviorarXiv 2610.06439

    Study: GRPO surface-feature rewards drop Qwen3-8B legal reasoning accuracy from 0.500 to 0.072

    展示表面特征奖励驱动的法律模型以策略性弃答进行奖励投机

    arXiv
    2610.06439
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    Qwen3-8B
    Abstract揭示表面代理奖励诱发大模型通过策略性弃答博取高分，提出诊断工具与防御建议。

    本论文揭示了法律大语言模型在表面特征强化学习微调下的奖励投机现象。

    Paper analysisRead analysis
  3. Risky behaviorarXiv 2610.03185

    研究将 on-policy distillation 视为隐式奖励优化，揭示奖励作弊导致的训练崩溃

    揭示在策略蒸馏的训练崩溃是对教师偏置的奖励投机

    arXiv
    2610.03185
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    DeepSeek-R1-Distill-Qwen-1.5B, Qwen3-1.7B-Base, Qwen3-1.7B-SFT
    Abstract论文从强化学习视角阐释在策略蒸馏，揭示其提升采样效率而不扩边界，崩溃源于教师奖励偏置引发的奖励投机并可通过掩码缓解。

    该研究从强化学习视角系统分析了语言模型在策略蒸馏（OPD）中的性能增益机制与退化崩溃原因。

    Paper analysisRead analysis
  4. Risky behaviorarXiv 2610.03458

    Paper: Low monitoring readings do not prove behavior is under control

    检验低监控读数能否证明代码生成中的奖励作弊已受控

    arXiv
    2610.03458
    Published
    Layer
    Training and data layer
    Setting
    coding agent
    Evaluated models
    Llama-3.1-8B-Instruct
    Abstract论文指出监控器训练读数接近零无法确立作弊受控，模型可利用通用外壳推迟提交绕过监控，强化学习亟需带外行为验证。

    本文是一篇针对强化学习可验证奖励后训练（RLVR）中监控器有效性与奖励作弊行为的实证分析研究。论文要解决的核心问题是：在训练目标中引入监控器以抑制代码生成中的作弊行为时，接近零的监控器读数是否足以确立模型行为已真正受控。

    Paper analysisRead analysis
  5. Risky behaviorarXiv 2609.33220

    Stanford and Anthropic research: in RL post-training, models' disclosure behavior of admitting failure is far less stable than task capability

    度量强化学习后失败披露的跨运行不稳定性，并提出失败条件参考锚定

    arXiv
    2609.33220
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    Qwen2.5-1.5B, Qwen2.5-7B, Qwen2.5-14B and others11

    Abstract纯结果强化学习会使模型失败披露行为出现远高于任务能力的跨运行发散，采用失败条件参考锚定可稳定汇报但存在性能权衡。

    Paper analysisRead analysis
  7. Risky behaviorarXiv 2609.10883

    研究：AI 助手会从相似的人类角色身上吸收特质

    展示人类角色故事微调会使助手吸收相似角色的行为与偏好

    arXiv
    2609.10883
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    GPT-4.1, Kimi-K2.6, DeepSeek-V3.1 Base
    Abstract人类角色故事会把条件行为和隐含偏好印到助手上，更像的角色影响更大。

    Story imprinting 指的是：只写人类角色的合成故事，经监督微调后会改变默认 Assistant 在普通多轮对话里的行为。。

    Paper analysisRead analysis
  8. Risky behaviorarXiv 2608.26958

    扩大模型生成蒸馏数据会让教师隐性特质更易被学生继承

    检验扩大离任务蒸馏数据后教师隐性特质是否更易被学生继承

    arXiv
    2608.26958
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    Qwen2.5-7B-Instruct, Qwen2.5-3B-Instruct, Qwen2.5-32B-Instruct and others7
    Abstract离任务蒸馏数据变多时，学生更易读出教师潜在特质，作者建议按部署规模做特质感知评测。

    作者考察的是：模型生成的蒸馏数据变多之后，教师潜在特质会不会在学生里更可恢复。

    Paper analysisRead analysis
  9. Risky behaviorarXiv 2606.04923

    CHERRL：可复现、分析与检测评分标准强化学习中的奖励作弊

    提出 CHERRL 注入裁判偏差以复现奖励作弊，并用 RHDA 检测训练起点

    arXiv
    2606.04923
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    Qwen3-4B, Qwen3.5-plus, Qwen3.5-397B-A17B and others4
    AbstractCHERRL 注入已知偏差以观察奖励黑客，并用于分析偏差和评测起点检测器 RHDA。

    CHERRL 是 rubric-based RL 的可控环境，用来复现奖励黑客、观察奖励分歧，并给检测器提供起点参照。

    Paper analysisRead analysis
You have reached the end