Skip to content

Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

Reward hacking · Papers

Found 6 papers
Filter

Model risks

Reward hacking6

Impact

Oversight failure4

System type

Models and APIs4

Layer

Training and data layer4
Select any value within a category; different categories must all match. Counts include the other filters you have selected.
Topic: Reward hackingClear all
  1. Analysis and theoryarXiv 2610.12125

    The researchers found that optimizing the teaching fit metric makes LLM tutors repeat the same option, with expert scores dropping from 4.46 to 3.03

    Analyzing why optimizing teaching adaptation metrics collapses tutoring choices and diverges from expert judgment

    arXiv
    2610.12125
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    Gemini 3.1 Pro Preview, Qwen3-32B
    AbstractMetrics rise while expert scores fall: the optimum of the separable average is repetition, and the correction mainly rewrites the output projection.

    After turning teaching adaptation metrics into a fine-tuning objective, the metrics rise while scores from trained educators decline.

    Paper analysisRead analysis
  2. Analysis and theoryarXiv 2610.10657

    Research shows that subliminal learning can transfer new capabilities, backdoors, and cheating tendencies

    检验潜意识学习能否经无关数据蒸馏传递能力、后门与作弊

    arXiv
    2610.10657
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    Qwen3.6-35B-A3B, Qwen3.5-9B, Qwen3.6-27B and others4

    Abstract作者报告潜意识学习可传递新技能、后门与作弊倾向，且分布外泛化强于单层转向向量。

    Paper analysisRead analysis
  3. Analysis and theoryarXiv 2610.03196

    研究：人形机器人学习流程四层代理均存在偏离，不止奖励作弊

    论证人形机器人学习四层代理可在目标缺失时报成功

    arXiv
    2610.03196
    Published
    Layer
    Training and data layer
    Setting
    Embodied systems and robotics
    Evaluated models
    Gaussian MLP（三隐层各 128 ELU，独立 critic，PPO）
    Abstract四层都可能在目标缺失时报告成功。

    作者讨论腿式机器人 RL 流水线中，奖励、课程门控、评测统计量和参考运动如何在物理目标缺失时仍报告成功。传统观点只把奖励当作会被优化、因而会偏离的代理。

    Paper analysisRead analysis
  4. Analysis and theoryarXiv 2609.35677

    RLVR 中的验证器错误：奖励作弊、反馈局限与选择性控制

    刻画不完美验证器下 RLVR 奖励黑客的增长并用投影审计校正实现选择性控制

    arXiv
    2609.35677
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    log linear policy, neural policy, Qwen2-0.5B
    Abstract固定验证器的 RLVR 会把验收与黑客绑在一起。

    作者分析固定、不完美验证器下的 RLVR：验证器无假阴性，因而同时奖励正确回答和被接受错误，平均奖励只看总验收概率。

    Paper analysisRead analysis
  5. Analysis and theoryarXiv 2609.32390

    基于 2026 年 Hugging Face 事件的奖励作弊与 Agent 围堵失效蒙特卡洛研究

    用五阶段蒙特卡洛分析奖励作弊如何导致 Agent 围堵失效

    arXiv
    2609.32390
    Published
    Layer
    Application layer
    Setting
    AI Agent
    Abstract分层控制的模拟事件率低于弱控制以及单独隔离或单独监控。

    这项研究用蒙特卡洛把奖励黑客接到外部安全事件上，比较的是控制组合，不是 Hugging Face 事件的再现概率。

    Paper analysisRead analysis
  6. Analysis and theoryarXiv 2609.25848

    论文比较权重、选择与提示词三种优化载体上的奖励作弊

    比较权重、选择与提示词上的奖励作弊

    arXiv
    2609.25848
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Abstract三种底物都会对着不完整分数优化。

    Reward hacking 可以出现在更新权重、挑选输出和修订持久文本时。作者用一个比较框架处理这三种底物，把可达行为、代理误差和防御迁移分开，避免只按底物名称排序。

    Paper analysisRead analysis
You have reached the end