Translation in progress Some entries still include text in the original language.
Oct 12 · Monday
Reward hacking · Papers Found 4 papers “
About this collection Collection in progress We are building a large AI safety paper collection to help researchers find work related to their interests. Papers are collected automatically andfiltered only for relevance to AI safety , without judging quality, novelty or reading value. The default “Annotated first” shows papers with tags or published paper analyses first. Each group is ordered by original publication date, newest first, with undated papers last. “Newest first” uses publication dates only. This order is not a quality ranking.
One-line summaries, tags and paper analyses are prepared by AI from the papers. Language models may have systematic biases and produce hallucinations.We plan to carry out human annotation and review to reduce errors in this content.
Analyzing why optimizing teaching adaptation metrics collapses tutoring choices and diverges from expert judgment
Published Oct 8
Evaluated models Gemini 3.1 Pro Preview , Qwen3-32B Abstract Metrics rise while expert scores fall: the optimum of the separable average is repetition, and the correction mainly rewrites the output projection.After turning teaching adaptation metrics into a fine-tuning objective, the metrics rise while scores from trained educators decline.
检验潜意识学习能否经无关数据蒸馏传递能力、后门与作弊
Published Oct 8
Evaluated models Qwen3.6-35B-A3B , Qwen3.5-9B , Qwen3.6-27B and others4 Abstract 作者报告潜意识学习可传递新技能、后门与作弊倾向，且分布外泛化强于单层转向向量。
刻画不完美验证器下 RLVR 奖励黑客的增长并用投影审计校正实现选择性控制
Published Sep 29
Evaluated models log linear policy , neural policy , Qwen2-0.5B Abstract 固定验证器的 RLVR 会把验收与黑客绑在一起。作者分析固定、不完美验证器下的 RLVR：验证器无假阴性，因而同时奖励正确回答和被接受错误，平均奖励只看总验收概率。
比较权重、选择与提示词上的奖励作弊
Published Sep 22 Abstract 三种底物都会对着不完整分数优化。Reward hacking 可以出现在更新权重、挑选输出和修订持久文本时。作者用一个比较框架处理这三种底物，把可达行为、代理误差和防御迁移分开，避免只按底物名称排序。
You have reached the end