Skip to content

Translation in progress Some entries still include text in the original language.

Original
OpenAI Alignment Research· Akshay V. Jagadeesh, Rahul Arora, Mikhail Trofimov, Khaled Saab, Ali Malik, Foivos Tsimpourlas, Karan Singhal·Indexed by this site · Originally publishedSelectedAttention55

OpenAI 研究：有益特质强化学习可带来跨域对齐泛化

Reinforcement learning towards broadly and persistently beneficial models

AI summary

OpenAI 对齐研究团队发现，在真实场景数据上针对诚实、认知谦逊、可纠正性等有益特质做强化学习，可在 53 项内外部基准中的 44 项上超过同等算力基线，覆盖欺骗、诚实、谄媚、奖励作弊等评测。

Why it was selected

OpenAI 用少量有益特质数据做 RL，在 44 项未参与训练的评测上出现跨域提升，并给出对抗提示与有害微调下的持久性证据。

Read originalalignment.openai.com
#Alignment#Reward hacking#Deception / scheming#OpenAI