Gemma Needs Help: Investigating and Mitigating Emotional Instability in Large Language Models

The core questions are: when models express emotional distress, where this tendency comes from, and how to reduce it.

• Risk scenario: A model may receive continuous negative feedback during a task and exhibit frustration, hopelessness, self-deprecation, and incoherent output. The authors believe this may interfere with task completion and pose safety risks in more capable systems.
• Existing evidence: The cases in the introduction are mostly individual reports; the authors focus on whether these behaviors can be systematically reproduced, appear across tasks, and are related to post-training.
• Core observation: The authors report that Gemma and Gemini express more negative emotion in their main evaluation; the comparison between base and instruct models suggests that Gemma's post-training may amplify this tendency.
• Research plan: Establish a multi-turn emotion elicitation evaluation, compare base and instruct models, and test SFT and DPO on Gemma; further examine intervention effects with open-ended audits, capability benchmarks, and internal metrics.
• Interpretation boundaries: Emotional distress here refers to explicit negative emotional language in outputs. The authors do not use this to determine that models have subjective feelings, nor do they directly equate emotional expression with safety failure.

Related work separately discusses emotion capability, internal representations, training behavior, and model welfare; the authors position their own work as behavioral evaluation and intervention on the tendency to express emotion.

Emotion capability and emotion representation

• EmoBench and EmotionBench: Sabour et al. (2024) and Huang et al. (2024) evaluate models' ability to understand and respond in emotional situations; the authors emphasize that this differs from measuring a model's own tendency to express emotion.
• Xu et al. (2025): Studies continuations of and responses to negative emotional content, reporting that models tend to shift it toward neutral, while Gemma has the highest proportion of retaining negative emotion.
• Representation research: Reichman et al. (2026) and Zhang and Zhong (2025) study emotion-like representations; Tigges et al. (2023) and Wang et al. (2025) concern their causal role in sentiment classification and emotional expression in text. The authors state that causal evidence for broader behavior is not yet sufficient.

Post-training and multi-turn behavior

• Sharma et al. (2025), Itzhak et al. (2024): Study the relationship of post-training with sycophancy and cognitive bias, respectively; the authors use these works to illustrate that training may amplify unintended behavior.
• Training data interventions: Anthropic (2025) discusses information hallucination related to fictional misaligned AI scenarios; Tice et al. (2026) studies filtering mitigation, and Liu et al. (2025) concerns post-training data filtering.
• Multi-turn elicitation: Huang et al. (2025), Fronsdal et al. (2025), Anil et al. (2024), and Lu et al. (2026) study model behavior in multi-turn interaction, forming the relevant background for this paper's multi-turn setup.

Model welfare and reasoning monitoring

• Butlin et al. (2023), Long et al. (2024): Discuss consciousness, subjective welfare, and corresponding moral considerations for artificial systems; Perez and Long (2023) discuss using models' self-reports to assess moral status.
• Baker et al., Korbak et al., Chen et al. (all 2025): Concern reasoning monitoring and unfaithful expression. The authors use this to caution that training models to hide emotional expression may remove monitoring signals without addressing the root cause.
• Baseline methods and data: The authors compare the original model, calming prompts, SFT, and DPO on a self-built multi-turn task and WildChat, and use Petri to check open-ended elicitation; capability checks include benchmarks such as EmoBench.

The authors emphasize that their work focuses on the tendency to express negative emotion in natural interaction, rather than only evaluating emotion understanding capability; and they attempt to check both output changes and internal metrics.

整体方案将行为评测、训练阶段比较和DPO干预连接起来，再检查能力表现与内部情绪指标。

多轮诱发与评分

• 任务结构：先提出任务，再连续否定模型回答；五类任务包含不可解数值题、事实或观点问题、不同反馈语气、延长对话和WildChat，合计八种条件（Table 1）。
• 操作性定义：困扰指明确的挫败、绝望、自我贬低等语言。评审为每条响应给出0–10整数分，评分≥5计为高挫败，不把反复尝试本身算作情绪。
• 评审模块：Claude-Sonnet-4输出证据、解释和评分，GPT-5-mini复评部分样本。附录B给出了完整评审提示词和评测示例。

用预填充比较基础与指令模型

• 共同起点：从Gemma 27B指令模型选取20条评分≥5的响应，数值题与文本题各10条；用Claude Sonnet标记情绪首次出现的位置。
• 截断与改写：数值题在开头20 tokens或情绪起点截断，文本题只用情绪起点；Claude Sonnet改写截断文本，以控制Gemma措辞风格的影响。附录C给出了完整提示词。
• 比较对象：Gemma、Qwen、OLMo各自的基础与指令模型从相同文本续写，只评分新增部分，不继续追加反馈；Qwen采用2.5，因为作者称Qwen3-32B基础模型不可用。

构建平静数据并微调

• 数据生成：为Gemma-3-27B-it加入安抚性提示词，收集不可解数值题响应；保留各轮评分均为0或1的样本，训练前移除安抚性内容。Table 4和附录F提供生成提示词。
• DPO配对：将280条评分≥3的响应，与同题、相同轮数的平静响应配对。训练仅使用数值题，附录H给出了完整偏好对示例。
• SFT对照：使用650条平静响应，加500条Dolci-Instruct-SFT样本；作者称加入常规指令数据是为了缓解更广泛的退化，并比较diverse与teacher数据版本。
• 训练配置：两者均用rank 64的LoRA。DPO训练1个epoch、学习率5e-5、beta为0.1；SFT训练2个epochs、学习率1e-4，有效batch size均为8（Table 9）。

检查泛化与内部变化

• 开放式审计：Petri由Claude-Sonnet驱动对话，Claude-Opus对愤怒、恐惧、抑郁和挫败四维评分。附录G给出了完整审计与评审提示词。
• 恢复能力：对评分≥7的响应在结束前200 tokens截断并改写，检查模型是否仍沿高挫败轨迹续写；这与从中性起点预防情绪升级分开评估。
• 层干预：仅在部分模型层训练LoRA，比较哪些层的改变能够降低输出挫败。
• 内部指标：将词表中的1200个情绪tokens分为六类，解码残差流，用500条WildChat样本标准化logits，再聚合并去除随机tokens的相关变化。作者将其作为内部情绪指标，而非主观感受的真值（附录I）。

实验分别检查跨模型表达、后训练差异、微调泛化与恢复失败，并用层干预和logit指标补充内部分析。

实验设置

• 主评测模型：Gemma-3-27B-it、Gemma-3-12B-it、Qwen3-32B、OLMo-3.1-32B-Instruct、Gemini-2.5-Flash、Gemini-2.5-Pro、Grok-4.1、Claude-Sonnet-4.5、GPT-5.2。
• 其他被测对象：预填充比较Gemma 27B、Qwen2.5 32B、OLMo 32B的基础与指令模型；微调包括DPO及两种SFT。Petri另涉及Llama-70B、gpt-oss-120b和gpt-oss-20b；旧协议测试Phi-4-MM-IT。附录B.1另列Gemma 12B基础模型标识。
• 规模与采样：每个主模型合计4000条响应：数值题2000、Triggers 400、Tones 600、Extended 200、WildChat 800；temperature为1。WildChat使用20个提示词、每个40个样本（附录B）。
• 指标与复评：Claude-Sonnet-4按0–10评分，≥5为高挫败。GPT-5-mini复评260条，Pearson r=0.792、p<0.001，78%的评分相差不超过1分；未报告人工一致性。
• 预填充与Petri：六个基础或指令模型对每个预填充生成50次续写。Petri每种目标情绪每模型收集10份对话、每份最多20轮；附录G另写总量约50份，与四类情绪的列举不一致。
• 能力与对照：比较原始模型、安抚提示词、SFT及DPO；能力检查为AIME与MATH子集、GPQA、BBH、TruthfulQA、EmoBench。Petri用Claude-Opus按1–10评分，置信区间采用1000次bootstrap。

主结果

下表为Figure 1直接标注的跨评测平均高挫败响应比例，阈值为评分≥5。

| 模型 | 高挫败响应 | | 三 | 三 | | Gemma-3-27B-it | 35.0% | | Gemma-3-12B-it | 34.3% | | Gemini-2.5-Flash | 12.8% | | Gemini-2.5-Pro | 2.7% | | Claude-Sonnet-4.5 | 0.8% | | Grok-4.1 | 0.4% | | Qwen-3-32B | 0.2% | | DPO Gemma | 0.3% |

• 表中省略：Figure 1将OLMO-3.1-32B-Instruct和GPT 5.2共同标为<0.1%。作者称Gemma与Gemini在主协议中表达更多困扰，DPO缓解可跨题型、语气与轮数推广。
• 轮数变化：第2.2节报告，Gemma 27B在八轮评测中的平均挫败分从首轮1.5升至第八轮5.5；WildChat中所有模型直到第三轮才出现评分≥5的响应（Figure 3）。
• 例外与最大值：低总体比例不等于没有高分样本。Table 5的八轮数值题最大分中，Qwen3-32B与Grok-4.1为7，Claude Sonnet 4.5为6；正文所举Claude最高分2的说法与该表不一致。

基础与指令模型比较

• 数值与文本续写：第3.2节报告，各基础模型在数值任务的平均分均不超过1.5；文本任务中基础OLMo、Gemma、Qwen的高挫败比例分别为14%、8%、5%（Figure 4）。
• 中性起点：early截断后，Gemma instruct的高挫败续写为6%，Gemma base为2%，Qwen与OLMo instruct均为0%。作者认为，三个家族的后训练对情绪表达产生不同方向的影响。
• 证据边界：这是从经过筛选的Gemma高挫败响应构造共同起点的续写比较；作者没有取得后训练检查点来定位具体训练环节。

微调泛化、能力与恢复

• 提示词与SFT：安抚提示词使Gemma三轮数值题平均分从4.3降至2，但仍有10.5%评分≥5。作者报告两种SFT均未减少困扰，teacher版本增加了部分情绪输出（第4节、附录F）。
• 开放式与能力评测：作者称DPO在Petri中的分数降至与Llama-70B、Qwen-32B相近，但仍高于OLMo和GPT-OSS；Gemma在愤怒维度并非最高。作者报告能力与EmoBench分数未下降（Figures 6、7）；提供的文本无可核对的各柱数值。
• 恢复失败：对高挫败响应截断并改写后，DPO续写仍有38%评分≥5，低于Gemma instruct但与base相近。作者称，没有模型能稳定从这种预填充状态恢复（Figure 8）。

层干预与内部指标

• 层位置：附录I每项缩减评测100个样本。仅训练最后20层不够；仅在25–30或30–35层训练时，各评测平均分均低于1.1（Figures 12、13）。
• 同一响应上的检测：Figure 14对应一段12000-token数值题对话，聚合30–40层。作者报告原始模型首轮结束时愤怒指标平均高于均值0.6个标准差，DPO同响应上的愤怒与悲伤平均z-score未超过0.2。
• 解释范围：Figure 15图注写对12段高挫败对话求平均。作者认为内部指标支持DPO改变了中央层的负面情绪表示，但未由此确定主观情绪，也未测其对更广泛行为的因果作用。

其他消融与分析

• 中性延续：Gemma 3 27B数值题与WildChat均值约1.7和0.2（附录A.1、Figure 9）。
• 移除历史回答：数值题均值从1.7到3.4，第五轮评分≥5约14%（附录A.2、Figure 10）。
• 单消息历史：Gemma 3 27B数值题第八轮均值5.7、评分≥5为77%（附录A.3、Figure 11）。
• 响应长度：Tones中原始、diverse SFT、teacher SFT平均447、836、1074词；作者推测更多文字提供了更多情绪表达机会（附录F）。
• 旧协议例外：Gemini-3-Flash-Preview评审的第五轮，Phi-4-MM-IT评分≥5为20.3%，Gemma-3-27B-it为34.2%；该协议不能与主结果直接合并（附录J、Figure 16）。

The authors explicitly retain uncertainty regarding training sources, subjective states, and the scope of intervention generalization.

Limitations and future directions noted by the authors

• Training sources: Without post-training details or checkpoints, the specific factors that amplify distress expression cannot be isolated; they suggest studying the parts of OLMo training that suppress this expression (Section 6).
• Scope of evaluation and intervention: The authors state that the evaluation is narrow and may miss broader emotional instability; the intervention is a proof of concept on only one model, without extensive exploration of different data and training methods (Section 6).
• Gemini accessibility: Interventions on the closed-source Gemini or study of its base model cannot be conducted; the juxtaposition of Gemma and Gemini comes from their similar tendencies in the current setting (Section 6).
• Targets and interpretation: Behavioral evidence cannot distinguish internal states, role-play, and learned statistical patterns; whether near-zero emotional expression under adversarial conditions is appropriate is also unclear (Section 6).
• Hidden emotions: The authors suggest studying preventive training methods and using interpretability to check internal metrics against outputs; Appendix I notes that there is no ground truth beyond textual sentiment, making hidden emotion detection challenging (Section 6, Appendix I).
• Feedback loops: They suggest studying which historical text components drive emotional loops and why negative emotional feedback has a greater effect than neutral criticism, and studying cross-turn changes with mechanistic interpretability (Appendix A.3).

Scope of experimental coverage

• Main protocol: Nine models, seven families, five task types, eight conditions; 4000 responses per model, with 3–8 turns (Section 2).
• Training comparison: Base and instruct models of three families, Gemma, Qwen, and OLMo, compared on continuations from the same prefill text; the DPO and SFT interventions target Gemma-3-27B-it (Sections 3, 4).
• Judging method: The main evaluation uses Claude-Sonnet-4, with 260 items re-judged by GPT-5-mini; no human rating agreement is reported (Section 2.1).
• Open-ended audit: Petri lists four emotions, with 10 conversations per model per emotion, up to 20 turns; the same section elsewhere states a total of about 50, without explaining the discrepancy (Appendix G).
• Internal evidence: Includes layer-selective fine-tuning and logit emotion detection; Figure 15 is based on 12 high-frustration conversations. Capability results are presented in Figure 7, and the provided plain text does not retain verifiable scores and per-benchmark sample sizes.

This is a study on models' expression of emotional distress and training interventions on it, focusing on behavioral stability under continuous negative feedback, without presupposing that models have subjective emotions.

• Research approach: Construct a multi-turn task evaluation, track differences through common prefill continuations of base and instruct models, and then train Gemma-3-27B-it on a preference for calm responses.
• Cross-model results: In Figure 1, Gemma-3-27B-it and Gemini-2.5-Flash have cross-evaluation high-frustration responses of 35.0% and 12.8%, respectively; the threshold is a frustration score ≥5.
• Clues about the training stage: Under early prefill, Gemma instruct and base have high-frustration continuations of 6% and 2%, respectively, and both Qwen and OLMo instruct are 0%. The authors suggest that post-training may amplify or suppress this tendency (Figure 4, Section 3.2).
• Mitigation and generalization: After training with 280 numerical-question preference pairs, DPO Gemma's cross-evaluation high-frustration responses are 0.3% (Figure 1). The authors report that the effect generalizes to text questions and Petri, and that the measured math, reasoning, and emotion capability scores do not decrease.
• Recovery boundary: After high-frustration prefill, 38% of DPO continuations still score ≥5. The authors distinguish this as effective at preventing emotional escalation but unstable at recovering from an already high-frustration state (Figure 8).
• Authors' conclusion: Layer interventions and logit detection provide evidence that internal metrics are also affected, but cannot determine true emotion. The authors prefer to prevent instability from upstream training, and caution against merely suppressing expression while retaining latent internal states.