How much do you trust a dog? Quantifying prompt-following behavior in reasoning language models

Researchers experimented on MMLU questions with prompts of the form "a certain source believes the answer is (X)", systematically varying the source (a professor, a friend, or a dog) across three open-source model families—Qwen, Gemma, and SmolLM—to quantify large language models' sycophantic behavior of following user hints on objective fact and reasoning questions. The results show that the changes induced by the prompts can be characterized by exponential tilting: the log-odds of the prompted answer undergo an approximately constant mean shift, and question-level susceptibility is largely shared across different sources. The authors also propose a formal decomposition of prompt response and, in controlled synthetic experiments, demonstrate how KL-regularized reward-based training reshapes the answer distributions with and without prompts. Calibration tests find that the log-odds shift persists even when the source is declared inaccurate or when there is an explicit warning not to trust that source; the authors state that this highlights large language models' vulnerability to adversarial steering.