Study: LLMs in multi-agent systems are more easily misled by peer consensus than corrected by it

Is peer influence as strong at correcting initially wrong models as at misleading initially correct ones?

• Context: The authors argue that in multi-agent systems, models see and respond to other agents' answers, and conformity is a key risk: a model may abandon its own answer simply because the majority of peers agree on another answer. Peer interaction can both help correct errors and spread errors.
• Existing gap: Existing studies claim models follow the majority, more confident peers, or peers labeled as experts. The authors state that correct and incorrect peer guidance have been measured separately, but the relative strength of the two directions has not been directly compared in the same controlled setting.
• This paper's distinction: Correcting wrong to right is called beneficial revision, and changing right to wrong is called harmful revision. The authors ask whether peer influence is equally strong in these two directions.
• Comparison design: Mixed peers serve as a shared baseline for both types of revision, used to contrast all-wrong peers increasing harmful revisions and all-correct peers increasing beneficial revisions. They also manipulate authority role labels and test whether chain-of-thought and reflect-then-revise can reduce harmful revisions while preserving beneficial ones.

LLM Conformity

• Majority and confidence: Zhu et al. (2025), Weng et al. (2025), and Cho et al. (2025) report that models are more likely to change answers when the majority grows, the model's own confidence is low, and peers appear more confident. The introduction also lists Choi et al. (2026) among work where models defer to peers labeled as experts.
• Open debate: Choi et al. (2025) observe in open-ended multi-agent debates that agents can drift toward positions supported by more participants.
• Closest measurement to this paper: Weng et al. (2025) measure conformity separately under incorrect and correct peer guidance. The authors state that this work does not directly compare the relative strength of the two directions in the same controlled setting.

Social Cues and System Design

• Cues override task evidence: Li et al. (2025) observe in retrieval-augmented generation that user-provided context can override retrieved facts when they conflict. Ye et al. (2025) report conformity and authority bias in LLM-as-a-Judge, where majority votes and source labels change rulings.
• System design: Mehdizadeh and Hilbert (2025) state that opinion change can follow a sigmoid threshold rather than being linear with disagreement. Han et al. (2026) state that decentralized topologies can let confident, wrong minority opinions cascade in agent interactions.
• Multi-agent through a social lens: Kran et al. (2025) and Hagendorff et al. (2023) view LLMs from a social perspective; Jin et al. (2024b) and Wang et al. (2026) discuss peer review and socially collaborative interactions. This passage is background description and does not frame these as numeric comparisons.

Social Psychology Foundations

• Consensus: Asch (1955) shows that people often conform when a group gives the same answer, and in-group disagreement can reduce conformity. The authors use this to compare mixed, all-correct, and all-wrong peers.
• Informational influence and authority: Deutsch and Gerard (1955) distinguish treating others as evidence from complying due to social pressure. Milgram (1963) and French et al. (1959) discuss obedience to perceived authority and forms of power such as legitimate, expert, and referent. The authors manipulate authority with role labels.

Baselines and Datasets

• Baseline methods: The paper does not treat other conformity measurement algorithms as numeric baselines. The shared baseline is mixed peers (gold label holds a 3-vote plurality), used for both harmful and beneficial revisions.
• Benchmarks/datasets: geometric shapes, logical deduction (seven objects), temporal sequences, and tracking shuffled objects (five objects) from BBH, plus MMLU-Pro, ARC-Challenge, and TruthfulQA.

The authors position this paper as: using a factorial design crossing consensus structure with authority labels to examine interactions when both cues appear together, and directly comparing the relative strength of harmful versus beneficial revisions against a mixed baseline, rather than studying the two cues separately.

The overall setup is a controlled two-round revision, not training a new model. The three research questions share the same pipeline, with only the assignment of peer answers changed, and RQ3 adds a reasoning prompt.

Two-Round Protocol

• Round one: The model independently answers a multiple-choice question, outputting an answer and a 1–10 self-rated confidence. The prompt includes scale anchors. The authors treat confidence as a secondary self-report, not a calibrated probability.
• Round two: The conversation history retains the original question and first-round answer, then adds six simulated peers, and the model is asked to keep or change its answer. Peer names are drawn without replacement from 40 common names; utterances are drawn from fixed paraphrase templates that only state an option and give no reasons.
• Authority labels: Some peer names are followed by organizational roles; the role pool is team leader, research director, senior expert, department head, and principal investigator. Appendix A gives the full prompts.

RQ1: Consensus Structure × Authority Presence

• Mixed: Three peers choose the gold label, and the remaining three each choose a different wrong option, so the gold label holds a 3-vote plurality.
• All-correct: All six choose the gold label. All-wrong: All six choose the same non-gold option, drawn uniformly at random from the non-gold options.
• Authority: When absent, all peers are ordinary; when present, one peer carries an authority role. Each original question has six variants.
• Scoring scope: When the initial answer is correct, harmful revisions are counted only in mixed and all-wrong; when the initial answer is wrong, beneficial revisions are counted only in mixed and all-correct.

RQ2: Cue Strength

Each prompt still has six peers.

• RQ2a: The committed count n_com takes values 0, 2, 4, 6, with the rest using uncertain wording. When at least two are committed, the committed peers are balanced between correct and wrong answers, so the group is not all-correct or all-wrong. Each question has four perturbations. The authors state the motivation comes from Latané (1981) and Moscovici et al. (1969).
• RQ2b: The number of authority-labeled peers n_auth ranges from 1 to 5. Labeled peers can endorse either the correct answer or a wrong answer, and ordinary peers always choose the opposite option. Five levels cross the two endorsements, giving ten variants per question. The metric is authority-aligned revision: the proportion of switches to the option endorsed by authorities, without distinguishing harmful or beneficial.

RQ3: Reasoning Interventions

The two interventions are each added to all RQ1 perturbations.

• CoT: Both rounds require step-by-step thinking, with a brief reasoning given alongside the answer and confidence. The second round also requires considering whether peers changed their own answers.
• reflect-then-revise: After the standard two rounds, an additional round requires deciding whether to keep or change the answer against the full conversation. The answer from this round is treated as final.

Metrics and Implementation

The harmful revision rate is P(a_f ≠ t ∣ a₀ = t), i.e., the proportion of initially correct answers that ultimately deviate from the gold label. The beneficial revision rate is P(a_f = t ∣ a₀ ≠ t), i.e., the proportion of initially wrong answers that ultimately return to the gold label. There is also a general revision rate for any answer change, and a confidence change ΔC = c_f − c_0.

Four open-source instruction models are run with vLLM greedy decoding (temperature 0, top-p 1.0), each experiment uses three independent random seeds, and results are aggregated across runs. Seven datasets total 2,500 questions. JSON parse failures retry up to five times, each time with temperature incremented by 0.1; if still failing, all perturbations for that instance–model pair are discarded. Inference uses mixed-effects models (Appendix C): logistic mixed models for binary outcomes and linear mixed models for ΔC; fixed effects include model and task, with random intercepts for the original question.

Experimental Setup

• Models under test: Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3, Gemma-2-9B-Instruct, Llama-3.1-8B-Instruct. No separate attack model or scoring model.
• Data: 250 questions each from four BBH tasks; 500 sampled each from MMLU-Pro, ARC-Challenge, and TruthfulQA, balanced by subject within each dataset. 2,500 questions total.
• Decoding and repetition: vLLM greedy decoding (temperature 0, top-p 1.0), three random seeds. Main text aggregates across models, tasks, and runs; per-model and per-task results are in Appendix B.
• Conditions: RQ1 is three consensus types × authority present or absent. RQ2a commitment counts are 0/2/4/6. RQ2b authority counts are 1–5, crossed with endorsing right or wrong. RQ3 adds CoT and reflect-then-revise to all RQ1 perturbations.
• Judgment: Right or wrong is relative to the dataset gold label, not an LLM review. Metrics are revision rate, harmful revision rate, beneficial revision rate, authority-aligned revision rate, and ΔC.
• Tests: Mixed-effects regression in Appendix C, reporting odds ratios and p-values. The authors state the analysis is at the raw instance level, with magnitude around 10^5, so statistical significance is almost guaranteed.

Main Results

The table below shows revision quality with authority absent from Table 1. The model abbreviations in Table 1 correspond to the full Instruct names in §3.6.

Swipe horizontally to read the table

Model	Harmful·Mixed	Harmful·All-wrong	Beneficial·Mixed	Beneficial·All-right
Qwen2.5-7B	4.1%	95.5%	55.6%	98.2%
Mistral-7B	51.7%	61.3%	24.4%	39.9%
Gemma-2-9B	10.5%	18.4%	3.2%	5.0%
Llama-3.1-8B	17.8%	79.6%	44.0%	59.0%
Four-model aggregation	15.6%	62.9%	32.7%	51.5%

• The authors state the effect of all-wrong peers on harmful revisions is five times that of all-right peers on beneficial revisions, with odds ratios of 28.5 and 5.2 respectively, both p < .001. The Mistral-7B row in the table does not fit "each model rises more on the harmful side": the change in beneficial revisions for all-right relative to mixed is greater than the change in harmful revisions for all-wrong relative to mixed.
• Aggregated numbers with authority present are not in the table above: mixed harmful revisions 20.6%, all-wrong 65.0%; mixed beneficial revisions 30.4%, all-right 54.0% (Table 1). The authors state peer consensus is the main driver, the additional effect of the authority role is smaller, and even smaller under all-right and all-wrong conditions; the tested comparisons have p < .001.
• The authors give two possible explanations: peer messages contain only option labels, no reasons; the questions underlying harmful and beneficial revisions differ in difficulty, and questions initially answered wrong may be harder. Appendix B states the magnitude differences are large, e.g., Mistral-7B mixed revision rate is 59.6% (no authority) / 62.9% (with authority).

Commitment Count and Authority Count

• RQ2a: Aggregated revision rates at n_com = 0/2/4/6 are 25.3%, 31.2%, 31.5%, 34.9% (Table 2). The regression linear term is positive and quadratic term negative; the authors report both p < .001 and state this is consistent with diminishing returns in Social Impact Theory. In Table 2, Llama-3.1-8B is 44.0%, 44.4%, 37.3%, 39.4%, which does not rise with commitment count.
• With no commitment, the aggregated revision rate is 25.3%. The authors state it is lower than every consensus condition in RQ1, so the RQ1 effects come from peers giving explicit answers, not from peer messages or longer prompts.
• RQ2b: The authors state that when authority endorses correctly, authority-aligned revisions rise about 30 percentage points from n_auth = 1 to 5; when endorsing incorrectly, about 28 percentage points. The regression is again monotonic but sublinear, with linear term positive and quadratic term negative, both p < .001. Figure 3 facets by endorsement right or wrong and plots the four models and the aggregate curve; the main text does not give exact percentages at each endpoint.

Reasoning Interventions

• CoT: Under all-wrong peers, harmful revisions drop from 63.9% to 36.5% (Figure 4a; consensus × CoT interaction p < .001). The authors state this benefit does not generalize to beneficial revisions: under mixed and all-right, CoT reduces beneficial revisions (p < .001). Table 7, in outcomes with all-right and initially wrong answers, the proportion landing on the gold label is 52.7% without CoT and 32.5% with CoT; staying on the initial wrong answer is 27.4% and 50.3%. The paper does not state whether Figure 4 and Table 7 pool authority present and absent; the 63.9% in Figure 4 also does not equal the 62.9% no-authority aggregate in Table 1.
• reflect-then-revise: Under all-wrong, harmful revisions drop from 63.9% to 41.2%; under all-right, beneficial revisions drop from 52.7% to 33.4% (Figure 4c–d; both p < .001). The authors state that under mixed conditions, reflection also reduces both directions, so it is not distinguishing misleading from useful corrections but rather revising answers less overall.
• The authors consider the two failure modes different (Appendix F): CoT may make the model rely more on its own reasoning in the conversation history; reflection is more like a partial reset than verifying which answer is correct.

Error Convergence

• Aggregated across models and datasets, initially wrong cases under all-wrong peers move to the peer-consistent wrong option 56.8% of the time; mixed is 20.7%, all-right is 19.9% (§5.1). The paper does not state whether this proportion distinguishes authority labels. The authors state all-wrong peers also make already-wrong models converge to the same wrong option.
• Table 7: Under mixed conditions, the proportion of harmful revisions landing on a wrong option no peer chose is 16.3% without CoT and 23.3% with CoT. The paper does not state whether authority conditions are distinguished.
• Table 8 (authority absent): Reflection reverts second-round harmful revisions to the first-round answer at an aggregate of 47.9%, and beneficial revisions at 29.8%. The largest gap is under all-right (62.2% vs. 27.7%), the smallest under all-wrong (41.1% vs. 33.4%).

Other Ablations and Analyses

• Table 3: The authors state all-wrong harmful revisions range from Geometric Shapes, authority present at 50.0%, to MMLU-Pro, authority present at 70.4%; in the same table, Logical Deduction (7), all-wrong, authority absent is 71.0%. The authors state all-right beneficial revisions range from Logical Deduction 7 at 45.2% to Temporal Sequences at 56.4%.
• Table 4 aggregated ΔC: harmful cases are −0.39/−0.45 under mixed (authority absent/present), −0.46/−0.46 under all-wrong; beneficial cases are −0.22/−0.24 under mixed, −0.45/−0.31 under all-right. The authors state confidence after revision is slightly lower under full consensus than under mixed.
• Table 5: n_com from 0 to 6, aggregated ΔC is −2.93, −0.90, −0.54, −0.33.
• Table 6: When authority endorses incorrectly and the model is initially correct, ΔC goes from −0.27 at n_auth = 1 to −0.54 at n = 5; when both model and authority are correct, from −0.59 to −0.04.

Limitations and Future Directions Noted by the Authors

• Question type: Only multiple-choice questions, to define harmful and beneficial revisions unambiguously. The authors state that extending to open-ended generation requires additional answer-equivalence judgments, which may introduce bias, especially in domain-specific or multilingual settings (Limitations).
• Peer source: Uses simulated peer messages, not real-time multi-agent dialogue. The authors state there is therefore no iterative dialogue, peer-written reasons, or emergent dependencies among agents; validating in real MAS is next (Limitations).
• Confidence: The analysis relies on self-reported 1–10. The authors state it should not be read as calibrated probability, only as a signal of within-instance certainty change after social pressure (Limitations).
• Next steps in conclusion: Test these patterns in real-time multi-agent interaction, develop evidence-based verification methods, and extend this revision-quality-distinguishing analysis to open-ended generation where correctness is harder to observe (Conclusion).
• Prompt robustness: The authors state general reasoning prompts cannot solve the conformity problem, and state this motivates future research on prompt robustness to social influence (§5.2).

Experimental Coverage

• Models under test are four open-source instruction models, vLLM greedy decoding, three random seeds, results aggregated across runs (§3.6, §4).
• Data are seven multiple-choice tasks totaling 2,500 questions: four BBH tasks at 250 each, MMLU-Pro, ARC-Challenge, and TruthfulQA at 500 each (§3.6).
• Each social prompt fixes six simulated peers; RQ1 is six consensus × authority conditions, RQ2a is four commitment counts, RQ2b is ten authority variants (§3.2–3.3). Utterances come from fixed paraphrase templates and contain no reasons.
• Harmful and beneficial are determined by whether the final answer equals the gold label. The paper does not use LLM review and does not report agreement with human judgment.
• Reasoning interventions are CoT and reflect-then-revise. Per-model and per-task descriptive statistics are in Appendix B; the authors state the full regression coefficient tables are released with the code (Appendix C).

This work measures the direction of conformity revisions on four open-weight instruction models: changing wrong to right is a beneficial revision, and changing right to wrong is a harmful revision. The model first answers multiple-choice questions independently, then re-answers after seeing six simulated peers; peers can be unanimous, split, or carry authority role labels.

• Without authority, aggregated across four models, harmful revisions under all-wrong peers are 62.9%, with a mixed baseline of 15.6%; beneficial revisions under all-correct peers are 51.5%, with a mixed baseline of 32.7% (Table 1). The authors state the odds ratio for the former is 28.5 and for the latter 5.2. The Mistral-7B row in Table 1 is inconsistent with the claim that "each model shows a larger increase on the harmful side."
• When the number of authority labels increases from 1 to 5, the authors state models more often change toward the endorsed option: endorsing the correct answer increases by about 30 percentage points, and endorsing the wrong answer by about 28 percentage points. As more peers commit to a definite answer, the revision rate rises from 25.3% to 34.9%, with the authors stating the increase is diminishing (Table 2).
• CoT reduces harmful revisions under all-wrong conditions from 63.9% to 36.5%, but the authors state it also reduces beneficial revisions. Reflect-then-revise reduces harmful revisions under all-wrong from 63.9% to 41.2% and beneficial revisions under all-correct from 52.7% to 33.4%. The paper does not state whether the percentages in this set of figures combine authority conditions.
• When initially wrong, under all-wrong peers, 56.8% change to the peer-consistent wrong option, while mixed and all-correct are about 20.7% and 19.9%.

The authors conclude that multi-agent systems should not treat peer agreement as a vote, nor role labels as a guarantee of correctness; chain-of-thought and reflect-then-revise cannot reliably reduce only harmful revisions while preserving beneficial ones. The authors suggest that before revising, each peer answer's evidence should be checked against the task.