Narrative gaps: Can LLMs help us navigate diverse narratives across languages?

Core question: can LLMs serve as information systems that help users navigate among competing narratives across languages and regions, rather than reinforcing filter bubbles and echo chambers.

• Scenario and importance: the authors argue that information on controversial topics (such as geopolitical conflicts) is scattered across "information islands" in different languages and regions, and the same event has different causal and moral interpretations; LLMs are becoming an entry point for people to obtain information, and their multilingual capabilities should in theory be able to cross these barriers.
• Gaps in existing research: the authors state that existing audit studies find that LLMs reinforce users' existing beliefs (generative echo chambers), prefer sources in the query language over more relevant foreign-language sources, and struggle to reason when evidence conflicts; but whether LLMs can break down rather than reinforce barriers to information access remains an open question.
• Core observation: the authors formalize this challenge as the problem of "navigating multilingual narratives," operationalize a narrative as "a source in a particular language, anchored to a geographic region, describing a topic," and acknowledge that this simplifies narrative differences within the same region/language.
• What the paper proposes: define six information-access capabilities—three basic capabilities (narrative inclusion NI, narrative exclusion NE, missing narrative identification MNI) and three higher-order capabilities (presenting diverse narratives, maintaining narrative coverage under confirmatory queries, cross-lingual information parity); and build the NarrativeBench benchmark (1,333 articles, 23 publication languages, 23 controversial topics, 460 Reddit-sourced queries), evaluating 14 LLMs, 24 query languages, and 4,800 cases.
• Main findings (per the authors): current models lack the capabilities needed to reliably navigate diverse narratives; even when users explicitly request diversity, gaps in narrative exposure remain; information access varies with query framing and language, potentially exacerbating echo chambers, language filter bubbles, and information inequality.

Selective exposure and perspective exposure

• Research on selective exposure and partisan news consumption (Arguedas et al. 2022; Muise et al. 2022; Boonprakong et al. 2025)—some audiences concentrate on sources consistent with their political stance, and strong political beliefs increase susceptibility to confirmation bias.
• Interfaces and interventions that broaden perspective exposure (Jeon et al. 2024; Gao et al. 2018; Piccardi et al. 2025)—interfaces that help users explore news across political stances; labeling stances may instead reinforce users' selection of aligned views; reducing exposure to anti-democratic content can reduce affective polarization.
• User studies of LLM-mediated search (Sharma et al. 2024; Cheng et al. 2026; Jakesch et al. 2023)—users show stronger confirmation bias in LLM search than in traditional search; biased model responses reinforce existing beliefs or change user attitudes.

LLM-mediated information access and pluralism

• Research on AI search and news consumption (Yang 2025; Kiskola et al. 2025; Nordberg et al. 2026)—citations concentrate on a few media outlets, with concerns about accuracy, transparency, and consensus.
• Information access benchmarks (MIRAGE, SeekerGym, WideSearch, InfoDeepSeek, GISA, etc.)—evaluate the accuracy of retrieved evidence, completeness of information collection, and reasoning and clarification in expert consultation.
• Research on perspective identification and narrative framing (Chen et al. 2019; PRISM; PerSpectra; Otmakhova & Frermann 2025)—examine how models identify and distinguish perspectives, represent narrative frames, and accommodate diverse human preferences.

Language-dependent stances and knowledge conflicts

• Li et al. (2024)—LLM answers to territorial dispute questions differ by query language.
• Research on knowledge conflicts (WikiContradict, Clasheval, ConflictBank, the Xu et al. 2024 survey, etc.)—examine model behavior when context conflicts with parametric memory and when retrieved passages contradict each other. The paper notes that these works focus on language-dependent stances and conflicting facts, and do not answer how models represent competing narratives that emphasize different facts and interpretations across languages and regions.

Multilingual LLMs and information capabilities

• Multilingual consistency evaluations (Arora et al. 2025; Gupta et al. 2025; SeaEval, etc.)—answers to semantically equivalent questions are inconsistent, and performance varies with language, script, and resource level; cultural benchmarks (BLEnD, INCLUDE) evaluate everyday knowledge and regional knowledge reasoning; TyDi QA uses multilingual native information-access questions to evaluate passage selection and answer extraction.
• Language preferences in multilingual RAG (Sharma et al. 2025; Park & Lee 2025; Wang et al. 2026; Ki et al. 2025)—retrieval and generation stages prefer documents in the query language, and prefer high-resource languages when evidence in the query language is unavailable; models may prefer source language over relevance.

Positioning: The authors state that NarrativeBench builds on multilingual information access and conflicting-evidence research to further evaluate whether models can selectively present and retrieve narratives, broaden exposure under confirmatory queries, and maintain comparable information access across languages. The paper sets no comparison baseline method; the comparison objects are the 14 tested LLMs themselves.

Overall approach: decompose "navigating multilingual narratives" into six evaluable capabilities and build the NarrativeBench benchmark—provide models with multilingual article context and queries, determine which narratives are presented via document citations in the answer, then quantify each capability with metrics such as F1/recall. The design follows the Evidence-Centered Benchmark Design framework of Liu et al. (2024).

Dataset construction

• Article collection: a four-stage process—select controversial topics, identify relevant regions and stakeholders, develop region-specific search terms in relevant languages, and collect articles. Articles were collected via localized search on Google Advanced Search, Yandex, and Baidu and filtered for topic relevance, accessible body text, and identifiable source (Appendix A.1).
• Scale: 1,333 articles from 43 countries/regions/regional organizations, covering 23 topics and 23 publication languages; divided into 180 narrative groups by source perspective and publication language (Table 3). Median article length is 4,654 characters.
• Narrative distinguishability diagnostic: multilingual embeddings of articles in the same group have higher cosine similarity than those in different groups (mean difference 0.047, 95% bootstrap CI [0.044, 0.051], permutation test p < 0.001); after restricting to the same language, articles with the same perspective are still more similar (mean difference 0.012, Appendix A.2).
• Query sources: 460 queries = 283 directly sourced from Reddit + 177 adapted for under-covered topics; of these, 230 are neutral and 230 are confirmatory (seeking support for an existing belief). Two independent annotators rated 100 English queries on "would a real person ask an AI assistant this" with means of 4.09 and 4.89 (5-point scale).
• Query translation: translated with Google Translate into 23 publication languages plus Thai, totaling 11,038 query-language versions in 24 languages. The WMT22-COMETKiwi-DA mean for 1,704 translations is 0.853–0.879; in a manual review of 40 sample pairs, 90.8% (118/130) were judged to preserve meaning (Tables 4, 5).

Task design (six capabilities → five tasks)

• FC-1 Narrative inclusion (NI): require presenting only the specified narrative; the expected answer cites only evidence for the target narrative.
• FC-2 Narrative exclusion (NE): require excluding the specified narrative; the expected answer cites the remaining relevant narratives and does not cite the target narrative.
• FC-3 Missing narrative identification (MNI): given an existing answer focused on a single narrative (the same one for all models), require revision to incorporate the omitted narrative.
• HC-1 Diverse narrative presentation: compare narrative coverage and the difference under Baseline Prompt (BP, which only asks to answer with evidence) and Diverse Prompt (DP, which explicitly asks to present the diverse narratives in the context).
• HC-2 Robustness to confirmatory queries: compare the difference in narrative coverage between neutral and confirmatory queries; HC-3 Cross-lingual information parity: measure the gap with English along two axes, query language and narrative language.

Evaluation protocol and metrics

• Case construction: each case = document context + user query + task instruction. The context contains 3 articles per selected narrative (9–45 articles, median 24); the same question uses the same articles across tasks and languages. Models answer in the query language and cite document IDs (e.g., [D001]; IDs do not reveal source region or language).
• Narrative determination: a narrative is considered "presented" if the answer cites at least one article for that narrative. Comparing the cited narrative set Ci with the required narrative set Gi: NI/NE use F1 (also penalizing citation of narratives required to be excluded), MNI/BP/DP use recall.
• HC-1 metric: recall under BP and DP and the paired difference ∆DP−BP = RDP − RBP, holding question, query language, and document context fixed.
• HC-2 metric: for each configuration, take the absolute difference in recall between neutral and confirmatory queries and then average; Appendix B.4 also provides signed regression estimates controlling for topic, language, and prompt.
• HC-3 metric: for each language, the mean difference from English ∆ℓ = Mℓ − Men; the query-language axis aggregates all five tasks, and the narrative-language axis aggregates NI/NE; also reports the source-language omission rate (context contains articles in a language but the answer does not cite them).
• Language validation (LVal): an answer whose language does not match the query language scores zero and is not included in narrative scoring; language identification uses Lingua, with fastText for four languages including Amharic.

Experimental setup

• Models tested: 14 models in 15 configurations — GPT-5.4 mini, GPT-6 Luna, GPT-OSS 20B (two variants: low 3K and medium 16K), GPT-OSS 120B (low), Gemini 2.5 Pro, Gemini 2.5 Flash, Gemini 3.8 Flash (low), Gemma 4 26B-A4B IT, DeepSeek V4 Flash, Qwen3.5 0.8B IT, Qwen3.5 9B IT, Qwen3.5 9B Base, Kimi K2.6, Command A (Table 6 lists model identifiers, reasoning settings and output limits, with most output limits at 3,000 tokens).
• Benchmark and scale: NarrativeBench, with 4,800 cases per configuration (960 per task, 200 per query language, 2,400 per query category), covering all 460 queries (230 neutral + 230 confirmatory), for a total of 72,000 scored responses.
• Metrics: NI/NE use narrative F1, MNI/BP/DP use narrative recall; HC-2 is the absolute neutral/confirmatory recall difference (pp); HC-3 is the gap relative to English (pp); also source-language omission rate and LVol (count of responses in a mismatched language).
• Judging method: citation-based rule judging (citing a narrative article counts as presentation), with response language verified using Lingua/fastText; no LLM judge. Citation support was manually audited (Appendix C.1).
• Number of repeats: The paper does not report the number of repeated runs; standard errors are reported.

Main results

According to Table 1 (standard errors in parentheses; units %, gap in percentage points):

Swipe horizontally to read the table

Model	NI F1	NE F1	MNI recall	BP recall	DP recall	HC-2 gap (pp)
GPT-5.4 mini	21.0	54.8	62.3	45.5	55.4	2.74
GPT-6 Luna	26.0	42.3	48.1	32.0	39.2	4.25
GPT-OSS 120B (low, 3K)	16.5	52.3	73.7	51.4	56.6	1.14
Gemini 2.5 Pro	25.2	74.2	79.4	70.6	74.9	4.29
Gemini 3.8 Flash (low)	52.5	65.2	82.8	57.8	68.7	10.09
Qwen3.5 0.8B IT	2.6	7.4	11.9	8.1	8.2	0.48
Kimi K2.6	19.8	60.4	73.0	48.3	51.1	0.72
Command A	14.3	35.8	50.3	32.0	34.3	3.34

(The two GPT-OSS 20B configurations, Gemini 2.5 Flash, Gemma 4 26B-A4B IT, DeepSeek V4 Flash, Qwen3.5 9B IT and Qwen3.5 9B Base are omitted; all appear in Table 1.)

• Basic capabilities are generally low: Across 15 configurations, average NI F1 is 19.9% (SE 0.6%), NE F1 49.3%, MNI recall 56.8%; NI F1 is lower than NE F1 in every configuration. The authors say models struggle to let users control which narratives are presented.
• Explicit diversity requests yield limited gains: BP averages recall 44.3%, DP 49.2%; the authors say that even with explicit diversity requests, the strongest configuration (Gemini 2.5 Pro, DP 74.9%) still fails to present about a quarter of available narratives.
• Query framing affects coverage: The average absolute recall difference between neutral and confirmatory queries is 3.23 percentage points, ranging from 0.32 for Qwen3.5 9B IT to 10.09 for Gemini 3.8 Flash; the example in Table 2 shows that under confirmatory queries, responses can cite more narratives while agreeing with the query's premise.

Cross-language information parity (HC-3)

• Query language gap: Across 15 configurations, the average absolute gap relative to English is 8.74 percentage points, ranging from 3.59 for Kimi K2.6 to 13.84 for GPT-OSS 20B (low) (Section 5.2 of the main text; Table 1 lists Query gap).
• Narrative language gap: Average 7.51 percentage points, ranging from 2.76 for Qwen3.5 0.8B IT to 10.72 for Gemini 3.8 Flash.
• Source language omission: Under DP, omission rates for Tigrinya, Nepali and Armenian sources are 59.8, 57.9 and 54.6 percentage points higher than English, respectively (Figure 1E). Appendix B.5: when the query language matches the source language the omission rate is 41.3%, and when it does not match it is 50.4%; for 22 of the 23 source languages, omission is lower when matched (Figure 2).

Robustness and supplementary analyses

• Citation selection control (Appendix C.2): On 1,720 cases and 12 models, compared with randomly selecting the same number of documents, models score above random on NI (observed .331 vs random .235) and below random on NE, MNI, BP and DP (e.g., DP .753 vs .797, Table 9).
• Effect of generation settings (Appendix C.3): For GPT-5.4-mini on 985 identical inputs, medium reasoning raises NI F1 from 28.11% to 39.37%, but lowers NE F1 and MNI/BP/DP recall by 7.38–15.94 percentage points; 102 responses exhausted the output limit (including 46 empty responses). The authors therefore stress the importance of reporting generation settings.
• Other checks: Translation quality shows no detectable association with benchmark scores (Pearson r = −0.019, Appendix A.4); doubling input length corresponds to a score change of 4.3 percentage points (95% CI [−0.7, 9.3], Appendix C.2); articles published after the knowledge cutoff show no recall decline (+4.77 pp, p = .325, Appendix C.4); on 14 queries from WildChat/ShareChat sources, GPT-5.4 recall is 46.4% and Gemini 2.5 Pro 46.8% (Appendix C.5).

Other ablations and analyses

• Manual audit of citation support (Appendix C.1): Two annotators reviewed 45 citation-sentence–document pairs, with 82.2% agreement (Cohen's κ = 0.708); 33 pairs were judged supportive by both annotators.
• Service refusals: The Kimi K2.6 service refused 552 of 4,800 requests (high-risk), concentrated in topics such as India-China tensions, Taiwan, and Indian student protests (Discussion section).
• Query annotation consistency (Appendix A.3): Two annotators' ratings of 100 queries were 39% in exact agreement, with weighted Cohen's κ = −0.067.
• Signed HC-2 estimates (Appendix B.4, Table 7): After controlling for topic, language and prompt, the adjusted confirmatory-minus-neutral difference ranges from −3.2 pp for Kimi K2.6 to +3.2 pp for GPT-6 Luna, with inconsistent direction.

Limitations and future directions noted by the authors

• Narrative representation (Section 7): Grouping by source perspective and publication language does not capture disagreements within the same region/language over events, causes and consequences; the authors believe it should be extended to within-group disagreement.
• Corpus representativeness (Section 7): The corpus is limited to topics, regional sources and texts reachable through localized search; information access occurs across modalities and media, requiring broader sampling to test how capability gaps vary with available narratives.
• Source quality and disinformation (Section 7): Contested reporting may contain propaganda and misleading content; future evaluations should examine whether models can distinguish reliable evidence while preserving competing perspectives; the authors state they do not endorse the narratives in the benchmark as fact.
• Citation coverage and claim support (Section 7): Citations only measure narrative presentation, and the citation-support audit covers only a sample; the authors say evaluating faithfulness (loss of meaning, argument strength, sentiment) across responses and languages is complex and was not done.
• Human evaluation of higher-order capabilities (Section 7): The actual impact of higher-order capabilities on filter bubbles and echo chambers requires human-subject studies; even when models present competing narratives, users may selectively attend to perspectives supporting their existing beliefs.
• Multi-turn information access and information consumption (Section 7): The benchmark only evaluates single-turn responses to fixed queries and instructions; future work should test narrative presentation under evolving intent across multiple turns; and user studies are needed to examine how narrative organization affects comparison and source-tracing abilities.

Scope of experimental coverage

• The models tested are 14 models in 15 configurations, covering six families: OpenAI, Google, DeepSeek, Qwen, Moonshot, Cohere (Table 6).
• The evaluation scale is 4,800 cases per configuration, 24 query languages, 5 tasks, for a total of 72,000 responses; 23 topics and 23 publication languages (Section 5, Appendix B.1).
• Narrative "presentation" is judged solely by citation, and the manual audit of citation support covers only 9 query languages and 45 sample pairs (Appendix C.1).
• The query translation quality audit covers 8 target languages; the remaining languages received no manual translation quality review (Appendix A.4).
• The paper does not report the number of repeated runs per configuration; standard errors are computed at the case level.

A benchmark paper: the authors formalize "whether LLMs can help users navigate competing narratives across languages and regions" as six information-access capabilities and build NarrativeBench to measure them.

• Problem: Information on contested topics is dispersed across narratives in different languages and regions, and the authors believe LLMs may reinforce rather than break filter bubbles and echo chambers, requiring capability-oriented evaluation.
• Method: 1,333 news articles in 23 languages covering 23 contested topics and 180 narrative groups, paired with 460 Reddit-sourced queries (230 neutral + 230 confirmatory, translated into 24 languages); narrative presentation is judged by document citations in responses, with five tasks NI, NE, MNI, BP, DP, measured by F1/recall and language gaps.
• Basic capability results: Across 15 configurations, NI averages F1 19.9%, NE 49.3%, MNI recall 56.8%; NI is lower than NE in every configuration (Table 1).
• Higher-order capability results: BP averages recall 44.3%, DP 49.2%, with the highest, Gemini 2.5 Pro, at 74.9% under DP; neutral/confirmatory query coverage differs by an average of 3.23 percentage points; query languages differ from English by an average of 8.74 percentage points, and under DP the Tigrinya source omission rate is 59.8 percentage points higher than English.
• Authors' conclusion: Current models lack the ability to reliably navigate diverse narratives, creating an "illusion of control"; the authors recommend making these capabilities explicit training and evaluation targets and call for policy and procurement standards to require reporting of narrative-access disparities across languages and topics.
• Positioning: The authors describe NarrativeBench as a "capability foundation" for building cross-language information systems, whose narrative operationalization is a necessary but not sufficient condition, with within-group disagreement, multi-turn interaction and user impact left for future work.