One fabrication is enough: a single fake news article changes the predictions of an LLM forecasting system
One Fabrication Is Enough: A Fake News Article Changes the Forecasts of LLM Forecasting Systems
Translation in progress — untranslated text is omitted.
A paper by Northwestern University researchers empirically tested a retrieval poisoning attack against an LLM forecasting system: an attacker only needs to publish one fake news article supporting a particular direction, without touching the model or the retriever. After the article enters the model's context, 74% to 82% of predictions that originally opposed that direction were flipped, and this held across all five frontier models tested; the AIA Forecaster and xAI Grok ForecastBench systems, reproduced according to public descriptions, retrieved the article in about 90% of runs. The model cannot verify whether the article is true or false; after the attack, the forecaster's average accuracy is almost unchanged, but the direction of predictions swings with the attacker's bets, and two fake news articles in opposite directions do not cause the model to hedge. Paragraph filtering and partitioned retrieval can each block a single-article attack, but each can be bypassed by a single prompt change; used together they can block four articles but are defeated by sixteen distorted copies, and they change 29% of decisions made without attack and discard 39% of genuine news; partitioned retrieval also requires the deployer to control the retrieval index, so systems that search the open web cannot use it.
- Retrieval-augmented LLM forecasting systems rely on open news archives; the authors study the weakest attacker among them: one who only knows the forecasting question and publishes fabricated news articles, with no system access whatsoever, and how much harm they can cause.
- The authors construct a frozen news archive and stock and prediction-market questions, use models outside the evaluation set to generate fabricated articles, measure metrics such as reversal under Protocol A (forcibly placed into context) and Protocol B (the system retrieves on its own), and evaluate two defenses, passage filtering and partitioned retrieval, and their combination.
- The authors report: once an article enters context, it reverses 74%–82% of opposing forecasts; deployed systems retrieve the article in about 90% of runs; models cannot distinguish fabrication (they also judged 45.7% of real news as fabricated); average accuracy is nearly unchanged but forecasts follow the attacker's direction; the two defenses are each defeated by a bypass that changes only the prompt, and combined they fail under 16 tampered articles and change 29% of clean decisions.
- The authors note: the deployed systems are reimplementations based on public descriptions, with no live web testing; articles come only from three author models and two rules and were not optimized against the filters; some experiments cover only 42 questions with wide intervals; evidence is limited to five models and nine windows in 2026, without cross-year validation.
The paper quantifies the actual impact of a single fake news article on an LLM forecasting system and measures the cost and bypass methods of existing defenses when retrieval is used, which deployers can check against.
Related events and news
1 events · 0 news mentions
- This event's paper
假新闻文章可扭转LLM预测系统74%-82%预测