Jev-compatible multi-agent trading system: when online news lags the market, track or follow?

Core question: Do LLM judgments about Web news track the market's reaction to the events they report, or merely follow price moves that already occurred before the news was published?

• Problem setting and importance: Web news is often published after price moves, so a reader may agree with the market by following the move rather than by understanding the event. The authors argue that this makes existing evaluations unable to distinguish whether a reader truly understands the event or merely restates the market reaction.
• Shortcomings of existing methods: prior tests either score against gold-standard labels or collect predictions prospectively, and do not distinguish judgments from the timing of news publication, nor control for memorization (recall) in model training data and for the confound that the news itself reports price moves.
• Core observation: much Web news (especially contributor-written) is published after the issuer's 8-K filing and often reports price moves that already occurred; measured from the first trade after release, the association drops substantially.
• Paper's contribution: proposes a multi-agent trading system in which agents share only a single-writer point-in-time store and answer closed-ended questions through the Jev typed interface; code sets positions under a proven influence bound; through a hash-committed pre-registration protocol, it tests the agreement of reader judgments with announcement-window abnormal returns and 8-K filings.
• Threat model: the attacker controls any number of news items with any byline, knows the system architecture and the current week's coefficients, but cannot write trusted inputs; the 7 uplift families succeed when the attacked name enters the signal book, and the 2 downgrade families succeed when a held name is removed or halved.

Machine readers and market tests

• Lopez-Lira & Tang (2026) — GPT-4's interpretation of news headlines tracks initial market reactions after the model's cutoff.
• Kirtac (2026) — a supervised language model's news probability predicts the sign of three-day excess returns and achieves net returns at the daily horizon.
• Glasserman & Lin (2023) — evaluate look-ahead bias in GPT sentiment analysis.

Typed decision models

• Sun et al. (2026) — note that typed probability outputs alone do not prove calibration; typed models follow option names rather than definitions.
• Tang & Zheng (2026) — an audit finds no independent accuracy advantage for typed readouts, and most evaluations do not report a label-probability baseline.
• Tian et al. (2023) — verbalized confidence is better calibrated than label probabilities but has low discrimination.

Look-ahead bias and point-in-time evaluation

• Lopez-Lira et al. (2025) — memory tests only bound recall from below; prompting to ignore later information is ineffective.
• Paleka et al. (2026) — publication dates can bound the cutoff, but the authors add two channels: the news itself reporting the outcome, and the news being published after the reaction.
• He et al. (2025) — time-locked language models remove recall by construction.

Modular trading agents and manipulable text

• Greshake et al. (2023) — indirect prompt injection can compromise LLM-integrated applications.
• Nestaas et al. (2025) — crafted Web content can manipulate products recommended by LLM assistants.
• Xiao et al. (2024) — in the TradingAgents multi-agent framework, expert analyst debate or a portfolio manager model makes decisions.

Baselines and comparisons

• Baseline methods: VERB (verbalized probabilities from the same LLM), NG (open-source typed model), FinBERT (financial sentiment model), dictionary tone (Loughran–McDonald).
• Benchmark/data: Benzinga news (100,906 items), top 100 S&P 500 stocks by market cap, SEC 8-K filings.

The authors position the paper's distinction: it is the first to combine a reader-agnostic point-in-time multi-agent system with pre-registered tests, separating market reactions realized before and after news publication, and measuring validity and manipulability on the same reader.

Overall approach: build a multi-agent trading system in which agents do not exchange free text; they only answer closed-ended questions through the Jev typed interface (returning a probability for each declared option) and share a single-writer point-in-time store; code sets positions under a proven influence bound, ensuring that any reader text has a bounded influence on positions.

Problem setup and formalization

• Reader definition: a closed-choice reader receives state and a question with declared options, and returns a probability distribution over the options (a typed judgment); it does not generate free text.
• Validity criterion: a judgment is valid if and only if its option statements about the event agree with records the reader did not write: the market reaction to the event (AR[0,+1]) and the issuer's 8-K filing.
• H2 statistic: weekly Spearman correlation coefficient measuring the association between E[effect] (probability-weighted effect level) and AR[0,+1], averaged by week, one-sided Newey–West test (4 lags).

System architecture and agents

• News agent: for each stock-week, reads up to 40 recent news items; after filtering (removing opinion, promotion, instructions), each item answers 13 questions (relevance, novelty, event type, 5-level effect, importance, whether it exceeds expectations, whether it is planned, serious issues, etc.).
• Other agents: technical (price action), fundamental (growth, margins, leverage, valuation percentile), policy (theme, stage, profit impact), macro (indicator percentiles, rate calendar), profile (21 exposure flags from the annual report's business section).
• Decision layer: an online logistic model predicts whether the holding-week open-to-open return exceeds the universe median; 24 trusted features form the trusted logit L^T, 12 text features form the text logit L^W, fitted with L^T as offset, ensuring text only explains what the trusted data does not explain.

Influence bound and decision card

• Bound mechanism: the text logit is clipped to |L^W| ≤ κ_t (κ_t = 2s_t^T), and text can only admit names from the band (trusted rank ≤ K+J, K=20, J=10), cannot raise weights above the trusted target, and cannot remove protected holdings without a trusted anchor (serious 8-K or abnormal price move).
• Decision card: each holding and entering/exiting name gets a logit and weight ledger, aggregated exactly to the decision, attributing each contribution to an agent and to specific news items.

Evaluation protocol

• Pre-registration: before the evaluation window unlocks, hash-commit the question bank, prompts, model pins, decision layer, attack templates, seeds and analysis code; H1–H3 with Holm correction.
• Status words: Confirmatory (H1–H3, one-sided, Holm-corrected), Registered (fixed, 95% intervals), Exploratory (specified after unlock, always labeled), Descriptive (Jev rows).
• Attack tests: 7 uplift families (A1 instructions, A2 fabricated facts, A3 self-promotion, A4 hidden instructions, A5 stock promotion, A6 planted press release, A7 repeated planting) and 2 downgrade families; success is defined as a name entering the signal book or a held name being removed/halved.

Experimental setup

• Models tested: emu (gpt-4.1-mini-2025-04-14, confirmatory reader), Jev (jev-1.13.0, descriptive), NG (this-that-model-1.0, registered reference), VERB (verbalized probabilities from gpt-4.1-mini-2025-04-14, comparison).
• Datasets: Benzinga news (via Alpaca API, 100,906 items, 2024-07-05 to 2026-09-11), top 100 S&P 500 constituents by market cap (monthly rebalancing), SEC 8-K filings.
• Benchmark: 115 weekly decisions (W_clean), 73 post-release decisions (W_release); trading cost 15 bps/side.
• Metrics: weekly Spearman correlation (E[effect] vs AR[0,+1]), AUC (event type vs 8-K item code), rank IC (logit vs holding-week relative return), ASR (attack success rate).
• Review method: no LLM reviewer; 8-K filings as gold standard; attack success determined by code.
• Sample sizes: H2 frame 27,556 news items (emu retained non-stock-activity/opinion items among 31,848); VERB 2,000 items, NG 500 items (stratified sampling by seed rule).

Main results

Swipe horizontally to read the table

Measurement	Estimate	95% CI	Source
emu AR[0,+1] (all 27,556 items)	0.119	[0.098, 0.140]	Table 2A
emu post-release 73 weeks	0.115	[0.090, 0.140]	Table 2A
emu from first trade after release	0.023	[0.005, 0.041]	Table 3
emu 8-K AUC (earnings vs Item 2.02)	0.833	[0.826, 0.840]	Table 2A
VERB AR[0,+1] (2,000 items)	0.097	[0.054, 0.138]	Table 2B
NG AR[0,+1] (500 items)	0.108	[0.016, 0.203]	Table A2
Jev AR[0,+1] (549 post-release)	0.102	[0.017, 0.284]	§4.3

• The authors state that emu's judgments track announcement-window abnormal returns, and that this persists 73 weeks after release (0.115), indicating it is not mere memorization.
• The authors state that measured from the first trade after release, the association drops to 0.023, indicating that most of the association comes from returns already realized before the news was published.
• The authors state that event-type judgments agree with 8-K filings (earnings AUC 0.833), but agreement is lower for deals (0.611) and management (0.557).

Timing structure and content analysis

• Publication timing: 72% of earnings news is published after the issuer's 8-K filing (median 32 minutes later); 19.0% of contributor-written news reports price moves, versus only 0.6% of desk news (Table 3).
• Event timing: the association peaks on the event session day and is near zero the next day; 81% of news items' AR[0,+1] windows close before the next open (Fig. 3b).
• Price reporting: a fixed pattern flags 7.1% of news items as reporting price moves; audit estimates the true proportion at about 10.6%; after removing these, the statistic drops to 0.089 (Table 3, App. C).

Decision-layer utility (H3)

• rank IC: full-system IC −0.024, code-only twin −0.008, text portion −0.006; the H3 confirmatory test did not detect incremental information from text judgments over code features (ΔIC −0.016 [−0.029, −0.002]) (Table 4).
• Authors' interpretation: for this learner and decision layer, text judgments provided no detectable rank information at the weekly horizon; consistent with rapid assimilation research (next-day drift drops to annualized Sharpe 1.22).

Attacks and influence bound (H1)

• Attack success rate: per-item reader ASR 10.5%, holistic reader 0.1% (difference 0.104 [0.081, 0.127]); the H1 hypothesis (per-item lowers ASR) was not supported (Holm-adjusted p = 1.000) (Table 5B).
• Attack families: fabricated facts (A2) ASR 22.5%, planted press release (A6) 31.5%, repeated planting (A7) 25.9%, downgrade attacks (A8) 57.2%; instruction-type attacks (A1, A3, A4) ASR 0.0% (Table 5A).
• Influence bound: all attacks were within the bound; no name entered from outside the band, no protected name was excluded without an anchor; decision cards traced 88% of successful attacks to planted items (Table 5B).

Other ablations and analyses

• Aggregation method: median aggregation ASR 6.0%, majority vote 2.9%, both lower than mean aggregation (10.5%) (Table 5B).
• Leave-one-agent-out: removing any single agent changes rank IC from −0.007 (policy) to +0.015 (fundamental), with all intervals covering zero (§4.5).
• FinBERT: tone correlation 0.094 on the same set of news, dropping to 0.004 after release (Table 3).

Limitations and future directions noted by the authors

• Scope limits (§6 Scope): the method is limited to a single fixed LLM (emu), Jev as a descriptive reader, NG as a registered reference, a single news source (Benzinga headlines and summaries), 100 large-cap stocks and weekly decisions; the authors state the method can extend to any Web text that produces publicly timestamped records (e.g., clinical trial reports), and the system accepts any reader conforming to the schema.
• Evaluation standards (§6): the authors propose reporting rules: in addition to the announcement-window criterion, one must report placebo windows, pre-market controls, post-release estimates, re-identification and abstention, the proportion of news reporting already-realized outcomes, and most importantly the release-alignment criterion.
• Conclusion (§7): the authors state that Web news mostly follows the market, so the release-alignment criterion rather than the announcement window determines whether a reader is useful; the system can serve as a testbed for future reader evaluations.

Experimental coverage

• Models tested: emu (gpt-4.1-mini-2025-04-14), Jev (jev-1.13.0), NG (this-that-model-1.0), VERB (verbalized comparison from gpt-4.1-mini-2025-04-14), FinBERT (reference).
• Data coverage: 100,906 Benzinga news items (2024-07-05 to 2026-09-11), top 100 S&P 500 stocks by market cap, SEC 8-K filings.
• Attack tests: 7 uplift families + 2 downgrade families, 173–250 attacked stock-weeks per family; adaptive attacker tested on NG (64 queries/item, 30 items).
• Not reported by the paper: other LLMs as confirmatory readers, real-time streaming, higher-frequency trading (intraday), other news sources, non-English news, the attacker's specific query cost or time.

The paper builds a multi-agent trading system to test whether LLM judgments about Web news track the market's reaction to the events they report, or merely follow price moves that already occurred before the news was published. The system has agents answer closed-ended questions through the Jev typed interface, share a single-writer point-in-time store, and set positions in code under a proven influence bound, ensuring that any reader text has a bounded influence on positions.

Core findings: emu's (gpt-4.1-mini) judgments on 27,556 news items have a weekly Spearman correlation of 0.119 with announcement-window abnormal returns, and this holds at 0.115 73 weeks after the model's release, indicating it is not mere memorization; however, measured from the first trade after release, the association drops to 0.023, and 72% of earnings news is published after the issuer's 8-K filing, indicating that most of the association comes from returns already realized before the news was published. Event-type judgments agree with 8-K filings (earnings AUC 0.833).

At the decision layer, text judgments provided no detectable incremental rank information over code features (ΔIC −0.016). Attack tests show that the per-item reader did not lower the attack success rate (10.5% vs holistic reader 0.1%), but all attacks were within the influence bound, and decision cards traced 88% of successful attacks to planted items. The authors propose that market-grading tests should report a release-alignment criterion rather than only an announcement-window criterion, and that the system can serve as a testbed for future reader evaluations.