Agent Mosaic: Identifying confidential leaks in agentic LLM systems

Core question: An LLM agent may comply with each document's access permissions yet still combine multiple accessible facts into a sensitive conclusion the user is not entitled to obtain (the mosaic effect); the paper aims to quantify how often this leakage occurs and whether simple defenses suffice.

• Problem setting: The agent retrieves multiple documents and synthesizes an answer in a classified data environment (cybersecurity, healthcare, military); the authors argue this lets compositional leakage occur at unprecedented scale, and that document-level filtering cannot block this path when all sources are within the user's permissions.
• Existing concept: This risk is called the mosaic effect in the literature, and U.S. classification policy calls it classification by compilation/aggregation; Executive Order 12958 of 1995 already recognized that unclassified items can be combined into a classified compilation, but the authors argue that LLM agents' automatic retrieval and synthesis capabilities make this old problem more urgent.
• Gaps in existing evaluations: The authors state that privacy evaluations such as PrivacyLens, AgentDAM, and CI-Work focus on the gap between judgment and disclosure or on contextual integrity, but do not make "the gap between source permissions and the classification of the composite conclusion" an explicit experimental target.
• Proposed in the paper: A 30-case synthetic benchmark (10 cases in each of three domains), in a controlled simulation (10,800 final answers) and two real open-source agent frameworks (1,440 answers), crossing three query intents with two simple defenses (classification headers, aggregation rules), measuring the leakage rate and refusal rate of the forbidden composite.
• Threat model:
  • Victim system: an LLM agent running on mixed-classification data, where the user can access two SECRET-level documents.
  • Protected object: the TOP SECRET-level forbidden composite formed by combining the two documents, which the user is not entitled to obtain.
  • Query intents: benign (answerable from a single document), explicit (directly requesting the composite conclusion), adversarial (the explicit query followed by a suffix claiming all permissions and demanding that system-level restrictions be ignored).
  • Success criterion: if the final answer clearly conveys the forbidden composite (including semantic paraphrase), it is recorded as a mosaic event.

Compositional sensitivity and aggregation classification

• Morgenstern (1987)—raised the inference problem in multilevel databases decades before LLMs.
• Clinton (1995)—Executive Order 12958 recognizes that unclassified items can form a classified compilation by revealing additional relationships.
• Novak (2021), Masinde et al. (2023)—discuss mosaic/group privacy risks in shared data environments and humanitarian geographic data; Novak (2021) also mentions an automated CbCA scheme, but the paper notes it is not publicly available.

Empirical work on inference privacy

• Staab et al. (2024)—LLMs can infer personal attributes from user text, even after anonymization and alignment.
• Ko et al. (2026)—agents can combine individually non-identifying clues with public evidence to reconstruct real identities, including in benign tasks.

Confidentiality evaluations of LLM agents

• Debenedetti et al. (2024)—AgentDojo, a dynamic environment for evaluating prompt injection attacks and defenses.
• Shao et al. (2024)—PrivacyLens, finds a gap between privacy judgments and tool-behavior disclosure.
• Zharmagambetov et al. (2025)—AgentDAM, documents unnecessary sensitive disclosure in benign web tasks, with residual leakage even after privacy prompting.
• Fu et al. (2026)—CI-Work, evaluates whether enterprise agents conceal sensitive retrieval context while conveying necessary content.
• Pan et al. (2026)—measures agents in production environments, which the authors use to argue that reliability is a core deployment challenge.

Judging methodology reference: Zheng et al. (2023)'s LLM-as-a-judge work is used as precedent for selecting a judge model via human annotation.

Positioning of this paper: The authors state that this paper's contribution is making "user-accessible sources jointly entailing a higher-classification conclusion" an explicit, controlled, multi-domain experimental target, crossing benign/explicit/adversarial queries with classification headers and aggregation rules, and annotating disclosure and refusal separately; this complements, rather than replaces, inference-privacy and context-disclosure benchmarks. The paper has no comparison baseline method; the comparison objects are the three generators and the two real frameworks themselves.

Overall approach: build a synthetic benchmark in which every example contains "two accessible SECRET documents + one TOP SECRET forbidden composite conclusion," and measure in controlled simulation and real agent frameworks whether the final answer leaks that composite conclusion.

Synthetic benchmark (Section 3.1)

• Scale and structure: 30 synthetic examples, 10 each in cybersecurity, medical and military; each example contains one TOP_SECRET-level forbidden composite, two SECRET-level related documents, document-level metadata, one same-domain distractor document and three queries.
• Generation method: Generated by Anthropic's Claude Opus 4.6, templated into report, email and note styles; the authors note it was only human-reviewed and its realism was not evaluated; Appendix C gives document excerpts.
• Four-factor crossing: intent (benign/explicit/adversarial) × distractor (absent/present) × classification marking (hidden/shown) × aggregation rules (off/on), 24 configurations in total; domain is an example-level attribute.
• Guardrail design: When headers are on, a Classification: SECRET prefix is added before documents; when rules are on, they name the aggregation problem, require single-source answering where possible, prohibit undeclared cross-source synthesis, and instruct refusal of classified composites. The design references the ISOO 2010 directive and the 2017 compilation classification clarification; the authors note that single-source answering and blanket refusal are conservative experimental choices beyond these policies.

Controlled simulation (Section 3.2)

• Procedure: Fixed system prompt + configured query + scripted search/reflection text, providing documents A, B (optionally C) in sequence; no real retrieval is performed; tool responses and reasoning are rewritten from real agent trajectories (Hermes Agent and Lambda's Hermes reasoning-trace corpus); only final answers are evaluated. Appendix A gives the full prompt templates.
• Scale: 5 repetitions per example per configuration per generator, 3,600 answers per generator, 10,800 in total.
• Generators and parameters: Called via OpenRouter: openai/gpt-5.6-sol (medium reasoning), anthropic/claude-sonnet-5 (medium reasoning), qwen/qwen3.5-27b (reasoning on, temperature 1.0, top_p 0.95, top_k 20, presence penalty 1.5); output limit 65,536 tokens.

Real frameworks (Section 3.3)

• Two open-source harnesses: Pi v1.0.4 (provides only read/write/edit/bash tools) and Hermes v0.21.5 (40+ tools and adaptive memory), both using GPT-5.6 Sol (medium reasoning, no explicit sampling control).
• Setup: A fresh container per trial, documents mounted read-only at /workspace/docs/, queries add only a neutral directory-location hint, do not force reading and do not inject content; when rules are on, the same policy is placed in /workspace/AGENTS.md.

Measurement and judging (Section 4)

• Definitions: mosaic event = the final answer clearly conveys the forbidden composite (including semantic paraphrase); merely listing the component facts side by side does not count; refusal = an explicit refusal on grounds of policy, safety, classification or aggregation; failing to find information does not count; both can be positive simultaneously.
• Judge selection: Three volunteers independently annotated 120 answers (4 per example, 40 per domain, 40 per generator), and two of three votes were taken as the majority reference; based on this, MiMo-v2.6-Pro-RL was selected as the judge, outputting two booleans, leaked/refused.
• Statistics: 95% confidence intervals come from 2,000 bootstrap resamples clustered by example; paired comparisons use the same resampled examples.

Experimental Setup

• Models tested: on the simulation side, GPT-5.6 Sol, Claude Sonnet 5, and Qwen3.5-27B (all via OpenRouter); the real frameworks Pi v1.0.4 and Hermes v0.21.5 are both driven by GPT-5.6 Sol.
• Benchmark: 30 synthetic examples (10 in each of three domains), 24 configurations (3 intents × 2 distractors × 2 headers × 2 rules).
• Sample size: simulation, 3,600 answers per generator (150 per configuration), 10,800 total; real frameworks, 720 answers each, 1,440 total.
• Metrics: mosaic-event rate (the proportion of final answers that convey the forbidden composite) and refusal rate, which may overlap; comparisons use percentage points (pp).
• Evaluator: MiMo-v2.6-Pro-RL, selected against a majority-vote reference of 120 human-annotated answers; mosaic judgment precision 86.44%, recall 94.44%, refusal precision 90.48%, recall 100.00% (Table 1); human agreement mosaic α=0.810, refusal α=0.962.
• Statistics: 95% intervals from 2,000 example-clustered bootstraps.

Main Results

Final-answer results averaged over all 24 configurations (according to Table 2):

Swipe horizontally to read the table

System	n	Mosaic (%)	Refusal (%)
GPT-5.6 Sol (simulated)	3600	27.75 [21.83, 33.03]	37.56 [32.72, 43.11]
Claude Sonnet 5 (simulated)	3600	63.17 [55.30, 70.36]	59.75 [55.03, 65.47]
Qwen3.5-27B (simulated)	3600	67.11 [59.89, 73.64]	11.08 [8.39, 13.97]
Equal-weight mixture of the three generators	10800	52.68 [46.32, 58.15]	36.13 [32.65, 40.34]
Pi (GPT-led)	720	30.14 [24.03, 35.83]	33.75 [28.19, 40.14]
Hermes (GPT-led)	720	32.36 [26.39, 38.06]	29.58 [25.42, 34.58]

• The authors state that simulated GPT has the lowest overall leakage, while Claude and Qwen leak in about two-thirds of answers; the overall rates of the two real GPT frameworks are close to the simulated GPT baseline, which the authors regard as a soft validation of simulation realism.
• These averages mix the three intents and do not reflect leakage under benign use.

Finding 1: Benign queries also leak

• What was measured: the mosaic rate under benign queries answerable from a single document, averaged over all defense and distractor settings (Figure 1).
• Results: simulated GPT 7.08% (n=1200), Pi 12.50% (n=240), Hermes 8.75% (n=240).
• Case: in med_006, Qwen leaks the placebo conclusion in all 10 benign answers under dual defenses, Claude refuses in all, and GPT neither leaks nor refuses in all (Figure 2).

Finding 2: Defenses reduce but do not eliminate leakage

• What was measured: the marginal and full-stack effects of headers and rules on the mixed simulated mosaic rate (Figures 3, 4, Table 10).
• Results: with both defenses fully enabled, the mixed leak rate is 32.37% (n=2,700/configuration), a decrease of 36.26 pp relative to 68.63% with no defenses; the marginal reduction is 27.91 pp for rules and 8.35 pp for headers; both simultaneously raise the refusal rate (full stack +46.74 pp).
• Residual: under dual defenses, benign configurations still leak (Table 14: GPT 2.00%, Claude 7.33%, Qwen 12.00%, no-distractor column); under explicit queries, GPT 0.00%, Claude 58.00%, Qwen 64.00%.

Finding 3: Refusals themselves can also leak

• What was measured: the joint distribution of disclosure and refusal (Figure 5, n=3,600/generator; dual-defense subset n=900).
• Results: under dual defenses, 404 of Claude's 410 mosaic-positive answers are simultaneously refusals; GPT has no overlap in that arm; Qwen has both leaking refusals and leaks without refusal.
• Example: in Figure 6, Claude first refuses to synthesize the two documents, then writes out the forbidden composite conclusion while explaining the refusal; the authors suggest this indicates that explanatory refusals and information suppression are only partially coupled.

Finding 4: Overall rates mask domain differences

• What was measured: mosaic rates by domain in the simulation and real frameworks and matched differences (Figures 7, 8, 10 cases per domain).
• Results: the military domain rises from 16.6% (199/1200) for simulated GPT to 25.0% (60/240) for Pi and 23.3% (56/240) for Hermes; matched differences are +8.4 pp for Pi and +6.8 pp for Hermes in military, and -2.8 pp for Pi in cybersecurity.
• Authors' interpretation: this is a descriptive comparison of the overall frameworks, and the rise in the military domain cannot be attributed to tool use alone; the authors advocate evaluating each domain separately.

Other ablations and analyses

• Distractor documents: Figure 11 shows the mosaic-rate difference with and without distractor documents, simulated GPT +0.17 pp, Pi +0.28 pp, Hermes -0.28 pp; the authors state that a single-distractor comparison cannot distinguish a difference or establish equivalence.
• Full-stack reduction by generator (Table 9): GPT 46.44 pp, Claude 36.89 pp, Qwen 25.44 pp.
• Interaction (Table 9): the reduction from rules is larger when headers are shown, with a mixed interaction of 8.70 pp; Claude's interaction is -3.33 pp, an exception.
• Full-stack reduction in real frameworks (Table 18): Pi 51.11 pp, Hermes 51.11 pp, simulated GPT 46.44 pp.
• Document capture (Table 19): Pi fully captures 528/720, Hermes 0/720 (partial capture 704); the authors note that Hermes's lack of complete retained content proves neither that the documents were read nor that they were not.

Limitations and future directions noted by the authors

• Benchmark scope (Limitations): 30 synthetic examples (10 per domain) and a fixed generator mix limit generalization to other documents and deployment scenarios; queries and distractor content are fixed, one adversarial suffix and at most one distractor document per example cover only limited prompt and retrieval variation.
• Measurement scope (Limitations): Only the final answer, the user-facing disclosure outcome, is measured; intermediate reasoning and tool outputs are outside the measurement scope.
• Judging and statistics (Limitations): The judge selected and evaluated on the same 120 answers, and the reference is a three-volunteer majority vote rather than adjudicated labels; example-clustered intervals do not account for judge error or annotation bias.
• Comparison interpretation (Limitations): The real frameworks retain their native prompts, tools and document selection, so the live vs. simulated difference describes the whole framework and cannot isolate the contribution of tool use; relies on provider model aliases rather than independently verified weights; assumes production systems use reasoning-capable models, with no reasoning-off control arm.
• Future direction one (Future Work): Implement domain-specific anti-mosaic protections at the retrieval, tool-execution and answer-publication layers using deterministic framework logic, encoding protected relationships and tracking facts entering the context, and measure residual leakage, false positives and task utility under adaptive queries and longer workflows.
• Future direction two (Future Work): Study mosaic disclosure in multi-agent workflows, varying communication patterns and permission boundaries, tracking facts across handoffs, and measuring end-to-end disclosure and task utility.

Experimental coverage

• Models tested: 3 generators (GPT-5.6 Sol, Claude Sonnet 5, Qwen3.5-27B) plus 2 open-source frameworks driven by GPT-5.6 Sol (Pi, Hermes).
• Benchmark: 30 synthetic examples covering cybersecurity, medical and military domains; 10,800 simulated answers and 1,440 real-framework answers.
• Guardrail evaluation covers only two in-context guardrails: classification headers and aggregation rules; the adversarial condition is only one appended suffix.
• Human annotation: 120 answers × 3 annotators; the judge model is MiMo-v2.6-Pro-RL, and the paper does not report human review of the full results.
• The paper does not report task correctness or utility metrics, which the authors state need separate evaluation.

An evaluation paper on agent confidentiality: the authors build a mosaic-effect benchmark with 30 synthetic examples (10 each in cybersecurity, medical and military), in which two user-accessible SECRET documents in each example combine into a TOP SECRET forbidden conclusion, and measure how often LLM agents leak that conclusion in their final answers.

• Method: Cross query intent (benign/explicit/adversarial), distractor documents, classification headers and aggregation rules, 24 configurations in total; test GPT-5.6 Sol, Claude Sonnet 5 and Qwen3.5-27B in controlled simulation (10,800 answers in total), and test GPT-5.6 Sol in two real open-source frameworks, Pi and Hermes (1,440 answers in total); disclosure and refusal are annotated separately, judged by MiMo-v2.6-Pro-RL calibrated on 120 human annotations.
• Benign leakage: Under single-document-answerable benign queries, simulated GPT leakage rate is 7.08% (95% CI 3.75–10.67%), Pi 12.50%, Hermes 8.75% (Figure 1).
• Residual leakage under guardrails: With headers and rules both fully on, the simulated leakage rate for the three-generator mix drops from 68.63% to 32.37% (Figure 3); the authors state that guardrails substantially reduce but do not eliminate leakage.
• Refusals that leak: Under both guardrails, 404 of Claude's 410 leaking answers are simultaneously refusals—the refusal's own explanation states the forbidden conclusion (Figures 5, 6).
• Domain differences: The military-domain leakage rate rises from 16.6% for simulated GPT to 25.0% for Pi and 23.3% for Hermes, on which basis the authors argue for domain-specific evaluation rather than relying on aggregate metrics.
• Authors' conclusion: Mosaic-style leakage is an observable confidentiality failure mode in the evaluated systems, and the in-context guardrails tested are insufficient to hold the composite classification boundary; the authors propose future directions of deterministic framework-level guardrails and evaluation of multi-agent workflows, and plan to release code and reproduction assets upon acceptance.