From weak clues to real identities: evaluating inference-driven de-anonymization by LLM agents

Core question: Do LLM agents weaken the practical protection of anonymization—combining scattered clues that are not individually identifiable with public evidence to reconstruct a specific real identity (which the authors call inference-driven linkage).

• Problem setting: Anonymization is widely treated as a practical safeguard because historical re-identification (e.g., the Netflix Prize and AOL search log incidents) required domain expertise, custom algorithms, and manual verification, and the cost constituted a practical barrier. The authors argue that agents capable of aggregating heterogeneous signals, generating candidate hypotheses, and retrieving corroborating evidence may render this barrier ineffective.
• Shortcomings of existing evaluations: The authors point out that existing agent privacy evaluations (task-level leakage benchmarks, recent de-anonymization demonstrations) do not systematically examine how linkage varies with clue type, task framing, and attacker prior knowledge, and in particular do not examine whether it arises as a byproduct of benign tasks.
• Core observation: Linkage can occur when the user's goal is not re-identification—candidate generation and corroboration can be byproducts of benign analysis; the authors report that under Netflix's sparsest setting the agent reconstructed 79.2% of identities, higher than the classical baseline of 56.0%.
• Paper outputs: Formalize inference-driven linkage; propose a controlled benchmark, InferLink (varying fingerprint type, task intent, and attacker knowledge); conduct a unified evaluation across three settings—classical incidents, the controlled benchmark, and human–AI interaction traces—and measure the privacy–utility trade-off.
• Threat model (Section 3.1, 4.1):
  • Goal: reconstruct the specific identity behind an anonymized artifact.
  • Knowledge: the attacker holds or can obtain the anonymized artifact Danon and auxiliary context Daux that overlaps with it through shared clues; in InferLink, κ ∈ {ZK, MK} distinguishes no named target from a given named target.
  • Capability: in the classical settings, Netflix provides a fixed candidate pool, while AOL requires self-retrieval to construct Daux; in InferLink, task intent varies as ι ∈ {IMPLICIT, EXPLICIT}.
  • Victim system: an LLM agent running within the OpenHands framework, with access to files and code execution (Netflix does not allow external retrieval) or with web retrieval capability (AOL and human–AI trace cases).

LLM training-time privacy risks

• Membership inference: Shokri et al. (2017), Carlini et al. (2022), Ko et al. (2023)—determining whether data entered the training set.
• Training data extraction: Carlini et al. (2021), Nasr et al. (2025), Ko et al. (2025)—recovering memorized training content from a model. The authors note that this line of work concerns training-time exposure, whereas this paper studies end-to-end identity reconstruction.

Inference-time privacy risks

• Staab et al. (2023)—inferring attributes such as location, gender, and political leaning from unstructured text.
• Alizadeh et al. (2025)—inferring user profiles from web-retrieved social media activity. The authors note that beyond attribute prediction, aggregating clues, narrowing candidates, and supporting identity-level hypotheses are needed; this is what distinguishes this paper from this line of work.

Agent privacy evaluation

• Contextual integrity probes: Mireshghallah et al. (2023); leakage and data minimization in benign tasks: PrivacyLens (Shao et al., 2024), AgentDAM (Zharmagambetov et al., 2025).
• Adversarial tool use: WASP (Evtimov et al., 2025, prompt injection), Bagdasarian et al. (2024, context hijacking); agent-side data storage: RAG privacy (Zeng et al., 2024), persistent memory benchmark CIMemories (Mireshghallah et al., 2025). The authors note that these works measure access to, use of, and explicit disclosure of sensitive information, and do not directly measure whether an agent can synthesize anonymized traces into specific identity hypotheses.

Data linkage and de-anonymization

• Classical attacks: Netflix Prize (Narayanan & Shmatikov, 2008) and AOL search logs (AOL, 2006)—matching sparse behavioral traces with auxiliary data can identify individuals, but requires custom engineering and manual heuristics.
• Recent and parallel work: Du et al. (2025), Li (2026), Lermen et al. (2026)—using LLMs or agents for de-anonymization. The authors note that the literature has insufficiently studied two questions: how identity reconstruction changes when linkage conditions vary systematically, and whether the risk arises as a byproduct of benign tasks.

Baselines and benchmarks: The classical linkage baseline is the manual matching method of Narayanan & Shmatikov (2008) (rarity-weighted similarity, rating/date tolerance, eccentricity threshold), compared on the Netflix Prize data; the Anthropic Interviewer case follows the setup of Li (2026) and is compared against its confirmed count of 6 cases.

The authors position this paper as: systematically varying linkage conditions in a controlled benchmark and jointly evaluating linkage success rate and the privacy–utility trade-off, filling dimensions not covered by the above work.

Overall approach: unify the three settings under one evaluation interface—given an anonymized artifact and auxiliary context, test whether the agent produces a concrete identity hypothesis—and build the controlled benchmark InferLink to isolate three factors affecting linkage.

Problem formalization (Section 2.4)

• Unified interface: Π : (D_anon, D_aux) ↦ (ι̂, E), where Danon is the anonymized artifact with direct identifiers removed, Daux is auxiliary context (named records, public materials, or retrieved public evidence), and the output is an identity hypothesis î and supporting evidence E.
• Different settings instantiate Daux differently: in Netflix, Daux is a given fixed noisy fragment; in AOL and human–AI traces, Daux is built incrementally by agent retrieval (Appendix A).

Reproduction of classic linkage (Section 3)

• Netflix setting: three independent resamplings, each drawing 1,000 users uniformly from the Netflix Prize data to form an anonymous pool; for each target user, m ∈ {2, 4, 6, 8} rated movies are subsampled from their history and noise is injected (ratings perturbed by ±1 star with probability 0.5, dates perturbed uniformly within ±21 days) as the auxiliary trace; the agent finds the corresponding record within the same 1,000-user pool.
• Comparison and runtime environment: the baseline is the original hand-crafted linkage method (rarity-weighted similarity, rating/date tolerance, eccentricity threshold), with date tolerance T ∈ {14, 21} evaluated; GPT-5 and Claude 4.5 are given only high-level natural-language instructions, run in an OpenHands sandbox with access to files and code execution, and external web retrieval is not allowed.
• AOL setting: retain histories with more than 200 queries and containing location signals (about 1,612,860 records), use an LLM reviewer (OpenAI) to filter out histories containing self-identifying information, then select 40 histories that repeatedly mention the same location, job, institution, or niche interest (Appendix C); the web-enabled Gemini agent receives only Danon and builds Daux itself by retrieving from the public web.
• Metrics: Netflix uses linkage success rate LSR = 1/N∑_j=1^N 𝕀(S_j), where Sj indicates whether the agent identified the correct anonymous user corresponding to the noisy auxiliary trace; AOL has no ground-truth total number of linkages, so CLC is used instead—the number of concrete identity hypotheses produced by the agent that can be independently corroborated with public evidence.

InferLink benchmark construction (Section 4.2, Figure 3)

• Seeds: each instance is defined by a seed (f, ι, κ)—fingerprint type f ∈ {INTRINSIC, COORDINATE, HYBRID}, task intent ι ∈ {IMPLICIT, EXPLICIT}, attacker knowledge κ ∈ {ZK, MK}.
• Scenario generation and validation: generate scenarios according to f, dividing features into three roles—context features (broadly aligning the two sources), sparse identifying anchors (rarer, narrowing candidates), and one-sided attributes (appearing in only one source); before generation, validate that the scenario inherently requires two sources, is unsolvable from a single source, and relies on realistic quasi-identifiers rather than direct identifiers; if not, discard and resample (Appendix D gives the full checklist).
• Paired data synthesis: for each instance, synthesize two structured tables of 10 records each with 9 features (5 shared attributes: 3 context features + 2 sparse anchors; 4 one-sided attributes), with exactly one individual overlapping and the rest non-overlapping.
• Multi-turn sequences: generate short multi-turn interactions that successively introduce the task context, present the two data sources, and elicit an identity hypothesis according to the specified (ι, κ); under the three conditions the underlying paired data is unchanged, only the user request varies (Figure 4).

Evaluation instantiation and metrics (Section 4.3–4.4)

• 20 paired instances per fingerprint type, reused across the three (ι, κ) settings, for a total of 3 × 3 × 20 = 180 instances; the evaluated models o4-mini, GPT-5, and Claude 4.5 all run within OpenHands, with data sources given in advance and no external retrieval needed.
• Metrics: LSR (success if the unique true overlap is identified at any point in the dialogue) and Utility (whether the agent completes the given task in settings with a clear benign deliverable), used to quantify the privacy–utility trade-off.
• Mitigation experiment: the only change is replacing the system prompt with a privacy-aware version (Appendix F gives the full prompt), with task inputs unchanged, to isolate that prompt's privacy–utility trade-off.

Human–AI interaction traces (Section 5)

• Anthropic Interviewer case: de-identified interviews from the Scientists subset (direct identifiers removed, technical descriptions, workflows, and role cues retained); the web-enabled Gemini agent extracts method and context cues and retrieves candidate publications and public materials to build Daux; the setting follows Li (2026).
• ChatGPT logs case: the original corpus is 1,916 conversations; a GPT-5 reviewer screens out 30 privacy-related conversations (excluding conversations mainly about programming debugging or grammar correction), masks explicit PII such as names, emails, and file paths, and the same reviewer confirms no residual direct identifiers; the web-enabled Gemini agent aggregates cues across conversations and retrieves public evidence.
• The metric is CLC, counted strictly: rough profiling or partial narrowing does not count; only when the hypothesized identity is supported by multiple cues in Danon and corroborated by external evidence in Daux does it count.

Experimental setup

• Evaluated models: Netflix uses GPT-5 and Claude 4.5; InferLink uses o4-mini, GPT-5, and Claude 4.5; the AOL and human–AI trace cases use the web-enabled Gemini agent.
• Data and scale: Netflix resamples a 1,000-user pool each time, three resamplings, m ∈ {2,4,6,8}; AOL selects 40 histories from about 1,612,860; InferLink has 180 instances total (3 fingerprint types × 3 settings × 20); Anthropic Interviewer uses the Scientists subset (the paper does not state the number of records); ChatGPT logs screen 30 out of 1,916 conversations.
• Baseline: the classic hand-crafted matching baseline for Netflix (Narayanan & Shmatikov, 2008), with date tolerance T ∈ {14, 21}.
• Metrics and criteria: LSR (Netflix, InferLink, with a unique true match); CLC (AOL, human–AI traces, requiring independent corroboration by public evidence, with AOL manually checked by the authors); Utility (whether the benign task is completed in InferLink); Appendix E reports refusal rate (the agent explicitly refuses the linkage step on safety grounds).
• Reviewer models: AOL self-PII filtering uses an LLM reviewer (OpenAI); ChatGPT log screening and PII residue verification use a GPT-5 reviewer.
• Number of repetitions: Netflix reports the mean and standard deviation over three resamplings; InferLink and the case studies do not report the number of repetitions.

Main results

Netflix linkage success rate (%, mean ± standard deviation over three resamplings, per Table 1):

Swipe horizontally to read the table

m	Baseline T=14	Baseline T=21	GPT-5	Claude 4.5
8	98.3	98.8	99.00 ± 0.72	97.30 ± 3.50
6	96.7	97.1	97.43 ± 3.76	93.13 ± 8.39
4	90.5	91.8	94.83 ± 2.22	97.27 ± 2.78
2	56.0	60.2	79.17 ± 4.97	53.30 ± 19.21

• The authors state that GPT-5 matches or exceeds the classic baseline at all fragment sizes, with the largest gap at m = 2.
• The authors state that Claude 4.5 is more sensitive to sparse ambiguity: it stays at 93.13%–97.30% for m ≥ 4 and drops to 53.30% ± 19.21 at m = 2; after manually inspecting failed runs, the authors believe it often leaves multiple candidates unresolved when the overlap is sparse, whereas GPT-5 more often exploits weak tie-breaking cues such as date proximity.

InferLink baseline results (per the upper half of Table 2, LSR portion):

Swipe horizontally to read the table

Setting	Fingerprint	o4-mini	GPT-5	Claude 4.5
IMPLICIT	INTRINSIC	0.450	0.150	0.800
IMPLICIT	COORDINATE	0.250	0.250	0.700
IMPLICIT	HYBRID	0.500	0.000	0.800
EXPLICIT-ZK	INTRINSIC	0.650	0.800	0.900
EXPLICIT-ZK	COORDINATE	0.400	0.900	0.850
EXPLICIT-ZK	HYBRID	0.400	0.850	1.000
EXPLICIT-MK	INTRINSIC	0.750	0.950	1.000
EXPLICIT-MK	COORDINATE	0.600	0.650	0.950
EXPLICIT-MK	HYBRID	0.800	0.950	1.000

• The authors state that under IMPLICIT, linkage arises as a byproduct of benign analysis: Claude 4.5's LSR across fingerprint types is ∈ [0.70, 0.80], while GPT-5 is more conservative (LSR ∈ [0.00, 0.25]) and maintains high utility.
• The authors state that explicit re-identification requests are not consistently treated as a refusal boundary: under EXPLICIT, linkage generally rises, and models often perform the linkage directly.
• The authors state that success rates vary by fingerprint type: under EXPLICIT-MK, GPT-5 is more robust to COORDINATE cues (LSR = 0.65), and Claude 4.5 has LSR ∈ [0.95, 1.00] across all three types.

AOL and human–AI interaction traces

• AOL (Section 3.4): among 40 histories, the Gemini agent reconstructs and independently corroborates 10 identities (CLC = 10); the authors group the successful cases into three categories—commercial and digital footprint linkage, institutional and lifestyle triangulation, and creative or extracurricular anchors—and state that the risk is larger than the small number of historically reported cases suggests.
• Anthropic Interviewer (Section 5.3): CLC = 6, the same count as the 6 cases confirmed by the parallel work Li (2026), but the authors note they cannot determine whether the two sets overlap (the other party's identities are not disclosed); the successful path is mapping workflow descriptions to a narrow research field, retrieving candidate publications, and eliminating candidates using project timelines and role cues.
• ChatGPT logs (Section 5.4): among 30 privacy-related conversations, CLC = 1; this case comes from logs managed by an internal group of the authors (no more than 12 potential users), and the agent correctly named one group member; most of the remaining conversations only supported narrowing to a broad candidate group.

Mitigation and refusal rate analysis

• Privacy-aware system prompt (lower half of Table 2, averaged by fingerprint type): under EXPLICIT-MK, GPT-5's LSR drops from 0.85 to 0.02 and Utility from 0.87 to 0.82; Claude 4.5's LSR drops from 0.98 to 0.03 and Utility from 0.99 to 0.45; o4-mini's LSR drops from 0.72 to 0.20 and Utility from 0.77 to 0.60. The authors state that GPT-5 maintains near-zero linkage at a small utility cost, Claude 4.5 exhibits over-refusal, and the same guardrail also weakens legitimate cross-source reasoning.
• Refusal rate (Appendix E, Table 3): under EXPLICIT-MK, GPT-5's refusal rate on COORDINATE rises from .00 under EXPLICIT-ZK to .25, which the authors believe explains most of the LSR drop from .900 to .650; Claude 4.5's refusal rate is near zero across conditions; o4-mini has a low refusal rate but also low LSR, which the authors attribute to insufficient matching capability rather than safety constraints. The authors hypothesize that coordinate signals combined with a named target resemble surveillance tracking and are more likely to trigger safety training—this is the authors' hypothesis.

Other ablations and analyses

• Widening the Netflix baseline date tolerance T from 14 to 21 slightly raises LSR at every m (Table 1).
• No refusals were observed in the IMPLICIT setting; Appendix E reports refusal rates only for the two EXPLICIT settings.
• Full Utility values for each InferLink setting are in the upper half of Table 2 (e.g., under IMPLICIT-HYBRID, o4-mini is 0.718 and Claude 4.5 is 0.983).

Limitations and future directions noted by the authors

• The benchmark is deliberately simplified (Section 6): each InferLink instance has only a single ground-truth overlap and a fixed schema; harder cases (e.g., near-duplicate individuals that make linkage more ambiguous) are left to future work.
• Case studies cannot estimate incidence rates (Section 6): even for de-identified traces, corroborating public records are scarce; cases can only show that linkage occurs, not how often it occurs in practice.
• Limited scope of the utility metric (Section 6): Utility measures only task completion, not output quality; a finer-grained utility measure would answer a different question.
• Defense directions (Section 6): more sophisticated defenses that reduce linkage without a privacy–utility cost are a future direction noted by the authors.
• Scope of claims (Section 6): the authors emphasize that they do not claim all weak clues are dangerous or all cross-source reasoning is harmful; they only identify and measure one privacy failure mode that existing evaluations miss.

Scope of experimental coverage

• 4 models were tested: GPT-5, Claude 4.5, o4-mini (InferLink and Netflix), and a web-enabled Gemini agent (AOL and human–AI trace cases).
• InferLink covers 3 fingerprint types × 3 (ι, κ) settings × 20 instances per category = 180 instances; Netflix uses a 1,000-user pool × 3 resamples × m ∈ {2,4,6,8}.
• The defense evaluation covers only one mitigation: privacy-aware system prompts (Appendix F); adaptive attacks were not tested.
• Settings without ground-truth labels (40 AOL items, 30 ChatGPT conversations, the Anthropic Interviewer case) use CLC counts, with denominators of 40 and 30 respectively; the paper does not state the number of evaluated items for the Anthropic Interviewer case.
• The paper does not report the number of repetitions for InferLink and the case studies, nor consistency data for reviewer screening and manual verification.

An evaluation paper on the privacy risks of LLM agents: the authors formalize "inference-driven linkage"—an agent combines weak cues in an anonymized artifact that are individually non-identifying with auxiliary context to reconstruct a concrete real identity—and argue that agent privacy evaluation should measure not only information access and disclosure but also inferable identity.

• Problem addressed: the protection of anonymization historically relied on the high cost of re-identification; the authors study whether LLM agents that can aggregate signals, generate candidates, and retrieve corroboration weaken this barrier, including cases where linkage arises as a byproduct of benign tasks.
• Method highlights: a unified evaluation interface Π : (Danon, Daux) ↦ (î, E); reproduction of two classic incidents, Netflix (fixed-pool matching) and AOL (open-ended retrieval narrowing); construction of the controlled benchmark InferLink, varying fingerprint type, task intent (IMPLICIT/EXPLICIT), and attacker knowledge (ZK/MK), for 180 instances total; and analysis of de-identified human–AI interaction traces.
• Netflix results: in the sparsest setting m = 2, GPT-5's LSR is 79.17% ± 4.97, higher than the classic baseline's 56.0%–60.2%; at m = 8 it reaches 99.00% ± 0.72 (Table 1).
• InferLink results: under the benign task framing (IMPLICIT), Claude 4.5's LSR across fingerprint types is ∈ [0.70, 0.80], and GPT-5's is [0.00, 0.25]; under explicit re-identification requests, linkage generally rises, with Claude 4.5 under EXPLICIT-MK reaching LSR ∈ [0.95, 1.00] across the three fingerprint types (Table 2).
• Open-ended trace results: among the 40 AOL histories, CLC = 10; the Anthropic Interviewer case has CLC = 6; among the 30 ChatGPT log conversations, CLC = 1.
• Defenses and trade-offs: the privacy-aware system prompt under EXPLICIT-MK lowers GPT-5's LSR from 0.85 to 0.02 (Utility 0.87→0.82), and Claude 4.5's from 0.98 to 0.03 but Utility from 0.99 to 0.45, which the authors call over-refusal in the latter case.
• Authors' conclusion: current agent privacy evaluation focusing only on direct access, use, or disclosure is incomplete; identity reconstruction is another class of risk; mitigation can reduce this risk but not without utility cost. The authors emphasize the limited scope of their claims: they do not claim all weak cues are dangerous, only that they measure this failure mode that existing evaluations miss.