From Representations to Computational Trajectories: An Intrinsic Fingerprinting Method TrFP for Large Language Model Provenance Verification

1. 01What problem does this paper attempt to solve?

   Addresses the difficulty of provenance after LLM weights are modified: parametric fingerprints are not robust enough, and representational fingerprints lack uniqueness.

   Core problem: after model weights are obtained without authorization and modified by fine-tuning, pruning, quantization, parameter permutation, etc., how to reliably verify whether a suspect model originates from a protected source model.

   • Scenario and importance: the authors state that training LLMs is costly and constitutes an important digital asset; after obtaining the weights, attackers can modify the model with common techniques, retaining main capabilities while concealing the relationship with the original model, potentially causing unauthorized use and commercialization.
   • Shortcomings of existing methods (as stated in the paper): watermarks need to be embedded before release and may affect performance, and cannot protect already-released models; parametric fingerprints change greatly under continued training and structured pruning, limiting robustness; representational fingerprints rely on CKA similarity, and the authors find that CKA overestimates similarity under high dimension and small sample size, and that unrelated models may also have high representational similarity, lacking uniqueness.
   • Core observation: similar representations need not go through the same layer-by-layer update process. The authors call the sequence of layer-by-layer updates of representations along network depth the computational trajectory; in the UMAP visualization of Figure 1, trajectories of different model families are separated from each other, and derived models follow the trajectory of their source model, based on which the authors consider that trajectories can provide both uniqueness and robustness.
   • Proposed in the paper: TrFP, an intrinsic fingerprint method based on computational trajectories, which constructs compact fingerprints using permutation-invariant trajectory descriptors and systematically evaluates its performance under various model transformations and attacks.
   • Threat model (Section V-A):
     • Goal: given a source model Mv and a suspect model Ms, determine whether Ms shares a common origin with Mv (identical or derived through transformations).
     • Defender capabilities: white-box access to Mv, pre-stored reference fingerprints; access to Ms's intermediate hidden states during verification; use of a private probe set.
     • Attacker capabilities: obtains the source model, knows the fingerprint algorithm; can perform fine-tuning, pruning, merging, quantization, parameter permutation, and can also explicitly optimize the model to reduce fingerprint similarity.
     • Limitation: the defender's private probe set is kept secret from the attacker.

   Collapse this answer

2. 02What related research exists?

   Related work is divided into three groups: DNN fingerprinting, parametric LLM fingerprinting and representational LLM fingerprinting; this paper uses PCS, ICS, REEF, CREF and AttnDiff as baselines.

   DNN fingerprinting (behavioral and parametric lines)

   • IPGuard [25], Lukas et al. [26], Peng et al. [27]: characterize the model from decision behavior—extracting fingerprint samples near the classification boundary, constructing transferable adversarial examples that transfer only to the surrogate model, and using universal adversarial perturbations to characterize the geometry of the decision boundary.
   • Zheng et al. [28]: project discriminative model weights into an owner-determined random space to generate compact fingerprints.
   • GMFIP [2], FIT-Print [3]: focus on the robustness and security of fingerprints; the former trains a generator to synthesize fingerprint samples to resist modification and extraction attacks, while the latter proposes targeted fingerprints against false ownership claims.

   LLM parametric fingerprints

   • PCS, ICS [13]: PCS uses cosine similarity of parameter vectors to determine provenance; ICS exploits parameter invariants in Transformer attention and feed-forward modules to enhance robustness to parameter permutation.
   • SELF [14], GhostSpec [15], AWM [16]: construct fingerprints from the singular values/eigenvalues of attention weights, the spectral structure of attention weight matrices, and parameter coordinate alignment, respectively. The paper points out that such methods rely on parameter features remaining stably comparable after transformations, and may be limited under large parameter updates, structural changes, or weak parameter correspondence.

   LLM representational fingerprints

   • EasyDetector [17], REEF [18]: the former trains a classifier to see whether it can transfer to the suspect model; the latter directly compares the intermediate representations of two models on the same input, using CKA to measure similarity.
   • FNF [19], CREF [20], AttnDiff [21]: compare the activation trends of co-activated neuron groups, use model responses along semantic concept directions as fingerprints, and use the singular values of attention map differences for semantically conflicting prompt pairs as fingerprints, respectively. The paper points out that such methods use representational similarity as evidence of common origin, while similarity may not be specific to provenance, limiting uniqueness.

   Baselines and benchmarks: the experiments compare five baselines: PCS, ICS (parametric) and REEF, CREF, AttnDiff (representational); probe data use ToxiGen (TrFP default), TruthfulQA (REEF etc.), and others.

   Positioning: the authors state that this paper reveals two problems of representational fingerprints (CKA estimation bias, and high similarity caused by representation convergence), and proposes replacing parameters or single-layer representations with computational trajectories, claiming stronger provenance discrimination than several existing SOTA methods.

   Collapse this answer

3. 03How does the paper address this problem?

   TrFP uses three permutation-invariant descriptors—layer, neighborhood, and cumulative—to characterize how representations evolve with depth, constructs compact fingerprints, and compares them after alignment by relative depth.

   Overall idea: instead of comparing parameters or single-layer representations, compare the computational trajectory—how representations are updated layer by layer along network depth. TrFP extracts hidden states at each layer from a private probe set, constructs trajectories, extracts permutation-invariant trajectory descriptors and normalizes them into fingerprints, and finally computes similarity after alignment by relative depth to determine provenance (Figure 5).

   Computational trajectory construction

   • Hidden state sequence: for probe xi, take the hidden state of the last token at each layer (the authors state that under causal attention it aggregates information from the entire input), yielding h(0)…h(L).
   • Layer-wise displacement: the difference between hidden states of adjacent layers d(l) = h(l+1) − h(l); since h(l+1) is determined by the current hidden state and the parameters of that layer, d(l) is jointly determined by both.
   • Trajectory definition: the computational trajectory of probe xi is the displacement sequence T_M(xi) = (d(0), …, d(L−1)).

   Three-level trajectory descriptors

   • Layer descriptor ρ: the ratio of each layer's displacement magnitude to the input representation magnitude, characterizing the update strength of that layer on the representation.
   • Neighborhood descriptor δ: the difference between ρ of adjacent layers, characterizing how update strength changes across layers (taken as 0 when l=0).
   • Cumulative descriptor e: the magnitude of the vector sum of displacements of the first l layers, i.e., e_i^(l) = ‖Σ_k d_i^(k)‖₂, characterizing the net displacement from the starting point; the vector sum makes it reflect directional interactions among displacements.
   • Cross-model comparability: all three depend only on Euclidean norms and do not require coordinate correspondence between hidden dimensions; the trajectory state of a probe at depth l is q(l) = [ρ(l), δ(l), e(l)].

   Fingerprint construction and comparison

   • Normalization: for each layer position, compute the descriptor center r(l) (mean) over the probe set and a global scale σM, and normalize as q̂ = (q − r)/(σM + ϵ); the authors state this is to eliminate the interference of scale differences among descriptors and of shared baselines across probes on discrimination.
   • Fingerprint: FM ∈ R^{L×N×3}, i.e., the set of normalized descriptors for all layer positions and all probes.
   • Depth alignment and decision: models of different depths are uniformly sampled at L̄ = min(Lv, Ls) aligned positions by relative depth, and cosine similarity of probe descriptors is computed and averaged position by position to obtain the total similarity S; S ≥ τ is judged as common origin, and the paper follows prior work in setting τ = 0.5.

   Permutation invariance analysis

   • Three types of permutation: the paper theoretically analyzes attention permutation (query/key and value dimensions), FFN intermediate dimension permutation, and global hidden dimension permutation.
   • Conclusion: using the orthogonality of permutation matrices, under all three permutations the attention and FFN outputs are unchanged or the trajectory descriptors remain unchanged; the authors state that TrFP is permutation-invariant by construction; experiments use BF16 precision, which may introduce small deviations from the theoretical invariance.

   Evaluation method

   • Metrics: average similarity of related/unrelated model pairs, TPR and TNR at a fixed threshold of 0.5; robustness experiments examine the similarity between source and derived models; adaptive attacks additionally evaluate model utility; storage overhead is measured by fingerprint size.

   Collapse this answer

4. 04What experiments did the paper conduct?

   Compare 5 baselines on 56 models from 7 families, and measure robustness under fine-tuning, pruning, quantization, merging, permutation, and adaptive attacks.

   Experimental setup

   • Models under test: seven open-source families Falcon3-7b, Llama-3.1-8b, Mistral-7b, OLMo-2-7b, Qwen2.5-14b, Qwen2.5-7b, Qwen3-1.7b, each with 8 candidate models: Base, Instruct, Fine-tuned, Adapter-based, Merged, GPTQ-quantized, Distilled, totaling 56; the Base and Instruct of each family serve as source models, 14 source models in total.
   • Model pairs: a 14×56 similarity matrix, with 98 related pairs and 672 unrelated pairs; PCS cannot compare GPTQ-quantized models, so its matrix is 14×49 (84 related, 588 unrelated).
   • Baselines: PCS, ICS (parametric), REEF, CREF, AttnDiff (representational), following the original settings as much as possible (e.g., REEF uses 200 TruthfulQA probes and takes the last-token representation at layer 18).
   • TrFP settings: 200 ToxiGen probes by default; REEF/CREF/AttnDiff use linear CKA, PCS/ICS/TrFP use cosine similarity.
   • Metrics and decision: average similarity of related/unrelated pairs, [email protected], [email protected]; similarity ≥ 0.5 is judged as common origin.
   • Number of repetitions: the parameter permutation experiment is repeated with 3 seeds, reporting mean±standard deviation; the paper does not report the number of repetitions for other experiments.

   Main results

   Overall provenance results of Table I (56-model pool):

   Swipe horizontally to read the table

   Method	Rel. Mean ↑	Unrel. Mean ↓	[email protected] ↑	[email protected] ↑
   PCS	0.906	0.034	91.7%	100%
   ICS	0.933	0.003	93.9%	100%
   REEF	0.889	0.728	100%	0%
   CREF	0.898	0.692	100%	2.1%
   AttnDiff	0.988	0.751	100%	28.6%
   TrFP	0.887	0.248	100%	100%

   • Authors' interpretation: PCS/ICS have good uniqueness but miss some related pairs at the 0.5 threshold; REEF/CREF/AttnDiff have high means for unrelated pairs, limiting uniqueness; TrFP achieves both 100% TPR and TNR.
   • Illustration: in the similarity matrices of Figure 6, PCS, ICS, and TrFP show a diagonal block structure, while REEF, CREF, and AttnDiff also have high similarity outside the blocks.

   Two validation experiments on representational similarity (Section III)

   • CKA estimation bias: for two sets of independent Gaussian random representations, with n=200 and d=8192, the biased CKA reaches 0.976 and drops to 0.001 after debiasing (Figure 2); the authors state that under high dimension and small sample size, the biased HSIC estimator overestimates similarity.
   • Provenance specificity: the debiased CKA of 7 pretrained models with no derivation relationship on 200 TruthfulQA probes all fall in 0.61–0.80 (mean 0.69), exceeding REEF's unrelated threshold of 0.5; after randomly reinitializing parameters, it drops to 0.31–0.54 (mean 0.46) (Figure 3). The authors state that high similarity is more likely related to the training process (representation convergence) rather than provenance relationships.
   • Re-test of REEF: on 14 models (7 families Base+Instruct), under biased CKA the unrelated-pair mean is 0.718 and the related-pair mean is 0.843; after debiasing they are 0.646 and 0.801, respectively, and the distributions overlap under both settings; for example, Qwen3-1.7b-Base and the unrelated Qwen2.5-7b-Base score 0.685, higher than its 0.674 with its own Instruct version (Figure 4).

   Robustness to model transformations (Table II)

   • Fine-tuning: with Llama-2-7b as the source and 6 fine-tuned derived models (5M–700B tokens). TrFP similarity is 0.9993–0.9999; PCS drops to 0.5333 on CodeLlama-7b (500B tokens), and ICS drops to 0.4626, 0.0175, and 0.0159 on Chinese-llama-2-7b, CodeLlama-7b, and Llemma-7b, respectively.
   • Pruning: Llama-2-7b and the Sheared-Llama 1.3b/2.7b series. PCS is all below 0.03, ICS is all below 0.58; TrFP is 0.8840–0.9996, AttnDiff is 0.9903–0.9945.
   • GPTQ quantization: Falcon3-7b, OLMo-2-7b, Qwen3-1.7b. TrFP is 0.9196, 0.8924, 0.8286; ICS drops to 0.0070 on OLMo-2-7b; PCS is not applicable.
   • Model merging: Evollm-jp-7b and its three component models. TrFP is 0.8002–0.8100, lower than other methods (e.g., AttnDiff 0.9940–0.9966); the authors state that merging makes the trajectory partially deviate from each source model, but all similarities remain above 0.5.

   Parameter permutation and adaptive attacks

   • Parameter permutation (Table III): Qwen2.5-7b and Llama-3.1-8b, with 3 seeds each for FFN/attention/hidden dimension/combined permutation. TrFP is all ≥ 0.9968 (e.g., Qwen2.5-7b combined permutation 0.9970±0.0003); ICS, REEF, CREF, AttnDiff are also close to 1; PCS drops to 0.0427±0.0001 and 0.0426±0.0001 under hidden dimension and combined permutation on Llama-3.1-8b, respectively.
   • Adaptive attack (Figure 7): the attacker knows the full fingerprint pipeline, uses 200 TruthfulQA samples as a surrogate probe set, fine-tunes Falcon3-7b and Qwen3-1.7b with LoRA on Alpaca, with the objective Ltask + λ·Levade, λ taking 0/1/5/10, for 1000 steps in total. The attack pushes the similarity on the surrogate probes toward −1, but the similarity on the defender's private ToxiGen probes remains above the threshold of 0.5 throughout; increasing λ from 5 to 10 yields limited additional reduction in private-probe similarity, while model performance degradation becomes larger (specific utility values are not annotated on the figure). The authors state that TrFP remains robust even when the mechanism is fully public.

   Other ablations and analyses

   • Descriptor combinations (Table IV): the full combination ρ+δ+e achieves 100% TPR/TNR and the highest robustness under the four transformations (fine-tuning 0.9996, pruning 0.9121, GPTQ 0.8803, merging 0.8055); using δ alone gives only 0.3122 under pruning.
   • Probe datasets (Figure 8): under the five probes ToxiGen, ConfAIde, TruthfulQA, StereoSet, PKU-SafeRLHF, related-model similarity is higher than unrelated-model similarity, with the Gap annotated on the figure at 0.49–0.64.
   • Number of probes (Figure 8): results fluctuate when N is 10–20, and approach the N=1000 results at about 100–200; the default is N=200.
   • Storage overhead (Table V): TrFP fingerprints range from 0.0732 MiB for Llama2-7b to 0.1396 MiB for KimiK2-1043b; by comparison, PCS is 3978749.2500 MiB on KimiK2-1043b, ICS about 384 MiB, and CREF 0.0076 MiB.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors state that TrFP is limited to white-box settings, that broader architectures and stronger adaptive attacks remain to be studied, and that they plan to extend it to agent systems.

   Limitations and future directions noted by the authors

   • White-box dependence (Conclusion): The authors state that TrFP requires access to intermediate representations and is therefore mainly applicable to white-box provenance verification.
   • Architecture coverage (Conclusion): The authors state that its robustness on broader model architectures remains to be further studied.
   • Stronger adaptive attacks (Conclusion): The authors state that robustness against stronger adaptive attacks remains to be further studied.
   • Extension to agents (Conclusion): The authors plan to extend TrFP from LLMs to agent systems, exploring provenance features in planning, tool use and interaction trajectories.

   Scope of experiments

   • Models tested: The overall evaluation covers 7 open-source model families and 56 models (including Base/Instruct/fine-tuned/Adapter/merged/GPTQ-quantized/distilled variants); storage overhead is additionally measured on Llama2-70b, GLM-4.5-Air-106b, GLM-4.5-355b, DeepSeek-V3-671b and KimiK2-1043b.
   • Transformations and attacks: Fine-tuning (5M–700B tokens), structured pruning (Sheared-Llama 1.3b/2.7b), GPTQ quantization, model merging (Evollm-jp-7b), four types of parameter permutation (3 seeds), LoRA adaptive attack (λ=0/1/5/10, 1000 steps, only Falcon3-7b and Qwen3-1.7b).
   • Probe settings: 200 ToxiGen probes by default; ablations cover 5 probe datasets and sizes from N=10 to 1000.
   • Repetition and statistics: Only the parameter permutation experiments report mean±standard deviation over 3 seeds; for the remaining experiments the paper does not report the number of repetitions or statistical tests.
   • Utility evaluation: In the adaptive attack, model performance is evaluated every 100 steps, but the paper does not state the name of the utility metric used or specific values.

   Collapse this answer

6. 06Summarize the main content of the paper

   TrFP uses computation trajectories as white-box provenance fingerprints for LLMs, achieving 100% TPR/TNR on a pool of 56 models and resisting multiple transformations and adaptive attacks.

   A fingerprinting paper on LLM ownership protection, targeting the white-box provenance verification scenario where weights are obtained and modified.

   • Problem: The authors state that parametric fingerprints lack robustness under large-scale fine-tuning, pruning and other transformations; representational fingerprints rely on CKA similarity and have two problems—CKA overestimates similarity under high dimension and small sample (with n=200, d=8192, unrelated random representations reach 0.976), and debiased CKA between unrelated pretrained models still reaches 0.61–0.80, so uniqueness is insufficient.
   • Method: TrFP uses the sequence of layer-by-layer updates of representations along network depth (computation trajectory) as the fingerprint source, characterizes the trajectory with three descriptors—layer, neighborhood, cumulative—that depend only on norms and are permutation-invariant, and after standardization aligns by relative depth and computes cosine similarity, with a threshold of 0.5 to determine same origin; the authors theoretically analyze its invariance to three types of parameter permutations.
   • Main results: On 56 models from 7 families, TrFP gives a mean of 0.887 for related pairs and 0.248 for unrelated pairs, with [email protected] and [email protected] both 100%; the representational baselines REEF/CREF/AttnDiff have TNR of only 0%/2.1%/28.6% (Table I).
   • Robustness: Similarity is 0.9993 for CodeLlama-7b fine-tuned on 500B tokens; 0.8840–0.9996 for pruned models; ≥ 0.9968 under combined parameter permutations; a LoRA adaptive attack that knows the full algorithm cannot push similarity on the private probes below the threshold (Table II, III, Figure 7). In the model merging scenario similarity is 0.8002–0.8100, lower than other baselines but still above the decision threshold.
   • Cost: The fingerprint is less than 0.3 MiB (0.0732 MiB for Llama2-7b) and varies little with model scale (Table V).
   • Authors' conclusion: Computation trajectories can provide both uniqueness and robustness, and TrFP is suitable for large-scale model provenance; its limitations are dependence on access to intermediate representations and primary applicability to white-box scenarios, with broader architectures and stronger adaptive attacks left for future work, and plans to extend to agent systems.

   Collapse this answer

Related papers