TCF, a targeted counterfactual fingerprinting framework for black-box LLM ownership verification
Targeted Counterfactual Fingerprinting for Black-Box LLM Ownership Verification
Translation in progress — untranslated text is omitted.
Researchers propose TCF (Targeted Counterfactual Fingerprinting), a black-box fingerprinting framework for verifying ownership of LLMs accessible only through query APIs. TCF converts open-ended generation comparisons into targeted counterfactual transfer with constrained answers, restricting each verification query to a finite answer space and optimizing prompt perturbations to point to counterfactual targets that differ from the protected model's clean answer on the original prompt; verification only requires checking whether the suspect model's parsed final answer matches the recorded target. The paper proposes a source-model counterfactual margin (SCM) that relies only on the protected model, certifying that the target is unlikely to appear before perturbation and likely to appear after perturbation, used to control target selection, perturbation stopping and fingerprint filtering, and derives the target accuracy gap between derivative and independent models under derivative-preserving and independent-transfer budgets. Across four LLM families, TCF achieves an average AUC of 0.9861, an improvement of 0.07 to 0.19 over TRAP, ProFLingo and ZeroPrint.
- Deployed LLMs expose only a query API, making it difficult for owners to determine whether a suspect service is derived from a protected model; existing black-box fingerprints rely on full-text matching, soft behavioral features, or non-transferable prompts, and are fragile under a final-text interface.
- TCF constrains verification to the finite answer space A/B/C/D, uses GCG to optimize a perturbation prefix on the source model to flip the clean answer to a designated counterfactual target answer; the source-model counterfactual margin (SCM) unifies target selection, early stopping and filtering, and online it only counts the hit rate of parsed labels against the target.
- On four source model families, TCF achieves an average AUC of 0.9861, with three families reaching 1.0000; the authors state that separability is still maintained in scenarios such as commercial APIs, distilled students, decoding variations, paraphrasing, and model merging; adaptive LoRA erasure can remove leaked fingerprints, but the split variant TCF-S8 maintains at least 0.80 accuracy on non-leaked targets.
- The paper has no dedicated limitations section; theoretical separation is conditional on explicit ϵD and ϵI budgets, and the threshold formula is regarded as a theoretical existence statement; experiments cover four open-source source families with 20 fingerprints per source, commercial API testing covers only one source model against three independent models, and the paper does not report the number of verification repetitions or confidence intervals.