Skip to content

TCF, a targeted counterfactual fingerprinting framework for black-box LLM ownership verification

Targeted Counterfactual Fingerprinting for Black-Box LLM Ownership Verification

arXiv· arXiv:2608.08195· Yutong Wu, Xiaofan Bai, Shixin Li, Pingyi Hu, Ziqi Zhou, Zilong Wang, Xiaojing Ma, Songfeng Lu, Yuhong Li, Jin Xuan, Yi Wang, Dongmei Zhang, Bin Benjamin ZhuHuazhong University of Science and Technology; Alibaba Group; Microsoft CorporationIndexed by this site · 19 days ago· Originally publishedDefenses

Translation in progress — untranslated text is omitted.

Quick read

Defenses

Based on the paper PDF (AI-generated); refer to the original

AI summaryResearchers propose TCF (Targeted Counterfactual Fingerprinting), a black-box fingerprinting framework for verifying ownership of LLMs accessible only through query APIs. TCF converts open-ended generation comparisons into targeted counterfactual transfer with constrained answers, restricting each verification query to a finite answer space and optimizing prompt perturbations to point to counterfactual targets that differ from the protected model's clean answer on the original prompt; verification only requires checking whether the suspect model's parsed final answer matches the recorded target. The paper proposes a source-model counterfactual margin (SCM) that relies only on the protected model, certifying that the target is unlikely to appear before perturbation and likely to appear after perturbation, used to control target selection, perturbation stopping and fingerprint filtering, and derives the target accuracy gap between derivative and independent models under derivative-preserving and independent-transfer budgets. Across four LLM families, TCF achieves an average AUC of 0.9861, an improvement of 0.07 to 0.19 over TRAP, ProFLingo and ZeroPrint.

Questions
Deployed LLMs expose only a query API, making it difficult for owners to determine whether a suspect service is derived from a protected model; existing black-box fingerprints rely on full-text matching, soft behavioral features, or non-transferable prompts, and are fragile under a final-text interface.
Methods
TCF constrains verification to the finite answer space A/B/C/D, uses GCG to optimize a perturbation prefix on the source model to flip the clean answer to a designated counterfactual target answer; the source-model counterfactual margin (SCM) unifies target selection, early stopping and filtering, and online it only counts the hit rate of parsed labels against the target.
Experiments and results
On four source model families, TCF achieves an average AUC of 0.9861, with three families reaching 1.0000; the authors state that separability is still maintained in scenarios such as commercial APIs, distilled students, decoding variations, paraphrasing, and model merging; adaptive LoRA erasure can remove leaked fingerprints, but the split variant TCF-S8 maintains at least 0.80 accuracy on non-leaked targets.
Limitations and future work
The paper has no dedicated limitations section; theoretical separation is conditional on explicit ϵD and ϵI budgets, and the threshold formula is regarded as a theoretical existence statement; experiments cover four open-source source families with 20 fingerprints per source, commercial API testing covers only one source model against three independent models, and the paper does not report the number of verification repetitions or confidence intervals.

Related papers

Show 3 more papers
Read originalarxiv.org