Proof-of-Learning Is Currently More Brittle Than We Think
Proof-of-Learning is Currently More Broken Than You Think
Translation in progress — untranslated text is omitted.
The paper proposes a forgery strategy that can be reproduced across different PoL verification configurations, and can generate training proofs for a stolen model at a cost far lower than previous methods. Proof-of-Learning proves the compute consumed in training a model by recording training checkpoints; the original authors argued for its effectiveness by showing that the cost of forgery is comparable to honest training. The authors point out that recent work has underestimated the non-robustness of PoL verification to attackers, and that existing forgery strategies either cannot be reproduced or only target weakened PoL instances, failing once the verification hyperparameters are changed. The authors claim their strategy is among the first forgery methods reproducible across configurations, and systematically analyze the assumptions that robust verification relies on; theoretically realizing these assumptions still reduces to open problems in learning theory. Based on this, the authors argue that without further understanding of optimization in deep learning, it is impossible to build provably robust PoL verification mechanisms.
- Proof-of-Learning (PoL) uses recorded training checkpoints to prove training compute investment, but the robustness of the verification mechanism lacks a formal guarantee; whether the spoofing counterexample proposed by Zhang et al. holds and where PoL's fundamental vulnerability lies had previously been undecided.
- The authors split PoL verification into two roles (efficiently verifying the validity of updates, and establishing precedence), formalize the reproducibility and representativeness assumptions they depend on, and for these two assumptions respectively propose two structurally correct deception attacks: infinitesimal update and blindfold top-Q.
- On CIFAR-10/100, both new attacks pass verification under settings such as δ=0.008, with a per-step cost of about 1 FP, lower than the (43·n+1)·k FPs of the Zhang et al. attack; the stochastic spoofing experiments targeting precedence (data reordering, synthetic data, adversarial update rules) all failed.
- The authors state that the construction of the optimal verification strategy, the selection of the representative subset, and whether random deception exists are all open problems; the experiments cover only CIFAR-10/100 and ResNet-20/50, and the authors themselves admit that any empirical study of the random deception part is incomplete.