Privacy-preserving mechanisms enable low-cost verifiable inference for LLMs

The researchers propose two protocols that use privacy-preserving inference to provide verification guarantees for LLM inference services, requiring only the computation of a few additional tokens. Such mechanisms target the problem in third-party hosting scenarios where a service provider may replace a large model with a weaker model that has lower operating cost and return the weaker model's results. Compared with existing verification methods that rely on zero-knowledge proofs (ZKPs), this scheme has very small additional overhead and almost no impact on downstream performance; the authors state that because the fastest privacy-preserving inference methods are usually faster than ZKP methods, verification runtime is also improved.