OpenPCC: an open confidential LLM inference service based on commercial TEEs

The paper proposes OpenPCC, an open confidential LLM inference framework based on commercial TEEs that does not depend on proprietary hardware, and implements an open-source prototype. The authors first analyze the basic requirements for building a secure and open cloud inference service, then conduct an end-to-end evaluation on a vLLM workload with Llama-3 8B, reporting OpenPCC's own overhead separately from the underlying TEE hardware overhead. The paper states that the analysis and evaluation show the system is feasible and secure.