Private LLM Inference: Optimizing Tricycle
Abstract
The adoption of Large Language Models (LLM) in scenarios with sensitive data requires the use of privacy-preserving technologies to guarantee confidentiality. Private LLM inference using Fully-Homomorphic Encryption (FHE) guarantees data privacy by encrypting the input beforehand and performing inference directly over the encrypted data. LLM inference is known to require substantial computational resources, and the use of FHE introduces additional memory and runtime overhead beyond these already demanding requirements. To optimize the number of computations performed on encrypted data, different strategies have been proposed. We adopt tricyclic encodings, which support matrix arithmetic with optimal multiplicative depth and minimal key-switching (one of the most costly operations in FHE). We propose a system that combines this strategy with SHIP, a bootstrapping algorithm that minimizes level consumption and features a large degree of parallelism in the computations. This combination avoids the slots-to-coefficients conversion required by traditional bootstrapping methods, while still keeping the result in the slots domain, making it directly compatible with our encoding strategy. In addition, we leverage an optimization from the Batched Relinearization (BRL) technique that improves accuracy and latency by multiplying several ciphertexts in groups of two or three.