CHIP: Efficient Homomorphic Encryption-Based CNN Batch Inference Using Channel-Interleaved Packing with Small Rotation Key Set

Abstract

As privacy concerns rise, numerous laws require machine learning-based applications to comply with stringent privacy regulations. While Homomorphic Encryption (HE) allows computation directly on encrypted data, existing HE-based inference solutions suffer from significant computational and memory overhead for both single and multiple samples. Additionally, current methods require many rotation keys, which limits their practicality in a broader range of scenarios.To address these challenges, we propose channel-interleaved packing (CHIP) to embed three-dimensional (3-D) data into 2-D ciphertexts, enabling 3-D HE convolution to be performed as a 2-D HE convolution combined with channel aggregations via ciphertext rotations. To further improve the performance of CHIPbased convolution, we introduce an efficient 2-D convolution that halves the number of HE multiplications. For computationally intensive inference tasks, we employ partial-kernel and minibatch strategies that iteratively process sliced kernels and subsets of samples, aggregating the results to produce the final output. Experimental results demonstrate the superior efficiency of our method compared to the state-of-the-art HE-based approaches by Lee et al. (ICML’22) and Cheon et al. (IEEE TDSC’24) in both single-sample and multi-sample scenarios. Using ResNet18, VGG11, and VGG16 with a batch size of 64, our solution achieves speedups of up to $4.7 \times$. When processing a single test sample, the speedup increases to $\mathbf{6 0} \times$. Moreover, our method requires only 29 rotation keys for evaluation, which is at least 35% fewer than previous works, resulting in an overall memory reduction of up to 45%.