FHE-EESI: A Lightweight Fully Homomorphic Encryption-Based End-Edge Collaborative Split Inference Framework

Abstract

End-edge collaborative inference has emerged as an important trend for deploying deep learning applications on resource-constrained end devices. However, the continuous data interaction between end devices and edge server inevitably raises privacy concerns. Fully homomorphic encryption (FHE) provides a privacy-preserving solution by enabling inference directly on encrypted data, but deploying full FHE inference on the edge suffers from high latency. To address these issues, we propose FHE-EESI, an FHE-based end-edge collaborative split inference framework. In FHE-EESI, the end device executes the initial layers on plaintext, encrypts intermediate features using the residue number system Cheon-Kim-Kim-Song (RNS-CKKS) scheme, and transmits them to the edge server for the remaining FHE inference. To construct an FHE-compatible inference network and enable flexible partitionability, we adopt a stage-wise key management strategy and the Chebyshev polynomial approximation. Furthermore, considering dynamic channel conditions and heterogeneous computational capabilities, we design a dueling double deep Q-network (D3QN)-based dynamic split mechanism to adaptively determine the optimal split point. Experimental results on the CIFAR-10 dataset show that FHE-EESI achieves 83.3% inference accuracy and 183.1 s total latency, effectively providing privacy-preserving inference while maintaining inference efficiency.