EHEIR: Efficient Homomorphic Encrypted Inference via Architectural Redesign

Memory-Efficient Designs for Word-Wise Universal Fully Homomorphic Encryption

Fully Homomorphic Encryption (FHE) enables computation on encrypted data, preserving privacy throughout analysis. While its privacy is very strong, FHE is much slower to execute than the original computation. In particular, due to the recent success in accelerating its compute, the performance bottleneck shifts to the memory, especially considering that FHE magnifies the data size by orders of magnitude, resulting in a low arithmetic intensity. We propose BXT, an FHE optimization framework that mitigates the memory bottleneck through four techniques: (1) ciphertext compression, which regenerates ciphertext components from seeds during execution; (2) ciphertext serialization, which packs coefficients as bit arrays and unpacks them during L2-to-L1 transfer; (3) delayed seed generation, which defers PRNG-heavy offline work across aggregated operations; and (4) ciphertext digit pruning guided by fault-aware training tailored for Universal FHE. On CNN inference, the BXT-CSO50 configuration effectively achieves up to 3.8$\times$ speedup over the 100x GPU baseline with less than 1% accuracy loss at 50% comparison precision.

How to Watermark the RLWE Homomorphic Ciphertexts

GoldenRetriever: Non-Interactive Homomorphic Encrypted Retrieval for Privacy-Preserving RAG

This work proposes a practical non-interactive encrypted retrieval framework for RAG based on threshold selection, and introduces a precision-stable mask polarization method that ensures accurate recovery of selected documents.

Ciphertext-Native Watermarking for RLWE-Based Homomorphic Encryption

This work proposes a watermarking technique for RLWE-based HE ciphertexts by exploiting the algebraic structure of RLWE polynomials and introduces two practical schemes that preserve the original security of HE while maintaining correctness and watermark robustness.

CHIP: Efficient Homomorphic Encryption-Based CNN Batch Inference Using Channel-Interleaved Packing with Small Rotation Key Set

As privacy concerns rise, numerous laws require machine learning-based applications to comply with stringent privacy regulations. While Homomorphic Encryption (HE) allows computation directly on encrypted data, existing HE-based inference solutions suffer from significant computational and memory overhead for both single and multiple samples. Additionally, current methods require many rotation keys, which limits their practicality in a broader range of scenarios.To address these challenges, we propose channel-interleaved packing (CHIP) to embed three-dimensional (3-D) data into 2-D ciphertexts, enabling 3-D HE convolution to be performed as a 2-D HE convolution combined with channel aggregations via ciphertext rotations. To further improve the performance of CHIPbased convolution, we introduce an efficient 2-D convolution that halves the number of HE multiplications. For computationally intensive inference tasks, we employ partial-kernel and minibatch strategies that iteratively process sliced kernels and subsets of samples, aggregating the results to produce the final output. Experimental results demonstrate the superior efficiency of our method compared to the state-of-the-art HE-based approaches by Lee et al. (ICML’22) and Cheon et al. (IEEE TDSC’24) in both single-sample and multi-sample scenarios. Using ResNet18, VGG11, and VGG16 with a batch size of 64, our solution achieves speedups of up to $4.7 \times$. When processing a single test sample, the speedup increases to $\mathbf{6 0} \times$. Moreover, our method requires only 29 rotation keys for evaluation, which is at least 35% fewer than previous works, resulting in an overall memory reduction of up to 45%.

S2MM: Scalable FPGA Acceleration of Secure Matrix Multiplication with Homomorphic Encryption

Homomorphic Encryption (HE) enables secure computation on encrypted data, addressing privacy concerns in cloud computing. However, the high computational cost of HE operations, particularly matrix multiplication (MM), remains a major barrier to its practical deployment. Accelerating Homomorphic Encrypted MM (HE MM) is crucial for applications such as privacy-preserving machine learning. In this paper, we present S2MM, the first scalable FPGA-based accelerator designed for HE MM. We show that HE MM is fundamentally bottlenecked by excessive off-chip memory traffic when operating with practical HE parameter sets. To analyze this bottleneck, we develop a cost model that characterizes the on-chip memory requirements of ciphertexts during HE MM, revealing the scalability limits of CPU-based implementations. Guided by the analysis, we propose a novel datapath for Homomorphic Linear Transformation (HLT), the dominant workload in HE MM. The proposed datapath enables fine-grained on-chip data reuse, substantially reducing both off-chip memory traffic and on-chip memory demand. To further improve scalability, S2MM supports tiled HE MM, removing the matrix size limitation imposed by the HE encoding capacity and enabling efficient computation on large matrices. In addition, the architecture is highly configurable, adapting to diverse matrix sizes and HE parameter sets while maximizing throughput within FPGA resource constraints. We also introduce a systematic configuration methodology to determine optimal design parameters for efficient deployment across varying workloads and platforms. We implement S2MM on an Alveo U280 and conduct comprehensive evaluations across a wide range of matrix sizes, shapes, and HE parameters. Experimental results demonstrate that S2MM achieves average speedups of 219× and 37.4× over state-of-the-art CPU and GPU implementations, respectively. We further evaluate S2MM in an application-level encrypted CNN inference pipeline, achieving a 6.08× end-to-end speedup by accelerating the HE MM-dominated fully connected layers.