A Hybrid SNN Accelerator for 2X Video Frame Interpolation on FPGA for High Frame Rate Applications

Abstract

We present a hardware-efficient hybrid spiking neural network (SNN) accelerator for video frame interpolation (VFI) on a Zynq-7020 FPGA SoC (53,200 LUTs, 140 BRAM36K, 5 W or less). A five-layer leaky integrate-and-fire (LIF) convolutional SNN is partitioned across programmable logic (PL) and the ARM processing system (PS): Conv1+LIF1 runs in PL as a pipelined Q15 fixed-point streaming engine at 100 MHz, while four deeper LIF layers and an optical-flow head execute on the ARM Cortex-A9 in mixed float32/int8 precision. The PL kernel processes 86.7 million MACs per frame pair at 1.13 percent LUT and 1.43 percent BRAM utilisation, achieving 2.43 nJ per MAC (DMA-inclusive). A controlled Cython-vs-NumPy benchmark identifies the PS as DDR3-bandwidth-bound (0.99 times speedup), bounding PS-only optimisation gains below 5 percent and motivating a PL layer-migration roadmap. The system delivers 2 times frame-rate VFI at 41.37 dB PSNR (SSIM of 0.989) over 50 test triplets, verified by co-simulation with zero discrepancies. With more than 90 percent fabric headroom, migrating two further layers to PL is projected to exceed 10 frames per minute at under 2 W. To our knowledge this is the first characterised hybrid SNN VFI PS-PL co-design on a low-cost reconfigurable edge SoC.