SIFMap: Parallel CNN Inference Using Sub-Input-Feature Maps in PIM

Abstract

Processing-in-memory (PIM) accelerators perform in-situ dot-product operations between input feature maps (IFMs) and convolutional neural network (CNN) weights to accelerate inference. The efficiency of these computations is heavily influenced by the weight mapping strategy. However, existing approaches often adopt IFM-oriented mapping schemes that provide limited parallelism, leading to high latency and low energy efficiency. To address this issue, we propose <sc>SIFMap</sc>, a subIFM-oriented weight mapping framework designed to enable highly parallel CNN inference. <sc>SIFMap</sc> leverages the observation that different regions within an IFM contribute unevenly to computation. It first partitions the IFM into fine-grained sub-maps, or subIFMs, based on their computational characteristics. Then, it applies a reinforcement learning (RL) technique to determine optimal weight mapping strategies tailored to each subIFM, optimizing global performance. Furthermore, <sc>SIFMap</sc> introduces a subIFM-oriented resource allocation scheme to further boost computational parallelism across subIFMs. Experimental results show that <sc>SIFMap</sc> reduces inference latency by <inline-formula><tex-math notation="LaTeX">$\mathbf{4.8}{\boldsymbol{\times}}$</tex-math><alternatives><mml:math><mml:mrow><mml:mn mathvariant="bold">4.8</mml:mn></mml:mrow><mml:mrow><mml:mo mathvariant="bold">×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="he-ieq1-3727031.gif"/></alternatives></inline-formula> and improves energy efficiency by <inline-formula><tex-math notation="LaTeX">$\mathbf{5.2}{\boldsymbol{\times}}$</tex-math><alternatives><mml:math><mml:mrow><mml:mn mathvariant="bold">5.2</mml:mn></mml:mrow><mml:mrow><mml:mo mathvariant="bold">×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="he-ieq2-3727031.gif"/></alternatives></inline-formula> compared to state-of-the-art mapping methods under equivalent resource constraints.