ReRAM-based in-memory computing (IMC) architectures are widely regarded as a promising approach to alleviating the computational bottleneck of conventional architectures. Since ReRAM crossbars perform matrix-vector multiplication (MVM) in the analog domain, their computational energy consumption is highly dependent on weight and activation distributions. However, most existing ReRAM accelerators focus primarily on weight optimization while paying limited attention to the impact of activations on computational energy consumption, leaving the energy-saving potential of activation sparsity largely underexploited. In this paper, we propose a voltage-aware ReRAM-based accelerator (VARA), along with its accompanying design methodology. Specifically, we first introduce a voltage-aware training (VAT) algorithm that incorporates a preset threshold into the activation function to steer the activation distribution toward zero values, thereby enhancing activation sparsity. Building upon this, we further propose a co-zero activation reordering (CAR) scheme for crossbar-level computation skipping. CAR clusters activation dimensions based on their co-zero correlations and consistently reorders both the activation matrix and its corresponding weights. This process consolidates scattered zero activations into contiguous zero-valued regions to maximize the benefits of crossbar-level computation skipping. Extensive experimental results demonstrate that, with only marginal accuracy loss, VARA reduces the average total system energy consumption by 60.12\% and improves the average system energy efficiency by 2.68$\times$ compared to the baseline, outperforming existing state-of-the-art accelerators for sparse-activation optimization.
Monte Carlo tree search (MCTS) enables artificial intelligence (AI) decision-making, but requires 55-300 W on conventional processors, limiting edge deployment. In-memory computing (IMC) is energy-efficient on regular workloads but has been considered incompatible with irregular multi-phase algorithms. We introduce phase-to-primitive decomposition, which reformulates each algorithmic phase as a hardware-native IMC primitive. Applied to MCTS, selection, expansion, rollout and backpropagation map to content-addressable memory, combinational logic, a resistive random-access memory (RRAM) crossbar and static random-access memory, keeping search on chip. At 22 nm with fabricated RRAM-array parameters, IMC-MCTS consumes ~60 mW for 9x9 Go, achieving 96x energy efficiency over a central processing unit (CPU) and 65x-2,059x over an H100 graphics processing unit (GPU). It reaches a European Go Federation rating within sample-size uncertainty of open-source Go engines (Pachi-UCT and Michi-C). The same substrate runs eight applications across four AI domains.
Tergel Molom-Ochir, Benjamin F. Morris, Yintao He et al.· arXiv.org· 0 citations
The growing demand for long-context LLM inference has exposed a critical bandwidth–capacity trade-off in memory systems, rendering single-tier PIM architectures ineffective. HBM-PIMs offer high bandwidth but limited capacity, while DIMM-PIMs provide scalability at the cost of lower bandwidth; neither satisfies the throughput-latency requirements of long-context serving workloads for LLMs. To address this, we propose <bold>HydraPIM</bold>, a heterogeneous multi-tier PIM architecture that decomposes attention computation across HBM-PIM and DIMM-PIM tiers by exploiting the <bold>inherent sparsity</bold> of attention. HydraPIM introduces <bold>HydraAttention</bold>, a tiled attention mechanism with hierarchical reduction, enabling efficient cross-tier execution through lightweight on-chip reduction units. To maintain load balance under dynamic access patterns of sparse attention, HydraPIM features an <bold>importance-aware KV migration</bold> mechanism that monitors token relevance and relocates hot tokens to high-bandwidth tiers at runtime. This software-hardware co-design helps improve the utilization of both bandwidth and capacity. Evaluations show that HydraPIM achieves 1.66<inline-formula><tex-math notation="LaTeX">$\boldsymbol{\times}$</tex-math><alternatives><mml:math><mml:mo mathvariant="bold">×</mml:mo></mml:math><inline-graphic xlink:href="wang-ieq1-3710733.gif"/></alternatives></inline-formula> higher throughput than HBM-based NPU-PIM and 1.96<inline-formula><tex-math notation="LaTeX">$\boldsymbol{\times}$</tex-math><alternatives><mml:math><mml:mo mathvariant="bold">×</mml:mo></mml:math><inline-graphic xlink:href="wang-ieq2-3710733.gif"/></alternatives></inline-formula> lower latency than DIMM-based NPU-PIM across diverse long-context workloads.
Shixin Zhao, Lian Liu, Xiangwen An et al.· IEEE transactions on compute...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
