Skip to content
Open access

Improving Accuracy and Efficiency in DNNs with Approximate Multipliers: Insights from Information Bottleneck Theory

Sep 2026 · Applied Informatics · Vol 7, pp. 385 · 0 citations · 24 references

TL;DR

A layer-wise heterogeneous approach using quantized approximate multipliers (INT8) in DNNs, applying varying levels of approximation across layers, achieves estimated energy savings and improves accuracy while improving computational and energy efficiency.

Abstract

Approximate multipliers have potential to improve energy efficiency in Deep Neural Networks but introduce computational errors that degrade accuracy. This paper introduces a novel method, leveraging approximate multipliers to enhance accuracy, while improving computational and energy efficiency. We propose a layer-wise heterogeneous approach using quantized approximate multipliers (INT8) in DNNs, applying varying levels of approximation across layers. This approach achieves estimated energy savings of up to 44.75% across all evaluated configurations, including up to 43.72% for the VGG models, while improving Top-1 accuracy by up to 2.07 percentage points relative to the corresponding exact INT8 baseline. Using Information Bottleneck (IB) theory, we analyze the enhanced information flow and feature extraction capabilities enabled by approximate multipliers. Through Information Plane (IP) analysis, we gain insights into DNN behavior and demonstrate how this method can overcome accuracy limitations. An analytical MAC-count comparison indicates that, under the experimental settings considered, the forward-pass-only GA search requires 146× to 22,500× fewer MAC operations than the corresponding gradient-based training schedules; this comparison reflects computational workload rather than an equivalent-objective algorithmic speedup.

Read PDF

Similar papers

Book Open access Jul 2026

HAP: Efficient Quantization Harnessing Adaptive Precision for DNN Hardware Acceleration

A novel quantization and accelerator design harnessing adaptive-precision (HAP), leverages a per-group dynamic zero-point to generalize APQ and achieves superior accuracy compared with existing PTQ solutions at the level of 4 or 5 bits for a variety of DNN families.

Er-Jing Luo, Xin-Kuang Geng, Hong-Lan Jiang et al. · 0 citations
#machine learning Preprint Sep 2026

RAMP: Robust Adaptive Mixed-Precision Quantization for Edge CPU Vision Models

K-Means clustering is addressed with K-Means clustering, achieving near-lossless accuracy and a mean speed-up over the full-precision model, and it is revealed that excluding from quantization the layers whose speed-up is negligible, regardless of their sensitivity, can be counterproductive.

D. Población-Criado, D. García-Gasulla, Eduardo Quiñones · 0 citations
#machine learning Preprint Oct 2026

Q-PACE: Dynamic Precision Allocation for Quantization-Aware Training

A new approach, called Q-PACE, consisting of a second-order sensitivity model that predicts the loss increase as a sum of quantization noise MSE weighted by per-layer curvature coefficients, and its stability during training allows for infrequent, cheap recalibration.

Alexandra Volkova, Matin Ansaripour, Erik Schultheis et al. · 0 citations
#machine learning Preprint Oct 2026

TR-PTQ: High-Accuracy Integer-Only Transformer Post Training Quantization via Taylor Region Reformulation

TR-PTQ, a unified integer-only formulation using shared Taylor Region exponential and logarithm primitives, is introduced, which eliminates the need for floating-point hardware units for nonlinearities and achieves less than 1.5\% absolute accuracy degradation across vision and language benchmarks.

Eli S. Levy, Adam Teman, Yoni Pugachov · 0 citations
#artificial intelligence Preprint Aug 2026

FAMPWQ: Fisher Information-based Adaptive Mixed Precision Weight Quantization for Effective LLM Inference

A novel Fisher information-based Adaptive Mixed Precision Weight Quantization approach, i.e., FAMPWQ, which performs layer-adaptive weight quantization for effective LLM inference on commodity GPUs and a reinforcement learning-based bit-width allocator in FAMPWQ, which generates an adaptive bit-width allocation strateg...

G. Lee, Ji Liu, Jun-Cheng Jia et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.