Hardware-Aware Optimization of Large Language Models: A System-Level Analysis
TL;DR
A hardware-aware, system-level analysis of key optimization techniques, including pruning, quantization, knowledge distillation, Low-Rank Adaptation (LoRA), and Neural Architecture Search (NAS), shows that quantization consistently achieves the highest inference speedups and memory efficiency.
Abstract
Deep neural networks (DNNs), particularly large language models (LLMs), have achieved remarkable performance; however, they introduce significant computational and memory challenges that hinder practical deployment. This paper presents a hardware-aware, system-level analysis of key optimization techniques, including pruning, quantization, knowledge distillation, Low-Rank Adaptation (LoRA), and Neural Architecture Search (NAS). The study evaluates these methods in terms of scalability, hardware compatibility, and deployment efficiency. The results show that quantization consistently achieves the highest inference speedups and memory efficiency, while LoRA enables scalable fine-tuning with minimal resource overhead. In contrast, pruning provides limited real-world benefits unless supported by structured sparsity and specialized hardware. These findings highlight the importance of integrated optimization and hardware–algorithm co-design for efficient large-scale AI systems.