On-Device Multimodal Small Language Models for Privacy- Preserving Edge Intelligence Through Quantization Pruning Distillation and Energy-Efficient Inference

Abstract

On-device multimodal small language models are emerging as a practical foundation for privacypreserving edge intelligence, enabling devices to interpret and generate language while processing images, audio, video, sensor streams, and contextual signals without continuous cloud dependence. Their deployment can reduce data exposure, network latency, connectivity costs, and centralized infrastructure demand; however, limited memory, compute capability, battery capacity, thermal headroom, and accelerator support constrain model design and inference. This review examines architectures, compression techniques, privacy mechanisms, and energy-efficient inference strategies for multimodal small language models operating on smartphones, wearables, vehicles, robots, cameras, and embedded Internet of Things platforms. It develops a deploymentcentric taxonomy covering weight, activation, cache, and modality-encoder quantization; structured, unstructured, token, channel, and modality-aware pruning; response, representation, cross-modal, and task-specific knowledge distillation; and hybrid compression pipelines. The review analyzes how these methods affect language quality, visual and acoustic understanding, cross-modal alignment, robustness, fairness, privacy leakage, memory consumption, latency, throughput, energy per inference, and thermal stability. Particular attention is given to local retrieval, adaptive modality selection, speculative decoding, cache management, early exiting, hardware-aware scheduling, and heterogeneous CPU, GPU, neural, and digital-signal processing. The paper also evaluates secure personalization, federated updates, trusted execution, encrypted storage, sensor permission controls, and data-minimizing telemetry. Comparative evidence is organized by device class, workload, compression target, and evaluation objective. The gap analysis identifies limited reproducibility, inconsistent energy measurement, weak lowresource language coverage, underdeveloped multimodal privacy benchmarks, and insufficient study of sustained inference under thermal throttling. It distinguishes theoretical compression savings from measurable device-level efficiency improvements. The review concludes with standardized reporting recommendations and research priorities for compact, private, reliable, and energy-efficient multimodal intelligence at the edge.