MOSAIC: Efficient Collaborative Inference for Multi-Modal Large Language Models with Distributed Edge Computing

Abstract

Multi-modal large language models (MLLMs) enable natural language reasoning over heterogeneous sensory data, appealing to many edge AI scenarios, yet their deployment faces competing demands on latency, energy, and quality. We present MOSAIC, which adopts a component-level decomposition and distributed deployment paradigm for efficient MLLM inference at the edge, realizing three core ideas: parallel encoding, token pruning, and device-server collaboration. Specifically, for the offline stage, modality-specific encoders distilled from a unified foundation model are co-deployed with Q-Former projection on edge devices for parallel encoding, while LoRA instruction fine-tuning adapts the frozen LLM backbone to downstream tasks. For the online stage, we formulate a multi-objective optimization over latency and energy that jointly tunes token budgets, device frequencies, and quantization precision under long-term semantic accuracy constraints, achieving dynamic token pruning and device-server collaboration. We leverage Lagrangian relaxation to decompose the original problem into tractable per-slot subproblems and design an online algorithm with provable near-optimal performance and guaranteed constraint satisfaction. We implement a realistic testbed for extensive performance evaluation, which demonstrates that MOSAIC reduces latency by up to 49.8% and energy by up to 65.1% over advanced baselines while maintaining high semantic accuracy.