Large language model (LLM) services increasingly operate on edge infrastructure, enabling low-latency and privacy-preserving AI services. However, efficiently serving LLM requests across heterogeneous and resource-constrained edge devices require orchestration mechanisms that jointly determine model configuration (family, size, and quantization level) and execution placement while satisfying user- and system-level quality of service (QoS) requirements. This paper introduces LMEdge, a QoS-aware orchestration service that dynamically makes these decisions across heterogeneous edge devices. We formulate the problem as a binary integer linear programming (BILP) optimization that minimizes response time under accuracy, network, and resource constraints. To enable scalable online scheduling, we employ five lightweight machine learning (ML) models to predict query-specific latency, accuracy, resource usage, and response size for each model-size-quantization-device combination, and design a lightweight heuristic that approximates the BILP solution. We collect a comprehensive benchmarking dataset of over 59000 rows to train models and support reproducibility. Evaluation on a Kubernetes-based edge testbed with 57 instances and diverse query categories shows that LMEdge reduces latency, preserves accuracy, improves resource utilization, and increases serving ratio compared to two baselines.
Reza Farahani, Zoha Azimi, Mario Colosi et al.· 0 citations
Satellite edge computing (SEC) has emerged as a promising paradigm to enhance in-orbit data processing capabilities and reduce transmission latency. However, satellite image processing tasks in SEC environments face critical challenges in efficient data handling, resource coordination, and transmission scheduling. The dynamic network topology and time-varying resource availability in satellite constellations further degrade the quality and stability of SEC services. To address these challenges, we propose a deep learning-based Collaborative Image Feature-extraction Task Optimization (CIFTO) framework. CIFTO dynamically distributes image processing workloads across multiple Low Earth Orbit (LEO) satellites, enabling continuous temporal updates for task allocation while significantly accelerating convergence and reducing computational overhead. By integrating temporal modeling and iterative optimization, CIFTO effectively mitigates the NP-hard nature of satellite task allocation. Furthermore, a lightweight satellite image processing model is designed to meet the strict constraints of on-orbit computation, achieving efficient image inference with minimal parameters. Extensive experimental evaluations demonstrate that the proposed framework ensures timely task completion, substantially lowers system-wide energy consumption, and enhances the adaptability and training efficiency of SEC services.
Xiaoteng Yang, Jie Feng, Lei Liu et al.· IEEE transactions on compute...· 0 citations