A Measurement Study of LLM Inference Trade-offs Across Edge Continuum Hardware
A controlled measurement study of self-hosted LLM inference across edge and near-edge deployment nodes: an NVIDIA Jetson AGX Orin and a near-edge server with CPU-only and GPU-enabled inference modes, highlighting that compute-side inference metrics alone can lead to suboptimal placement for latency-sensitive interactiv...
Maysam Khatib, Moysis Symeonides, Demetris Trihinas et al.
· 0 citations