Jordi Torres

SLIM: Saturation-Aware Lightweight Performance Modeling for LLM Serving

SLIM (Saturation-Aware Lightweight Performance Model), a semi-analytical model that predicts LLM inference throughput and latency from analytical formulations of Transformer computation and memory traffic, is introduced, which outperforms representative performance-modeling baselines while successfully generalizing to...