Taming Generation Quality and Latency for Text-to-Image Serving at the Edge

Abstract

Serving text-to-image (T2I) generation on edge servers can reduce the serving latency and enhance the preservation of user privacy. Currently, mainstream T2I models generate images iteratively, and the more iterations, usually the higher the generation quality, but the longer the generation latency. Existing T2I serving approaches simply adopt a fixed iteration number or employ large iteration numbers to merely optimize the generation quality, failing to adapt to the time-varying workload at the edge. Besides, they do not consider the edge computing environment where computing capabilities and network connections of edge servers are always heterogeneous. To make up for these drawbacks, we propose the problem of latency optimization for T2I serving at the edge under the long-term generation quality constraint and design EdgeT2I to address it. EdgeT2I employs Lyapunov optimization to decompose the long-term problem into a series of real-time sub-problems and leverages a Markov approximation (MA)-based algorithm to jointly determine the iteration number and offloading scheme of requests for each sub-problem. By doing so, EdgeT2I can adaptively adjust the iteration number to handle the time-varying workload by appropriately sacrificing generation quality to reduce serving latency during the high workload and compensating for generation quality during the low workload, as well as optimizing request offloading at the heterogeneous edge. Extensive trace-driven experiments confirm that compared to baselines, EdgeT2I can decrease the serving latency by up to 99.8% while maintaining the generation quality.