MIGServe: Layout-Aware Multi-Instance GPU Management for Efficient LLM Serving
MIGServe treats the physical layout of MIG instances as a first-class scheduling dimension through three techniques: buddy-aware partition placement, which preserves large contiguous free blocks by allocating next to existing occupied buddies; proactive pair-matching migration, which consolidates fragmented half-full buddy pairs off the critical path of inference.
Jian-Wen Chen, Yun-Kai Liang, Bin Gao et al.
· Proceedings of the Internati... · 0 citations