Ming Tang

ZeroLock: Concurrent Memory-Efficient LLM Training via Modular Update Decoupling

This work proposes a BP-free algorithm, called ZeroLock, that decouples the model updates into independent chunk updates by local objective construction and provides the first theoretical framework for such local objective construction-based approaches under general model chunk division by mapping local objectives to t...