Scheduling Data to Compute: Hierarchical Data Pipelining for LLM Training on AMD GPUs
Large language models (LLM) have been the key focus of recent machine learning breakthroughs. As the model size grows exponentially, the computation needed for training is astronomical. A GPU can do a huge amount of arithmetic. However, the data its compute units need is not always available when they are ready. This d...