Skip to content

Author

Masahiro Tanaka

3 papers indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

Nov 2026

Pipelining the Memory Hierarchy for Ultra-Long-Context Language Model Training

Long-context language models are attractive because they can read whole documents, code repositories, conversations, and scientific sequences. However, a long inference window is not the same as a model that was trained to use millions of tokens reliably. Direct long-context training is blocked by temporary memory spik...

Jing-Han Yao, Sam Ade Jacobs, Masahiro Tanaka et al. · 0 citations
Open access May 2025

ZenFlow: Enabling Stall-Free Offloading for LLM Training

ZenFlow is presented, a new offloaded training framework that decouples updates between GPU and CPU by prioritizing important gradients and achieves up to 5× end-to-end speedup, 2× lower PCIe traffic, 85% less GPU stalls, and up to 72% lower cloud GPU cost, all while preserving accuracy.

Ting-Feng Lan, Yu-Sen Wu, Bin Ma et al. · 8 citations · ⚡3
Open access Sep 2026

ZenFlow: Enabling Stall-Free Offloading for LLM Training

Fine-tuning large language models (LLMs) often exceeds GPU memory limits, prompting systems to offload model states to CPU memory. However, existing offloaded training frameworks like ZeRO-Offload treat all parameters equally and update the full model on the CPU, causing severe GPU stalls, where fast, expensive GPUs si...

Ting-Feng Lan, Yu-Sen Wu, Bin Ma et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.