Preprint
Aug 2026
SubZero+: Efficient Zeroth-Order LLM Fine-Tuning via Large Learning Rates
Experiments show that SubZero+, an improved SubZero framework that improves stability in three complementary ways, consistently outperforms prior ZO baselines, enlarges the stable learning-rate range, and narrows the gap to first-order methods with minimal extra memory overhead.
Ziming Yu, Shu-Yao Xiao, Xingyu Zhao et al.
· 0 citations