Vector quantization (VQ), in particular lattice-based methods, offer strong rate-distortion trade-offs for low-bit large language models (LLMs), yet their behavior under quantization-aware training (VQ-QAT) remains poorly understood. We scale lattice VQ-QAT to billion-parameter LLMs and multi-billion-token training reg...
Sebastian Loeschcke, Ivan Koryakovskiy, T. V. D. van der Ouderaa et al.· 0 citations
Satellite observations, precomputed embeddings, and map products describe the same evolving Earth, yet are stored as independent, petabyte-scale data products. Their continued growth calls for compact representations of multiple products while preserving spatial and temporal detail. We introduce Planetary Feature Field...
Arjun Rao, Sebastian Loeschcke, A. Fuller et al.· 0 citations
Method is introduced, which augments SOAP-style preconditioning with a scalar secant-energy correction adapted to Kronecker geometry and an adaptive basis update followed by variance-state downscaling, and is positioned as a scalable option for stiff, high-accuracy physics-informed training, rather than a uniform repla...
Guang-Yuan Wang, Mads Toftrup, Sebastian Loeschcke et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.