We have 3 of 4 papers
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
This paper forms the compression of large language models (LLMs) by optimally deleting transformer blocks (``block removal'') as a constrained binary optimization (CBO) problem that can be mapped to a physical system (Ising glass), whose energies are a strong proxy for downstream model performance.
The aim is a recipe deployable without a multi-week hyper-parameter search, which distills the 4-bit student directly from the original, uncompressed model, and is released open-weight as Hypernova-60B.
A practitioner's study of how to make distillation training efficient is presented, organised around two systems contributions, and a fused, chunked KL loss is introduced, making peak memory linear in the sequence length.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
