The results show that data composition controls the safety and usability trade-off, and that safety alignment should be evaluated on both sides of the intended refusal boundary.
Alejo Lopez-Avila, Iker García-Ferrero, Jezabel R. Garcia et al.· 0 citations
The aim is a recipe deployable without a multi-week hyper-parameter search, which distills the 4-bit student directly from the original, uncompressed model, and is released open-weight as Hypernova-60B.
Bakbergen Ryskulov, Iker García-Ferrero, David Montero et al.· 0 citations
A practitioner's study of how to make distillation training efficient is presented, organised around two systems contributions, and a fused, chunked KL loss is introduced, making peak memory linear in the sequence length.
Bakbergen Ryskulov, Iker García-Ferrero, David Montero et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.