#machine learning
May 2026
Memory by Design: Probabilistic Sequence Layers
Distilling Bayesian Layers into a pretrained 340M Gated DeltaNet improves RULER long-context retrieval over a matched-compute control, and improves robustness beyond the training regime in controlled collision studies, learned associative recall, and the Zoology MQAR benchmark.
Matthew Dowling, H. Jeon, Cristina Savin et al.
· arXiv.org · 0 citations