We have 4 of 4 papers
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
An exploratory, single-seed ablation of TALH (Adaptive Latent Hybrid), a decoder-only language model with parallel Multi-head Latent Attention (MLA) and a custom recurrent state-space (SSM) branch supports implementation-specific hypotheses rather than general conclusions about MLA, SSMs, or mixture-of-experts models.
Slicing a byte-level BPE tokenizer allows one language model to operate at several vocabulary sizes, use a control token to indicate the active size, and be deployed at any trained size by slicing its embedding and output head, yielding a falsifiable prediction for future work.
A conventional all-attention model of the same size on the same data and a conventional all-attention hybrid that beats GPT-2 124M, Pythia-160M, OPT-125M and GPT-neo-125M, and exceeds MobileLLM-125M's published score despite that model seeing a trillion tokens.
Intent-Driven Dynamic Chunking (IDC) is introduced, a novel approach that uses predicted user queries to guide document segmentation and aligning document structure with anticipated information needs significantly boosts retrieval performance, particularly for long and heterogeneous documents.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
