We describe an inference-time architectural enhancement for off-the-shelf foundation models that systematically reduces perplexity and boosts accuracy across generation and reasoning tasks. Our approach incurs essentially no additional latency during generation, though it requires serial processing in the prefill phase...
M. Mozer, Shoaib Ahmed Siddiqui, Danny P. Sawyer et al.· Catalysis from A to Z· 11 citations· ⚡1
It is argued that temporally extended cognition requires refocusing from explicit thought traces to implicit activation dynamics via recurrent architectures via recurrent architectures, and a taxonomy of recurrent and continuous-thought transformer architectures is introduced.
M. Mozer, Shoaib Ahmed Siddiqui, Rosanne Liu· arXiv.org· 3 citations
This work describes an inference-time architectural enhancement for off-the-shelf foundation models that markedly reduces perplexity and boosts accuracy across generation and reasoning tasks, and proposes and evaluates an adaptive variant of recirculation which requires only light tuning of hyperparameters while freezi...
Michael C. Mozer, Shoaib Ahmed Siddiqui, Danny Sawyer et al.· Rhinology· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.