The scaling relationship of compute and performance for language models (LMs) has been extensively studied; more recently, models are being designed to scale memory use, as well. In particular, adding embeddings for n-grams scales memory separately from compute use, because each pass only processes individual embedding...
Ian Magnusson, Noah A. Smith, P. W. Koh· 0 citations
A central challenge in de novo protein design is generating proteins with desired structural and functional properties. Existing design models typically require a dedicated input representation for each property, limiting the design objectives they support. We develop Prompteina, the first all-atom structure generation...
Joseph Lee, Yan-Jing Li, Bo Qiang et al.· 1 citation
Recent advances in artificial intelligence (AI) have largely been driven by large language models, deep neural networks that operate over discrete units called tokens. To represent text, most large language models use words or word fragments as the tokens, known as subword tokenization1. Subword tokenization obscures f...
Benjamin Minixhofer, Tyler Murray, Tomasz Limisiewicz et al.· Nature· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.