What is the Role of Small Models in the LLM Era: A Survey
Abstract Large Language Models (LLMs) have demonstrated remarkable capabilities in various reasoning tasks, which leads to the development of increasingly large models. However, scaling up model sizes results in significantly higher computational costs and energy consumption, which makes these models impractical for ac...
Lihu Chen, Gaël Varoquaux
· Computational Linguistics · 6 citations