On the smallness of the large language model scaling exponents
We discuss reasons why the scaling exponents of current large language model (LLM) applications are pointing towards a hardly sustainable regime in terms of energy resources. We further show that attributing the smallness of such exponents to a numerical bias due to the neglect of a non-zero value of the loss function...
Sauro Succi, Peter V. Coveney, Alex Hansen
· Frontiers in Physics · 0 citations