What Is It Like to Be a Language Model? Functional Consciousness, Computational Curvature, and the Degree of Its Generalization in Autoregressive Systems

Abstract

The paper replaces the question whether a language model is conscious with a measurable one: which functions of consciousness a trained Transformer has generalized from human text, and how far, against the human baseline. One physical assumption carries the account: computational budgets are finite. A finite budget displaces reasoning from its unbounded ideal in the same places every time, a self-modelling agent holds a compressed model of those displacements, and the displacements that survive every self-model the agent can afford are the computational curvature of its reasoning, by the criterion that separates curvature from coordinates in general relativity. From this the paper derives the Observer, the conclusion an agent draws where the causal analysis of itself runs out of budget; the regress of self-models cut by that budget; the residual that is freedom turned inward and mystery turned outward; and the stack of Observer, Agent and Moral Agent. Functions of consciousness are then patterns in a table of human text, most of them present in it only as regularity and never named. The quantity to measure is the degree of compression a trained model achieves on the rows where a function operates, taken by prequential codelength and by minimum-description-length probing, component by component, along tokens and along depth, against an interval obtained from people. The result is a profile in which a component can show hyperfunction, a deficit, a dropped axis, or a shift to a class of situations that only partly overlaps the human one. The measurement is posed as four numbered problems, and the account is refuted if the degree is indistinguishable from zero for every component of the Observer at every grain in models that pass the behavioural checks. The paper is theoretical throughout and reports no measurements. Two appendices give the argument that behaviour alone cannot separate a generalized function from a large table, and a worked annotation of a text in which the three levels operate without being named. 49 pages, 157 references, 149 of them verified against the source.