Prediction and Entropy of Evolving Sequences: A Transformer-Based Estimator of Functional Information
It is demonstrated that transformer derived surprisal systematically distinguishes functionally constrained sequences from unconstrained ones across digital and biological substrates, establishing transformer surprisal as a potential measure of functional information in evolving systems.