IRIS: Internal Representation and Insight System — Sequence-Level Evidence for the Linear Representation Hypothesis and a Geometric Instrument for Alignment Monitoring

Abstract

We present IRIS (Internal Representation and Insight System), a geometric instrument for observing and monitoring the internal state of language models. IRIS provides initial sequence-level empirical evidence for the linear representation hypothesis — demonstrating that the relational arrangement of tokens in activation space reflects semantic structure without supervision, across model families and processing depths. IRIS extracts six properties from the residual stream of a transformer model at a chosen layer, producing a per-sequence activation constellation — a PCA projection of token representations that reflects the relational geometry of meaning as the model has internally organised it. We demonstrate that this constellation is empirically deterministic under fixed seed conditions (zero variance across repeated extractions), validating IRIS as a reliable measurement instrument. We report seven empirical findings. In diffusion model space, we demonstrate that visual output similarity does not predict latent geometric proximity. In language model space, we show that tokenisation boundaries determine which concepts exist as geometric units, that transformer processing proceeds through two distinct geometric regimes, and that instruction tuning encodes geometric valence separation invisible to output-level evaluation — a ×997 expansion of positive-negative geometric distance after fine-tuning, replicated across three independent runs. A seventh finding, from 252 extractions across 12 models, shows scale amplifies geometric divergence following data curation. Full details, methodology, and limitations in the attached paper.