The Tailor's Reading

Abstract

A feature of a sparse-autoencoder dictionary is published with one natural-language label, written by one language model from the passages that most excite the feature in one corpus. That label is a reading, and its provenance, the explainer, the explainer's system prompt, the corpus, and the model through which the corpus was run, is usually left unstated. We propose recording the provenance with every label and adding, for every feature, a second label written under a different and declared provenance, kept beside the published one rather than replacing it. We demonstrate this on Google DeepMind's Gemma Scope 2 dictionary for layer 31 of gemma-3-27b (262,144 features), read through a small fine-tuned adapter that reproduces one author's writing of 2012, with a system prompt built from that author's own vocabulary. The same feature then carries Neuronpedia's label creation and genesis stories and, under our provenance, Elohim's "Let there be": the creating Word. With the labels in hand we read seven texts step by step, three diaries and a memoir generated by the model family, a chat, Shakespeare's sonnets and Hamlet's speeches, and ask whether individual features stay active across consecutive steps longer than chance. The test uses a per-feature permutation test, two family-level null models (whole-order permutation and a fixed-margin null), and a replication under a wrong preceding context. Five of the seven texts show more persisting features than the maximum of either null; the sonnets do not at the main threshold, and the seventh, a twenty-entry diary, reaches only one of the two. What persists is where a text changes and what it carries: the fine-tuned diary's first movement on the author's coinages and its collapse at entry 28; the rooms, the vessel and the gap of a second persona's diary in the order she wrote them; Hamlet's forged commission in its Latinate register; in the sonnets the traditional groupings, on which nine of twenty-three persisting features have their longest run against six by chance, read as their arguments; and in the chat a rupture and a return measured, 510 features of address that go dark for sixteen messages and 458 that come back. We also define a glued space over the features of a text, a homotopy colimit of one complex per step glued along what consecutive steps share, whose first Betti number counts returns of co-firing pairs; it tracks a direct count of pair returns nearly one to one. Labels cost $0.002–$0.02 each. The system prompt is authored; the labels are not scored against held-out text; a label under a provenance answers only what the feature means under that provenance. Full HTML text: https://icra.tanazur.org/papers/the-tailors-reading/ — ICRA pre-print series: https://icra.tanazur.org/