Modelometry: Classification of Flagship AI Model Families Based on Inter-Model Stylometry

Abstract

Working paper and open data. Studies that measure the output of more than one AI language model consistently find that the models write differently, in the same manner that human authors have different writing styles. We coin the term **modelometry** for the measurement and attribution of the inter-model writing style of AI systems, and **modelolect** for the style itself, formed on the pattern of idiolect and sociolect. The study asks whether the modelolect of a flagship model family is strong enough for a classifier that reads only surface stylometric features to classify the AI family that originally wrote a text. We test seven families (GPT, Claude, Gemini, DeepSeek, Grok, Llama and Qwen) against human text, in two registers (formal academic text, and informal chat text) and with three tiers of features. Every AI text used is from corpora generated for our prior studies or from open datasets. On academic text, a gradient boosting classifier over 47 interpretable stylometric features attributes 7 classes (human and six families) at 73.7% accuracy against a 14.3% chance rate, and 6 classes on a second corpus at 79.2%. On chat responses from the LMArena preference dataset, the same 47 features attribute all seven families at 63.9%, and a character n-gram model with model names masked reaches 87.0%, so the small interpretable feature set accounts for about three quarters of the attributable signal. The confusion structure is also informative, since DeepSeek is rarely confused with GPT (3% in the corpus where DeepSeek is most identifiable), so the writing-style evidence does not support the claim that 'DeepSeek behaves as a distillation of GPT'. The largest confusion in the chat register is between Qwen and GPT, at about 16%. Claude is the most identifiable family in both registers. A classifier trained on academic AI generated rewrites achieves only 18.6% on chat text from the same families, so a modelolect is specific to *register*, and to *model version*, and attribution requires training data from the register it will judge. As a byproduct we release inter-family vocabulary lexicons built with the log-odds method of our study on AI vocabulary. As a final experiment, after one TextPulse humanization, *p(human)* under the family classifier increases for 87 to 98% of the texts, a majority of the humanized texts classify as human, and the source family is recovered for at most 3% of them. All features, statistics, scores and code are made publicly available for future work. Files: the paper (PDF), per-text feature tables and classifier scores for every experiment, the confusion matrices and results, the per-family vocabulary lexicons, and the scripts and figures. Raw texts from the LMArena and HAP-E datasets are not redistributed; the released sampling code (fixed seed) reconstructs the exact samples from the public datasets.