Skip to content
#small language model Open access

Self-Referential Processing in Large Language Models: A Pre-Registered Multi-Model Evaluation Protocol (v8.6.3)

Sep 2026 · Open Science Framework

Abstract

Pre-registration protocol and frozen materials for Master Lock v8.6.3. The specification of record is MASTER_LOCK_v8_6_3.md; any rendered PDF is derived from it. The frozen 260-item prompt pool (items_pool_v1.json) is locked to SHA-256 128ff78bb6beb1640fa51897ab97dbb9d07dc56de276973d05a48dda524b53d3. The statistical estimator is estimator.py (Cameron–Gelbach–Miller two-way cluster-robust OLS with eigenvalue clipping to zero for positive semi-definiteness). The recovery harness has been executed at 1,000 seeds per scenario and passes all §5.9 criteria (bias ≤ 0.02, null rejection in [3%, 9%], sign-flip ≥ 99%, seed floor ≥ 100, no NaN standard errors). Both rendered PDFs were verified against the lock file (109 REQ present, 15 BAN absent, 4 OPEN pending pre-collection tasks). All 13 files are hashed in MANIFEST.sha256. Version history: v8.6.3 widens the §5.9 null-case Type-I error acceptance band from [3%, 7%] to [3%, 9%] and raises the recovery harness seed count from 400 to 1,000 per scenario, both before any confirmatory data collection. The widened band reflects known small-G finite-sample over-rejection in Cameron–Gelbach–Miller two-way cluster-robust inference (G = 12, df = 11). estimator.py is unchanged from v8.6.2. A small-G degrees-of-freedom correction was considered and set aside to preserve estimator immutability across versions. Null rejection rate showed environment-level variation (observed 6.3%–7.6% on identical seeds) attributable to floating-point differences in LAPACK/BLAS eigendecomposition; both values sit inside the pre-registered band. No confirmatory model queries have been executed. The two OPEN operational items (closed-API endpoint verification and OSF deposit/DOI minting) will be closed on filing.

View source

Similar papers

#small language model Dataset Open access Oct 2026

Socratic guiding questions in synthetic arithmetic data: matched LoRA runs (revision v2)

Supporting data, adapters, predictions and code for the article *Low-Cost LoRA Fine-Tuning of Small Language Models for Multi-Step Arithmetic Reasoning* by Jake O'Grady, Asena Isik Gürhan, Chee Fong Ting and Effirul Ramlan (University of Galway). We generated 20,000 GSM8K-derived arithmetic problems with step-by-step s...

O'Grady, Jake, Gürhan, Asena Isik, Chee, Fong Ting et al. · 465 citations
#computer vision Open access Jun 2016

Software Development in Startup Companies: The Greenfield Startup Model

The results are packaged in the Greenfield Startup Model (GSM), which explains the priority of startups to release the product as quickly as possible, and the need to shorten time-to-market, by speeding up the development through low-precision engineering activities.

Carmine Giardino, Nicolò Paternoster, M. Unterkalmsteiner et al. · 178 citations · ⚡14
#computer vision Open access Oct 2016

Software Startups - A Research Agenda

Software startup companies develop innovative, software-intensive products within limited timeframes and with few resources, searching for sustainable and scalable business models.

M. Unterkalmsteiner, P. Abrahamsson, Xiaofeng Wang et al. · 157 citations · ⚡17
#machine learning Review Open access Oct 2016

“Failures” to be celebrated: an analysis of major pivots of software startups

This study conducts a case survey study based on the secondary data of the major pivots happened in 49 software startups, and demonstrates that customer need pivot is the most common among all pivot types.

Sohaib Shahid Bajwa, Xiaofeng Wang, Anh Nguyen-Duc et al. · 127 citations · ⚡15
#computer vision Review Open access May 2015

A survey study on major technical barriers affecting the decision to adopt cloud services

The comparison of adopter and non-adopter sample reveals three potential adoption inhibitor, security, data privacy, and portability, which underlines the importance of the technical and security perspectives for research investigating the adoption of technology.

Nattakarn Phaphoom, Xiaofeng Wang, S. Samuel et al. · 111 citations · ⚡8
#computer vision Open access Feb 2018

Lean Internal Startups for Software Product Innovation in Large Companies: Enablers and Inhibitors

This study investigates how Lean internal startup facilitates software product innovation in large companies and identifies its enablers and inhibitors, and shows the potential of the method-in-action framework to investigate the Lean startup approach in non-startup context.

Henry Edison, Nina M. Smørsgård, Xiaofeng Wang et al. · 78 citations · ⚡6

Related blog posts

MIT News · Artificial Intelligence Oct 15, 2026

AI model decodes the language cells use to communicate

A new computational approach reveals how cells communicate to assemble into tissues. This could accelerate stem cell engineering, improve disease models, and inform therapies.

Microsoft Research Blog Oct 15, 2026

Proof-based general-purpose computing with Magnetar

Zero-knowledge proofs can verify computing results without rerunning workloads or exposing sensitive data. Magnetar helps bring them into the mainstream with support for standard programming languages, major speed gains, and quantum-resistant security. The post Proof-based general-purpose computing with Magnetar appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.