Sep 2026· Zenodo (CERN European Organization for Nuclear Research)
Abstract
Summary This release documents a working method for injecting a chosen concept into a large language model's internal computation, at inference time, without changing the model's weights or its prompt. The starting point was AkbasCore's existing steering mechanism (Damped Resonance Alignment, "DRA"), which nudges a model's internal state toward a target direction using a decaying push across its early layers. Porting that mechanism to a larger 7B model produced no effect at all — the push was being silently swallowed. Diagnosing why led to two fixes that make up this release: 1. The push amount is now set directly, not derived from a similarity score. The original design scaled the push by how well-aligned the internal state already was with the target — measured by cosine similarity. At larger model sizes this similarity score turns out to collapse toward zero for structural reasons (a small number of internal dimensions dominate the model's activity and swamp the measurement), so the push collapsed with it. The fix: the push is now set as a direct, fixed percentage of the internal state's own magnitude, and the similarity score is used only as a safety brake, not as the control knob. 2. The injected direction is built differently. Directions built from the model's vocabulary embeddings — which worked at smaller scale — do not reliably carry the intended concept at 7B. The fix: the injection direction is now extracted from the model's own internal activity while it processes matched positive and negative example sentences, separately for each layer. What this release shows, with measurements: The injected amount can now be dialed precisely (e.g. "inject 10% of the internal magnitude") and the effect scales with that dial. Two different target concepts, injected in the same setting, each move only their own target and leave the other unaffected — a controlled, direction-specific effect rather than a general disruption. The result is reproducible byte-for-byte across independent runs. What this release does not claim: The technique has only been tested on one model family and size. The effect depends on the surrounding context — it biases what the model was already likely to say, it does not force new content regardless of context. There is a real tension between "safe" injection strength and "clearly effective" injection strength that is not yet resolved. Concept presence is currently measured by keyword frequency, not by human judgment. The attached PDF is the full technical write-up: complete mathematics, all 23 experimental runs with raw output logs, the failure modes discovered along the way, and a full accounting of what remains unproven. See the repository README (Development History → Phase V) for how this fits into the project's history, and the paper's own Limitations and Reproducibility sections for the precise scope of what this release does and does not establish.
Supporting data, adapters, predictions and code for the article *Low-Cost LoRA Fine-Tuning of Small Language Models for Multi-Step Arithmetic Reasoning* by Jake O'Grady, Asena Isik Gürhan, Chee Fong Ting and Effirul Ramlan (University of Galway). We generated 20,000 GSM8K-derived arithmetic problems with step-by-step s...
O'Grady, Jake, Gürhan, Asena Isik, Chee, Fong Ting et al.· Zenodo (CERN European Organi...· 465 citations
The results are packaged in the Greenfield Startup Model (GSM), which explains the priority of startups to release the product as quickly as possible, and the need to shorten time-to-market, by speeding up the development through low-precision engineering activities.
Carmine Giardino, Nicolò Paternoster, M. Unterkalmsteiner et al.· IEEE Transactions on Softwar...· 178 citations· ⚡14
Software startup companies develop innovative, software-intensive products within limited timeframes and with few resources, searching for sustainable and scalable business models.
M. Unterkalmsteiner, P. Abrahamsson, Xiaofeng Wang et al.· e-Informatica Software Engin...· 157 citations· ⚡17
This study conducts a case survey study based on the secondary data of the major pivots happened in 49 software startups, and demonstrates that customer need pivot is the most common among all pivot types.
Sohaib Shahid Bajwa, Xiaofeng Wang, Anh Nguyen-Duc et al.· Empirical Software Engineeri...· 127 citations· ⚡15
The comparison of adopter and non-adopter sample reveals three potential adoption inhibitor, security, data privacy, and portability, which underlines the importance of the technical and security perspectives for research investigating the adoption of technology.
Nattakarn Phaphoom, Xiaofeng Wang, S. Samuel et al.· Journal of Systems and Softw...· 111 citations· ⚡8
This study investigates how Lean internal startup facilitates software product innovation in large companies and identifies its enablers and inhibitors, and shows the potential of the method-in-action framework to investigate the Lean startup approach in non-startup context.
Henry Edison, Nina M. Smørsgård, Xiaofeng Wang et al.· Journal of Systems and Softw...· 78 citations· ⚡6
Related blog posts
Microsoft Research Blog· microsoft.comOct 12, 2026
StudentSim is a research system for evaluating and improving AI tutoring tools before they are tested with learners. It models how individuals may engage with tutors, giving feedback to improve tutor performance across subjects like chess, writing, and mathematics. The post Simulating student learning for better AI tutoring appeared first on Microsoft Research.