Toolkit for Confidence-Corpus Consistency, Corpus Absorption and Rule Learning via Fine-Tuning on a Fabricated Corpus
An open toolkit that fine-tunes a small causal language model on a corpus asserting one fabricated arithmetic answer for each of the 81 single-digit addition pairs, and compares the model's post-fine-tuning confidence in each fabricated answer against its own pre-fine-tuning confidence in the corresponding true answer,...