From Transfer Probes to Expert Grouping: Evidence from LoRA Post-Training

Abstract

A platform that post-trains one base model for many business lines often has more lines than experts it can maintain, so lines must share experts, yet lines seldom come with a taxonomy saying which should share. We ask whether transfer probes (train on one source, measure paired changes on all sources) can discover a grouping when the standard one is unknown. To test probes against a reference, we use seven public sources that do have a label tree (five mathematics subfields and two kinds of programming, standing in for business lines) and treat the tree's grouping as a label reference that the evaluator holds and the method does not. With four LoRA experts under fixed per-source exposure and per-expert budgets, we compare a purely probe-derived grouping and an uninformed uniform default (whose tied routing serves every line from one expert) against this reference. Trained once for one interval from a shared warm-up state and routed by a rule fixed by each allocation, neither reached the prespecified practical gain of 2 percentage points, and both had lower overall utility than the reference ($-2.43$ and $-2.54$ pp; 95\% intervals below zero). In a post hoc comparison, the probe-derived grouping was indistinguishable from the uniform default ($+0.11$ pp, $[-1.25,+1.54]$). In fixed-budget LoRA post-training, a grouping built from initial-state single-source probes thus did not deliver the advantage predicted from the probes once carried to a later training state, a finite budget, and specified routing.