Amplitude-Only FFN Intervention for Tool-Structured LLM Inference: Method, Gated Evaluation Protocol, and Cross-Model Empirical Results
TL;DR
An inference-time intervention that masks feed-forward network activation coordinates while keeping pretrained weights fixed provides a task-dependent intervention surface, with causal mechanisms, equal-budget advantages, repeated-run uncertainty, and serving cost still open.
Abstract
We study Amplitude Gating (AG), an inference-time intervention that masks feed-forward network (FFN) activation coordinates while keeping pretrained weights fixed. Six SwiGLU sites separate shared and gate-only inputs, expanded gate/value channels, and the FFN output. Magnitude-ranked entropy masks and a Newton-Schulz windowed ranking variant provide alternative forward paths without retraining. The corrected September collection covers three Qwen checkpoints and eight text/tool endpoints. Under frozen local text scorers, the displayed Qwen3.5-9B non-NS configurations improve two function-call endpoints by +2.61 and +2.67 percentage points (pp), and static ToolBench in/out-domain endpoints by +1.64 and +2.82 pp. Comparing the lower historical/September AG mean with the higher baseline mean leaves positive margins on these routes. This supports robustness to the two observed executions, rather than formal statistical significance. JSON gains are small and displayed L-Eval effects are negative. These endpoints measure lexical or answer/action-name agreement rather than executed-tool success. Protocol-specific offline selector and Per-Sample Fix-Harm Evaluation (PFHE) analyses complement the fixed-route results. Current experimental data do not establish a significant random-forest (RF) benefit; further experiments are needed. AG provides a task-dependent intervention surface, with causal mechanisms, equal-budget advantages, repeated-run uncertainty, and serving cost still open.
