Estimating Rare Events in Language Models with Proper Evaluation
This work introduces Gradient Activation Adaptive Multi-Level Splitting (GA-AMLS), which adapts rare-event Monte Carlo methods to the continuous activation space of language models and establishes activation space as a tractable domain for rare-event estimation in language models, circumventing the brittleness of discrete input-space search.