The Parser Already Knows: Lightweight Bias Correction in Constrained Decoding
SHIM is introduced, a lightweight, offline-trained correction of the LM's next-token probabilities, conditioned on this syntactic and lexical state together with candidate next tokens, which substantially narrows the gap to the LM's grammar-conditioned distribution compared to masking and online sampling, while running...
Işıl Özgü, Yaoxuan Wu, Guy Van den Broeck et al.
· 0 citations