Compiling Autoregressive LLM Control into Policy-Exact Parallel Inference for Hierarchical RAG
Large language models increasingly act as controllers in structured decision pipelines, yet their decisions are still executed by autoregressive text decoding. Hierarchical RAG is one such setting: at each frontier of a document tree, an LLM commits to $b$ of $k$ sections under an input-dependent action grammar, execut...
Zehao Lin, Qian-Feng Shen, Xi-Xuan Hao et al.
· 0 citations