InfoMamba: An Attention-Free Hybrid Mamba-Transformer Model
A consistency boundary analysis is presented that characterizes when diagonal short-memory SSMs can approximate causal attention and identifies structural gaps that remain and proposes InfoMamba, an attention-free hybrid architecture that consistently outperforms strong Transformer and SSM baselines.