Asif Ahmed Essak

Diagnosing Multi-Head Self-Attention: An Information-Theoretic Framework with Application to Time-Series Forecasting

Background: Multi-head self-attention is central to Transformer-based time-series forecasting, yet its head-level information-selection behavior lacks a unified information-theoretic characterization. How much information a single head selects, how inter-head redundancy should be measured, and under what conditions a h...