The Sirens'Song: When Proximal Background Context Overshadows Distant Evidence
This work identifies the Proximity Trap, and introduces LYRA (Long-context heavY-tailed Relevance Alignment), a t-distributed directional matching mechanism that reshapes the context retrieval distribution, directing more attention mass toward task-relevant evidence, while preserving the relative positional information...