Akshay S. Chaudhari

Gradient-Stable Attention Heads Signal LLM Correctness

This work introduces HeadEntropy, a method that predicts answer correctness by measuring how susceptible each attention head's pattern is to modification under further gradient updates, and evaluates broadly across 5 instruction-tuned LLMs and 5 datasets spanning general knowledge, multi-hop reasoning, mathematical rea...