CARAT: Do Materials LLMs Reason or Recite?
When a materials LLM answers a question about crystal structure, does it reason from the structure or copy an answer already printed in its input? Accuracy cannot tell: a structural description often prints the very field it is scored against. CARAT holds question and gold answer fixed across eight matched views, names...
Jia-Jun Wu, Jian Yang, Zi-Xiang Ni et al.
· 0 citations