Burcu Hudaverdi

Does CoT-Pass@k Really Check the CoT? A Multilingual Mathematical Audit

Pass@k measures whether a model reaches a correct answer under repeated sampling, but never how: a lucky guess counts the same as sound reasoning. CoT-Pass@k was proposed to close that gap, adding an LLM-as-judge that must assess a solution's reasoning chain before it counts. Its value rests entirely on one assumption:...