SyntaxBench: A Statistical Diagnostic Framework for Character-Level Reasoning in Large Language Models
Large language models are increasingly used where small syntactic errors matter, yet character-level reasoning is still evaluated mostly through isolated probes and aggregate accuracy. We introduce SyntaxBench, a diagnostic benchmark and statistical evaluation framework for character-level reasoning. It contains five c...
Mohsen Larni, Sobhan Ebrahimi Azar, Pouyan Nahed et al.
· 0 citations