FLoRE: A Formal Language Benchmark for Robustness and Effectiveness Evaluation of LLM Reasoning
Large language models (LLMs), especially recent reasoning-oriented models trained with reinforcement learning, achieve near-saturating accuracy on many reasoning benchmarks. However, it remains unclear whether these models truly reason over semantic structure or rely on superficial symbol patterns and familiar language...