Spider 2.0-AIFunc: Extending Real-World Text-to-SQL to AI-Native SQL Workflows
This work introduces Spider 2.0-AIFunc, a benchmark of 465 verified instances across 125 real-world databases covering six types of AI functions on the Snowflake platform, and finds that the strongest proprietary models reach 67-70% execution accuracy while the best open-source model achieves 58.1%, a gap driven primarily by errors in predicate specification, schema grounding, and AI function parameterization.