Jun 2026· Annual Meeting of the Association for Computational Linguistics· pp. 29625-29643· 2 citations· 43 references
Computer Science
TL;DR
This paper introduces FinED-Bench, the first publicly public benchmark for FinED-Bench, which covers nine real-world financial scenarios, and includes over 900 documents reported in 2025 that are unseen by existing language models.
Abstract
Ensuring the accuracy of financial documents is critical for economic analysis, regulatory compliance, and corporate decision-making. Several studies have shown that Large Language Models (LLMs) perform well in many financial tasks, such as stock price movements and financial analytics. However, a critical task remains unexplored: the ability of LLMs to identify errors in financial documents. In this paper, we introduce \textbf{FinED-Bench}, the first publicly \textbf{Bench}mark for \textbf{Fin}ancial \textbf{E}rror \textbf{D}etection across three levels of cognitive complexity. FinED-Bench covers nine real-world financial scenarios, and includes over 900 documents reported in 2025 that are unseen by existing language models. We detail the benchmark construction process and evaluate several advanced LLMs (e.g., GPT-4o, Qwen3-14B) on this tasks, which requires both financial domain knowledge and reasoning capabilities. Experimental results show that current LLMs still struggle with this task, especially in high-complexity cases. Besides, supervised fine-tuning can significantly improve the performance of weaker LLMs on this task. Our data and code are available at https://github.com/hedyHe/FinED-Bench.
The comparative analysis of banks'financial statements poses significant challenges for automated question answering systems due to their complexity, substantial length, technical language, and inhomogeneity of both textual and numerical content across different jurisdictions and institutions. We introduce FinRAG-QA, a novel benchmark dataset for financial question answering, which comprises 999 practitioner-curated questions on 10 standardised indicators, grounded in 209 annual and Pillar 3 reports from 24 major European and U.S. banks spanning 2019-2023. Unlike prior financial QA benchmarks, which centre on U.S. filings and single-institution analysis, FinRAG-QA targets cross-institutional retrieval over documents averaging 198k words, longer than any existing financial QA resource. On this benchmark we evaluate a multi-stage RAG pipeline and isolate the contribution of each component. Contextual chunk enrichment combined with a retrieval-optimised embedding model raises NDCG@10 from 0.322 to 0.710; conditional on the ground truth being retrieved, a reasoning-optimised generator raises answer accuracy from 44.6% to 79.0% (+34.4 percentage points), at roughly 20x the generation latency. We further show that cross-encoder reranking degrades retrieval when the first-stage ranking is already strong, and that a single top-ranked chunk outperforms larger contexts at generation time. Experiments were run in late 2024-early 2025 with the models available at that time.
Arianna Miola, Bruno Spaccavento, Lorenzo Silotto et al.· 0 citations
This work constructs and makes publicly available a comprehensive U.S. company dataset combining financial statements, summarized MD&A text, and fraud labels and achieves the best performance on the challenging CI-FSFD task, demonstrating the critical value of textual data and robust evaluation for reliable financial fraud detection.
Guy Stephane Waffo Dzuyo, Gaël Guibon, Christophe Cerisara et al.· arXiv.org· 0 citations
The Financial Regulatory Office has added significantly to its database of text data but has not increased staff. Large language models have strong natural language understanding and generation abilities, and a new technical path for automating regulatory Q&A and intelligent risk screening has been provided. Build an intelligent assistant system for financial regulation based on large language models in this paper, and introduce a regulatory question-answering module and a risk-screening module. A controlled experiment was carried out in the actual work environment of a local financial regulatory office during the 2025 fiscal year. According to the above experimental results, LLM assistance has reduced the regulatory question response time by 82.6%, improved the accuracy of risk screening by 19.3 percentage points, increased the average daily effective workload of regulatory staff to 2.8 times, and raised the overall regulatory efficiency index by 156.7%. Provide empirical support and operational references for the financial regulatory authorities in utilising artificial intelligence technology to enhance the efficiency of supervision.
Zhen-Yu Luo· International journal of com...· 0 citations
This project built a pipeline that pulls news articles from the News API, company background from Wikipedia, and stock price data from Yahoo Finance for ten major companies and developed a simple but effective template that converts stock data into natural language narratives.
It is concluded that while LLMs hold genuine promise within AI trading systems, robust deployment requires careful task decomposition, rigorous backtesting protocols, and domain-aware fine-tuning strategies.
Financial AI systems often train information extractors on one textual register and deploy them across filings, news, and user-generated content, and standard F1 scores do not indicate which predictions remain safe to automate when that input distribution changes. We study confidence estimation and selective prediction for financial named entity recognition (NER) on a three-tier stress test spanning SEC filings, financial news, and general-topic social media as an extreme out-of-domain condition, evaluating a BERT tagger and LoRA-tuned Qwen2.5-0.5B/1.5B models with five inference-time confidence signals, three training seeds, and bootstrap intervals. Confidence rankings themselves change under shift: whole-output probability is the strongest in-domain error detector but deteriorates out of domain, whereas entity-span probability and self-consistency are more robust; self-consistency is also better calibrated without post-hoc fitting. Abstention reduces sentence error from 34.3% to below 2% on the highest-confidence 40% of in-domain inputs and remains useful on financial news, but recovers no usefully large clean subset under the extreme social-media shift. These results motivate a staged deployment strategy that detects severe distribution shift upstream before applying prediction-level confidence gating.
Zihao Zheng, Baichuan Li, Junyi Yao et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
