The findings show that QLoRA is effective for financial sentiment adaptation, while also documenting a clear gap between classification accuracy and tradable cross-sectional signals.
Abstract
Financial sentiment classifiers are commonly evaluated against human labels, but strong linguistic performance does not necessarily imply economically useful return predictability. This study separates these questions through two experiments. First, we construct a unified three-class benchmark from five financial text datasets and compare TF--IDF Naive Bayes, off-the-shelf FinBERT and Financial-RoBERTa encoders, zero-shot Qwen2.5-7B, and QLoRA-adapted Qwen2.5-7B, LLaMA3-8B, and Mistral-7B models. Mistral-7B achieves the best test accuracy (0.8840) and macro-F1 (0.8771), while QLoRA raises Qwen2.5's macro-F1 from 0.7274 to 0.8615. An inverse-frequency class-weighted loss does not improve Qwen2.5. Second, we evaluate economic validity on a temporally separate 2019 Benzinga sample containing 10,637 unique headlines and 13,115 headline--stock observations for a fixed S\&P~100 universe. Model probabilities are converted into continuous sentiment scores, aggregated by stock and signal date, and aligned with next-session returns over one-, two-, three-, and five-day horizons. All seven downstream models produce positive but small mean rank information coefficients at the one-day horizon; the largest is 0.0143 for FinBERT. None of the 28 model--horizon tests remains significant after Newey--West inference and false-discovery-rate correction. Portfolio results likewise fail to establish a robust advantage for the best-performing classifiers. The findings show that QLoRA is effective for financial sentiment adaptation, while also documenting a clear gap between classification accuracy and tradable cross-sectional signals.
Financial sentiment analysis converts unstructured financial news into quantitative signals that can support market analysis and decision-making. Existing work on resource-efficient financial NLP has largely focused on compressing or adapting pretrained language models, with less attention to combining contextual representations with lightweight rule-derived features. This study develops Rule-Aware FinBERT (RA-FinBERT), a parameter-efficient framework that integrates low-rank adaptation (LoRA) with three continuous VADER-derived sentiment proportions (positive, negative, and neutral) and a source-level metadata feature. The standardized four-dimensional feature vector is directly concatenated with the 768-dimensional final-layer FinBERT [CLS] representation and passed through a lightweight classification head. This design introduces only 1,024 additional trainable weights relative to a structurally matched text-only FinBERT model. RA-FinBERT was evaluated against text-only FinBERT and a lightweight DistilBERT baseline for three-class sentiment classification of financial-news titles and descriptions. On the held-out test set, RA-FinBERT achieved 69.89% accuracy and a macro F1 score of 0.634, compared with 63.44% and 0.526 for text-only FinBERT. Neutral-class recall increased from 18.18% to 45.45%. The framework supports both CPU and GPU execution, offering a lightweight and practical approach to financial sentiment classification under constrained computational resources. These findings indicate that rule-derived sentiment information and source metadata can provide complementary signals to contextual FinBERT representations and improve performance with minimal additional model complexity.
Measuring sentiment from financial news is a central task in economics and finance, yet most existing indicators rely on dictionary-based approaches that infer sentiment from word counts and only partially capture context, negation, and semantic structure. This paper proposes a framework for constructing daily news mood indices using transformer-based language models and evaluates whether they better represent sentiment than dictionary-based alternatives. Using 143,755 financial news articles from Factiva, we classify sentiment at the sentence level with FinBERT and aggregate these predictions into article-level and daily sentiment measures through alternative normalization schemes. We compare the resulting indices with benchmark measures based on Shapiro et al., 2022 and Barbaglia et al., 2025. A central contribution is the validation of alternative sentiment measures against human judgments. We conducted an incentivized annotation exercise in which 444 participants evaluated a validation subsample of 588 financial news articles. Consensus ratings from independent human evaluations serve as an external benchmark for assessing the quality of automated sentiment measures. Across correlation, regression, and classification exercises, transformer-based measures show stronger agreement with human judgments than vocabulary-based alternatives and perform substantially better in distinguishing positive, neutral, and negative articles. Overall, the results suggest that incorporating contextual information through transformer-based language models produces sentiment measures that more closely reflect human assessments of financial news.
Maria Saveria Mavillonio, Stefano Borgioli, C. Giannetti et al.· 0 citations
Overall, LLMs offer a flexible and annotation-light alternative to traditional fine-tuning, but their successful deployment in finance will depend on robust prompt engineering and solid factual grounding.
Liyang Gao· Applied and Computational En...· 0 citations
Sentiment analysis identifies opinion polarity in textual data and supports decision-making in review-rich digital environments. This study compares three approaches for binary sentiment classification on the Internet Movie Database (IMDb) movie review dataset, namely Term Frequency-Inverse Document Frequency (TF-IDF)+Naive Bayes, TF-IDF+Logistic Regression, and DistilBERT, a distilled version of Bidirectional Encoder Representations from Transformers (BERT). The comparison focuses on the performance-efficiency trade-off rather than accuracy alone. Models are evaluated by accuracy, precision, recall, F1 score, training time, and inference time under 20%, 50%, and 100% training-data settings. A representative rule-based error analysis is also conducted. Under the full training-data setting, DistilBERT achieves the highest F1 score of 0.914496, ahead of Logistic Regression at 0.883284 and Naive Bayes at 0.846688. However, relative to Logistic Regression, DistilBERT is approximately 175.69 times slower in training and 51.05 times slower in inference. DistilBERT trained with only 20% of the training data still exceeds the full-data F1 scores of both traditional baselines. The findings indicate that model selection should jointly consider effectiveness, efficiency, data scale, and error characteristics.
Lekang Sun· Applied and Computational En...· 0 citations
It is taken as initial evidence for market time series as an input modality in financial text classification on the task of classifying sentences from Federal Reserve communication as hawkish, dovish, or neutral.
Michael Schlee, Fabian Lukassen, Christoph Weisser· 0 citations
— By comparing and analysing the performance of several machine learning algorithms on fine-grained sentiment classification problems to examine their suitability and shortcomings for use as models in sentiment analysis. Sentiment analysis remains a prominent research area in Natural Language Processing (NLP). However, systematic comparisons of whether these methods demonstrate superiority in fine-grained classification settings have yet to be conducted. Based on the GoEmotions dataset that includes 28 sentiment classes as our experiment’s reference for evaluating four kinds of models: logistic regression, BiLSTM, BERT, and the large-scale language model Claude (claude-haiku-4-5). The experiments employ TF-IDF feature extraction, sequence encoding, pre-training with fine-tuning, and zero-shot and few-shot prompting strategies. Based on this experiment, the model’s generalisation capabilities improve as architectural depth increases. The accuracy of logistic regression was only 55.11%; BERT obtained the highest F1 score and performed the best overall, and bidirectional pre-trained representations were most valuable among other methods. However, large language models perform significantly worse on the 28-class classification task in zero-shot settings, suggesting that they are better suited for generative and open-ended emotional interaction than for standardized classification benchmarks.
Shangjiafeng Guo· International journal of eng...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
