Citation Intent Classification via Exponential Borda Fusion and SciBERT

Abstract

This study presents a clear and reliable framework for classifying the intent behind scientific citations. It combines multi-model reasoning with concepts from social choice theory. Instead of using a single model, this framework employs three open Large Language Models Gemma, LLaMA, and Mistral. Additionally, we combine their ranked outputs using an exponentially weighted Borda method. By doing so, this approach increases agreement among high-confidence predictions, maintains ranking information, and produces stable, high-quality supervision signals. Consequently, it boosts reliability while remaining transparent. To create a strong experimental basis, we built a large, balanced dataset from the UnarXive corpus, which contains structured full-text scientific articles and citation networks. First, we automatically pulled citation contexts and organized them within a DuckDB-based analytical setup. Then, we rebalanced the dataset across rhetorical categories to enhance representativeness and minimize bias. Finally, we categorized each citation context into one of five roles: background, methodology, comparison, extension, or critique. As a result, the resulting dataset provides a robust foundation for training and evaluation. We trained a SciBERT classifier using these ensemble-generated annotations and tested it on a five-category citation intent classification task. The model achieved a macro F1-score of 0.83, an outstanding result for this type of classification. Indeed, this level of performance shows strong reliability given how challenging it is to differentiate closely related citation functions. Moreover, it demonstrates that combining multiple models produces valuable and distinct supervision signals, capturing subtle rhetorical and semantic patterns that single models often overlook. Furthermore, the framework enhances interpretability. Specifically, the explicit weighting system clarifies how each model contributes to the final outcome. In addition, the deterministic tie-breaking method ensures the outputs are consistent and reproducible. Taken together, these design choices maintain explainability without sacrificing effectiveness.