Overall, legal information processing benefits from different inductive biases across tasks, with cross-architecture ensembling, feature-based reranking and retrieval-augmented prompting each proving most effective in different settings.
Abstract
Legal information processing spans retrieval, entailment and judgment prediction problems, requiring text matching, reasoning and robust generalisation with limited supervision. We report Team DU's participation in all five tasks of COLIEE 2026, using open-weight systems for legal case retrieval, case entailment, statute retrieval and entailment, and legal judgment prediction. For Tasks 3 and 4, all models predate the 15 July 2025 cutoff required by the rules. For Task 4 (statute entailment), a cross-architecture ensemble of nine models from three families achieves 96.3% accuracy, placing first among 33 submissions from 11 teams. For the Pilot Task (tort prediction and rationale extraction), a multi-view system combining five claim-level models and refining the verdict using features derived from the claim predictions achieves 73.1% TP accuracy and 68.2% RE F1 as an unofficial submission, scoring above all official entries on TP and matching the highest on RE. For Task 2 (legal case entailment), changing only the prompt from single- to multi-selection raises F1 from 0.343 to 0.555 in post-competition evaluation on released gold labels, exceeding the best official submission (F1 = 0.490). For Task 3 (statute retrieval and entailment), replacing the entailment model with Qwen3-235B and a structured legal reasoning prompt raises accuracy from 79.3% to 91.5% in post-competition analysis. For Task 1 (legal case retrieval), a learning-to-rank system combining lexical and semantic retrieval with structural, citation authority, and temporal features (34 in total) achieves F1 = 0.314 (rank 11 of 54 submissions from 22 teams). Overall, legal information processing benefits from different inductive biases across tasks, with cross-architecture ensembling, feature-based reranking and retrieval-augmented prompting each proving most effective in different settings.
This paper presents the methodologies and results of the NOWJ team's participation across all five tasks of the COLIEE 2026 competition and adopts a retrieval-augmented generation framework with dense retrieval, attention-based reranking, and few-shot-prompted LLM reasoning.
T. Ngo, Hoang-Trung Nguyen, Huu-Dong Nguyen et al.· arXiv.org· 0 citations
Automated verification of numerical claims is a challenging problem, as it requires both language understanding and quantitative reasoning. This paper describes our system for CLEF 2026 CheckThat! Task 2, which focuses on ranking reasoning traces generated by large language models (LLMs) and predicting a final verdict for numerical claims in English and Arabic. We explore two approaches. The first approach fine-tunes an LLM-based verifier using LoRA to score each reasoning trace independently as a binary classification problem, and selects the final verdict using Best-of-N selection. We further experiment with adaptive sub-claim decomposition to break complex claims into simpler parts before verification. The second approach uses a lightweight TF-IDF reward model with handcrafted numeric and temporal overlap features to score traces, and aggregates scores by verdict group to determine the final prediction. For Arabic, we compare a general multilingual model against AraBERT, a language-specific model pretrained on Arabic text. Our results show that the LLM-based approach outperforms the lightweight reward model on most metrics, particularly Recall@5, while the reward-based approach shows stronger performance on the Conflicting class. Sub-claim decomposition did not improve performance, suggesting that claim splitting introduces noise rather than aiding reasoning. For Arabic, AraBERT outperforms the multilingual baseline across most metrics.
Legal information retrieval demands high precision, yet traditional ''Retrieve-then-Rerank'' pipelines with two separate models suffer from cascading error propagation and knowledge disconnects between stages. To address these issues, we propose RULER, a Robust Unified LLM-based Efficient Retrieval that integrates efficient Bi-Encoder retrieval and high-precision Cross-Encoder reranking within a parameter-sharing architecture. To mitigate the Phantom Hits problem that irrelevant documents are assigned unreasonably high confidence, we introduce a Distribution-Robust Data Construction strategy that explicitly simulates pure-negative candidate groups. This is coupled with a Dynamic Margin Ranking Objective and Maximum Entropy Regularization, which collectively enforce uncertainty on irrelevant samples and enhance robustness. Extensive experiments on the JuDGE and LeCaRDv2 benchmarks demonstrate that RULER achieves state-of-the-art performance, outperforming all independent retrievers in retrieval tasks and surpassing competing unified architectures—where retriever and reranker parameters are shared—in high-precision reranking.
Chenyu Hou, Ziyang Wang, Bin Cao et al.· Annual International ACM SIG...· 0 citations
With more than 45 million cases awaiting disposal across Indian courts as of 2024, the judicial system faces an acute
need for faster, smarter tools to support legal research. This work introduces an artificial-intelligence-driven legal research
assistant tailored to the jurisprudence of the Supreme Court of India. Starting from a case description written in ordinary
language, the system executes a four-stage pipeline: first, it performs dense semantic search by encoding the 26,688 judgments
of the Indian Legal Documents Corpus (ILDC) with InLegalBERT and indexing roughly 1.1 million resulting text segments
through a FAISS IVFFlat structure; second, it applies cross-encoder reranking to narrow the retrieved candidates down to the
five precedents judged most semantically relevant; third, it forecasts the judicial outcome across three possible categories—
Allowed, Dismissed, or Partly Allowed—via an InLegalBERT classification head paired with SHAP explainability; and fourth, it
produces a structured reasoning summary in Issue–Rule–Application–Conclusion (IRAC) form using the Llama 3 8B model
served locally through Ollama. By combining semantic search, neural reranking, interpretable outcome prediction, and AIgenerated legal reasoning inside one coherent architecture, the framework helps legal practitioners locate relevant precedents,
anticipate probable case outcomes, and streamline the overall research process. Constructed entirely from openly accessible
Indian legal resources and transformer-based architectures, the system establishes an extensible base for intelligent legal
support within the Indian judiciary
Krish P. Gokhale, Tanvi Kshirsagar, Anugraha Kasbe et al.· International Journal for Re...· 0 citations
TabRank is presented, a framework for training reasoning rerankers for Tabular Retrieval that generalizes effectively to multi-table reasoning and significantly improves performance across a variety of table retrieval datasets.
Adarsh Singh, K. Bhandari, Jianxi Gao et al.· arXiv.org· 0 citations
Ontology learning from text remains challenging despite significant progress in Large Language Models (LLMs), which can hallucinate domain terms, produce inconsistent formats, and favor hierarchical over associative relations. In the LLMs4OL 2026 Challenge, we address both the End-to-End Flagship Task (Task A) and Ontology Extension Reuse Task (Task B) using an offline retrieval-augmented few-shot prompting pipeline. Our system employs Qwen2.5-14B-Instruct with all-MiniLM-L6-v2 for demonstration retrieval, selecting the top-5 examples for Task A and top-2 for Task B. A left-truncated context-windowing strategy preserves task instructions within long prompts. For Task B, generated triples undergo deterministic vocabulary-constrained filtering, retaining triples when at least one endpoint belongs to the sample's closed term/type vocabulary and removing duplicates of the initial ontology. The approach achieves Semantic Graph Similarity of 0.8692, Term-Typing F1 of 0.9200, and Taxonomy Discovery F1 of 0.8540 on Task B, while Task A achieves 0.7416 Semantic Graph Similarity. However, no non-taxonomic relations are extracted, highlighting limitations of closed, taxonomy-oriented relation vocabularies.
Shivam Mishra, Dhannu Ram Meena, M. Ojha et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
