Skip to content
Conference Open access

RegTrack: A Fine-Grained Benchmark for Multi-Class Legal Change Detection

2026 · Annual Meeting of the Association for Computational Linguistics · pp. 764-778 · 0 citations · 45 references
Computer Science

TL;DR

A novel benchmark for change detection in EU regulations is introduced that comprises 4,772 manually annotated pairs of structurally distinct provisions, defined as Atomic Legal Units (ALUs), mapped to a six-class taxonomy of legal change types.

Abstract

Organizations must continuously monitor evolving regulations to maintain compliance. While current tools are limited to surface-level text comparison, existing models lack the fine-grained classification schemes to determine whether small changes impact legal obligations or merely update formatting. To address this gap, we introduce a novel benchmark for change detection in EU regulations. It comprises 4,772 manually annotated pairs of structurally distinct provisions, defined as Atomic Legal Units (ALUs), mapped to a six-class taxonomy of legal change types. We formalize three core tasks: structural alignment, change classification, and a combined task requiring simultaneous alignment and classification. Evaluating lexical algorithms, dense encoders, and Large Language Models (LLMs) as baselines, we find LLMs excel at isolated change classification, whereas domain-specific dense encoders offer the most robust combined performance. By providing fine-grained labeled data, this benchmark enables the development of AI systems that can help organizations analyze regulatory shifts and support version-aware retrieval in the legal domain.

Read PDF

Similar papers

Book Open access Aug 2026

Label Annotation for Tabular Anomaly Detection with Large Language Models

Tabular Anomaly Detection (TAD) plays a fundamental role in securing real-world applications. Despite rapid advances in TAD, the prohibitive cost of human-centric label annotation remains a primary bottleneck for large-scale production systems. To alleviate this bottleneck, we propose a novel ''coarse-to-fine'' label annotation pipeline to improve labor efficiency through a coarse-grained label annotation and fine-grained human verification. Specifically, Large Language Models (LLMs), with their strong cross-domain capabilities, serve as a promising solution for the coarse-grained annotation stage. However, effectively generalizing LLMs to coarse-grained annotation remains challenging due to the inability to ground semantic priors in rigorous deduction, as well as the overfitting risks inherent in single-domain fine-tuning. Accordingly, we introduce TaDGeneral, a large-scale cross-domain corpus constructed by fusing deductive reasoning paths from diverse domains. This design bridges the reasoning gap while preventing the memorization of local shortcuts. Building upon this, we develop TaDFM, a foundation model tailored to internalize generalizable deductive logic for effective zero-shot annotation. Extensive experiments on both public and large-scale real-world TAD datasets demonstrate the superiority of TaDFM over representative methods, with its practical value further validated by an industrial case study. Code: https://github.com/cshhzhao/TaDFM.

Haihong Zhao, Aochuan Chen, Miao Peng et al. · 0 citations
Review Jul 2026

Semantics of Subterfuge: Benchmarking Legal Deception Detection Against General-domain State-of-the-Art

A survey and comparative analysis of NLP-based Automatic Deception Detection focusing on the legal domain and the evolution from feature-based machine learning to Large Language Model (LLM) approaches are presented, showing strong domain sensitivity.

T. Samaradiwakara, Nisansa de Silva, George C. Lobb · 0 citations
Jul 2026

Diagnosing Fine-Grained Inconsistency Classification in Financial Disclosure Text

Findings identify evidence localization and fine-grained type discrimination as distinct challenges and show that compact supervised encoders are strong baselines for this task.

Aman Kumar, Lasitha Vidyaratne, Dipanjan Ghosh et al. · 0 citations
Jul 2026

ExReg: Using Automated Example Generation to Guide Candidate Disambiguation and Validation in Regular Expression Construction

ExReg, a human-in-the-loop workflow that automatically generates discriminative examples using SMT-based constraint solving, highlights how automated example generation guided by formal methods and mutations can improve the reliability, efficiency, and trustworthiness of LLM-assisted regex pattern generation.

Hadar Muchtar, Ilia Shevrin, S. Maoz · 0 citations
Preprint Aug 2026

ANNOTARES: A Dataset for Extracting Logical Structures from German Statutory Texts

This paper introduces the task of identifying and segmenting legal conditions (Tatbestand) and legal consequences (Rechtsfolge) within German statutory texts and presents ANNOTARES (Annotations of Tatbestand-Rechtsfolge Sequences), a novel dataset comprising German law texts with span-level annotations.

R. Schwarz, Jannik Strötgen · 0 citations
Preprint Jul 2026

Constraint-Aware Hierarchical Search for Regulation-Driven Fine-Grained Classification

Tasks such as customs tariff classification, export control categorization, and standards-based equipment coding require assigning an input instance to a fine-grained class under an explicit regulatory hierarchy. Unlike standard text classification, the correct label in these tasks is not determined by semantic similarity alone, but by rule-defined boundaries, threshold conditions, exclusion clauses, definitions, and local exceptions. As a result, two highly similar inputs may require different labels, while a retrieved passage that appears relevant may still be inapplicable under the governing rules. Existing flat classifiers, hierarchical text classification methods, and retrieval-augmented LLM systems are not designed to jointly enforce hierarchical validity, rule consistency, and fine-grained boundary reasoning. In this paper, we formulate this setting as regulation-driven fine-grained hierarchical classification, where an external instance must be assigned to a fine-grained class through a valid path in a regulatory hierarchy and supported by auditable evidence. We construct four benchmark datasets from representative regulation-intensive scenarios and validate the annotations through an expert-in-the-loop process. We further propose a constraint-aware hierarchical search framework that converts regulatory documents into a searchable tree, retrieves only valid local candidate nodes, and uses structured regulatory fields with evidence snippets to guide each next-hop decision. Experiments show that our method achieves the best mean accuracy on all four datasets and provides interpretable decision paths, with the largest gains on cases involving fine-grained neighboring categories and rule-based boundary conditions.

Siyu Wang, Weiwu Tan, Lu-Lu Chen · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.