This approach combines (i) role-set injection in prompts for schema compliance, (ii) parameter-efficient supervised fine-tuning (LoRA) using the same JSON-only interface used at inference, and (iii) deterministic decoding with post-processing that validates JSON, filters invalid roles, de-duplicates arguments, and aligns spans to the document window.
Abstract
Event Argument Extraction (EAE) converts documents into structured event records by identifying argument spans and assigning them schema-defined roles. Document-level EAE is challenging due to long-range dependencies between triggers and arguments, cross-sentence context, and strict role constraints, which often lead to boundary errors, uncertainty in roles, and inconsistencies with restricted schemas. In this paper, we study whether mid-sized open LLMs can perform schema-constrained EAE reliably at the document level on MAVEN-ARG. Our approach combines (i) role-set injection in prompts for schema compliance, (ii) parameter-efficient supervised fine-tuning (LoRA) using the same JSON-only interface used at inference, and (iii) deterministic decoding with post-processing that validates JSON, filters invalid roles, de-duplicates arguments, and aligns spans to the document window. Under the official MAVEN-ARG evaluator, fine-tuned mid-sized open models outperform previously reported GPT baselines across mention, entity-coreference, and event-coreference evaluations; our best model (Phi-4, 14B) reaches 42.39\% F1 at the event-coreference level. Code to reproduce experiments is publicly available at https://github.com/dessertlab/EAE/.
Document-level relation extraction (DocRE) finds relations across a whole document. It often needs evidence from several sentences. It also needs to link repeated entity mentions and follow multi-hop clues. Many large language model (LLM) based DocRE methods use fixed relation descriptions. They make little use of confident errors or samples with missing labels. We propose SPO-PA, which combines self-correcting prompt optimization (SPO) and preference alignment (PA). SPO compares LLM predictions with reference annotations. It then revises relation descriptions, clarifies relation boundaries, and adds role constraints. PA uses false negative (FN) conflicts as the main supervision signal. It also tests false positive (FP) conflicts under similar constraints. PA applies source-specific constraints to these signals. The constraints help smaller DocRE models separate correct relations from wrong candidates. On Re-DocRED, SPO improves F1 to 26.23% and Precision to 41.93%. It also achieves 37.60% F1 on HacRED. Both are the best LLM-only scores in our comparison and reproduce on a second, more recent backbone, though both stay far below supervised encoders; the extraction stage supplies conflict signals rather than final predictions. Among the PA variants, PAFN\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\text {PA}_{FN}$$\end{document} gives the most stable gains on ATLOP, KD-DocRE, DREEAM, and DAATF. The largest gain is 2.89%.
Fu-Yuan Wei, Wen-Zhong Yang, Ti-Quan Gu et al.· Journal of King Saud Univers...· 0 citations
Document-level relation extraction (DocRE) aims to extract relations among multiple entities across extended contexts while maintaining consistency across predicted triples. Although large language models (LLMs) show remarkable reasoning capabilities in information extraction, their predictions are typically generated independently for each candidate triple and may violate fundamental relational constraints such as transitivity, symmetry, and functional uniqueness, leading to contradictory and unreliable outputs. We propose CONSISTRE, a unified consistency-aware framework for DocRE that addresses this limitation through two complementary tracks. The first operates at inference time for black-box LLMs, combining constraint-aware prompting, constraint-based verification, and iterative self-reflection to refine predictions without task-specific fine-tuning. The second injects consistency knowledge into smaller open-source models via a knowledge distillation and reinforcement learning pipeline: reasoning traces from a powerful teacher are distilled into a student via supervised fine-tuning, followed by GRPO alignment using a composite reward that jointly optimizes extraction performance and relational consistency. Together, the two tracks cover both API-accessible and locally deployable scenarios under a unified consistency formulation. Experiments on DocRED show that both tracks outperform their baselines, with the inference-time track achieving competitive F1 using off-the-shelf black-box LLMs and the training-time track substantially narrowing the gap between 7--8B open-source models and state-of-the-art proprietary LLMs at a fraction of their inference cost. Ablation studies confirm that explicit consistency modeling mitigates relational contradictions and enhances the reliability of LLM-based DocRE across both deployment paradigms.
Constrained-guided mapping is proposed, a neuro-symbolic method with three stages: schema-grounded admissibility constraints with metadata mc =, where tau_c denotes the constraint type and delta_c provides executable relation and normalization logic, and constraint-restricted candidate generation with cascade relaxation to guarantee a nonempty feasible set under noise.
Sebastian Monka, Pramod Anantharam, Thị Minh et al.· 0 citations
Long-document question-answering experiments show that human-verified TOC hierarchies and contextual relationships improve reasoning, with their combination providing complementary benefits.
Yuefeng Zou, Yichen Lu, Jingxiao Yang et al.· 0 citations
A seven-stage graph-grounded pipeline that converts domain documents into a complete, auditable Web Ontology Language (OWL) Terminological Box (TBox) without any unconstrained generation step is presented, demonstrating that the pipeline produces stable, reusable domain representations from large document corpora.
Maruf Ahmed Mridul, A. Talukder, O. Seneviratne· 0 citations
LlamaExtract Agentic Plus ranks first on all three metrics, with accuracy comparable to coding agents at a fraction of the cost, and is the first to score value accuracy, record completeness at scale, grounding, and measured cost together.
Boyang Zhang, Adrian Lyjak, Elizabeth Stewart et al.· arXiv.org· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
