Document-level relation extraction via self-correcting prompt optimization and preference alignment
TL;DR
This paper proposes SPO-PA, which combines self-correcting prompt optimization (SPO) and preference alignment (PA), which compares LLM predictions with reference annotations and applies source-specific constraints to these signals.
Abstract
Document-level relation extraction (DocRE) finds relations across a whole document. It often needs evidence from several sentences. It also needs to link repeated entity mentions and follow multi-hop clues. Many large language model (LLM) based DocRE methods use fixed relation descriptions. They make little use of confident errors or samples with missing labels. We propose SPO-PA, which combines self-correcting prompt optimization (SPO) and preference alignment (PA). SPO compares LLM predictions with reference annotations. It then revises relation descriptions, clarifies relation boundaries, and adds role constraints. PA uses false negative (FN) conflicts as the main supervision signal. It also tests false positive (FP) conflicts under similar constraints. PA applies source-specific constraints to these signals. The constraints help smaller DocRE models separate correct relations from wrong candidates. On Re-DocRED, SPO improves F1 to 26.23% and Precision to 41.93%. It also achieves 37.60% F1 on HacRED. Both are the best LLM-only scores in our comparison and reproduce on a second, more recent backbone, though both stay far below supervised encoders; the extraction stage supplies conflict signals rather than final predictions. Among the PA variants, PAFN\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\text {PA}_{FN}$$\end{document} gives the most stable gains on ATLOP, KD-DocRE, DREEAM, and DAATF. The largest gain is 2.89%.
