A preliminary version of a framework that improves the most widespread DLA datasets quality by assessing and correcting layout coherence in scholarly documents and provides a more reliable ground truth with improved structural and semantic coherence for training and evaluating document segmentation and understanding models.
The results indicate that the proposed framework effectively enhances the quality and structural consistency of generated hyper-relational data while mitigating the effects of long-tail distributions and pseudo-negative samples without requiring additional manual annotations.
Xinzhi Du, Yan Chen, Siqi Xu et al.· Applied Sciences· 0 citations
This work introduces an ontology-driven framework to quantify and enforce structural consistency in DocRE datasets and reveals substantial structural noise in DocRED distant and demonstrates that such inconsistencies propagate to model predictions.
Laura Menotti, Stefano Marchesin, Gianmaria Silvello· 0 citations
This study explores a semantic variation methodology to augment training data by generating question-answer pairs with explicit control over semantic similarity, and shows that semantically controlled augmentation improves domain-specific knowledge acquisition while preserving consistency.
Alexander Chen, Caroline Tang, Jennifer Sleeman· TEXT2KG/BiKE@ESWC· 0 citations
Multimodal Large Language Models (MLLMs) have achieved strong performance on structured visual understanding tasks such as chart and document question answering. However, existing benchmarks typically evaluate these domains in isolation, leaving underexplored a key capability: whether models can use textual context to determine how chart evidence should be selected, interpreted, and aggregated. We introduce DocHop, a benchmark for integrated chart--context reasoning in document-style images. In DocHop, the document narrative specifies multi-step compositional constraints, while charts provide the corresponding data values. Questions are grounded on a semantic reference label defined in the narrative, requiring models to resolve target entities from context before aggregating evidence across multiple charts. To enable systematic evaluation, we construct DocHop via a stochastic logic-first generation pipeline with controllable reasoning depth and visual density, covering 2,074 examples across six task categories. Experiments on a wide range of proprietary and open-source MLLMs show a substantial gap to human performance: annotators achieve over 90% accuracy, while the best model reaches only 62.83%. Reasoning-enhanced models consistently show improved results, but performance degrades as reasoning complexity increases. Overall, DocHop provides a controlled testbed for challenging multi-hop document reasoning.
Zhuoran Yu, Le Thien Phuc Nguyen, Jaden Park et al.· 0 citations
A novel, interpretable metric for intrinsic KG quality assessment that measures how closely an automatically extracted graph approximates an"ideal"graph capturing the key noun phrases, predicate relations, and basic linguistic phenomena such as negation expressed in the source text is proposed.
Nipun Misra, Vikranth Udandarao, Aanchal Gupta et al.· 0 citations
It is asserted that the present contribution consists of an interpretable domain palette, a constructed benchmark of diverse tabular datasets, and reproducible code and data to enable further research on domain discovery and domain-aware tooling for tabular data.
Elizaveta Gamper, Irina Deeva· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
