Aug 2026· Proceedings of the 2026 ACM Symposium on Document Engineering· 0 citations· 40 references
TL;DR
Two language-model-based strategies are proposed for semantic code document segmentation, including a line-by-line approach that classifies each line of code separately before grouping the results into functional units, and a range-based approach that aims to directly determine groups of code lines from the input.
Abstract
A key task for better understanding and maintaining source code as a code document is semantic code document segmentation, i. e., dividing the code into coherent blocks of functional intent. Despite its potential to enhance code comprehension, navigation, and reuse, this task remains underexplored due to the lack of semantically segmented datasets. Prior work generally relied on syntactic signals (e. g., AST-based features) or manual heuristics, limiting scalability and generalization. We propose two language-model-based strategies: (i) a line-by-line approach that classifies each line of code separately before grouping the results into functional units, and (ii) a range-based approach that aims to directly determine groups of code lines from the input. The latter is particularly suitable for generative language models as they can take an entire code file as the input context. Furthermore, we release two expert-annotated datasets from real-world scientific code in both a low-resource language, R, and the widely used Python language. Experiments show that line-by-line strategy with a local context of K surrounding lines generally outperforms the range-based approach for both programming languages. Fine-tuning smaller models like CodeBERT and CodeT5+ for line-by-line classification generally outperforms larger, generative language models, even without R-specific pretraining. On a single GPU, the runtime of CodeBERT is 100-170x faster than those of the best competing LLMs, supporting the practical integration of semantic segmentation into modern development environments. The code, prompts, and datasets are available at: https://github.com/Dahouabdelhalim/CodeSeg
LifeSciBench is introduced, a benchmark of 750 expert-authored tasks designed to evaluate whether language models can handle realistic life science research work, with each constituent task paired with a human expert-written rubric.
Amelia Liu, Andrew Ho, Anne Marie Droste et al.· bioRxiv· 2 citations
TestifAI, a deep learning testing framework for efficient and accurate estimation of robustness against combinations of perturbations, is proposed and partial model tomography is introduced, a novel approach to reconstructing model behaviour in a multi-perturbation space from tests that apply only a small number of perturbations.
Arooj Arif, T. Hartung, E. Botoeva et al.· 1 citation
A rapidly advancing precision-therapy pipeline-including antisense oligonucleotides to upregulate the intact allele, AAV-based gene replacement, CRISPR-mediated transcriptional activation, epigenetic modulators, and rational pathway-targeted small molecules-offers realistic prospects for disease modification.
This paper proposes the Evaluation Context Protocol (ECP), an early-stage, vendor-neutral framework intended to act as a portable evaluation contract layer for agentic systems and describes an open-source reference implementation that includes adapters for LangChain, LlamaIndex, CrewAI, and PydanticAI.
Large Language Models show potential in their diagnostic accuracy and consequent ability to reduce clinician burden, and may provide the greatest benefit when used to optimise referral quality at source, improving both clinician and potentially LLM triage downstream.
K. Surendran, I. Aziz, Glyndwr Jenkins· Current Surgery Reports· 0 citations
Background Ambient AI documentation tools, known as scribes, are entering routine clinical practice at scale, but the evidence comparing the notes they produce against clinician-written notes is dominated by single-site, single-language studies that rely on human review to find errors, a method known to miss most documentation errors. Methods We conducted a paired simulation across five countries and languages (Cambridge/English, Barcelona/Spanish, Milan/Italian, Paris/French, Cologne/German; 385 paired consultations, 770 notes). From each actor-performed consultation, an AI scribe (Heidi) and a junior-to-middle-grade clinician independently produced a note. Notes were scored on the PDQI-9 by evaluators blinded to authorship. Documentation errors were identified by two methods of deliberately different sensitivity - clinician adjudication, and a calibrated automated reviewer externally validated against a blinded ten-clinician panel - then graded for clinical risk by a three-model panel. The co-primary outcomes were PDQI-9 total and Critical+High error burden, the latter reported under both detection arms. The analysis plan was registered before any pooling across sites. Results AI notes scored higher than clinician notes on the PDQI-9 (40.6 vs 35.6; difference +5.08, 95% CI 4.6-5.6; Cohen dz=0.55), consistently across all five sites (dz 0.41-0.75), and were less dispersed (5.7% of AI vs 27.8% of clinician notes fell below the study pre-specified low-score threshold (<32)). On the principal safety outcome - the paired probability that a note carried [≥]Critical+High error - clinician notes were affected more often under both detection arms: 61.0% versus 24.4% by the calibrated reviewer (relative risk 2.50, 95% CI 2.09-3.00) and 21.8% versus 6.2% by clinician adjudication (relative risk 3.50, 95% CI 2.32-5.27). The difference was largest for omissions. Unaided clinician review identified roughly 12% of the errors the calibrated reviewer retained, and a smaller fraction in AI notes than in clinician notes. Conclusions In this simulation, AI-generated notes scored higher on documentation quality, varied less, and carried fewer clinically significant errors than notes written on the same consultations by junior-to-middle-grade clinicians. The magnitude of the safety difference depends on the sensitivity of error detection, so we report both detection regimes and bound rather than point-estimate the absolute error rate. Extension to live practice, consultant-authored documentation, and notes as filed after clinician editing remains to be established.
H. Bergman, V. Liu, B. Austin et al.· medRxiv· 0 citations
A new method for surgically removing training examples from a model reveals that as datasets grow, the link between what a model learns and what it produces dissolves.
