Jul 2026· Vietnam Journal of Computer Science· pp. 1-40· 0 citations
TL;DR
A novel transformer-based approach for biomedical EQA that incorporates Named Entity Recognition (NER) to identify key medical terms, Relation Extraction (RE) to understand their interconnections, and a BiLSTM layer to enhance contextual comprehension is introduced.
Abstract
Extractive Question Answering (EQA) in the biomedical domain presents significant challenges due to complex medical terminology and intricate relationships between entities. Existing systems often perform suboptimally compared to their counterparts in general domains. Current biomedical QA models struggle with accurately identifying named entities and extracting meaningful relationships within medical contexts, thereby limiting their practical utility. This study introduces a novel transformer-based approach for biomedical EQA that incorporates Named Entity Recognition (NER) to identify key medical terms, Relation Extraction (RE) to understand their interconnections, and a BiLSTM layer to enhance contextual comprehension. These components are integrated within an ensemble framework that leverages pretrained biomedical language models. Our methodology systematically evaluated six transformer encoder models and identified Bio+ClinicalBERT as the optimal foundation for developing three progressively advanced biomedical EQA models. The final model—Bio+ClinicalBERT_NER_RE_BiLSTM—incorporating ensemble learning techniques, demonstrated substantial performance improvements: an F1 score of 91.69%, exact match of 88.35%, and lenient accuracy of 0.84, marking a 5.45% performance gain over baseline methods. These results highlight the model's superior ability to navigate complex biomedical language and extract accurate information from medical texts. The developed system directly benefits biomedical researchers, clinical practitioners, medical students, and healthcare professionals by delivering accurate and rapid access to precise information within extensive medical literature. By bridging the gap between intricate medical texts and effective information retrieval, this work significantly advances automated biomedical knowledge extraction and supports evidence-based medical decision-making.
Motivation Pharmacovigilance relies on accurate extraction of structured biomedical entities and their semantic relationships from scientific literature. However, most biomedical information extraction systems address named entity recognition (NER) and relation extraction as separate tasks trained on corpus-specific architectures, limiting scalability and cross-task knowledge sharing. Recent developments in instruction-tuned Large Language Models (LLMs) offer a promising alternative through unified generative extraction, but robust schema-grounded multitask adaptation for biomedical extraction is still understudied. Methods This study proposes a unified multitask instruction-tuned LLM framework that jointly performs biomedical NER and relation extraction across three benchmark corpora to identify chemical, disease, drug entities, as well as chemical-disease relations, drug-adverse event relations, and drug-drug interactions. Two general LLMs, Llama-3.2-3B-Instruct and Qwen3-8B, were fine-tuned using Low-Rank Adaptation (LoRA) under a shared generation interface that extracts both entity pairs and their underlying relation. Zero-shot and fine-tuned configurations were evaluated across all the tasks on their respective held-out test sets. Results Parameter-efficient fine-tuning substantially improved both entity and relation extraction performance across all tasks and model families. Fine-tuned Qwen3-8B achieved the strongest overall performance with 89.42% micro-averaged entity F1 and 62.32% micro-averaged relation F1. Fine-tuned Llama-3.2-3B achieved 87.63% entity F1 and 58.42% relation F1 despite its substantially smaller parameter count, outperforming the zero-shot 8B model on both tasks. Fine-tuning also reduced structured JSON parse failures from 23.5% to 0.11%, demonstrating stable schema internalization during supervised adaptation. Conclusion Schema-grounded multitask instruction tuning with LoRA provides a robust and computationally feasible framework for unified biomedical information extraction across heterogeneous benchmark corpora. The findings further demonstrate that schema-grounded adaptation is substantially more important than model scale alone for reliable extraction of structured biomedical relations. The gap between NER and relation extraction performance motivates future research on explicit negative-relation supervision and ontology-guided relation extraction.
A three-stage framework made up of neighborhood-aware retrieval, dual reranking, and score fusion that achieves the state of the art on average across five widely-used benchmarks and remains efficient at inference.
: Biomedical texts present significant challenges for natural language processing (NLP) due to their complex terminology, intricate contextual dependencies, and highly domain-specific semantics. This study investigates the effectiveness of knowledge distillation (KD) for biomedical text classification, aiming to develop lightweight, resource-efficient models that remain competitive with larger architectures. A balanced dataset of 25,000 PubMed records was constructed, equally distributed across five biomedical domains. Two teacher models (BERT and PubMedBERT) and five student models (DistilBERT, BioClinicalBERT, BioBERT, DistilBioBERT, and DistilRoBERTa) were evaluated across ten distinct KD configurations. Each student model was also directly fine-tuned to serve as a controlled baseline. Model performance was assessed using accuracy, precision, recall, and F1-score. The results show that KD can, under suitable teacher–student configurations, enable student models to surpass direct fine-tuning, while other configurations yield only marginal or comparable improvements. Among all configurations, BERT → DistilBERT achieved the highest performance, reaching 89% accuracy. Unexpectedly, the general-purpose BERT teacher outperformed the domain-specific PubMedBERT across multiple student models, suggesting that broader linguistic representations can transfer more effectively across diverse biomedical subdomains. Lower performance in certain KD settings, such as DistilBioBERT and DistilRoBERTa, was attributed to architectural mismatches and limited student capacity. These findings demonstrate that compact models can achieve strong biomedical classification performance through KD under compatible teacher–student pairings, while also highlighting that KD effectiveness varies substantially depending on the specific model combination.
Results demonstrate that InfoFlowEX equips LLMs with robust adaptability, achieving consistent gains over baselines with minimal task-specific customization, highlighting InfoFlowEX for real-world biomedical applications.
Wuyang Lan, Siqi Zhang, Wenzheng Wang et al.· Cell Reports Medicine· 0 citations
