Recent progress in Medical Visual Question Answering (VQA) has significantly aided clinical decision-making across various domains such as pneumonia, oncology, and neurology. However, spinal and musculoskeletal ailments remain critically underexplored. The development of reliable VQA models for spinal imaging is currently hindered by a lack of datasets and evaluation protocols that do not reflect the descriptive, diagnostic interpretations used in clinical practice. In this resource paper, we introduce SpineXR-VQA, an open-source, clinically grounded, and verified benchmark comprising 2,187 X-rays and 8,272 expert-verified, open-ended Question-Answer pairs. Unlike standard classification-based datasets, SpineXR-VQA features six expert-validated categories: abnormality, severity, location, diagnosis, treatment, and reasoning. Ten orthopedic specialists from India and Thailand validated all pairs, ensuring geographic diversity and high inter-rater agreement (Cohen's Kappa: 0.96 for questions, 0.93 for answers). We benchmark 15 state-of-the-art Multimodal Large Language Models (MLLMs), including proprietary systems such as Claude Sonnet, GPT-o4 mini, and Gemini Flash, alongside medical-specific and general open-weight models. While these models achieve moderate semantic alignment (median similarity: 0.72), a detailed analysis reveals that they consistently fail to capture essential diagnostic nuances, such as anatomical fidelity and clinical completeness. These results underscore the necessity for specialized models in spinal VQA, a gap SpineXR-VQA fills.
Deepali Mishra, Dr.Sorayouth Chumnanvej, Vikas Trivedi et al.· ACM Transactions on Computin...· 0 citations
Institutional policies stay in natural language while the systems that check compliance demand machine-readable constraints. Bridging that gap is still done by hand. PolicyKG closes the loop. It is an LLM pipeline that reads a policy PDF, classifies each sentence as an obligation, permission, or prohibition, lifts the label into first-order deontic logic, and emits SHACL constraints. Four stages run on a LangGraph state machine with per-stage validators. The piece that matters most is the Corpus Adapter: a YAML vocabulary registry that grounds LLM predicates in a target ontology. Retargeting to a new domain means swapping the registry, not retraining a model. On the Asian Institute of Technology Policies and Procedures corpus (1,663 sentences, 443 rules), PolicyKG reaches 86.9% deontic classification accuracy (Cohen's kappa = .709). Three annotators independently re-label a 50-item sample and agree at Fleiss'kappa = .844. SHACL shape correctness on a 69-shape subset is F1 = .866. The FOL path handles 79.2% of rules; the rest go through a direct NL-to-SHACL fallback. We audited every one of the 443 rules for second- or higher-order constructs. An automated regex checklist flagged none, and a first-author pass on the 92 FOL-fallback cases confirmed the same. The exact upper 95% Clopper-Pearson bound on the true HOL rate is 0.67%. This is an audit finding for one corpus, not a proof of FOL sufficiency for institutional policy. Swapping the AIT registry for a GDPR registry raises exact property alignment from 1/15 to 11/15 (Fisher's exact p<.001; Cohen's h = 1.53). On the LexDeMod lease-contract benchmark (N = 200), Macro F1 drops to .370 because lease English uses"shall be entitled"for permission -- exactly the vocabulary mismatch registry swap is meant to fix. Repeated runs produce hash-identical SHACL outputs.
