Clinically Structured Retrieval: Can a Well-Designed RAG Layer Add Value Over a Base Language Model in Clinical Oncology?

Abstract

Study 1 (currently under review) found that retrieval-augmented generation (RAG) and fine-tuning both *degraded* clinical answer quality relative to an un-augmented base model in oncology. That study bounded its own retrieval conclusion explicitly: its index used 500-character chunks with 100-character overlap, small relative to the structure of oncology guidelines, where staging tables and treatment algorithms span whole sections. The observed degradation might therefore reflect chunking-induced fragmentation rather than an intrinsic limit of retrieval. This study tests that alternative explanation directly. It rebuilds the retrieval layer with section-level chunking, a modern multilingual embedding model, clinical metadata, and state-of-the-art retrieval engineering, and asks whether a *well-designed* RAG layer adds value over the base model — using an ablation ladder with an explicit floor (a sham distractor condition) and ceiling (a hand-selected oracle condition). An empirical audit of the study-1 index motivates the design: 41.9% of chunks begin mid-sentence and end without terminal punctuation, 73.5% lack closing punctuation, and **0% of NCCN chunks carry a section path** (versus 96–100% for every other source family). Tabular structure survived in 0.02% of chunks.