Accurate prediction of tumor recurrence in brain tumor patients following surgery is essential for optimizing adjuvant therapy, response assessment, and surveillance regimen. While MRI remains the gold standard for surveillance, integrating patient-specific clinical context may inform recurrence prediction. Traditional multimodal deep learning approaches often incorporate clinical data via simple fusion, failing to fully capture the semantic interdependencies between visual features and clinical context. Trained on over 5,000 scans from approximately 400 pediatric low-grade glioma subjects and validated across three institutional cohorts, including one clinical trial cohort, our experiments demonstrate incremental performance gains when progressing from vision-only to clinical-vision to a vision-language approach. Our results indicate that converting structured clinical covariates into natural language text allows for more effective synthesis of multimodal data, while providing a platform for incremental addition of clinical context without extending model complexity. We demonstrate that our proposed VLM architecture offers a promising direction for neuro-oncological prognosis by effectively encoding imaging cues and clinical context, with potential applicability to other longitudinal prognosis tasks.
D. Tak, D. Sreedhar, H. Aerts et al.· medRxiv· 0 citations
PURPOSE
To develop and validate a pediatric diffuse midline glioma (DMG) auto-segmentation tool optimized for longitudinal treatment response assessment across the disease course.
MATERIALS AND METHODS
In this multi-institutional retrospective study, we included patients aged 1-30 years with DMG from an institutional pediatric cancer center, BraTS-PEDs 2024, and PNOC007, a prospective trial of radiation followed by peptide vaccine plus poly-ICLC, and we trained nnU-Net-based DMGtracker using expert segmentations from 140 institutional pre- and post-treatment studies and all 261 BraTS-PEDs 2024 pre-treatment studies, using four-sequence multiparametric MRI (T1, T1 post-contrast, T2, and FLAIR). We externally validated the model on 88 annotated PNOC007 studies (n = 49 patients) and compared it with the BraTS-PEDs 2024 winning model using median Dice similarity coefficient (DSC) and relative volumetric difference (RVD) for whole-tumor and contrast-enhancing tumor segmentation using the Wilcoxon signed-rank test.
RESULTS
Training and internal testing used 153 scans (59 post-treatment) from 74 patients. Incorporating post-treatment data improved internal whole-tumor DSC for our trained model (0.94 [IQR 0.82-0.96] vs 0.93 [0.81-0.96]; p<0.001). On external validation, DMGtracker outperformed the BraTS-PEDs 2024 winning model for whole-tumor segmentation, with higher DSC (0.90 [0.72-0.95] vs 0.81 [0.66-0.90]) and lower RVD (9.6% [3.7%-31.6%] vs 16.8% [7.3%-39.2%]). This advantage was greatest in post-treatment scans (n = 50 scans, DSC 0.90 [0.73-0.94] vs 0.80 [0.58-0.88]; RVD 9.7% [3.8%-26.2%] vs 19.8% [12.6%-39.8%]; p<0.001 for both). In post-treatment scans, DMGtracker achieved clinically acceptable whole-tumor segmentation (DSC > 0.80) in 64.0% of cases, compared with 52.0% for the BraTS-PEDs winner.
CONCLUSION
Training DMG segmentation models with post-treatment scans substantially improves performance in longitudinal clinical trial imaging, enabling more accurate volumetric tracking and response assessment.
John Zielke, Francesca Romana Mussa, A. Zapaishchykova et al.· AJNR. American journal of ne...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
