Pretraining of Medical Visual Encoders Toward Multi-modal Large Language Models
MedMLIP is introduced, a framework that pretrains the visual encoder through report generation with a frozen LLM, while employing Local Relational Distillation (LRD) to preserve relationships among visual patches to avoid visual collapse.