A new vision-language framework, MedReCo-VLM, is introduced to enable entity-aware comparative reasoning in radiology, leveraging large-scale comparative imaging resources to improve clinical follow-up accuracy and analog retrieval.
Medical AI has shown strong performance in interpreting isolated medical images, but faces a gap when compared to real-world radiological practice, which relies on comparing diagnoses across prior studies. This research addresses this by formulating radiological comparison as an entity-aware cross-image reasoning problem. The work introduces MedReCo-DB, a massive comparative imaging resource comprising over 690,000 image-report pairs. Using this resource, the framework develops MedReCo, an entity-aware visual encoder, and MedReCo-VLM, a vision-language extension for generative interpretation of interval change.
Evaluations demonstrated that MedReCo achieved the highest Recall@1 in retrieval settings, and MedReCo-VLM significantly improved longitudinal follow-up accuracy. Specifically, it improved follow-up accuracy by 14.5-46.5 percentage points on chest radiographs and 13.0-27.9 percentage points on CT scans, suggesting a more clinically aligned foundation for medical imaging AI.