Advancements in Medical Image Classification through Fine-Tuning Natural Domain Foundation Models
Journal:
arXiv
Published Date:
May 26, 2025
Abstract
Using massive datasets, foundation models are large-scale, pre-trained models
that perform a wide range of tasks. These models have shown consistently
improved results with the introduction of new methods. It is crucial to analyze
how these trends impact the medical field and determine whether these
advancements can drive meaningful change. This study investigates the
application of recent state-of-the-art foundation models, DINOv2, MAE, VMamba,
CoCa, SAM2, and AIMv2, for medical image classification. We explore their
effectiveness on datasets including CBIS-DDSM for mammography, ISIC2019 for
skin lesions, APTOS2019 for diabetic retinopathy, and CHEXPERT for chest
radiographs. By fine-tuning these models and evaluating their configurations,
we aim to understand the potential of these advancements in medical image
classification. The results indicate that these advanced models significantly
enhance classification outcomes, demonstrating robust performance despite
limited labeled data. Based on our results, AIMv2, DINOv2, and SAM2 models
outperformed others, demonstrating that progress in natural domain training has
positively impacted the medical domain and improved classification outcomes.
Our code is publicly available at:
https://github.com/sajjad-sh33/Medical-Transfer-Learning.
