Bridging Contrastive Learning and Domain Adaptation: Theoretical Perspective and Practical Application
Journal:
arXiv
Published Date:
Jan 28, 2025
Abstract
This work studies the relationship between Contrastive Learning and Domain
Adaptation from a theoretical perspective. The two standard contrastive losses,
NT-Xent loss (Self-supervised) and Supervised Contrastive loss, are related to
the Class-wise Mean Maximum Discrepancy (CMMD), a dissimilarity measure widely
used for Domain Adaptation. Our work shows that minimizing the contrastive
losses decreases the CMMD and simultaneously improves class-separability,
laying the theoretical groundwork for the use of Contrastive Learning in the
context of Domain Adaptation. Due to the relevance of Domain Adaptation in
medical imaging, we focused the experiments on mammography images. Extensive
experiments on three mammography datasets - synthetic patches, clinical (real)
patches, and clinical (real) images - show improved Domain Adaptation,
class-separability, and classification performance, when minimizing the
Supervised Contrastive loss.
