MoSAiC: Multi-Modal Multi-Label Supervision-Aware Contrastive Learning for Remote Sensing
Journal:
arXiv
Published Date:
Jul 11, 2025
Abstract
Contrastive learning (CL) has emerged as a powerful paradigm for learning
transferable representations without the reliance on large labeled datasets.
Its ability to capture intrinsic similarities and differences among data
samples has led to state-of-the-art results in computer vision tasks. These
strengths make CL particularly well-suited for Earth System Observation (ESO),
where diverse satellite modalities such as optical and SAR imagery offer
naturally aligned views of the same geospatial regions. However, ESO presents
unique challenges, including high inter-class similarity, scene clutter, and
ambiguous boundaries, which complicate representation learning -- especially in
low-label, multi-label settings. Existing CL frameworks often focus on
intra-modality self-supervision or lack mechanisms for multi-label alignment
and semantic precision across modalities. In this work, we introduce MoSAiC, a
unified framework that jointly optimizes intra- and inter-modality contrastive
learning with a multi-label supervised contrastive loss. Designed specifically
for multi-modal satellite imagery, MoSAiC enables finer semantic
disentanglement and more robust representation learning across spectrally
similar and spatially complex classes. Experiments on two benchmark datasets,
BigEarthNet V2.0 and Sent12MS, show that MoSAiC consistently outperforms both
fully supervised and self-supervised baselines in terms of accuracy, cluster
coherence, and generalization in low-label and high-class-overlap scenarios.
