GeoLangBind: Unifying Earth Observation with Agglomerative Vision-Language Foundation Models
Journal:
arXiv
Published Date:
Mar 8, 2025
Abstract
Earth observation (EO) data, collected from diverse sensors with varying
imaging principles, present significant challenges in creating unified
analytical frameworks. We present GeoLangBind, a novel agglomerative
vision--language foundation model that bridges the gap between heterogeneous EO
data modalities using language as a unifying medium. Our approach aligns
different EO data types into a shared language embedding space, enabling
seamless integration and complementary feature learning from diverse sensor
data. To achieve this, we construct a large-scale multimodal image--text
dataset, GeoLangBind-2M, encompassing six data modalities. GeoLangBind
leverages this dataset to develop a zero-shot foundation model capable of
processing arbitrary numbers of EO data channels as input. Through our designed
Modality-aware Knowledge Agglomeration (MaKA) module and progressive multimodal
weight merging strategy, we create a powerful agglomerative foundation model
that excels in both zero-shot vision--language comprehension and fine-grained
visual understanding. Extensive evaluation across 23 datasets covering multiple
tasks demonstrates GeoLangBind's superior performance and versatility in EO
applications, offering a robust framework for various environmental monitoring
and analysis tasks. The dataset and pretrained models will be publicly
available.
