Post-Hurricane Debris Segmentation Using Fine-Tuned Foundational Vision Models
Journal:
arXiv
Published Date:
Apr 17, 2025
Abstract
Timely and accurate detection of hurricane debris is critical for effective
disaster response and community resilience. While post-disaster aerial imagery
is readily available, robust debris segmentation solutions applicable across
multiple disaster regions remain limited. Developing a generalized solution is
challenging due to varying environmental and imaging conditions that alter
debris' visual signatures across different regions, further compounded by the
scarcity of training data. This study addresses these challenges by fine-tuning
pre-trained foundational vision models, achieving robust performance with a
relatively small, high-quality dataset. Specifically, this work introduces an
open-source dataset comprising approximately 1,200 manually annotated aerial
RGB images from Hurricanes Ian, Ida, and Ike. To mitigate human biases and
enhance data quality, labels from multiple annotators are strategically
aggregated and visual prompt engineering is employed. The resulting fine-tuned
model, named fCLIPSeg, achieves a Dice score of 0.70 on data from Hurricane Ida
-- a disaster event entirely excluded during training -- with virtually no
false positives in debris-free areas. This work presents the first
event-agnostic debris segmentation model requiring only standard RGB imagery
during deployment, making it well-suited for rapid, large-scale post-disaster
impact assessments and recovery planning.
