FD2-Net: Frequency-Driven Feature Decomposition Network for Infrared-Visible Object Detection
Journal:
arXiv
Published Date:
Dec 12, 2024
Abstract
Infrared-visible object detection (IVOD) seeks to harness the complementary
information in infrared and visible images, thereby enhancing the performance
of detectors in complex environments. However, existing methods often neglect
the frequency characteristics of complementary information, such as the
abundant high-frequency details in visible images and the valuable
low-frequency thermal information in infrared images, thus constraining
detection performance. To solve this problem, we introduce a novel
Frequency-Driven Feature Decomposition Network for IVOD, called FD2-Net, which
effectively captures the unique frequency representations of complementary
information across multimodal visual spaces. Specifically, we propose a feature
decomposition encoder, wherein the high-frequency unit (HFU) utilizes discrete
cosine transform to capture representative high-frequency features, while the
low-frequency unit (LFU) employs dynamic receptive fields to model the
multi-scale context of diverse objects. Next, we adopt a parameter-free
complementary strengths strategy to enhance multimodal features through
seamless inter-frequency recoupling. Furthermore, we innovatively design a
multimodal reconstruction mechanism that recovers image details lost during
feature extraction, further leveraging the complementary information from
infrared and visible images to enhance overall representational capacity.
Extensive experiments demonstrate that FD2-Net outperforms state-of-the-art
(SOTA) models across various IVOD benchmarks, i.e. LLVIP (96.2% mAP), FLIR
(82.9% mAP), and M3FD (83.5% mAP).
