3D Reconstruction and Information Fusion between Dormant and Canopy Seasons in Commercial Orchards Using Deep Learning and Fast GICP
Journal:
arXiv
Published Date:
Jul 2, 2025
Abstract
In orchard automation, dense foliage during the canopy season severely
occludes tree structures, minimizing visibility to various canopy parts such as
trunks and branches, which limits the ability of a machine vision system.
However, canopy structure is more open and visible during the dormant season
when trees are defoliated. In this work, we present an information fusion
framework that integrates multi-seasonal structural data to support robotic and
automated crop load management during the entire growing season. The framework
combines high-resolution RGB-D imagery from both dormant and canopy periods
using YOLOv9-Seg for instance segmentation, Kinect Fusion for 3D
reconstruction, and Fast Generalized Iterative Closest Point (Fast GICP) for
model alignment. Segmentation outputs from YOLOv9-Seg were used to extract
depth-informed masks, which enabled accurate 3D point cloud reconstruction via
Kinect Fusion; these reconstructed models from each season were subsequently
aligned using Fast GICP to achieve spatially coherent multi-season fusion. The
YOLOv9-Seg model, trained on manually annotated images, achieved a mean squared
error (MSE) of 0.0047 and segmentation mAP@50 scores up to 0.78 for trunks in
dormant season dataset. Kinect Fusion enabled accurate reconstruction of tree
geometry, validated with field measurements resulting in root mean square
errors (RMSE) of 5.23 mm for trunk diameter, 4.50 mm for branch diameter, and
13.72 mm for branch spacing. Fast GICP achieved precise cross-seasonal
registration with a minimum fitness score of 0.00197, allowing integrated,
comprehensive tree structure modeling despite heavy occlusions during the
growing season. This fused structural representation enables robotic systems to
access otherwise obscured architectural information, improving the precision of
pruning, thinning, and other automated orchard operations.
