RoCo-Sim: Enhancing Roadside Collaborative Perception through Foreground Simulation
Journal:
arXiv
Published Date:
Mar 13, 2025
Abstract
Roadside Collaborative Perception refers to a system where multiple roadside
units collaborate to pool their perceptual data, assisting vehicles in
enhancing their environmental awareness. Existing roadside perception methods
concentrate on model design but overlook data issues like calibration errors,
sparse information, and multi-view consistency, leading to poor performance on
recent published datasets. To significantly enhance roadside collaborative
perception and address critical data issues, we present the first simulation
framework RoCo-Sim for road-side collaborative perception. RoCo-Sim is capable
of generating diverse, multi-view consistent simulated roadside data through
dynamic foreground editing and full-scene style transfer of a single image.
RoCo-Sim consists of four components: (1) Camera Extrinsic Optimization ensures
accurate 3D to 2D projection for roadside cameras; (2) A novel Multi-View
Occlusion-Aware Sampler (MOAS) determines the placement of diverse digital
assets within 3D space; (3) DepthSAM innovatively models foreground-background
relationships from single-frame fixed-view images, ensuring multi-view
consistency of foreground; and (4) Scalable Post-Processing Toolkit generates
more realistic and enriched scenes through style transfer and other
enhancements. RoCo-Sim significantly improves roadside 3D object detection,
outperforming SOTA methods by 83.74 on Rcooper-Intersection and 83.12 on
TUMTraf-V2X for AP70. RoCo-Sim fills a critical gap in roadside perception
simulation. Code and pre-trained models will be released soon:
https://github.com/duyuwen-duen/RoCo-Sim
