Data Quality Matters: Quantifying Image Quality Impact on Machine Learning Performance
Journal:
arXiv
Published Date:
Mar 28, 2025
Abstract
Precise perception of the environment is essential in highly automated
driving systems, which rely on machine learning tasks such as object detection
and segmentation. Compression of sensor data is commonly used for data
handling, while virtualization is used for hardware-in-the-loop validation.
Both methods can alter sensor data and degrade model performance. This
necessitates a systematic approach to quantifying image validity. This paper
presents a four-step framework to evaluate the impact of image modifications on
machine learning tasks. First, a dataset with modified images is prepared to
ensure one-to-one matching image pairs, enabling measurement of deviations
resulting from compression and virtualization. Second, image deviations are
quantified by comparing the effects of compression and virtualization against
original camera-based sensor data. Third, the performance of state-of-the-art
object detection models is analyzed to determine how altered input data affects
perception tasks, including bounding box accuracy and reliability. Finally, a
correlation analysis is performed to identify relationships between image
quality and model performance. As a result, the LPIPS metric achieves the
highest correlation between image deviation and machine learning performance
across all evaluated machine learning tasks.
