Is Your Training Pipeline Production-Ready? A Case Study in the Healthcare Domain
Journal:
arXiv
Published Date:
Jun 7, 2025
Abstract
Deploying a Machine Learning (ML) training pipeline into production requires
robust software engineering practices. This differs significantly from
experimental workflows. This experience report investigates this challenge in
SPIRA, a project whose goal is to create an ML-Enabled System (MLES) to
pre-diagnose insufficiency respiratory via speech analysis. The first version
of SPIRA's training pipeline lacked critical software quality attributes. This
paper presents an overview of the MLES, then compares three versions of the
architecture of the Continuous Training subsystem, which evolved from a Big
Ball of Mud, to a Modular Monolith, towards Microservices. By adopting
different design principles and patterns to enhance its maintainability,
robustness, and extensibility. In this way, the paper seeks to offer insights
for both ML Engineers tasked to productionize ML training pipelines and Data
Scientists seeking to adopt MLOps practices.
