Discussion on "Approval policies for modifications to machine learning-based software as a medical device: A study of biocreep" by Jean Feng, Scott Emerson, and Noah Simon.

I applaud the authors of Feng (2020) for tackling a challenging statistical problem on approval policies for software as a medical device (SaMD). Their work exploring methodology that could autonomously build algorithmic change protocols soundly extends and leverages related literatures in multiple testing and online learning, among others. While their paper appears in the section of the journal, I choose to focus on important practical considerations in this invited discussion, given that algorithms optimized and deployed in health care can directly impact human health. Thus, although not a paper, I aim to make the case that several broad issues are relevant for much of the algorithmic work of statisticians who are driven by health applications: the data and setting, whether the reference algorithm is an acceptable baseline, and metrics.