Regulating AI Adaptation: An Analysis of AI Medical Device Updates
Journal:
arXiv
Published Date:
Jun 22, 2024
Abstract
While the pace of development of AI has rapidly progressed in recent years,
the implementation of safe and effective regulatory frameworks has lagged
behind. In particular, the adaptive nature of AI models presents unique
challenges to regulators as updating a model can improve its performance but
also introduce safety risks. In the US, the Food and Drug Administration (FDA)
has been a forerunner in regulating and approving hundreds of AI medical
devices. To better understand how AI is updated and its regulatory
considerations, we systematically analyze the frequency and nature of updates
in FDA-approved AI medical devices. We find that less than 2% of all devices
report having been updated by being re-trained on new data. Meanwhile, nearly a
quarter of devices report updates in the form of new functionality and
marketing claims. As an illustrative case study, we analyze pneumothorax
detection models and find that while model performance can degrade by as much
as 0.18 AUC when evaluated on new sites, re-training on site-specific data can
mitigate this performance drop, recovering up to 0.23 AUC. However, we also
observed significant degradation on the original site after re-training using
data from new sites, providing insight from one example that challenges the
current one-model-fits-all approach to regulatory approvals. Our analysis
provides an in-depth look at the current state of FDA-approved AI device
updates and insights for future regulatory policies toward model updating and
adaptive AI.
