FairICP: identifying biases and increasing transparency at the point of care in post-implementation clinical decision support using inductive conformal prediction.

Journal: Journal of the American Medical Informatics Association : JAMIA
Published Date:

Abstract

OBJECTIVES: Fairness concerns stemming from known and unknown biases in healthcare practices have raised questions about the trustworthiness of Artificial Intelligence (AI)-driven Clinical Decision Support Systems (CDSS). Studies have shown unforeseen performance disparities in subpopulations when applied to clinical settings different from training. Existing unfairness mitigation strategies often struggle with scalability and accessibility, while their pursuit of group-level prediction performance parity does not effectively translate into fairness at the point of care. This study introduces FairICP, a flexible and cost-effective post-implementation framework based on Inductive Conformal Prediction (ICP), to provide users with actionable knowledge of model uncertainty due to subpopulation level biases at the point of care.

Authors

  • Xiaotan Sun
    Cardiovascular Innovation Research Center, Cleveland Clinic, Cleveland, OH 44195, United States.
  • Makiya Nakashima
    Heart, Vascular, and Thoracic Institute, Cleveland Clinic, Cleveland, Ohio, USA; Cardiovascular Innovations Research Center, Cleveland Clinic, Cleveland, Ohio, USA.
  • Christopher Nguyen
    Cardiovascular Research Center, Massachusetts General Hospital, Charlestown, MA 02129, USA Caroline.Burns@childrens.harvard.edu Geoff.Burns@childrens.harvard.edu Christopher.nguyen@mgh.harvard.edu.
  • Po-Hao Chen
    Department of Radiology, Perelman School of Medicine, Hospital of the University of Pennsylvania, 3400 Spruce Street, Philadelphia, PA, 19104, USA. po-hao.chen@uphs.upenn.edu.
  • W H Wilson Tang
    Robert and Suzanne Tomsich Department of Cardiovascular Medicine Sydell and Arnold Miller Family Heart and Vascular Institute Cleveland Clinic Cleveland OH.
  • Deborah Kwon
    Department of Cardiovascular Medicine, Heart, Vascular and Thoracic Institute, Cleveland Clinic, Cleveland, OH, USA.
  • David Chen
    Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada.