Evaluating Artificial Intelligence in Clinical Settings-Let Us Not Reinvent the Wheel.

Journal: Journal of medical Internet research
Published Date: Aug 7, 2024

Abstract

Given the requirement to minimize the risks and maximize the benefits of technology applications in health care provision, there is an urgent need to incorporate theory-informed health IT (HIT) evaluation frameworks into existing and emerging guidelines for the evaluation of artificial intelligence (AI). Such frameworks can help developers, implementers, and strategic decision makers to build on experience and the existing empirical evidence base. We provide a pragmatic conceptual overview of selected concrete examples of how existing theory-informed HIT evaluation frameworks may be used to inform the safe development and implementation of AI in health care settings. The list is not exhaustive and is intended to illustrate applications in line with various stakeholder requirements. Existing HIT evaluation frameworks can help to inform AI-based development and implementation by supporting developers and strategic decision makers in considering relevant technology, user, and organizational dimensions. This can facilitate the design of technologies, their implementation in user and organizational settings, and the sustainability and scalability of technologies.

Authors

  • Kathrin Cresswell
    Usher Institute of Population Health Sciences and Informatics, University of Edinburgh, Edinburgh, United Kingdom.
  • Nicolette de Keizer
    Amsterdam UMC, University of Amsterdam, Medical Informatics, Amsterdam, Netherlands.
  • Farah Magrabi
    Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Australia.
  • Robin Williams
    Institute for the Study of Science, Technology and Innovation, The University of Edinburgh, Edinburgh, United Kingdom.
  • Michael Rigby
    Keele University, School of Social Science and Public Policy, Keele, United Kingdom.
  • Mirela Prgomet
    Faculty of Medicine, Health and Human Sciences, Macquarie University, Sydney, Australia.
  • Polina Kukhareva
    Department of Biomedical Informatics, University of Utah, United States of America.
  • Zoie Shui-Yee Wong
    Graduate School of Public Health, St. Luke's International University, Tokyo, Japan.
  • Philip Scott
    School of Computing, University of Portsmouth, Portsmouth, UK philip.scott@port.ac.uk.
  • Catherine K Craven
    Institute for Healthcare Delivery Science, Dept. of Pop. Health Science and Policy, Icahn School of Medicine at Mount Sinai, New York, USA; Clinical Informatics Group, IT Department, Mount Sinai Health System, New York, USA.
  • Andrew Georgiou
    Macquarie University, Australian Institute of Health Innovation, Sydney, Australia.
  • Stephanie Medlock
    Amsterdam UMC location University of Amsterdam, Department of Medical Informatics, Amsterdam, The Netherlands.
  • Jytte Brender McNair
    Department of Health Science and Technology, Aalborg University, Aalborg, Denmark.
  • Elske Ammenwerth
    Institute of Medical Informatics, UMIT TIROL - Private University for Health Sciences and Health Technology, Eduard Wallnöfer Zentrum 1, Hall in Tirol, 6060 Austria.

Keywords

External Resources

Popular Topics
Recent Journals