Transparency and reproducibility in artificial intelligence.

Journal: Nature
Published Date: Oct 14, 2020

Abstract

Breakthroughs in artificial intelligence (AI) hold enormous potential as it can automate complex tasks and go even beyond human performance. In their study, McKinney et al. showed the high potential of AI for breast cancer screening. However, the lack of methods’ details and algorithm code undermines its scientific value. Here, we identify obstacles hindering transparent and reproducible AI research as faced by McKinney et al., and provide solutions to these obstacles with implications for the broader field.

Authors

  • Benjamin Haibe-Kains
    Princess Margaret Cancer Centre, University Health Network, Canada, Toronto, ON, Canada.
  • George Alexandru Adam
    Department of Computer Science, University of Toronto, Toronto, Ontario, Canada.
  • Ahmed Hosny
    Department of Radiation Oncology, Dana-Farber Cancer Institute, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts.
  • Farnoosh Khodakarami
    Princess Margaret Cancer Centre, University Health Network, Toronto, Ontario, Canada.
  • Levi Waldron
    Graduate School of Public Health and Health Policy, City University of New York, New York, New York, United States of America.
  • Bo Wang
    Department of Clinical Laboratory Medicine Center, Inner Mongolia Autonomous Region People's Hospital, Hohhot, Inner Mongolia, China.
  • Chris McIntosh
  • Anna Goldenberg
    SickKids Research Institute, 686 Bay Street, Toronto, ON M5G 0A4, Canada; Department of Computer Science, University of Toronto, 40 St. George Street, Toronto, ON M5S 2E4, Canada. Electronic address: anna.goldenberg@utoronto.ca.
  • Anshul Kundaje
    Department of Computer Science, Stanford University, Stanford, CA, USA.
  • Casey S Greene
    Institute for Biomedical Informatics, Perelman School of Medicine, University of Pennsylvania, United States; Department of Systems Pharmacology and Translational Therapeutics, Perelman School of Medicine, University of Pennsylvania, United States; Institute for Translational Medicine and Therapeutics, University of Pennsylvania, Perelman School of Medicine, University of Pennsylvania, United States. Electronic address: csgreene@upenn.edu.
  • Tamara Broderick
    Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA.
  • Michael M Hoffman
    Princess Margaret Cancer Centre, Toronto, Ontario, Canada.
  • Jeffrey T Leek
    Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.
  • Keegan Korthauer
    Department of Statistics, University of British Columbia, Vancouver, British Columbia, Canada.
  • Wolfgang Huber
    II. Medizinische Klinik und Poliklinik. Klinikum rechts der Isar der Technischen Universität München, 81675 München, Germany.
  • Alvis Brazma
    European Bioinformatics Institute, European Molecular Biology Laboratory, Wellcome Genome Campus, Cambridge, UK. brazma@ebi.ac.uk.
  • Joelle Pineau
    Department of Computer Science, McGill University, Montréal, Québec, Canada; Mila-Quebec Artificial Intelligence Institute, Montréal, Québec, Canada.
  • Robert Tibshirani
    Department of Statistics, Stanford University , Stanford, California 94305, United States.
  • Trevor Hastie
    Department of Statistics, Stanford University, Stanford, CA 94305, USA.
  • John P A Ioannidis
    Stanford Prevention Research Center, Department of Medicine, Stanford University, Stanford, California.
  • John Quackenbush
    Department of Biostatistics & Computational Biology, Dana-Farber Cancer Institute, Boston, Massachusetts.
  • Hugo J W L Aerts
    Department of Radiation Oncology, Brigham and Women's Hospital, Dana-Farber Cancer Institute, Harvard Medical School, Boston, MA, United States.

Keywords

External Resources

Popular Topics
Recent Journals