Evaluating the efficacy of artificial intelligence tools for the automation of systematic reviews in cancer research: A systematic review.

Journal: Cancer epidemiology
Published Date: Dec 9, 2023

Abstract

To evaluate the performance accuracy and workload savings of artificial intelligence (AI)-based automation tools in comparison with human reviewers in medical literature screening for systematic reviews (SR) of primary studies in cancer research in order to gain insights on improving the efficiency of producing SRs. Medline, Embase, the Cochrane Library, and PROSPERO databases were searched from inception to November 30, 2022. Then, forward and backward literature searches were completed, and the experts in this field including the authors of the articles included were contacted for a thorough grey literature search. This SR was registered on PROSPERO (CRD 42023384772). Among the 3947 studies obtained from search, five studies met the preplanned study selection criteria. These five studies evaluated four AI tools: Abstrackr (four studies), RobotAnalyst (one), EPPI-Reviewer (one), and DistillerSR (one). Without missing final included citations, Abstrackr eliminated 20%-88% of titles and abstracts (time saving of 7-86 hours) and 59% of the full-texts (62 h) from human review across four different cancer-related SRs. In comparison, RobotAnalyst (1% of titles and abstracts, 1 h), EPPI Review (38% of titles and abstracts, 58 h; 59% of full-texts, 62 h), DistillerSR (42% of titles and abstracts, 22 h) also provided similar or lower work savings for single cancer-related SRs. AI-based automation tools exhibited promising but varying levels of accuracy and efficiency during the screening process of medical literature for conducting SRs in the cancer field. Until further progress is made and thorough evaluations are conducted, AI tools should be utilized as supplementary aids rather than complete substitutes for human reviewers.

Authors

  • Xiaomei Yao
    Department of Oncology, Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Center for Clinical Practice Guideline Conduction and Evaluation, Children's Hospital of Fudan University, Shanghai, China. Electronic address: yaoxia@mcmaster.ca.
  • Mithilesh V Kumar
    Faculty of Engineering, McMaster University, Hamilton, ON, Canada; Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada.
  • Esther Su
    Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada.
  • Athena Flores Miranda
    Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada.
  • Ashirbani Saha
    Department of Radiology, Duke University School of Medicine, 2424 Erwin Road, Suite 302, Durham, NC, 27705, USA. ashirbani.saha@duke.edu.
  • Jonathan Sussman
    Department of Oncology, McMaster University, Hamilton, Ontario, Canada.

Keywords

External Resources

Popular Topics
Recent Journals