Comparative validation of machine learning algorithms for surgical workflow and skill analysis with the HeiChole benchmark.

Journal: Medical image analysis
Published Date:

Abstract

PURPOSE: Surgical workflow and skill analysis are key technologies for the next generation of cognitive surgical assistance systems. These systems could increase the safety of the operation through context-sensitive warnings and semi-autonomous robotic assistance or improve training of surgeons via data-driven feedback. In surgical workflow analysis up to 91% average precision has been reported for phase recognition on an open data single-center video dataset. In this work we investigated the generalizability of phase recognition algorithms in a multicenter setting including more difficult recognition tasks such as surgical action and surgical skill.

Authors

  • Martin Wagner
    Department of Biology, Norwegian University of Science and Technology, 5 Høgskoleringen, 7491 Trondheim, Norway. Electronic address: martin.wagner@ntnu.no.
  • Beat-Peter Müller-Stich
    Department of General, Visceral and Transplantation Surgery, Heidelberg University Hospital, Im Neuenheimer Feld 420, 69120, Heidelberg, Germany.
  • Anna Kisilenko
    Department for General, Visceral and Transplantation Surgery, Heidelberg University Hospital, Im Neuenheimer Feld 420, 69120 Heidelberg, Germany; National Center for Tumor Diseases (NCT) Heidelberg, Im Neuenheimer Feld 460, 69120 Heidelberg, Germany.
  • Duc Tran
    Department of Computer Science and Engineering, University of Nevada Reno, Reno, NV, USA.
  • Patrick Heger
    Department for General, Visceral and Transplantation Surgery, Heidelberg University Hospital, Im Neuenheimer Feld 420, 69120 Heidelberg, Germany.
  • Lars Mündermann
    KARL STORZ SE & Co. KG, Tuttlingen, Germany.
  • David M Lubotsky
    Department for General, Visceral and Transplantation Surgery, Heidelberg University Hospital, Im Neuenheimer Feld 420, 69120 Heidelberg, Germany; National Center for Tumor Diseases (NCT) Heidelberg, Im Neuenheimer Feld 460, 69120 Heidelberg, Germany.
  • Benjamin Müller
    Department for General, Visceral and Transplantation Surgery, Heidelberg University Hospital, Im Neuenheimer Feld 420, 69120 Heidelberg, Germany; National Center for Tumor Diseases (NCT) Heidelberg, Im Neuenheimer Feld 460, 69120 Heidelberg, Germany.
  • Tornike Davitashvili
    Department for General, Visceral and Transplantation Surgery, Heidelberg University Hospital, Im Neuenheimer Feld 420, 69120 Heidelberg, Germany; National Center for Tumor Diseases (NCT) Heidelberg, Im Neuenheimer Feld 460, 69120 Heidelberg, Germany.
  • Manuela Capek
    Department for General, Visceral and Transplantation Surgery, Heidelberg University Hospital, Im Neuenheimer Feld 420, 69120 Heidelberg, Germany; National Center for Tumor Diseases (NCT) Heidelberg, Im Neuenheimer Feld 460, 69120 Heidelberg, Germany.
  • Annika Reinke
    German Cancer Research Center DKFZ, Division of Computer Assisted Medical Interventions, Heidelberg, Germany. Electronic address: a.reinke@dkfz.de.
  • Carissa Reid
    Division of Biostatistics, German Cancer Research Center, Im Neuenheimer Feld 280, Heidelberg, Germany.
  • Tong Yu
  • Armine Vardazaryan
    ICube, University of Strasbourg, CNRS, IHU Strasbourg, France.
  • Chinedu Innocent Nwoye
    ICube, University of Strasbourg, CNRS, IHU, Strasbourg, France. nwoye.chinedu@gmail.com.
  • Nicolas Padoy
    IHU Strasbourg, Strasbourg, France.
  • Xinyang Liu
    Sheikh Zayed Institute for Pediatric Surgical Innovation, the Children's National Medical Center, Washington DC, USA.
  • Eung-Joo Lee
    Department of Electrical and Computer Engineering and the Institute for Advanced Computer Studies, University of Maryland, College Park, MD, USA.
  • Constantin Disch
    Fraunhofer Institute for Digital Medicine MEVIS, Max-von-Laue-Str. 2, 28359 Bremen, Germany.
  • Hans Meine
    Fraunhofer Institute for Medical Image Computing MEVIS, Bremen, Germany.
  • Tong Xia
    Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.
  • Fucang Jia
    Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, 1068 Xueyuan Boulevard, Shenzhen, 518055, China.
  • Satoshi Kondo
    Konica Minolta, Inc., Osaka, 569-8503, Japan.
  • Wolfgang Reiter
    Wintegral GmbH, München, Germany. wolfgang.reiter@wintegral.net.
  • Yueming Jin
    Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China.
  • Yonghao Long
    Department of Computer Science and Engineering, Ho Sin-Hang Engineering Building, The Chinese University of Hong Kong, Sha Tin, NT, Hong Kong.
  • Meirui Jiang
    Department of Computer Science and Engineering, Ho Sin-Hang Engineering Building, The Chinese University of Hong Kong, Sha Tin, NT, Hong Kong.
  • Qi Dou
  • Pheng Ann Heng
  • Isabell Twick
    Caresyntax GmbH, Komturstraße 18A, 12099, Berlin, Germany. isabell.twick@caresyntax.com.
  • Kadir Kirtac
    Caresyntax GmbH, Komturstr. 18A, 12099 Berlin, Germany.
  • Enes Hosgor
    Caresyntax GmbH, Komturstraße 18A, 12099, Berlin, Germany.
  • Jon Lindström Bolmgren
    Caresyntax GmbH, Komturstr. 18A, 12099 Berlin, Germany.
  • Michael Stenzel
    Caresyntax GmbH, Komturstr. 18A, 12099 Berlin, Germany.
  • Björn von Siemens
    Caresyntax GmbH, Komturstr. 18A, 12099 Berlin, Germany.
  • Long Zhao
    Department of Respiratory Medicine and Intensive Care Unit.Peking University People's Hospital, Beijing 100044, China.
  • Zhenxiao Ge
    Hikvision Research Institute, Hangzhou, China.
  • Haiming Sun
    Hikvision Research Institute, Hangzhou, China.
  • Di Xie
    School of Life Science, Liaoning University, Shenyang, 110036, China.
  • Mengqi Guo
    School of Computing, National University of Singapore, Computing 1, No.13 Computing Drive, 117417, Singapore.
  • Daochang Liu
    Department of Computer Science, Peking University, Beijing, China.
  • Hannes G Kenngott
  • Felix Nickel
    Department of General, Visceral, and Transplantation Surgery, University of Heidelberg, Im Neuenheimer Feld 110, 69120, Heidelberg, Germany. felix.nickel@med.uni-heidelberg.de.
  • Moritz von Frankenberg
    Department of Surgery, Salem Hospital of the Evangelische Stadtmission Heidelberg, Zeppelinstrasse 11-33, 69121 Heidelberg, Germany.
  • Franziska Mathis-Ullrich
    Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology (KIT), 76131, Karlsruhe, Germany.
  • Annette Kopp-Schneider
    Division of Biostatistics, German Cancer Research Center, Im Neuenheimer Feld 581, 69210, Heidelberg, Germany.
  • Lena Maier-Hein
    German Cancer Research Center (DKFZ), Computer Assisted Medical Interventions, Heidelberg, Germany.
  • Stefanie Speidel
    Division of Translational Surgical Oncology, National Center for Tumor Diseases (NCT), Partner Site Dresden, Dresden, Germany.
  • Sebastian Bodenstedt
    Division of Translational Surgical Oncology, National Center for Tumor Diseases (NCT), Partner Site Dresden, Dresden, Germany.