Towards multimodal graph neural networks for surgical instrument anticipation.

Journal: International journal of computer assisted radiology and surgery
PMID: 38985412

Abstract

PURPOSE: Decision support systems and context-aware assistance in the operating room have emerged as the key clinical applications supporting surgeons in their daily work and are generally based on single modalities. The model- and knowledge-based integration of multimodal data as a basis for decision support systems that can dynamically adapt to the surgical workflow has not yet been established. Therefore, we propose a knowledge-enhanced method for fusing multimodal data for anticipation tasks.

Authors

  • Lars Wagner
    Department of Pediatrics, Division of Pediatric Hematology and Oncology, Duke University School of Medicine, Durham, NC.
  • Dennis N Schneider
    Technical University of Munich, TUM School of Medicine and Health, Klinikum rechts der Isar, Research Group MITI, Munich, Germany.
  • Leon Mayer
    Medical Information Technology, Helmholtz Institute for Biomedical Engineering, RWTH Aachen University, 52074 Aachen, Germany.
  • Alissa Jell
    Department of Surgery, Research Group Minimally Invasive Interdisciplinary Therapeutical Intervention (MITI), Klinikum rechts der Isar, Technical University Munich (TUM), Munich, Germany, Department of Surgery, Klinikum rechts der Isar, Technical University Munich (TUM), Munich, Germany.
  • Carolin Müller
    Technical University of Munich, TUM School of Medicine and Health, Klinikum rechts der Isar, Research Group MITI, Munich, Germany.
  • Alexander Lenz
    Chair of Robotics, Artificial Intelligence and Real-time Systems, Technical University of Munich, Boltzmannstr. 3, 85748, Garching, Germany.
  • Alois Knoll
    Institut für Informatik VI, Technische Universität München, Boltzmannstraße 3, 85748 Garching bei München, Germany. Electronic address: knoll@in.tum.de.
  • Dirk Wilhelm
    Department of Surgery, Research Group Minimally Invasive Interdisciplinary Therapeutical Intervention (MITI), Klinikum rechts der Isar, Technical University Munich (TUM), Munich, Germany, Department of Surgery, Klinikum rechts der Isar, Technical University Munich (TUM), Munich, Germany.

Keywords

External Resources

Popular Topics
Recent Journals