Adapting content-based image retrieval techniques for the semantic annotation of medical images.

Journal: Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society
Published Date: Apr 1, 2016

Abstract

The automatic annotation of medical images is a prerequisite for building comprehensive semantic archives that can be used to enhance evidence-based diagnosis, physician education, and biomedical research. Annotation also has important applications in the automatic generation of structured radiology reports. Much of the prior research work has focused on annotating images with properties such as the modality of the image, or the biological system or body region being imaged. However, many challenges remain for the annotation of high-level semantic content in medical images (e.g., presence of calcification, vessel obstruction, etc.) due to the difficulty in discovering relationships and associations between low-level image features and high-level semantic concepts. This difficulty is further compounded by the lack of labelled training data. In this paper, we present a method for the automatic semantic annotation of medical images that leverages techniques from content-based image retrieval (CBIR). CBIR is a well-established image search technology that uses quantifiable low-level image features to represent the high-level semantic content depicted in those images. Our method extends CBIR techniques to identify or retrieve a collection of labelled images that have similar low-level features and then uses this collection to determine the best high-level semantic annotations. We demonstrate our annotation method using retrieval via weighted nearest-neighbour retrieval and multi-class classification to show that our approach is viable regardless of the underlying retrieval strategy. We experimentally compared our method with several well-established baseline techniques (classification and regression) and showed that our method achieved the highest accuracy in the annotation of liver computed tomography (CT) images.

Authors

  • Ashnil Kumar
    School of Information Technologies, University of Sydney, Australia; Institute of Biomedical Engineering and Technology, University of Sydney, Australia. Electronic address: ashnil.kumar@sydney.edu.au.
  • Shane Dyer
    School of Electrical and Information Engineering, University of Sydney, Australia. Electronic address: sdye9175@uni.sydney.edu.au.
  • Jinman Kim
    School of Information Technologies, University of Sydney, Australia; Institute of Biomedical Engineering and Technology, University of Sydney, Australia. Electronic address: jinman.kim@sydney.edu.au.
  • Changyang Li
    School of Information Technologies, University of Sydney, Australia; Institute of Biomedical Engineering and Technology, University of Sydney, Australia. Electronic address: changyang.li@sydney.edu.au.
  • Philip H W Leong
    Department of Electrical and Information EngineeringThe University of Sydney.
  • Michael Fulham
    School of Information Technologies, University of Sydney, Australia; Institute of Biomedical Engineering and Technology, University of Sydney, Australia; Department of Molecular Imaging, Royal Prince Alfred Hospital, Sydney, Australia; Sydney Medical School, University of Sydney, Australia. Electronic address: michael.fulham@sydney.edu.au.
  • Dagan Feng
    School of Information Technologies, University of Sydney, Australia; Institute of Biomedical Engineering and Technology, University of Sydney, Australia; Med-X Research Institute, Shanghai Jiao Tong University, China. Electronic address: dagan.feng@sydney.edu.au.

Keywords

External Resources

Popular Topics
Recent Journals