Mask-Guided Convolutional Neural Network for Breast Tumor Prognostic Outcome Prediction on 3D DCE-MR Images.

Journal: Journal of digital imaging
Published Date:

Abstract

In this proof-of-concept work, we have developed a 3D-CNN architecture that is guided by the tumor mask for classifying several patient-outcomes in breast cancer from the respective 3D dynamic contrast-enhanced MRI (DCE-MRI) images. The tumor masks on DCE-MRI images were generated using pre- and post-contrast images and validated by experienced radiologists. We show that our proposed mask-guided classification has a higher accuracy than that from either the full image without tumor masks (including background) or the masked voxels only. We have used two patient outcomes for this study: (1) recurrence of cancer after 5 years of imaging and (2) HER2 status, for comparing accuracies of different models. By looking at the activation maps, we conclude that an image-based prediction model using 3D-CNN could be improved by even a conservatively generated mask, rather than overly trusting an unguided, blind 3D-CNN. A blind CNN may classify accurately enough, while its attention may really be focused on a remote region within 3D images. On the other hand, only using a conservatively segmented region may not be as good for classification as using full images but forcing the model's attention toward the known regions of interest.

Authors

  • Gengbo Liu
    Department of Computer Engineering and Sciences, Florida Institute of Technology, Melbourne, FL, USA.
  • Debasis Mitra
    Department of Computer Engineering and Sciences, Florida Institute of Technology, Melbourne, FL, USA. dmitra@cs.fit.edu.
  • Ella F Jones
    Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA, USA.
  • Benjamin L Franc
    From the Department of Radiology and Biomedical Imaging (Y.D., J.H.S., H.T., R.H., N.W.J., T.P.C., M.S.A., C.M.A., S.C.B., R.R.F., S.Y.H., Y.S., R.A.H., M.H.P., B.L.F.) and Institute for Computational Health Sciences (J.H.S., M.G.K., H.T., D.L., K.A.Z., D.H.), University of California, San Francisco, 550 Parnassus Ave, San Francisco, CA 94143; Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, Calif (Y.D.); and Department of Radiology, University of California, Davis, Sacramento, Calif (L.N.).
  • Spencer C Behr
    From the Department of Radiology and Biomedical Imaging (Y.D., J.H.S., H.T., R.H., N.W.J., T.P.C., M.S.A., C.M.A., S.C.B., R.R.F., S.Y.H., Y.S., R.A.H., M.H.P., B.L.F.) and Institute for Computational Health Sciences (J.H.S., M.G.K., H.T., D.L., K.A.Z., D.H.), University of California, San Francisco, 550 Parnassus Ave, San Francisco, CA 94143; Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, Calif (Y.D.); and Department of Radiology, University of California, Davis, Sacramento, Calif (L.N.).
  • Alex Nguyen
    Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA, USA.
  • Marjan S Bolouri
    Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA, USA.
  • Dorota J Wisner
    Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA, USA.
  • Bonnie N Joe
    Department of Radiology, University of California, San Francisco, San Francisco, Calif.
  • Laura J Esserman
    Department of Surgery, University of California, San Francisco, CA, USA.
  • Nola M Hylton
    Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA, USA.
  • Youngho Seo
    Department of Radiology and Biomedical Imaging, University of California San Francisco, San Francisco, California.