Optic Disc Classification by Deep Learning versus Expert Neuro-Ophthalmologists.

Journal: Annals of neurology
Published Date:

Abstract

OBJECTIVE: To compare the diagnostic performance of an artificial intelligence deep learning system with that of expert neuro-ophthalmologists in classifying optic disc appearance.

Authors

  • Valérie Biousse
    From the Singapore National Eye Center (D.M., D.T., S.S., C.-Y.C., T.Y.W.), Singapore Eye Research Institute (D.M., R.P.N., D.T., C.V., S.S., C.-Y.C., T.Y.W.), Duke-NUS Medical School (D.M., R.P.N., D.T., S.S., C.-Y.C., T.Y.W.), Institute of High Performance Computing, Agency for Science, Technology, and Research (J.Z., X.X., Y.L.), and Yong Loo Lin School of Medicine, National University of Singapore (S.S., T.Y.W.) - all in Singapore; Farabi Eye Hospital, Tehran University of Medical Science, Tehran, Iran (M.A.F.); the Department of Ophthalmology, Centro Hospitalar e Universitário de Coimbra, and the Coimbra Institute for Biomedical Imaging and Translational Research, University of Coimbra, Coimbra, Portugal (P.F.); the Department of Ophthalmology, Ramathibodi Hospital, Mahidol University, Bangkok, Thailand (K.V.); the Eye Center, Medical Center, University of Freiburg, Freiburg (W.A.L.), and the Department of Ophthalmology, Ruprecht Karl University of Heidelberg, Mannheim (J.B.J.) - both in Germany; IRCCS Istituto delle Scienze Neurologiche di Bologna, Unità Operativa Complessa Clinica Neurologica, and Dipartimento di Scienze Biomediche e Neuromotorie, Università di Bologna, Bologna, Italy (C.L.M.); the Department of Ophthalmology and Visual Sciences, Chinese University of Hong Kong, Hong Kong (C.Y.C.), and Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou (H.Y.) - both in China; the Department of Ophthalmology, Rigshospitalet, University of Copenhagen, Glostrup, Denmark (S.H.); the Department of Ophthalmology, University Hospital of Grenoble-Alpes, and Grenoble-Alpes University, HP2 Laboratory, INSERM Unité 1042, Grenoble (C.C.), Service d'Ophtalmologie, Unité Rétine-Uvéites-Neuro-Ophtalmologie, Hôpital Pellegrin, Centre Hospitalier Universitaire de Bordeaux, Bordeaux (M.-B.R.), the Department of Ophthalmology, Lille Catholic Hospital, Lille Catholic University, and INSERM Unité 1171, Lille (T.T.H.C.), the Department of Ophthalmology, University Hospital Angers, Angers (P.G.), and Rothschild Foundation Hospital, Paris (C.C.-V.) - all in France; the Department of Clinical Neurosciences, Geneva University Hospital, Geneva (N.S.); the Department of Neurology, SUNY Upstate Medical University, Syracuse, NY (L.J.M.); the American Eye Center, Mandaluyong City, Philippines (R.K.); Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, University College London, London (P.Y.-W.-M.), and Cambridge Eye Unit, Addenbrooke's Hospital, Cambridge University Hospitals, and Cambridge Centre for Brain Repair and Medical Research Council Mitochondrial Biology Unit, Department of Clinical Neurosciences, University of Cambridge, Cambridge (P.Y.-W.-M.) - all in the United Kingdom; the Save Sight Institute, Faculty of Health and Medicine, University of Sydney, Sydney (C.L.F.); the Department of Ophthalmology and Neurology, Mayo Clinic, Rochester, MN (J.J.C.); the Department of Neuro-ophthalmology, Sankara Nethralaya, Medical Research Foundation, Chennai, India (S.A.); the Departments of Ophthalmology, Neurology, and Neurosurgery, Johns Hopkins University School of Medicine, Baltimore (N.R.M.); and the Departments of Ophthalmology and Neurology, Emory University School of Medicine, Atlanta (N.J.N., V.B.).
  • Nancy J Newman
    From the Singapore National Eye Center (D.M., D.T., S.S., C.-Y.C., T.Y.W.), Singapore Eye Research Institute (D.M., R.P.N., D.T., C.V., S.S., C.-Y.C., T.Y.W.), Duke-NUS Medical School (D.M., R.P.N., D.T., S.S., C.-Y.C., T.Y.W.), Institute of High Performance Computing, Agency for Science, Technology, and Research (J.Z., X.X., Y.L.), and Yong Loo Lin School of Medicine, National University of Singapore (S.S., T.Y.W.) - all in Singapore; Farabi Eye Hospital, Tehran University of Medical Science, Tehran, Iran (M.A.F.); the Department of Ophthalmology, Centro Hospitalar e Universitário de Coimbra, and the Coimbra Institute for Biomedical Imaging and Translational Research, University of Coimbra, Coimbra, Portugal (P.F.); the Department of Ophthalmology, Ramathibodi Hospital, Mahidol University, Bangkok, Thailand (K.V.); the Eye Center, Medical Center, University of Freiburg, Freiburg (W.A.L.), and the Department of Ophthalmology, Ruprecht Karl University of Heidelberg, Mannheim (J.B.J.) - both in Germany; IRCCS Istituto delle Scienze Neurologiche di Bologna, Unità Operativa Complessa Clinica Neurologica, and Dipartimento di Scienze Biomediche e Neuromotorie, Università di Bologna, Bologna, Italy (C.L.M.); the Department of Ophthalmology and Visual Sciences, Chinese University of Hong Kong, Hong Kong (C.Y.C.), and Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou (H.Y.) - both in China; the Department of Ophthalmology, Rigshospitalet, University of Copenhagen, Glostrup, Denmark (S.H.); the Department of Ophthalmology, University Hospital of Grenoble-Alpes, and Grenoble-Alpes University, HP2 Laboratory, INSERM Unité 1042, Grenoble (C.C.), Service d'Ophtalmologie, Unité Rétine-Uvéites-Neuro-Ophtalmologie, Hôpital Pellegrin, Centre Hospitalier Universitaire de Bordeaux, Bordeaux (M.-B.R.), the Department of Ophthalmology, Lille Catholic Hospital, Lille Catholic University, and INSERM Unité 1171, Lille (T.T.H.C.), the Department of Ophthalmology, University Hospital Angers, Angers (P.G.), and Rothschild Foundation Hospital, Paris (C.C.-V.) - all in France; the Department of Clinical Neurosciences, Geneva University Hospital, Geneva (N.S.); the Department of Neurology, SUNY Upstate Medical University, Syracuse, NY (L.J.M.); the American Eye Center, Mandaluyong City, Philippines (R.K.); Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, University College London, London (P.Y.-W.-M.), and Cambridge Eye Unit, Addenbrooke's Hospital, Cambridge University Hospitals, and Cambridge Centre for Brain Repair and Medical Research Council Mitochondrial Biology Unit, Department of Clinical Neurosciences, University of Cambridge, Cambridge (P.Y.-W.-M.) - all in the United Kingdom; the Save Sight Institute, Faculty of Health and Medicine, University of Sydney, Sydney (C.L.F.); the Department of Ophthalmology and Neurology, Mayo Clinic, Rochester, MN (J.J.C.); the Department of Neuro-ophthalmology, Sankara Nethralaya, Medical Research Foundation, Chennai, India (S.A.); the Departments of Ophthalmology, Neurology, and Neurosurgery, Johns Hopkins University School of Medicine, Baltimore (N.R.M.); and the Departments of Ophthalmology and Neurology, Emory University School of Medicine, Atlanta (N.J.N., V.B.).
  • Raymond P Najjar
    Singapore Eye Research Institute.
  • Caroline Vasseneix
    From the Singapore National Eye Center (D.M., D.T., S.S., C.-Y.C., T.Y.W.), Singapore Eye Research Institute (D.M., R.P.N., D.T., C.V., S.S., C.-Y.C., T.Y.W.), Duke-NUS Medical School (D.M., R.P.N., D.T., S.S., C.-Y.C., T.Y.W.), Institute of High Performance Computing, Agency for Science, Technology, and Research (J.Z., X.X., Y.L.), and Yong Loo Lin School of Medicine, National University of Singapore (S.S., T.Y.W.) - all in Singapore; Farabi Eye Hospital, Tehran University of Medical Science, Tehran, Iran (M.A.F.); the Department of Ophthalmology, Centro Hospitalar e Universitário de Coimbra, and the Coimbra Institute for Biomedical Imaging and Translational Research, University of Coimbra, Coimbra, Portugal (P.F.); the Department of Ophthalmology, Ramathibodi Hospital, Mahidol University, Bangkok, Thailand (K.V.); the Eye Center, Medical Center, University of Freiburg, Freiburg (W.A.L.), and the Department of Ophthalmology, Ruprecht Karl University of Heidelberg, Mannheim (J.B.J.) - both in Germany; IRCCS Istituto delle Scienze Neurologiche di Bologna, Unità Operativa Complessa Clinica Neurologica, and Dipartimento di Scienze Biomediche e Neuromotorie, Università di Bologna, Bologna, Italy (C.L.M.); the Department of Ophthalmology and Visual Sciences, Chinese University of Hong Kong, Hong Kong (C.Y.C.), and Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou (H.Y.) - both in China; the Department of Ophthalmology, Rigshospitalet, University of Copenhagen, Glostrup, Denmark (S.H.); the Department of Ophthalmology, University Hospital of Grenoble-Alpes, and Grenoble-Alpes University, HP2 Laboratory, INSERM Unité 1042, Grenoble (C.C.), Service d'Ophtalmologie, Unité Rétine-Uvéites-Neuro-Ophtalmologie, Hôpital Pellegrin, Centre Hospitalier Universitaire de Bordeaux, Bordeaux (M.-B.R.), the Department of Ophthalmology, Lille Catholic Hospital, Lille Catholic University, and INSERM Unité 1171, Lille (T.T.H.C.), the Department of Ophthalmology, University Hospital Angers, Angers (P.G.), and Rothschild Foundation Hospital, Paris (C.C.-V.) - all in France; the Department of Clinical Neurosciences, Geneva University Hospital, Geneva (N.S.); the Department of Neurology, SUNY Upstate Medical University, Syracuse, NY (L.J.M.); the American Eye Center, Mandaluyong City, Philippines (R.K.); Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, University College London, London (P.Y.-W.-M.), and Cambridge Eye Unit, Addenbrooke's Hospital, Cambridge University Hospitals, and Cambridge Centre for Brain Repair and Medical Research Council Mitochondrial Biology Unit, Department of Clinical Neurosciences, University of Cambridge, Cambridge (P.Y.-W.-M.) - all in the United Kingdom; the Save Sight Institute, Faculty of Health and Medicine, University of Sydney, Sydney (C.L.F.); the Department of Ophthalmology and Neurology, Mayo Clinic, Rochester, MN (J.J.C.); the Department of Neuro-ophthalmology, Sankara Nethralaya, Medical Research Foundation, Chennai, India (S.A.); the Departments of Ophthalmology, Neurology, and Neurosurgery, Johns Hopkins University School of Medicine, Baltimore (N.R.M.); and the Departments of Ophthalmology and Neurology, Emory University School of Medicine, Atlanta (N.J.N., V.B.).
  • Xinxing Xu
    A*STAR, Singapore, Singapore.
  • Daniel S Ting
    Vitreo-Retinal Department, Singapore National Eye Center, Singapore, Singapore.
  • Léonard B Milea
    University of California, Berkeley, Berkeley, California, USA.
  • Jeong-Min Hwang
    Department of Ophthalmology, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Seoul, Korea. Electronic address: hjm@snu.ac.kr.
  • Dong Hyun Kim
    Department of Ophthalmology, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Seoul, Korea.
  • Hee Kyung Yang
    Department of Ophthalmology, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Seoul, Korea.
  • Steffen Hamann
    From the Singapore National Eye Center (D.M., D.T., S.S., C.-Y.C., T.Y.W.), Singapore Eye Research Institute (D.M., R.P.N., D.T., C.V., S.S., C.-Y.C., T.Y.W.), Duke-NUS Medical School (D.M., R.P.N., D.T., S.S., C.-Y.C., T.Y.W.), Institute of High Performance Computing, Agency for Science, Technology, and Research (J.Z., X.X., Y.L.), and Yong Loo Lin School of Medicine, National University of Singapore (S.S., T.Y.W.) - all in Singapore; Farabi Eye Hospital, Tehran University of Medical Science, Tehran, Iran (M.A.F.); the Department of Ophthalmology, Centro Hospitalar e Universitário de Coimbra, and the Coimbra Institute for Biomedical Imaging and Translational Research, University of Coimbra, Coimbra, Portugal (P.F.); the Department of Ophthalmology, Ramathibodi Hospital, Mahidol University, Bangkok, Thailand (K.V.); the Eye Center, Medical Center, University of Freiburg, Freiburg (W.A.L.), and the Department of Ophthalmology, Ruprecht Karl University of Heidelberg, Mannheim (J.B.J.) - both in Germany; IRCCS Istituto delle Scienze Neurologiche di Bologna, Unità Operativa Complessa Clinica Neurologica, and Dipartimento di Scienze Biomediche e Neuromotorie, Università di Bologna, Bologna, Italy (C.L.M.); the Department of Ophthalmology and Visual Sciences, Chinese University of Hong Kong, Hong Kong (C.Y.C.), and Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou (H.Y.) - both in China; the Department of Ophthalmology, Rigshospitalet, University of Copenhagen, Glostrup, Denmark (S.H.); the Department of Ophthalmology, University Hospital of Grenoble-Alpes, and Grenoble-Alpes University, HP2 Laboratory, INSERM Unité 1042, Grenoble (C.C.), Service d'Ophtalmologie, Unité Rétine-Uvéites-Neuro-Ophtalmologie, Hôpital Pellegrin, Centre Hospitalier Universitaire de Bordeaux, Bordeaux (M.-B.R.), the Department of Ophthalmology, Lille Catholic Hospital, Lille Catholic University, and INSERM Unité 1171, Lille (T.T.H.C.), the Department of Ophthalmology, University Hospital Angers, Angers (P.G.), and Rothschild Foundation Hospital, Paris (C.C.-V.) - all in France; the Department of Clinical Neurosciences, Geneva University Hospital, Geneva (N.S.); the Department of Neurology, SUNY Upstate Medical University, Syracuse, NY (L.J.M.); the American Eye Center, Mandaluyong City, Philippines (R.K.); Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, University College London, London (P.Y.-W.-M.), and Cambridge Eye Unit, Addenbrooke's Hospital, Cambridge University Hospitals, and Cambridge Centre for Brain Repair and Medical Research Council Mitochondrial Biology Unit, Department of Clinical Neurosciences, University of Cambridge, Cambridge (P.Y.-W.-M.) - all in the United Kingdom; the Save Sight Institute, Faculty of Health and Medicine, University of Sydney, Sydney (C.L.F.); the Department of Ophthalmology and Neurology, Mayo Clinic, Rochester, MN (J.J.C.); the Department of Neuro-ophthalmology, Sankara Nethralaya, Medical Research Foundation, Chennai, India (S.A.); the Departments of Ophthalmology, Neurology, and Neurosurgery, Johns Hopkins University School of Medicine, Baltimore (N.R.M.); and the Departments of Ophthalmology and Neurology, Emory University School of Medicine, Atlanta (N.J.N., V.B.).
  • John J Chen
    From the Singapore National Eye Center (D.M., D.T., S.S., C.-Y.C., T.Y.W.), Singapore Eye Research Institute (D.M., R.P.N., D.T., C.V., S.S., C.-Y.C., T.Y.W.), Duke-NUS Medical School (D.M., R.P.N., D.T., S.S., C.-Y.C., T.Y.W.), Institute of High Performance Computing, Agency for Science, Technology, and Research (J.Z., X.X., Y.L.), and Yong Loo Lin School of Medicine, National University of Singapore (S.S., T.Y.W.) - all in Singapore; Farabi Eye Hospital, Tehran University of Medical Science, Tehran, Iran (M.A.F.); the Department of Ophthalmology, Centro Hospitalar e Universitário de Coimbra, and the Coimbra Institute for Biomedical Imaging and Translational Research, University of Coimbra, Coimbra, Portugal (P.F.); the Department of Ophthalmology, Ramathibodi Hospital, Mahidol University, Bangkok, Thailand (K.V.); the Eye Center, Medical Center, University of Freiburg, Freiburg (W.A.L.), and the Department of Ophthalmology, Ruprecht Karl University of Heidelberg, Mannheim (J.B.J.) - both in Germany; IRCCS Istituto delle Scienze Neurologiche di Bologna, Unità Operativa Complessa Clinica Neurologica, and Dipartimento di Scienze Biomediche e Neuromotorie, Università di Bologna, Bologna, Italy (C.L.M.); the Department of Ophthalmology and Visual Sciences, Chinese University of Hong Kong, Hong Kong (C.Y.C.), and Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou (H.Y.) - both in China; the Department of Ophthalmology, Rigshospitalet, University of Copenhagen, Glostrup, Denmark (S.H.); the Department of Ophthalmology, University Hospital of Grenoble-Alpes, and Grenoble-Alpes University, HP2 Laboratory, INSERM Unité 1042, Grenoble (C.C.), Service d'Ophtalmologie, Unité Rétine-Uvéites-Neuro-Ophtalmologie, Hôpital Pellegrin, Centre Hospitalier Universitaire de Bordeaux, Bordeaux (M.-B.R.), the Department of Ophthalmology, Lille Catholic Hospital, Lille Catholic University, and INSERM Unité 1171, Lille (T.T.H.C.), the Department of Ophthalmology, University Hospital Angers, Angers (P.G.), and Rothschild Foundation Hospital, Paris (C.C.-V.) - all in France; the Department of Clinical Neurosciences, Geneva University Hospital, Geneva (N.S.); the Department of Neurology, SUNY Upstate Medical University, Syracuse, NY (L.J.M.); the American Eye Center, Mandaluyong City, Philippines (R.K.); Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, University College London, London (P.Y.-W.-M.), and Cambridge Eye Unit, Addenbrooke's Hospital, Cambridge University Hospitals, and Cambridge Centre for Brain Repair and Medical Research Council Mitochondrial Biology Unit, Department of Clinical Neurosciences, University of Cambridge, Cambridge (P.Y.-W.-M.) - all in the United Kingdom; the Save Sight Institute, Faculty of Health and Medicine, University of Sydney, Sydney (C.L.F.); the Department of Ophthalmology and Neurology, Mayo Clinic, Rochester, MN (J.J.C.); the Department of Neuro-ophthalmology, Sankara Nethralaya, Medical Research Foundation, Chennai, India (S.A.); the Departments of Ophthalmology, Neurology, and Neurosurgery, Johns Hopkins University School of Medicine, Baltimore (N.R.M.); and the Departments of Ophthalmology and Neurology, Emory University School of Medicine, Atlanta (N.J.N., V.B.).
  • Yong Liu
    Department of Critical care medicine, Shenzhen Hospital, Southern Medical University, Guangdong, Shenzhen, China.
  • Tien Yin Wong
    Singapore National Eye Center, Duke-National University of Singapore Medical School, Singapore 168751, Singapore; National Institutes of Health Research Biomedical Research Centre Biomedical Centre, Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK.
  • Dan Milea
    Singapore National Eye Centre.