Early detection of visual impairment in young children using a smartphone-based deep learning system.

Journal: Nature medicine
Published Date: Jan 26, 2023

Abstract

Early detection of visual impairment is crucial but is frequently missed in young children, who are capable of only limited cooperation with standard vision tests. Although certain features of visually impaired children, such as facial appearance and ocular movements, can assist ophthalmic practice, applying these features to real-world screening remains challenging. Here, we present a mobile health (mHealth) system, the smartphone-based Apollo Infant Sight (AIS), which identifies visually impaired children with any of 16 ophthalmic disorders by recording and analyzing their gazing behaviors and facial features under visual stimuli. Videos from 3,652 children (≤48 months in age; 54.5% boys) were prospectively collected to develop and validate this system. For detecting visual impairment, AIS achieved an area under the receiver operating curve (AUC) of 0.940 in an internal validation set and an AUC of 0.843 in an external validation set collected in multiple ophthalmology clinics across China. In a further test of AIS for at-home implementation by untrained parents or caregivers using their smartphones, the system was able to adapt to different testing conditions and achieved an AUC of 0.859. This mHealth system has the potential to be used by healthcare professionals, parents and caregivers for identifying young children with visual impairment across a wide range of ophthalmic disorders.

Authors

  • Wenben Chen
    State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, Guangdong, China.
  • Ruiyang Li
    2State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, Guangdong China.
  • Qinji Yu
    Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University, Shanghai, China.
  • Andi Xu
    Zhongshan School of Medicine, Sun Yat-Sen University, Guangzhou, China.
  • Yile Feng
    VoxelCloud, Shanghai, China.
  • Ruixin Wang
    State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China.
  • Lanqin Zhao
    State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, Guangdong, 510060, China.
  • Zhenzhe Lin
    State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Centre, Sun Yat-sen University, Guangzhou, China.
  • Yahan Yang
    University of Pennsylvania, Philadelphia, PA.
  • Duoru Lin
    State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China.
  • Xiaohang Wu
    State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Xian Lie South Road 54#, Guangzhou, 510060, China.
  • Jingjing Chen
    Department of Cardiovascular Medicine, Affiliated Hospital of Guizhou Medical University, Guiyang, China.
  • Zhenzhen Liu
    Department of Functional Science, School of Medicine, Yangtze University, No.1 Nanhuan Road, Jingzhou City 434100, China.
  • Yuxuan Wu
    State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China.
  • Kang Dang
    Tencent Youtu Lab, Shanghai, People's Republic of China.
  • Kexin Qiu
    VoxelCloud, Shanghai, China.
  • Zilong Wang
    VoxelCloud, Shanghai, China.
  • Ziheng Zhou
    VoxelCloud, Los Angeles, CA, USA.
  • Dong Liu
    Department of Gastrointestinal Surgery, The Third Hospital of Hebei Medical University, Shijiazhuang, China.
  • Qianni Wu
    State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China.
  • Mingyuan Li
  • Yifan Xiang
    2State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, Guangdong China.
  • Xiaoyan Li
    Shulan International Medical College, Zhejiang Shuren University, Hangzhou, Zhejiang, China.
  • Zhuoling Lin
    State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Xian Lie South Road 54#, Guangzhou, 510060, China.
  • Danqi Zeng
    State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China.
  • Yunjian Huang
    State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China.
  • Silang Mo
    School of Medicine, Sun Yat-sen University, Shenzhen, China.
  • Xiucheng Huang
    Zhongshan School of Medicine, Sun Yat-Sen University, Guangzhou, China.
  • Shulin Sun
    Department of Urology, Peking University Third Hospital, Peking University Health Science Center, Beijing, China.
  • Jianmin Hu
    Department of Ophthalmology, The Second Affiliated Hospital of Fujian Medical University, Quanzhou, China.
  • Jun Zhao
  • Meirong Wei
    Dept. of Stomatology, Weifang People's Hospital, Affiliated to Weifang Medical College, Weifang 261042, China.
  • Shoulong Hu
    National Center for Children's Health, Department of Ophthalmology, Beijing Children's Hospital, Capital Medical University, Beijing, China.
  • Liang Chen
    Department of Neurosurgery, Huashan Hospital, Fudan University, Shanghai, China.
  • Bingfa Dai
    Department of Ophthalmology, The Second Affiliated Hospital of Fujian Medical University, Quanzhou, China.
  • Huasheng Yang
    State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China.
  • Danping Huang
    State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China.
  • Xiaoming Lin
    State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China.
  • Lingyi Liang
    State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China.
  • Xiaoyan Ding
    State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China.
  • Yangfan Yang
    State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China.
  • Pengsen Wu
    State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangdong Provincial Key Laboratory of Ophthalmology and Vision Science, Guangdong Provincial Clinical Research Center for Ocular Diseases, Guangzhou, China.
  • Feihui Zheng
    Singapore National Eye Centre, Singapore Eye Research Institute, Singapore.
  • Nick Stanojcic
    Department of Ophthalmology, St. Thomas' Hospital, London, United Kingdom.
  • Ji-Peng Olivia Li
    Moorfields Eye Hospital NHS Foundation Trust, London, United Kingdom.
  • Carol Y Cheung
    Department of Ophthalmology and Visual Sciences, Chinese University of Hong Kong, Hong Kong Special Administrative Region, China. Electronic address: carolcheung@cuhk.edu.hk.
  • Erping Long
    State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Xian Lie South Road 54#, Guangzhou, 510060, China.
  • Chuan Chen
    Information Security Center, State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China.
  • Yi Zhu
    2State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, Guangdong China.
  • Patrick Yu-Wai-Man
    From the Singapore National Eye Center (D.M., D.T., S.S., C.-Y.C., T.Y.W.), Singapore Eye Research Institute (D.M., R.P.N., D.T., C.V., S.S., C.-Y.C., T.Y.W.), Duke-NUS Medical School (D.M., R.P.N., D.T., S.S., C.-Y.C., T.Y.W.), Institute of High Performance Computing, Agency for Science, Technology, and Research (J.Z., X.X., Y.L.), and Yong Loo Lin School of Medicine, National University of Singapore (S.S., T.Y.W.) - all in Singapore; Farabi Eye Hospital, Tehran University of Medical Science, Tehran, Iran (M.A.F.); the Department of Ophthalmology, Centro Hospitalar e Universitário de Coimbra, and the Coimbra Institute for Biomedical Imaging and Translational Research, University of Coimbra, Coimbra, Portugal (P.F.); the Department of Ophthalmology, Ramathibodi Hospital, Mahidol University, Bangkok, Thailand (K.V.); the Eye Center, Medical Center, University of Freiburg, Freiburg (W.A.L.), and the Department of Ophthalmology, Ruprecht Karl University of Heidelberg, Mannheim (J.B.J.) - both in Germany; IRCCS Istituto delle Scienze Neurologiche di Bologna, Unità Operativa Complessa Clinica Neurologica, and Dipartimento di Scienze Biomediche e Neuromotorie, Università di Bologna, Bologna, Italy (C.L.M.); the Department of Ophthalmology and Visual Sciences, Chinese University of Hong Kong, Hong Kong (C.Y.C.), and Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou (H.Y.) - both in China; the Department of Ophthalmology, Rigshospitalet, University of Copenhagen, Glostrup, Denmark (S.H.); the Department of Ophthalmology, University Hospital of Grenoble-Alpes, and Grenoble-Alpes University, HP2 Laboratory, INSERM Unité 1042, Grenoble (C.C.), Service d'Ophtalmologie, Unité Rétine-Uvéites-Neuro-Ophtalmologie, Hôpital Pellegrin, Centre Hospitalier Universitaire de Bordeaux, Bordeaux (M.-B.R.), the Department of Ophthalmology, Lille Catholic Hospital, Lille Catholic University, and INSERM Unité 1171, Lille (T.T.H.C.), the Department of Ophthalmology, University Hospital Angers, Angers (P.G.), and Rothschild Foundation Hospital, Paris (C.C.-V.) - all in France; the Department of Clinical Neurosciences, Geneva University Hospital, Geneva (N.S.); the Department of Neurology, SUNY Upstate Medical University, Syracuse, NY (L.J.M.); the American Eye Center, Mandaluyong City, Philippines (R.K.); Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, University College London, London (P.Y.-W.-M.), and Cambridge Eye Unit, Addenbrooke's Hospital, Cambridge University Hospitals, and Cambridge Centre for Brain Repair and Medical Research Council Mitochondrial Biology Unit, Department of Clinical Neurosciences, University of Cambridge, Cambridge (P.Y.-W.-M.) - all in the United Kingdom; the Save Sight Institute, Faculty of Health and Medicine, University of Sydney, Sydney (C.L.F.); the Department of Ophthalmology and Neurology, Mayo Clinic, Rochester, MN (J.J.C.); the Department of Neuro-ophthalmology, Sankara Nethralaya, Medical Research Foundation, Chennai, India (S.A.); the Departments of Ophthalmology, Neurology, and Neurosurgery, Johns Hopkins University School of Medicine, Baltimore (N.R.M.); and the Departments of Ophthalmology and Neurology, Emory University School of Medicine, Atlanta (N.J.N., V.B.).
  • Ruixuan Wang
    School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, P. R. China. wangruix5@mail.sysu.edu.cn.
  • Wei-Shi Zheng
    School of Information Science and Technology, Sun Yat-sen University, Guangzhou 510006, China; Guangdong Province Key Laboratory of Computational Science, Guangzhou 510275, China. Electronic address: wszheng@ieee.org.
  • Xiaowei Ding
    VoxelCloud, Los Angeles, CA, USA.
  • Haotian Lin
    State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou.

Keywords

External Resources

Popular Topics
Recent Journals