Deep learning-based point-scanning super-resolution imaging.

Journal: Nature methods
Published Date: Mar 8, 2021

Abstract

Point-scanning imaging systems are among the most widely used tools for high-resolution cellular and tissue imaging, benefiting from arbitrarily defined pixel sizes. The resolution, speed, sample preservation and signal-to-noise ratio (SNR) of point-scanning systems are difficult to optimize simultaneously. We show these limitations can be mitigated via the use of deep learning-based supersampling of undersampled images acquired on a point-scanning system, which we term point-scanning super-resolution (PSSR) imaging. We designed a 'crappifier' that computationally degrades high SNR, high-pixel resolution ground truth images to simulate low SNR, low-resolution counterparts for training PSSR models that can restore real-world undersampled images. For high spatiotemporal resolution fluorescence time-lapse data, we developed a 'multi-frame' PSSR approach that uses information in adjacent frames to improve model predictions. PSSR facilitates point-scanning image acquisition with otherwise unattainable resolution, speed and sensitivity. All the training data, models and code for PSSR are publicly available at 3DEM.org.

Authors

  • Linjing Fang
    Nancy E. and Peter C. Meinig School of Biomedical Engineering, Cornell University, Ithaca, NY, United States of America.
  • Fred Monroe
    Wicklow AI Medical Research Initiative, San Francisco, CA, USA.
  • Sammy Weiser Novak
    Waitt Advanced Biophotonics Center, Salk Institute for Biological Studies, La Jolla, CA, USA.
  • Lyndsey Kirk
    Department of Neuroscience, Center for Learning and Memory, Institute for Neuroscience, University of Texas at Austin, Austin, TX, USA.
  • Cara R Schiavon
    Waitt Advanced Biophotonics Center, Salk Institute for Biological Studies, La Jolla, CA, USA.
  • Seungyoon B Yu
    Neurobiology Section, Division of Biological Sciences, University of California San Diego, La Jolla, CA, USA.
  • Tong Zhang
    Beijing University of Chinese Medicine, Beijing, China.
  • Melissa Wu
    Waitt Advanced Biophotonics Center, Salk Institute for Biological Studies, La Jolla, CA, USA.
  • Kyle Kastner
    Montreal Institute for Learning Algorithms, Université de Montréal, Montréal, Canada.
  • Alaa Abdel Latif
    Fast.AI, University of San Francisco Data Institute, San Francisco, CA, USA.
  • Zijun Lin
    Fast.AI, University of San Francisco Data Institute, San Francisco, CA, USA.
  • Andrew Shaw
    Fast.AI, University of San Francisco Data Institute, San Francisco, CA, USA.
  • Yoshiyuki Kubota
    Division of Cerebral Circuitry, National Institute for Physiological Sciences, Okazaki, Japan.
  • John Mendenhall
    Department of Neuroscience, Center for Learning and Memory, Institute for Neuroscience, University of Texas at Austin, Austin, TX, USA.
  • Zhao Zhang
  • Gulcin Pekkurnaz
    Neurobiology Section, Division of Biological Sciences, University of California San Diego, La Jolla, CA, USA.
  • Kristen Harris
    Department of Neuroscience, Center for Learning and Memory, Institute for Neuroscience, University of Texas at Austin, Austin, TX, USA.
  • Jeremy Howard
    Fast.AI, University of San Francisco Data Institute, San Francisco, CA, USA.
  • Uri Manor
    Waitt Advanced Biophotonics Center, Salk Institute for Biological Studies, La Jolla, CA, USA. umanor@salk.edu.

Keywords

External Resources

Popular Topics
Recent Journals