A Primer on Motion Capture with Deep Learning: Principles, Pitfalls, and Perspectives.

Journal: Neuron
Published Date: Oct 14, 2020

Abstract

Extracting behavioral measurements non-invasively from video is stymied by the fact that it is a hard computational problem. Recent advances in deep learning have tremendously advanced our ability to predict posture directly from videos, which has quickly impacted neuroscience and biology more broadly. In this primer, we review the budding field of motion capture with deep learning. In particular, we will discuss the principles of those novel algorithms, highlight their potential as well as pitfalls for experimentalists, and provide a glimpse into the future.

Authors

  • Alexander Mathis
    Institute for Theoretical Physics and Werner Reichardt Centre for Integrative Neuroscience, Eberhard Karls Universität Tübingen, Tübingen, Germany.
  • Steffen Schneider
    The Rowland Institute at Harvard, Harvard University, Cambridge, MA, USA; University of Tübingen and International Max Planck Research School for Intelligent Systems, Tübingen, Germany.
  • Jessy Lauer
    Center for Neuroprosthetics, Center for Intelligent Systems, Swiss Federal Institute of Technology (EPFL), Lausanne, Switzerland; Brain Mind Institute, School of Life Sciences, Swiss Federal Institute of Technology (EPFL), Lausanne, Switzerland; The Rowland Institute at Harvard, Harvard University, Cambridge, MA, USA.
  • Mackenzie Weygandt Mathis
    Institute for Theoretical Physics and Werner Reichardt Centre for Integrative Neuroscience, Eberhard Karls Universität Tübingen, Tübingen, Germany. mackenzie@post.harvard.edu.

Keywords

External Resources

Popular Topics
Recent Journals