Real-time face & eye tracking and blink detection using event cameras.

Journal: Neural networks : the official journal of the International Neural Network Society
PMID:

Abstract

Event cameras contain emerging, neuromorphic vision sensors that capture local-light​ intensity changes at each pixel, generating a stream of asynchronous events. This way of acquiring visual information constitutes a departure from traditional frame-based cameras and offers several significant advantages - low energy consumption, high temporal resolution, high dynamic range and low latency. Driver monitoring systems (DMS) are in-cabin safety systems designed to sense and understand a drivers physical and cognitive state. Event cameras are particularly suited to DMS due to their inherent advantages. This paper proposes a novel method to simultaneously detect and track faces and eyes for driver monitoring. A unique, fully convolutional recurrent neural network architecture is presented. To train this network, a synthetic event-based dataset is simulated with accurate bounding box annotations, called Neuromorphic-HELEN. Additionally, a method to detect and analyse drivers' eye blinks is proposed, exploiting the high temporal resolution of event cameras. Behaviour of blinking provides greater insights into a driver level of fatigue or drowsiness. We show that blinks have a unique temporal signature that can be better captured by event cameras.

Authors

  • Cian Ryan
    Xperi Corporation, Galway, Ireland. Electronic address: cian.ryan@xperi.com.
  • Brian O'Sullivan
  • Amr Elrasad
    Xperi Corporation, Galway, Ireland.
  • Aisling Cahill
    Xperi Corporation, Galway, Ireland.
  • Joe Lemley
    National University of Ireland Galway, Ireland.
  • Paul Kielty
    National University of Ireland Galway, Ireland.
  • Christoph Posch
  • Etienne Perot
    Prophesee, Paris, France.