Dual-Stream Spatiotemporal Networks with Feature Sharing for Monitoring Animals in the Home Cage.
Journal:
Sensors (Basel, Switzerland)
Published Date:
Nov 30, 2023
Abstract
This paper presents a spatiotemporal deep learning approach for mouse behavioral classification in the home-cage. Using a series of dual-stream architectures with assorted modifications for optimal performance, we introduce a novel approach that jointly processes the streams at regular intervals throughout the network. The dataset in focus is an annotated, publicly available dataset of a singly-housed mouse. We achieved even better classification accuracy by ensembling the best performing models; an Inception-based network and an attention-based network, both of which utilize this attribute. Furthermore, we demonstrate through ablation studies that for all models, the architectures consistently outperform the conventional dual-stream having standalone streams. In particular, the inception-based architectures showed higher gains with their increase in accuracy anywhere between 6.59% and 15.19%. The best-performing models were also further evaluated on other mouse behavioral datasets.
