Deep Learning-Based Violin Bowing Action Recognition.

Journal: Sensors (Basel, Switzerland)
Published Date:

Abstract

We propose a violin bowing action recognition system that can accurately recognize distinct bowing actions in classical violin performance. This system can recognize bowing actions by analyzing signals from a depth camera and from inertial sensors that are worn by a violinist. The contribution of this study is threefold: (1) a dataset comprising violin bowing actions was constructed from data captured by a depth camera and multiple inertial sensors; (2) data augmentation was achieved for depth-frame data through rotation in three-dimensional world coordinates and for inertial sensing data through yaw, pitch, and roll angle transformations; and, (3) bowing action classifiers were trained using different modalities, to compensate for the strengths and weaknesses of each modality, based on deep learning methods with a decision-level fusion process. In experiments, large external motions and subtle local motions produced from violin bow manipulations were both accurately recognized by the proposed system (average accuracy > 80%).

Authors

  • Shih-Wei Sun
    Department of New Media Art, Taipei National University of the Arts, Taipei 112, Taiwan. swsun@newmedia.tnua.edu.tw.
  • Bao-Yun Liu
    Deptartment of Communication Engineering, National Central University, Taoyuan 32001, Taiwan.
  • Pao-Chi Chang