EGCN++: A New Fusion Strategy for Ensemble Learning in Skeleton-Based Rehabilitation Exercise Assessment.

Journal: IEEE transactions on pattern analysis and machine intelligence
Published Date:

Abstract

Skeleton-based exercise assessment focuses on evaluating the correctness or quality of an exercise performed by a subject. Skeleton data provide two groups of features (i.e., position and orientation), which existing methods have not fully harnessed. We previously proposed an ensemble-based graph convolutional network (EGCN) that considers both position and orientation features to construct a model-based approach. Integrating these types of features achieved better performance than available methods. However, EGCN lacked a fusion strategy across the data, feature, decision, and model levels. In this paper, we present an advanced framework, EGCN++, for rehabilitation exercise assessment. Based on EGCN, a new fusion strategy called MLE-PO is proposed for EGCN++; this technique considers fusion at the data and model levels. We conduct extensive cross-validation experiments and investigate the consistency between machine and human evaluations on three datasets: UI-PRMD, KIMORE, and EHE. Results demonstrate that MLE-PO outperforms other EGCN ensemble strategies and representative baselines. Furthermore, the MLE-PO's model evaluation scores are more quantitatively consistent with clinical evaluations than other ensemble strategies.

Authors

  • Bruce X B Yu
  • Yan Liu
    Department of Clinical Microbiology, Shanghai Tenth People's Hospital, School of Medicine, Tongji University, Shanghai, 200072, People's Republic of China.
  • Keith C C Chan
    Department of Computing, Hong Kong Polytechnic University, Hong Kong.
  • Chang Wen Chen
    The Department of Computer Science, State University of New York at Buffalo, NY, USA.