Adaptive Attention Memory Graph Convolutional Networks for Skeleton-Based Action Recognition.

Journal: Sensors (Basel, Switzerland)
Published Date:

Abstract

Graph Convolutional Networks (GCNs) have attracted a lot of attention and shown remarkable performance for action recognition in recent years. For improving the recognition accuracy, how to build graph structure adaptively, select key frames and extract discriminative features are the key problems of this kind of method. In this work, we propose a novel Adaptive Attention Memory Graph Convolutional Networks (AAM-GCN) for human action recognition using skeleton data. We adopt GCN to adaptively model the spatial configuration of skeletons and employ Gated Recurrent Unit (GRU) to construct an attention-enhanced memory for capturing the temporal feature. With the memory module, our model can not only remember what happened in the past but also employ the information in the future using multi-bidirectional GRU layers. Furthermore, in order to extract discriminative temporal features, the attention mechanism is also employed to select key frames from the skeleton sequence. Extensive experiments on Kinetics, NTU RGB+D and HDM05 datasets show that the proposed network achieves better performance than some state-of-the-art methods.

Authors

  • Di Liu
    Laboratory of Nutrition and Functional Food, College of Food Science and Engineering, Jilin University, Changchun, China.
  • Hui Xu
    No 202 Hospital of People's Liberation Army, Liaoning 110003, China.
  • Jianzhong Wang
  • Yinghua Lu
    School of Information Science and Technology , Northeast Normal University , Changchun , 130117 , China.
  • Jun Kong
    Stony Brook University, Stony Brook, NY.
  • Miao Qi
    College of Computer Science and Information Technology, Northeast Normal University, Changchun, China.