Cheng, Jiale;
Shi, Dongzi;
Li, Chenyang;
Li, Yu;
Ni, Hao;
Jin, Lianwen;
Zhang, Xin;
(2023)
Skeleton-Based Gesture Recognition With Learnable Paths and Signature Features.
IEEE Transactions on Multimedia
10.1109/tmm.2023.3318242.
(In press).
Preview |
Text
Skeleton-Based_Gesture_Recognition_With_Learnable_Paths_and_Signature_Features.pdf - Accepted Version Download (1MB) | Preview |
Abstract
For the skeleton-based gesture recognition, graph convolutional networks (GCNs) have achieved remarkable performance since the human skeleton is a natural graph. However, the biological structure might not be the crucial one for motion analysis. Also, spatial differential information like joint distance and angle between bones may be overlooked during the graph convolution. In this paper, we focus on obtaining meaningful joint groups and extracting their discriminative features by the path signature (PS) theory. Firstly, to characterize the constraints and dependencies of various joints, we propose three types of paths, i.e., spatial, temporal, and learnable path. Especially, a learnable path generation mechanism can group joints together that are not directly connected or far away, according to their kinematic characteristic. Secondly, to obtain informative and compact features, a deep integration of PS with few parameters are introduced. All the computational process is packed into two modules, i.e., spatial-temporal path signature module (ST-PSM) and learnable path signature module (L-PSM) for the convenience of utilization. They are plug-and-play modules available for any neural network like CNNs and GCNs to enhance the feature extraction ability. Extensive experiments have conducted on three mainstream datasets (ChaLearn 2013, ChaLearn 2016, and AUTSL). We achieved the state-of-the-art results with simpler framework and much smaller model size. By inserting our two modules into the several GCN-based networks, we can observe clear improvements demonstrating the great effectiveness of our proposed method.
| Type: | Article |
|---|---|
| Title: | Skeleton-Based Gesture Recognition With Learnable Paths and Signature Features |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1109/tmm.2023.3318242 |
| Publisher version: | https://doi.org/10.1109/TMM.2023.3318242 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | Gesture recognition, Graph convolutional network, Path signature features |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences > Dept of Mathematics |
| URI: | https://discovery-pp.ucl.ac.uk/id/eprint/10178174 |
Archive Staff Only
 |
View Item |
