Title | ||
---|---|---|
Online Detection And Classification Of Dynamic Hand Gestures With Recurrent 3d Convolutional Neural Networks |
Abstract | ||
---|---|---|
Automatic detection and classification of dynamic hand gestures in real-world systems intended for human computer interaction is challenging as: 1) there is a large diversity in how people perform gestures, making detection and classification difficult; 2) the system must work online in order to avoid noticeable lag between performing a gesture and its classification; in fact, a negative lag (classification before the gesture is finished) is desirable, as feedback to the user can then be truly instantaneous. In this paper, we address these challenges with a recurrent three-dimensional convolutional neural network that performs simultaneous detection and classification of dynamic hand gestures from multi-modal data. We employ connectionist temporal classification to train the network to predict class labels from inprogress gestures in unsegmented input streams. In order to validate our method, we introduce a new challenging multimodal dynamic hand gesture dataset captured with depth, color and stereo-IR sensors. On this challenging dataset, our gesture recognition system achieves an accuracy of 83.8%, outperforms competing state-of-the-art algorithms, and approaches human accuracy of 88.4%. Moreover, our method achieves state-of-the-art performance on SKIG and ChaLearn2014 benchmarks. |
Year | DOI | Venue |
---|---|---|
2016 | 10.1109/CVPR.2016.456 | 2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR) |
Field | DocType | Volume |
Computer vision,Pattern recognition,Convolutional neural network,Computer science,Gesture,Gesture recognition,Speech recognition,Artificial intelligence,Connectionism | Conference | 2016 |
Issue | ISSN | Citations |
1 | 1063-6919 | 20 |
PageRank | References | Authors |
0.63 | 8 | 6 |
Name | Order | Citations | PageRank |
---|---|---|---|
Pavlo O. Molchanov | 1 | 198 | 11.96 |
Xiaodong Yang | 2 | 1094 | 41.92 |
Shalini Gupta | 3 | 299 | 20.42 |
Kihwan Kim | 4 | 409 | 28.22 |
Stephen Tyree | 5 | 548 | 38.32 |
Jan Kautz | 6 | 3615 | 198.77 |