Name
Abstract | ||
|---|---|---|
Head movements are an integral part of human nonverbal communication. As such, the ability to detect various types of head gestures from video is important for robotic systems that need to interact with people or for assistive technologies that may need to detect conversational gestures to aid communication. To this end, we propose a novel Multi-Scale Deep Convolution-LSTM architecture, capable of recognizing short and long term motion patterns found in head gestures, from video data of natural and unconstrained conversations. In particular, our models use Convolutional Neural Networks (CNNs) to learn meaningful representations from short time windows over head motion data. To capture longer term dependencies, we use Recurrent Neural Networks (RNNs) that extract temporal patterns across the output of the CNNs. We compare against classical approaches using discriminative and generative graphical models and show that our model is able to significantly outperform baseline models. |
Year | DOI | Venue |
|---|---|---|
2018 | 10.1109/WACV.2018.00050 | 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) |
Keywords | Field | DocType |
conversational gestures,novel MultiScale Deep Convolution-LSTM architecture,short term motion patterns,long term motion patterns,head gestures,video data,natural conversations,unconstrained conversations,Convolutional Neural Networks,short time windows,head motion data,longer term dependencies,Recurrent Neural Networks,visual signatures,spontaneous head,head movements,human nonverbal communication,robotic systems,assistive technologies,RNN,CNN | Pattern recognition,Gesture,Convolutional neural network,Computer science,Recurrent neural network,Gesture recognition,Nonverbal communication,Artificial intelligence,Graphical model,Hidden Markov model,Discriminative model | Conference |
ISSN | ISBN | Citations |
2472-6737 | 978-1-5386-4887-2 | 0 |
PageRank | References | Authors |
0.34 | 10 | 4 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Mohit Sharma | 1 | 25 | 4.09 |
| Dragan Ahmetovic | 2 | 166 | 21.09 |
| László A. Jeni | 3 | 235 | 16.45 |
| Kris M. Kitani | 4 | 630 | 72.32 |