Name
Abstract | ||
|---|---|---|
We revisit the study of a wrist-mounted camera system (referred to as HandCam) for recognizing activities of hands. HandCam has two unique properties as compared to egocentric systems (referred to as HeadCam): (1) it avoids the need to detect hands; (2) it more consistently observes the activities of hands. By taking advantage of these properties, we propose a deep-learning-based method to recognize hand states (free vs. active hands, hand gestures, object categories), and discover object categories. Moreover, we propose a novel two-streams deep network to further take advantage of both HandCam and HeadCam. We have collected a new synchronized HandCam and HeadCam dataset with 20 videos captured in three scenes for hand states recognition. Experiments show that our HandCam system consistently outperforms a deeplearning-based HeadCam method (with estimated manipulation regions) and a dense-trajectory-based HeadCam method in all tasks. We also show that HandCam videos captured by different users can be easily aligned to improve free vs. active recognition accuracy (3.3% improvement) in across-scenes use case. Moreover, we observe that finetuning Convolutional Neural Network consistently improves accuracy. Finally, our novel two-streams deep network combining HandCam and HeadCam achieves the best performance in four out of five tasks. With more data, we believe a joint HandCam and HeadCam system can robustly log hand states in daily life. |
Year | DOI | Venue |
|---|---|---|
2016 | 10.1007/978-3-319-46493-0_31 | COMPUTER VISION - ECCV 2016, PT IV |
Keywords | Field | DocType |
Activity recognition, Wearable camera | Computer vision,Activity recognition,Convolutional neural network,Computer science,Gesture,Speech recognition,Artificial intelligence,Deep learning | Conference |
Volume | ISSN | Citations |
9908 | 0302-9743 | 3 |
PageRank | References | Authors |
0.38 | 27 | 5 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| chengsheng chan | 1 | 4 | 1.08 |
| shouzhong chen | 2 | 3 | 0.72 |
| peixuan xie | 3 | 3 | 0.72 |
| chiungchih chang | 4 | 3 | 0.72 |
| Min Sun | 5 | 1083 | 59.15 |