Title
Concurrent Activity Recognition with Multimodal CNN-LSTM Structure.
Abstract
We introduce a system that recognizes concurrent activities from real-world data captured by multiple sensors of different types. The recognition is achieved in two steps. First, we extract spatial and temporal features from the multimodal data. We feed each datatype into a convolutional neural network that extracts spatial features, followed by a long-short term memory network that extracts temporal information in the sensory data. The extracted features are then fused for decision making in the second step. Second, we achieve concurrent activity recognition with a single classifier that encodes a binary output vector in which elements indicate whether the corresponding activity types are currently in progress. We tested our system with three datasets from different domains recorded using different sensors and achieved performance comparable to existing systems designed specifically for those domains. Our system is the first to address the concurrent activity recognition with multisensory data using a single model, which is scalable, simple to train and easy to deploy.
Year
Venue
Field
2017
arXiv: Computer Vision and Pattern Recognition
Activity recognition,Pattern recognition,Convolutional neural network,Computer science,Artificial intelligence,Sensory system,Classifier (linguistics),Multiple sensors,Machine learning,Scalability,Binary number
DocType
Volume
Citations 
Journal
abs/1702.01638
0
PageRank 
References 
Authors
0.34
28
7
Name
Order
Citations
PageRank
Xinyu Li18837.72
Yanyi Zhang2296.40
Jianyu Zhang302.03
Shuhong Chen462.84
Ivan Marsic571691.96
Richard A. Farneth6114.44
Randall S. Burd712221.53