Paper Info
Title
Multimodal Human Activity Recognition for Industrial Manufacturing Processes in Robotic Workcells
Abstract
We present an approach for monitoring and interpreting human activities based on a novel multimodal vision-based interface, aiming at improving the efficiency of human-robot interaction (HRI) in industrial environments. Multi-modality is an important concept in this design, where we combine inputs from several state-of-the-art sensors to provide a variety of information, e.g. skeleton and fingertip poses. Based on typical industrial workflows, we derived multiple levels of human activity labels, including large-scale activities (e.g. assembly) and simpler sub-activities (e.g. hand gestures), creating a duration- and complexity-based hierarchy. We train supervised generative classifiers for each activity level and combine the output of this stage with a trained Hierarchical Hidden Markov Model (HHMM), which models not only the temporal aspects between the activities on the same level, but also the hierarchical relationships between the levels.
Year
DOI
Venue
2015
10.1145/2818346.2820738
ACM International Conference on Multimodal Interaction
Keywords
Field
DocType
Human activity recognition, Hierarchical Hidden Markov Model, Industrial robotics, Cognitive robotics
Cognitive robotics,Manufacturing,Computer science,Gesture,Human–computer interaction,Artificial intelligence,Hierarchy,Workflow,Computer vision,Activity recognition,Hierarchical hidden Markov model,Generative grammar,Machine learning
Conference
Citations 
PageRank 
References 
12
0.71
17
Authors
5
Name
Order
Citations
PageRank
Alina Roitberg1241.80
Nikhil Somani2437.34
Alexander Clifford Perzylo3786.55
Markus Rickert421722.78
Alois Knoll Knoll51700271.32