Title
User-oriented Natural Human-Robot Control with Thin-Plate Splines and LRCN
Abstract
We propose a real-time vision-based teleoperation approach for robotic arms that employs a single depth-based camera, exempting the user from the need for any wearable devices. By employing a natural user interface, this novel approach leverages the conventional fine-tuning control, turning it into a direct body pose capture process. The proposed approach is comprised of two main parts. The first is a nonlinear customizable pose mapping based on Thin-Plate Splines (TPS), to directly transfer human body motion to robotic arm motion in a nonlinear fashion, thus allowing matching dissimilar bodies with different workspace shapes and kinematic constraints. The second is a Deep Neural Network hand-state classifier based on Long-term Recurrent Convolutional Networks (LRCNs) that exploits the temporal coherence of the acquired depth data. We validate, evaluate and compare our approach through both classical cross-validation experiments of the proposed hand state classifier; and user studies over a set of practical experiments involving variants of pick-and-place and manufacturing tasks. Results revealed that LRCNs outperform single image Convolutional Neural Networks; and that users’ learning curves were steep, thus allowing the successful completion of the proposed tasks. When compared to a previous approach, the TPS approach revealed no increase in task complexity and similar times of completion, while providing more precise operation in regions closer to workspace boundaries.
Year
DOI
Venue
2022
10.1007/s10846-021-01560-6
Journal of Intelligent & Robotic Systems
Keywords
DocType
Volume
Human-robot interaction, Teleoperation, Natural user interface, Kinematics mapping, Thin-plate spline, Long-term recurrent convolutional networks
Journal
104
Issue
ISSN
Citations 
3
0921-0296
0
PageRank 
References 
Authors
0.34
18
7