Abstract | ||
---|---|---|
We propose a real-time vision-based teleoperation approach for robotic arms that employs a single depth-based camera, exempting the user from the need for any wearable devices. By employing a natural user interface, this novel approach leverages the conventional fine-tuning control, turning it into a direct body pose capture process. The proposed approach is comprised of two main parts. The first is a nonlinear customizable pose mapping based on Thin-Plate Splines (TPS), to directly transfer human body motion to robotic arm motion in a nonlinear fashion, thus allowing matching dissimilar bodies with different workspace shapes and kinematic constraints. The second is a Deep Neural Network hand-state classifier based on Long-term Recurrent Convolutional Networks (LRCNs) that exploits the temporal coherence of the acquired depth data. We validate, evaluate and compare our approach through both classical cross-validation experiments of the proposed hand state classifier; and user studies over a set of practical experiments involving variants of pick-and-place and manufacturing tasks. Results revealed that LRCNs outperform single image Convolutional Neural Networks; and that users’ learning curves were steep, thus allowing the successful completion of the proposed tasks. When compared to a previous approach, the TPS approach revealed no increase in task complexity and similar times of completion, while providing more precise operation in regions closer to workspace boundaries. |
Year | DOI | Venue |
---|---|---|
2022 | 10.1007/s10846-021-01560-6 | Journal of Intelligent & Robotic Systems |
Keywords | DocType | Volume |
Human-robot interaction, Teleoperation, Natural user interface, Kinematics mapping, Thin-plate spline, Long-term recurrent convolutional networks | Journal | 104 |
Issue | ISSN | Citations |
3 | 0921-0296 | 0 |
PageRank | References | Authors |
0.34 | 18 | 7 |
Name | Order | Citations | PageRank |
---|---|---|---|
Bruno Lima | 1 | 0 | 0.34 |
Lucas Amaral | 2 | 0 | 0.34 |
Givanildo Nascimento-Jr | 3 | 0 | 0.34 |
Victor Mafra | 4 | 0 | 0.34 |
Bruno Georgevich Ferreira | 5 | 0 | 0.34 |
Tiago Vieira | 6 | 0 | 0.34 |
Thales Vieira | 7 | 0 | 0.34 |