Title
Catch & Carry: reusable neural controllers for vision-guided whole-body tasks
Abstract
AbstractWe address the longstanding challenge of producing flexible, realistic humanoid character controllers that can perform diverse whole-body tasks involving object interactions. This challenge is central to a variety of fields, from graphics and animation to robotics and motor neuroscience. Our physics-based environment uses realistic actuation and first-person perception - including touch sensors and egocentric vision - with a view to producing active-sensing behaviors (e.g. gaze direction), transferability to real robots, and comparisons to the biology. We develop an integrated neural-network based approach consisting of a motor primitive module, human demonstrations, and an instructed reinforcement learning regime with curricula and task variations. We demonstrate the utility of our approach for several tasks, including goal-conditioned box carrying and ball catching, and we characterize its behavioral robustness. The resulting controllers can be deployed in real-time on a standard PC.1
Year
DOI
Venue
2020
10.1145/3386569.3392474
ACM Transactions on Graphics
Keywords
DocType
Volume
reinforcement learning, physics-based character, motor control, object interaction
Journal
39
Issue
ISSN
Citations 
4
0730-0301
1
PageRank 
References 
Authors
0.35
0
9
Name
Order
Citations
PageRank
Josh S. Merel114311.34
Saran Tunyasuvunakool210.35
Arun Ahuja3727.45
Yuval Tassa4109752.33
Leonard Hasenclever5205.42
Vu Pham6152.28
Tom Erez7102750.56
Greg Wayne859231.86
Nicolas Heess9176294.77