Title
A Practical Approach to Insertion with Variable Socket Position Using Deep Reinforcement Learning
Abstract
Insertion is a challenging haptic and visual control problem with significant practical value for manufacturing. Existing approaches in the model-based robotics community can be highly effective when task geometry is known, but are complex and cumbersome to implement, and must be tailored to each individual problem by a qualified engineer. Within the learning community there is a long history of insertion research, but existing approaches are either too sample-inefficient to run on real robots, or assume access to high-level object features, e.g. socket pose. In this paper we show that relatively minor modifications to an off-the-shelf Deep-RL algorithm (DDPG), combined with a small number of human demonstrations, allows the robot to quickly learn to solve these tasks efficiently and robustly. Our approach requires no modeling or simulation, no parameterized search or alignment behaviors, no vision system aside from raw images, and no reward shaping. We evaluate our approach on a narrow-clearance peg-insertion task and a deformable clip-insertion task, both of which include variability in the socket position. Our results show that these tasks can be solved reliably on the real robot in less than 10 minutes of interaction time, and that the resulting policies are robust to variance in the socket position and orientation.
Year
DOI
Venue
2018
10.1109/ICRA.2019.8794074
2019 International Conference on Robotics and Automation (ICRA)
Keywords
Field
DocType
variable socket position,visual control problem,model-based robotics community,task geometry,off-the-shelf Deep-RL algorithm,narrow-clearance peg-insertion task,deformable clip-insertion task,deep reinforcement learning,haptic control problem
Small number,Parameterized complexity,Machine vision,Control engineering,Human–computer interaction,Artificial intelligence,Engineering,Robot,Visual control,Haptic technology,Robotics,Reinforcement learning
Journal
Volume
Issue
ISSN
abs/1810.01531
1
1050-4729
ISBN
Citations 
PageRank 
978-1-5386-8176-3
2
0.47
References 
Authors
13
6
Name
Order
Citations
PageRank
Mel Vecerík1122.28
Oleg O. Sushkov252.26
David Barker3122.14
Thomas Rothörl4893.54
Todd Hester533031.53
Scholz, Jonathan61045.42