Paper Info
Title
SafeAPT: Safe Simulation-to-Real Robot Learning Using Diverse Policies Learned in Simulation
Abstract
The framework of sim-to-real learning, i.e., training policies in simulation and transferring them to real-world systems, is one of the most promising approaches towards data-efficient learning in robotics. However, due to the inevitable reality gap between the simulation and the real world, a policy learned in the simulation may not always generate a safe behaviour on the real robot. As a result, during policy adaptation in the real world, the robot may damage itself or cause harm to its surroundings. In this work, we introduce SafeAPT, a multi-goal robot learning algorithm that leverages a diverse repertoire of policies evolved in simulation and transfers the most promising safe policy to the real robot through episodic interaction. To achieve this, SafeAPT iteratively learns probabilistic reward and safety models from real-world observations using simulated experiences as priors. Then, it performs Bayesian optimization to select the best policy from the repertoire with the reward model, while maintaining the specified safety constraint using the safety model. SafeAPT allows a robot to adapt to a wide range of goals safely with the same repertoire of policies evolved in the simulation. We compare SafeAPT with several baselines, both in simulated and real robotic experiments, and show that SafeAPT finds high-performing policies within a few minutes of real-world operation while minimizing safety violations during the interactions.
Year
DOI
Venue
2022
10.1109/LRA.2022.3177294
IEEE ROBOTICS AND AUTOMATION LETTERS
Keywords
DocType
Volume
Evolutionary robotics, learning from experience, machine learning for robot control
Journal
7
Issue
ISSN
Citations 
3
2377-3766
0
PageRank 
References 
Authors
0.34
0
3
Name
Order
Citations
PageRank
Rituraj Kaushik100.34
Karol Arndt200.34
V. Kyrki365261.79