Paper Info
Title
Shaping Progressive Net of Reinforcement Learning for Policy Transfer with Human Evaluative Feedback
Abstract
Deep reinforcement learning has achieved significant success in many fields, but will confront sampling efficiency and safety problems when applying to robot control in the real world. Sim-to-real transfer learning was proposed to make use of samples in the simulation and overcome the gap between simulation and real world. In this paper, we focus on improving Progressive Neural Network - an effective sim-to-real learning method, by proposing Interactive Progressive Network Learning (IPNL). IPNL integrates progressive network and interactive reinforcement learning (interactive RL) which learns from evaluative feedback provided by an observing human trainer. We test our method using five RL tasks with discrete or continuous actions in OpenAI Gym and a sinusoids curve following task with AUV simulator on the Gazebo platform. Our results suggest that while Progressive Network has good performance when transferring from tasks with low-dimensional state space to those with high-dimensional one but has little effect for transferring from high-dimensional tasks to low-dimensional ones, IPNL allows an agent to learn a more stable policy with better performance faster for both cases. More importantly, our further analysis indicate that there is a synergy between Progressive Network and interactive RL for improving the agent's learning. Our results in the path following of AUV shed light on the potential of applying our method in the real world tasks.
Year
DOI
Venue
2021
10.1109/IROS51168.2021.9636061
2021 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS)
DocType
ISSN
Citations 
Conference
2153-0858
0
PageRank 
References 
Authors
0.34
0
7
Name
Order
Citations
PageRank
Rongshun Juan100.34
Jie Huang200.34
Randy Gomez37628.11
Keisuke Nakamura418928.91
Qixin Sha500.34
Bo He67713.20
Guangliang Li773.98