Name
Abstract | ||
|---|---|---|
Machines are a long way from robustly solving open-world perception-control tasks, such as first-person view (FPV) aerial navigation. While recent advances in end-to- end Machine Learning, especially Imitation Learning and Reinforcement appear promising, they are constrained by the need of large amounts of difficult-to-collect labeled real- world data. Simulated data, on the other hand, is easy to generate, but generally does not render safe behaviors in diverse real-life scenarios. In this work we propose a novel method for learning robust visuomotor policies for real-world deployment which can be trained purely with simulated data. We develop rich state representations that combine supervised and unsupervised environment data. Our approach takes a cross-modal perspective, where separate modalities correspond to the raw camera data and the system states relevant to the task, such as the relative pose of gates to the drone in the case of drone racing. We feed both data modalities into a novel factored architecture, which learns a joint lowdimensional embedding via Variational Auto Encoders. This compact representation is then fed into a control policy, which we trained using imitation learning with expert trajectories in a simulator. We analyze the rich latent spaces learned with our proposed representations, and show that the use of our cross-modal architecture significantly improves control policy performance as compared to end-to-end learning or purely unsupervised feature extractors. We also present real-world results for drone navigation through gates in different track configurations and environmental conditions. Our proposed method, which runs fully onboard, can successfully generalize the learned representations and policies across simulation and reality, significantly outperforming baseline approaches. |
Year | DOI | Venue |
|---|---|---|
2020 | 10.1109/IROS45743.2020.9341049 | 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) |
Keywords | DocType | ISSN |
cross-modal representations,robustly solving open-world perception-control tasks,first-person view aerial navigation,end-to- end Machine Learning,imitation learning,real- world data,safe behaviors,real-life scenarios,robust visuomotor policies,real-world deployment,rich state representations,unsupervised environment data,cross-modal perspective,separate modalities correspond,raw camera data,system states,drone racing,data modalities,joint lowdimensional embedding,compact representation,rich latent spaces,cross-modal architecture,control policy performance,end-to-end learning,purely unsupervised feature extractors,real-world results,drone navigation,learned representations | Conference | 2153-0858 |
ISBN | Citations | PageRank |
978-1-7281-6213-3 | 2 | 0.42 |
References | Authors | |
0 | 5 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Rogerio Bonatti | 1 | 15 | 3.69 |
| Ratnesh Madaan | 2 | 2 | 0.76 |
| Vibhav Vineet | 3 | 2 | 0.76 |
| Sebastian Scherer | 4 | 522 | 57.76 |
| Ashish Kapoor | 5 | 1833 | 119.72 |