Paper Info
Title
Deep Episodic Memory For Verbalization Of Robot Experience
Abstract
The ability to verbalize robot experience in natural language is key for a symbiotic human-robot interaction. While first works approached this problem using template-based verbalization on symbolic episode data only, we explore a novel way in which deep learning methods are used for the creation of an episodic memory from experiences as well as the verbalization of such experience in natural language. To this end, we first collected a complex dataset consisting of more than a thousand multimodal robot episode recordings both from simulation as well as real robot executions, together with representative natural language questions and answers about the robot's past experience. Second, we propose and evaluate an episodic memory verbalization model consisting of a speech encoder and decoder based on the Transformer architecture, combined with an LSTM-based episodic memory auto-encoder, and evaluate the model on simulated and real data from robot execution examples. Our experimental results provide a proof-of-concept for episodic-memory-based verbalization of robot experience.
Year
DOI
Venue
2021
10.1109/LRA.2021.3085166
IEEE ROBOTICS AND AUTOMATION LETTERS
Keywords
DocType
Volume
Learning from experience, AI-enabled robotics, natural dialog for HRI, sensorimotor learning
Journal
6
Issue
ISSN
Citations 
3
2377-3766
0
PageRank 
References 
Authors
0.34
0
5
Name
Order
Citations
PageRank
Leonard Bärmann100.34
Fabian Peller-Konrad200.34
Stefan Constantin321.13
tamim asfour41889151.86
Alex Waibel563431980.68