Paper Info
Title
Deep Reinforcement Learning For Audio-Visual Gaze Control
Abstract
We address the problem of audio-visual gaze control in the specific context of human-robot interaction, namely how controlled robot motions are combined with visual and acoustic observations in order to direct the robot head towards targets of interest. The paper has the following contributions: (i) a novel audio-visual fusion framework that is well suited for controlling the gaze of a robotic head; (ii) a reinforcement learning (RL) formulation for the gaze control problem, using a reward function based on the available temporal sequence of camera and microphone observations; and (iii) several deep architectures that allow to experiment with early and late fusion of audio and visual data. We introduce a simulated environment that enables us to learn the proposed deep RL model without the need of spending hours of tedious interaction. By thoroughly experimenting on a publicly available dataset and on a real robot, we provide empirical evidence that our method achieves state-of-the-art performance.
Year
DOI
Venue
2018
10.1109/IROS.2018.8594327
2018 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS)
Field
DocType
ISSN
Computer vision,Gaze,Computer science,Visualization,Robot kinematics,Artificial intelligence,Robot vision systems,Robot,Microphone,Reinforcement learning
Conference
2153-0858
Citations 
PageRank 
References 
0
0.34
0
Authors
4
Name
Order
Citations
PageRank
Stéphane Lathuilière1335.98
B.R Mâsse2368.46
Pablo Mesejo3826.74
Radu Horaud42776261.99