Paper Info
Title
Two-layered audio-visual speech recognition for robots in noisy environments.
Abstract
Audio-visual (AV) integration is one of the key ideas to improve perception in noisy real-world environments. This paper describes automatic speech recognition (ASR) to improve human-robot interaction based on AV integration. We developed AV-integrated ASR, which has two AV integration layers, that is, voice activity detection (VAD) and ASR. However, the system has three difficulties: 1) VAD and ASR have been separately studied although these processes are mutually dependent, 2) VAD and ASR assumed that high resolution images are available although this assumption never holds in the real world, and 3) an optimal weight between audio and visual stream was fixed while their reliabilities change according to environmental changes. To solve these problems, we propose a new VAD algorithm taking ASR characteristics into account, and a linear-regression-based optimal weight estimation method. We evaluate the algorithm for auditory-and/or visually-contaminated data. Preliminary results show that the robustness of VAD improved even when the resolution of the images is low, and the AVSR using estimated stream weight shows the effectiveness of AV integration.
Year
DOI
Venue
2010
10.1109/IROS.2010.5651205
IROS
Keywords
Field
DocType
audio-visual systems,hearing,human-robot interaction,image resolution,mobile robots,regression analysis,robust control,speech recognition,visual perception,automatic speech recognition,high resolution images,human-robot interaction,linear-regression-based optimal weight estimation,noisy environments,perception,robots,robustness,two-layered audio-visual speech recognition,visually-contaminated data,voice activity detection
Computer vision,Computer science,Visualization,Voice activity detection,Signal-to-noise ratio,Feature extraction,Speech recognition,Robustness (computer science),Audio-visual speech recognition,Artificial intelligence,Visual perception,Mobile robot
Conference
ISSN
Citations 
PageRank 
2153-0858
1
0.36
References 
Authors
0
3
Name
Order
Citations
PageRank
Takami Yoshida1474.85
Kazuhiro Nakadai21342155.91
Hiroshi G. Okuno32092233.19