Paper Info
Title
Automatic Spoken Language Acquisition Based on Observation and Dialogue
Abstract
Human babies are born without knowledge of any specific language. They acquire language directly from observation and dialogue without being limited by the availability of labeled data. We propose spoken language acquisition agents that simulate the process. Such an ability requires multiple types of learning, including 1) word discovery, 2) symbol grounding, 3) message generation, and 4) pronunciation generation. Several studies have targeted one or combined learning types to elucidate human intelligence and aimed to equip spoken dialogue systems with human-like flexible language learning ability. However, their language ability was partially lacking some of the components. Our agents are the first to integrate them all. Our key concept is to design an architecture to integrate unsupervised, self-supervised, and reinforcement learning to utilize clues naturally existing in raw sensory signals and drive the learning based on the agent’s intrinsic motivation. Experimental results show agents successfully acquire spoken language from scratch by interacting with an environment to act by speaking. Our proposed focusing mechanism significantly improves learning efficiency. We also demonstrate that our agents can learn neural vocoder and the concept of logical negation as a part of language acquisition.
Year
DOI
Venue
2022
10.1109/JSTSP.2022.3189279
IEEE Journal of Selected Topics in Signal Processing
Keywords
DocType
Volume
Autonomous agent,reinforcement learning,self-supervised learning,spoken language acquisition,unsupervised learning
Journal
16
Issue
ISSN
Citations 
6
1932-4553
0
PageRank 
References 
Authors
0.34
15
12
Name
Order
Citations
PageRank
Ryota Komatsu100.34
Shengzhou Gao200.34
Wenxin Hou300.34
Mingxin Zhang400.34
Tomohiro Tanaka500.34
Keisuke Toyoda600.34
Yusuke Kimura700.34
Kent Hino800.34
Yu Iwamoto900.34
Kosuke Mori1000.34
Takuma Okamoto1110.70
Takahiro Shinozaki1200.34