Paper Info
Title
Audiovisual speech recognition for Kannada language using feed forward neural network
Abstract
Audiovisual speech recognition is one of the promising technologies in a noisy environment. In this work, we develop the database for Kannada Language and develop an AVSR system for the same. The proposed work is categorized into three main components: a. Audio mechanism. b. Visual speech mechanism. c. Integration of audio and visual mechanisms. In the audio model, MFCC is used to extract the features and a one-dimensional convolutional neural network is used for classification. In the visual module, Dlib is used to extract the features and long short-term memory recurrent neural network is used for classification. Finally, integration of audio and visual module is done using feed forward neural network. Audio speech recognition of Kannada dataset training accuracy achieved is 93.86 and 91.07% for testing data using seventy epochs. Visual speech recognition for Kannada dataset training accuracy is 77.57%, and testing accuracy is 75%. After integration, audiovisual speech recognition for Kannada dataset train accuracy is 93.33% and for testing is 92.26%.
Year
DOI
Venue
2022
10.1007/s00521-022-07249-7
Neural Computing and Applications
Keywords
DocType
Volume
Audiovisual speech recognition, Dlib, Feed forward neural network, Kannada Language, LSTM, MFCC
Journal
34
Issue
ISSN
Citations 
18
0941-0643
0
PageRank 
References 
Authors
0.34
3
2
Name
Order
Citations
PageRank
R. Shashidhar100.34
S. Patilkulkarni200.34