Name
Abstract | ||
|---|---|---|
A multi-modal framework for emotion recognition using bag-of-words features and undirected, replicated softmax topic models is proposed here. Topic models ignore the temporal information between features, allowing them to capture the complex structure without a brute-force collection of statistics. Experiments are performed over face, speech and language features extracted from the USC IEMOCAP database. Performance on facial features yields an unweighted average recall of 60.71%, a relative improvement of 8.89% over state-of-the-art approaches. A comparable performance is achieved when considering only speech (57.39%) or a fusion of speech and face information (66.05%). Individually, each source is shown to be strong at recognizing either sadness (speech) or happiness (face) or neutral (language) emotions, while, a multi-modal fusion retains these properties and improves the accuracy to 68.92%. Implementation time for each source and their combination is provided. Results show that a turn of 1 second duration can be classified in approximately 666.65ms, thus making this method highly amenable for real-time implementation. |
Year | DOI | Venue |
|---|---|---|
2014 | 10.1109/ISCAS.2014.6865245 | Circuits and Systems |
Keywords | Field | DocType |
emotion recognition,face recognition,feature extraction,natural language processing,sensor fusion,speech recognition,USC IEMOCAP database,bag-of-words features,brute-force collection,complex structure,emotion recognition,face feature extraction,facial feature,happiness recognition,language feature extraction,multimodal approach,multimodal fusion,neutral emotions,sadness recognition,speech feature extraction,speech-face information fusion,statistics,temporal information,undirected replicated softmax topic models,undirected topic models,unweighted average recall | Sadness,Softmax function,Pattern recognition,Computer science,Speech recognition,Feature extraction,Speaker recognition,Feature (machine learning),Artificial intelligence,Topic model,Recall,Modal | Conference |
ISSN | Citations | PageRank |
0271-4302 | 3 | 0.44 |
References | Authors | |
10 | 3 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Mohit Shah | 1 | 27 | 3.11 |
| Chaitali Chakrabarti | 2 | 1978 | 184.17 |
| Andreas S. Spanias | 3 | 528 | 87.90 |