Name
Abstract | ||
|---|---|---|
In this paper, we propose a Long Short-Term Memory Recurrent Neural Network (LSTM-RNN) and multiple kernel learning (MKL) based multi-modal affect recognition scheme (LSTM-MKL). It takes the LSTM-RNN advantage to model the long range dependencies between successive observations, and uses the MKL power to model the non-linear correlations between the inputs and outputs. For each of the affect dimensions (arousal, valence, expectancy, and power), two LSTM-RNN models are trained, one for each modality. In the recognition phase, the audio and visual features are input to the corresponding learned LSTM models, which in turn produce initial estimates of the affect dimensions. The LSTM outputs are further input into a multi-kernel support vector regression (MK-SVR) for the final recognition. Experimental results carried out on the AVEC2012 database, show that compared to the traditional SVR-LLR (Support Vector Machine - local linear regression) or MK-SVR fusion scheme, the proposed LSTM-MKL fusion scheme obtains higher recognition results, with an correlation coefficient (COR) of 0.354, compared to a COR of 0.124 for SVR-LLR, and 0.168 for MK-SVR, respectively. |
Year | DOI | Venue |
|---|---|---|
2014 | 10.1109/APSIPA.2014.7041743 | APSIPA |
Keywords | Field | DocType |
nonlinear correlation,lstm-mkl fusion,long short-term memory recurrent neural network,multikernel support vector regression,speech recognition,regression analysis,multiple kernel learning,image recognition,feature recognition,emotion recognition,audio recognition,feature extraction,multimodal continuous affect recognition,multimodal affect recognition scheme,visual recognition,support vector machines,neural nets,correlation methods,kernel,visualization,decision support systems,recurrent neural networks,correlation | Kernel (linear algebra),Pattern recognition,Computer science,Support vector machine,Multiple kernel learning,Recurrent neural network,Local regression,Speech recognition,Correlation,Feature (machine learning),Artificial intelligence,Kernel method | Conference |
ISSN | Citations | PageRank |
2309-9402 | 7 | 0.45 |
References | Authors | |
9 | 6 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Jiamei Wei | 1 | 7 | 0.45 |
| Ercheng Pei | 2 | 83 | 6.23 |
| Jiang Dongmei | 3 | 115 | 15.28 |
| Hichem Sahli | 4 | 475 | 65.19 |
| Lei Xie | 5 | 425 | 64.87 |
| Zhong-Hua Fu | 6 | 52 | 9.96 |