Name
Abstract | ||
|---|---|---|
We propose to use sparse inverse covariance matrices for acoustic model training when there is insufficient training data. Acoustic models trained with inadequate training data tend to over fit, generalizing poorly to unseen test data, especially when full covariance matrices are used. We address this problem by adding an L1 regularization term to the traditional objective function for maximum likelihood estimation, to penalize complex models. The structure of the inverse covariance matrices will be automatically sparsified using this new objective function. The Expectation Maximization algorithm is used to learn the parameters of the hidden Markov model using the new objective function. It is shown that the training procedures for all the hidden Markov model parameters are the same as that of maximum likelihood estimation except the inverse covariance matrices. The update equation for the inverse covariance matrices is concave and can be solved efficiently. Our experiments show that this proposed method can correctly learn the underlying correlations among the random variables of the speech feature vector. Experimental results on the Wall Street Journal data show that our proposed model significantly outperforms the diagonal covariance model and the full covariance model by 10.9% and 16.5% relative recognition accuracy, when only about 14 hours of training data are available. On our collected low resource language data-the Cantonese data set, the proposed model also significantly outperforms the diagonal covariance model and the full covariance model. |
Year | DOI | Venue |
|---|---|---|
2013 | 10.1109/TASL.2012.2221462 | IEEE Transactions on Audio, Speech, and Language Processing |
Keywords | Field | DocType |
training data,maximum likelihood estimation,hidden markov models,acoustics,expectation maximization algorithm,data models,speech recognition,objective function,hidden markov model,random variables,covariance matrix | Covariance function,Estimation of covariance matrices,Pattern recognition,Rational quadratic covariance function,Computer science,Covariance intersection,Speech recognition,CMA-ES,Artificial intelligence,Covariance matrix,Matérn covariance function,Covariance | Journal |
Volume | Issue | ISSN |
21 | 3 | 1558-7916 |
Citations | PageRank | References |
5 | 0.52 | 16 |
Authors | ||
2 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Weibin Zhang | 1 | 31 | 10.03 |
| Pascale Fung | 2 | 678 | 85.84 |