Name
Title | ||
|---|---|---|
Asymptotic convergence properties of the EM algorithm with respect to the overlap in the mixture |
Abstract | ||
|---|---|---|
The EM algorithm is generally considered as a linearly convergent algorithm. However, many empirical results show that it can converge significantly faster than those gradient based first-order iterative algorithms, especially when the overlap of densities in a mixture is small. This paper explores this issue theoretically on mixtures of densities from a class of exponential families. We have proved that as an average overlap measure of densities in the mixture tends to zero, the asymptotic convergence rate of the EM algorithm locally around the true solution is a higher order infinitesimal than a positive order power of this overlap measure. Thus, the large sample local convergence rate for the EM algorithm tends to be asymptotically superlinear when the overlap of densities in the mixture tends to zero. Moreover, this result has been detailed on Gaussian mixtures. |
Year | DOI | Venue |
|---|---|---|
2005 | 10.1016/j.neucom.2004.12.009 | Neurocomputing |
Keywords | Field | DocType |
empirical result,first-order iterative algorithm,asymptotic convergence rate,higher order infinitesimal,gaussian mixture,positive order power,em algorithm,asymptotic convergence property,linearly convergent algorithm,maximum likelihood,gaussian mixtures,mixture of densities from exponentialfamil ies,local convergence rate,asymptotically superlinear,iterative algorithm,first order,exponential family,local convergence,convergence rate,higher order | Convergence (routing),Applied mathematics,Maximum likelihood,Rate of convergence,Artificial intelligence,Mathematical optimization,Pattern recognition,Expectation–maximization algorithm,Exponential family,Gaussian,Local convergence,Infinitesimal,Mathematics | Journal |
Volume | ISSN | Citations |
68, | Neurocomputing | 11 |
PageRank | References | Authors |
0.96 | 3 | 2 |