Name
Abstract | ||
|---|---|---|
Belief networks are commonly used generative models of data, but require expensive posterior estimation to train and test the model. Learning typically proceeds by posterior sampling, variational approximations, or recognition networks, combined with stochastic optimization. We propose using an online Monte Carlo expectation-maximization (MCEM) algorithm to learn the maximum a posteriori (MAP) estimator of the generative model or optimize the variational lower bound of a recognition network. The E-step in this algorithm requires posterior samples, which are already generated in current learning schema. For the M-step, we augment with Polya-Gamma (PG) random variables to give an analytic updating scheme. We show relationships to standard learning approaches by deriving stochastic gradient ascent in the MCEM framework. We apply the proposed methods to both binary and count data. Experimental results show that MCEM improves the convergence speed and often improves hold-out performance over existing learning methods. Our approach is readily generalized to other recognition networks. |
Year | Venue | Field |
|---|---|---|
2016 | JMLR Workshop and Conference Proceedings | Convergence (routing),Gradient descent,Stochastic optimization,Random variable,Monte Carlo method,Mathematical optimization,Computer science,Artificial intelligence,Maximum a posteriori estimation,Machine learning,Generative model,Estimator |
DocType | Volume | ISSN |
Conference | 51 | 1938-7288 |
Citations | PageRank | References |
3 | 0.39 | 15 |
Authors | ||
4 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Zhao Song | 1 | 5 | 2.10 |
| Ricardo Henao | 2 | 286 | 23.85 |
| David E. Carlson | 3 | 182 | 15.35 |
| L. Carin | 4 | 4603 | 339.36 |