Name
Abstract | ||
|---|---|---|
Training of discrete latent variable models remains challenging because passing gradient information through discrete units is difficult. We propose a new class of smoothing transformations based on a mixture of two overlapping distributions, and show that the proposed transformation can be used for training binary latent models with either directed or undirected priors. We derive a new variational bound to efficiently train with Boltzmann machine priors. Using this bound, we develop DVAE++, a generative model with a global discrete prior and a hierarchy of convolutional continuous variables. Experiments on several benchmarks show that overlapping transformations outperform other recent continuous relaxations of discrete latent variables including Gumbel-Softmax (Maddison et al., 2016; Jang et al., 2016), and discrete variational autoencoders (Rolfe 2016). |
Year | Venue | DocType |
|---|---|---|
2018 | ICML | Conference |
Volume | Citations | PageRank |
abs/1802.04920 | 5 | 0.43 |
References | Authors | |
18 | 5 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Vahdat, Arash | 1 | 353 | 18.20 |
| William G. Macready | 2 | 161 | 39.07 |
| Zhengbing Bian | 3 | 20 | 2.20 |
| Amir Khoshaman | 4 | 13 | 1.39 |
| Evgeny Andriyash | 5 | 5 | 0.43 |