Paper Info
Title
An Infinite Restricted Boltzmann Machine.
Abstract
We present a mathematical construction for the restricted Boltzmann machine RBM that does not require specifying the number of hidden units. In fact, the hidden layer size is adaptive and can grow during training. This is obtained by first extending the RBM to be sensitive to the ordering of its hidden units. Then, with a carefully chosen definition of the energy function, we show that the limit of infinitely many hidden units is well defined. As with RBM, approximate maximum likelihood training can be performed, resulting in an algorithm that naturally and adaptively adds trained hidden units during learning. We empirically study the behavior of this infinite RBM, showing that its performance is competitive to that of the RBM, while not requiring the tuning of a hidden layer size.
Year
DOI
Venue
2015
10.1162/NECO_a_00848
Neural Computation
Field
DocType
Volume
Restricted Boltzmann machine,Well-defined,Maximum likelihood,Algorithm,Artificial intelligence,Mathematics,Machine learning,Hidden semi-Markov model
Journal
abs/1502.02476
Issue
ISSN
Citations 
7
0899-7667
3
PageRank 
References 
Authors
0.38
0
2
Name
Order
Citations
PageRank
Marc-Alexandre Côté11269.37
Hugo Larochelle27692488.99