Paper Info
Title
Learning Latent Space Models with Angular Constraints.
Abstract
The large model capacity of latent space models (LSMs) enables them to achieve great performance on various applications, but meanwhile renders LSMs to be prone to overfitting. Several recent studies investigate a new type of regularization approach, which encourages components in LSMs to be diverse, for the sake of alleviating overfitting. While they have shown promising empirical effectiveness, in theory why larger “diversity” results in less overfitting is still unclear. To bridge this gap, we propose a new diversity-promoting approach that is both theoretically analyzable and empirically effective. Specifically, we use near-orthogonality to characterize “diversity” and impose angular constraints (ACs) on the components of LSMs to promote diversity. A generalization error analysis shows that larger diversity results in smaller estimation error and larger approximation error. An efficient ADMM algorithm is developed to solve the constrained LSM problems. Experiments demonstrate that ACs improve generalization performance of LSMs and outperform other diversity-promoting approaches.
Year
Venue
Field
2017
ICML
Pattern recognition,Computer science,Artificial intelligence,Machine learning
DocType
Citations 
PageRank 
Conference
1
0.35
References 
Authors
23
7
Name
Order
Citations
PageRank
Pengtao Xie133922.63
yuntian deng224114.12
Yi Zhou36517.55
Abhimanu Kumar42279.76
Yaoliang Yu566934.33
James Y. Zou625126.63
Bo Xing77332471.43