Name
Abstract | ||
|---|---|---|
At the heart of machine learning lies the question of generalizability of learned rules over previously unseen data. While over-parameterized models based on neural networks are now ubiquitous in machine learning applications, our understanding of their generalization capabilities is incomplete. This task is made harder by the non-convexity of the underlying learning problems. We provide a general framework to characterize the asymptotic generalization error for single-layer neural networks (i.e., generalized linear models) with arbitrary non-linearities, making it applicable to regression as well as classification problems. This framework enables analyzing the effect of (i) over-parameterization and non-linearity during modeling; and (ii) choices of loss function, initialization, and regularizer during learning. Our model also captures mismatch between training and test distributions. As examples, we analyze a few special cases, namely linear regression and logistic regression. We are also able to rigorously and analytically explain the \emph{double descent} phenomenon in generalized linear models. |
Year | Venue | DocType |
|---|---|---|
2020 | ICML | Conference |
Citations | PageRank | References |
0 | 0.34 | 0 |
Authors | ||
5 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Emami Melikasadat | 1 | 0 | 0.34 |
| Mojtaba Sahraee-Ardakan | 2 | 8 | 2.61 |
| Pandit Parthe | 3 | 1 | 0.70 |
| Sundeep Rangan | 4 | 3101 | 163.90 |
| Alyson K. Fletcher | 5 | 552 | 41.10 |