Name
Abstract | ||
|---|---|---|
We look at the eigenvalues of the Hessian of a loss function before and after training. The eigenvalue distribution is seen to be composed of two parts, the bulk which is concentrated around zero, and the edges which are scattered away from zero. We present empirical evidence for the bulk indicating how over-parametrized the system is, and for the edges indicating the complexity of the input data. |
Year | Venue | Field |
|---|---|---|
2016 | arXiv: Learning | Algebra,Mathematical analysis,Singularity,Hessian matrix,Artificial intelligence,Deep learning,Mathematics |
DocType | Volume | Citations |
Journal | abs/1611.07476 | 0 |
PageRank | References | Authors |
0.34 | 0 | 3 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Levent Sagun | 1 | 0 | 2.03 |
| Léon Bottou | 2 | 11754 | 1364.56 |
| Yann LeCun | 3 | 26090 | 3771.21 |