Name
Abstract | ||
|---|---|---|
We propose a new notion of `non-linearityu0027 of a network layer with respect to an input batch that is based on its proximity to a linear system, which is reflected in the non-negative rank of the activation matrix. We measure this non-linearity by applying non-negative factorization to the activation matrix. Considering batches of similar samples, we find that high non-linearity in deep layers is indicative of memorization. Furthermore, by applying our approach layer-by-layer, we find that the mechanism for memorization consists of distinct phases. We perform experiments on fully-connected and convolutional neural networks trained on several image and audio datasets. Our results demonstrate that as an indicator for memorization, our technique can be used to perform early stopping. |
Year | Venue | Field |
|---|---|---|
2018 | arXiv: Learning | Early stopping,Linear system,Pattern recognition,Convolutional neural network,Matrix (mathematics),Network layer,Artificial intelligence,Factorization,Memorization,Mathematics,Machine learning |
DocType | Volume | Citations |
Journal | abs/1810.03372 | 0 |
PageRank | References | Authors |
0.34 | 0 | 3 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Edo Collins | 1 | 17 | 2.03 |
| Siavash Arjomand Bigdeli | 2 | 38 | 3.85 |
| Sabine Süsstrunk | 3 | 4984 | 207.02 |