Name
Abstract | ||
|---|---|---|
In i-theory a typical layer of a hierarchical architecture consists of HW modules pooling the dot products of the inputs to the layer with the transformations of a few templates under a group. Such layers include as special cases the convolutional layers of Deep Convolutional Networks (DCNs) as well as the non-convolutional layers (when the group contains only the identity). Rectifying nonlinearities ‐ which are used by present-day DCNs ‐ are one of the several nonlinearities admitted by i-theory for the HW module. We discuss here the equivalence between group averages of linear combinations of rectifying nonlinearities and an associated kernel. This property implies that present-day DCNs can be exactly equivalent to a hierarchy of kernel machines with pooling and non-pooling layers. Finally, we describe a conjecture for theoretically understanding hierarchies of such modules. A main consequence of the conjecture is that hierarchies of trained HW modules minimize memory requirements while computing a selective and invariant representation. |
Year | Venue | Field |
|---|---|---|
2015 | CoRR | Kernel (linear algebra),Linear combination,Pooling,Theoretical computer science,Equivalence (measure theory),Artificial intelligence,Invariant (mathematics),Dot product,Hierarchy,Conjecture,Mathematics,Machine learning |
DocType | Volume | Citations |
Journal | abs/1508.01084 | 8 |
PageRank | References | Authors |
0.74 | 13 | 4 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Fabio Anselmi | 1 | 73 | 6.18 |
| Lorenzo Rosasco | 2 | 1070 | 82.90 |
| Cheston Tan | 3 | 155 | 15.27 |
| Tomaso Poggio | 4 | 13488 | 3380.01 |