Name
Abstract | ||
|---|---|---|
Current algorithms for deep learning probably cannot run in the brain because they rely on weight transport, where forward-path neurons transmit their synaptic weights to a feedback path, in a way that is likely impossible biologically. An algorithm called feedback alignment achieves deep learning without weight transport by using random feedback weights, but it performs poorly on hard visual-recognition tasks. Here we describe two mechanisms - a neural circuit called a weight mirror and a modification of an algorithm proposed by Kolen and Pollack in 1994 - both of which let the feedback path learn appropriate synaptic weights quickly and accurately even in large networks, without weight transport or complex wiring. Tested on the ImageNet visual-recognition task, these mechanisms learn almost as well as backprop (the standard algorithm of deep learning, which uses weight transport) and they outperform feedback alignment and another, more-recent transport-free algorithm, the sign-symmetry method. |
Year | Venue | Keywords |
|---|---|---|
2019 | ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 32 (NIPS 2019) | deep learning |
Field | DocType | Volume |
Large networks,Standard algorithms,Artificial intelligence,Deep learning,Machine learning,Mathematics | Journal | 32 |
ISSN | Citations | PageRank |
1049-5258 | 0 | 0.34 |
References | Authors | |
0 | 5 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Mohsen Akrout | 1 | 34 | 4.12 |
| Wilson, Collin | 2 | 0 | 0.34 |
| Peter C. Humphreys | 3 | 0 | 0.34 |
| Timothy P. Lillicrap | 4 | 4377 | 170.65 |
| Douglas B. Tweed | 5 | 47 | 14.48 |