Abstract | ||
---|---|---|
Deep neural networks (DNN) have set new standards at predicting responses of neural populations to visual input. Most such DNNs consist of a convolutional network (core) shared across all neurons which learns a representation of neural computation in visual cortex and a neuron-specific readout that linearly combines the relevant features in this representation. The goal of this paper is to test whether such a representation is indeed generally characteristic for visual cortex, i.e. generalizes between animals of a species, and what factors contribute to obtaining such a generalizing core. To push all non-linear computations into the core where the generalizing cortical features should be learned, we devise a novel readout that reduces the number of parameters per neuron in the readout by up to two orders of magnitude compared to the previous state-of-the-art. It does so by taking advantage of retinotopy and learns a Gaussian distribution over the neuron’s receptive field position. With this new readout we train our network on neural responses from mouse primary visual cortex (V1) and obtain a gain in performance of 7% compared to the previous state-of-the-art network. We then investigate whether the convolutional core indeed captures general cortical features by using the core in transfer learning to a different animal. When transferring a core trained on thousands of neurons from various animals and scans we exceed the performance of training directly on that animal by 12%, and outperform a commonly used VGG16 core pre-trained on imagenet by 33%. In addition, transfer learning with our data-driven core is more data-efficient than direct training, achieving the same performance with only 40% of the data. Our model with its novel readout thus sets a new state-of-the-art for neural response prediction in mouse visual cortex from natural images, generalizes between animals, and captures better characteristic cortical features than current task-driven pre-training approaches such as VGG16. |
Year | DOI | Venue |
---|---|---|
2021 | 10.1101/2020.10.05.326256 | ICLR |
DocType | Citations | PageRank |
Conference | 0 | 0.34 |
References | Authors | |
0 | 12 |
Name | Order | Citations | PageRank |
---|---|---|---|
Konstantin-Klemens Lurz | 1 | 0 | 1.01 |
Mohammad Bashiri | 2 | 0 | 0.68 |
Konstantin Willeke | 3 | 0 | 0.34 |
Akshay Jagadish | 4 | 0 | 0.34 |
Eric Wang | 5 | 0 | 0.34 |
Edgar Y. Walker | 6 | 3 | 3.83 |
Santiago A. Cadena | 7 | 3 | 3.15 |
Muhammad, Taliah | 8 | 0 | 1.01 |
Erick Cobos | 9 | 0 | 1.69 |
a s tolias | 10 | 87 | 10.70 |
Alexander S. Ecker | 11 | 600 | 27.06 |
Fabian H. Sinz | 12 | 143 | 13.38 |