Name
Abstract | ||
|---|---|---|
It has long been known that neural networks can learn faster when their input and hidden unit activity is centered about zero; recently we have extended this approach to also encompass the centering of error signals (Schraudolph & Sejnowski, 1996). Here we generalize this notion to all factors involved in the network''s gradient, leading us to propose centering the slope of hidden unit activation functions as well. Slope centering removes the linear component of backpropagated error; this improves credit assignment in networks with shortcut connections. Benchmark results show that this can speed up learning significantly without adversely affecting the trained network''s generalization ability. |
Year | DOI | Venue |
|---|---|---|
2012 | 10.1007/978-3-642-35289-8_14 | Neural Networks: Tricks of the Trade, this book is an outgrowth of a 1996 NIPS workshop |
Keywords | DocType | Volume |
Centering Neural Network Gradient | Conference | 1524 |
ISSN | ISBN | Citations |
0302-9743 | 3-540-65311-2 | 1 |
PageRank | References | Authors |
0.41 | 0 | 2 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Nicol N. Schraudolph | 1 | 1185 | 164.26 |
| NN Schraudolph | 2 | 1 | 0.41 |