Name
Title | ||
|---|---|---|
An efficient learning algorithm using natural gradient and second order information of error surface |
Abstract | ||
|---|---|---|
Natural gradient learning algorithm, which originated from information geometry, is known to provide a good solution for the problem of slow learning speed of gradient descent learning methods. Whereas the natural gradient learning algorithm is inspired from the geometric structure of the space of learning systems, there have been other approaches to acceleration of learning by using the second order information of error surface. Although the second order methods cannot give as successful solutions as the natural gradient learning method, their results showed the usefulness of the second order information of error surface in the learning process. In this paper, we develop a method of combining these two different approaches to propose a more efficient learning algorithm. At each learning step, we calculate a search direction by means of the natural gradient. When we apply the search direction to parameter-updating process, the second order information of error surface is applied to determine an efficient learning rate. Through a simple experiment on a real world problem, we confirmed that the proposed learning algorithm show faster convergence than the pure natural gradient learning algorithm. |
Year | DOI | Venue |
|---|---|---|
2000 | 10.1007/3-540-44533-1_23 | PRICAI |
Keywords | Field | DocType |
error surface,search direction,slow learning speed,efficient learning rate,algorithm show,natural gradient,pure natural gradient,gradient descent,order information,efficient learning algorithm,second order | Semi-supervised learning,Active learning (machine learning),Probably approximately correct learning,Computer science,Wake-sleep algorithm,Unsupervised learning,Artificial intelligence,Population-based incremental learning,Online machine learning,Mathematical optimization,Stability (learning theory),Algorithm,Machine learning | Conference |
ISBN | Citations | PageRank |
3-540-67925-1 | 0 | 0.34 |
References | Authors | |
2 | 4 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Hyeyoung Park | 1 | 194 | 32.70 |
| kenji fukumizu | 2 | 1683 | 158.91 |
| shunichi amari | 3 | 5992 | 1269.68 |
| Yillbyung Lee | 4 | 190 | 24.13 |