Name
Abstract | ||
|---|---|---|
The performance of neural networks for which weights and signals are modeled by shot-noise processes is considered. Examples of such networks are optical neural networks and biological systems. We develop a theory that facilitates the computation of the average probability of error in binary-input/binary-output multistage and recurrent networks. We express the probability of error in terms of two key parameters: the computing-noise parameter and the weight-recording-noise parameter. The former is the average number of particles per clock cycle per signal and it represents noise due to the particle nature of the signal. The latter represents noise in the weight-recording process and is the average number of particles per weight. For a fixed computing-noise parameter, the probability of error decreases with the increase in the recording-noise parameter and saturates at a level limited by the computing-noise parameter. A similar behavior is observed when the role of the two parameters is interchanged. As both parameters increase, the probability of error decreases to zero exponentially fast at a rate that is determined using large deviations. We show that the performance can be optimized by a selective choice of the nonlinearity threshold levels. For recurrent networks, as the number of iterations increases, the probability of error increases initially and then saturates at a level determined by the stationary distribution of a Markov chain. |
Year | DOI | Venue |
|---|---|---|
1996 | 10.1109/72.501727 | IEEE Transactions on Neural Networks |
Keywords | Field | DocType |
markov processes,error statistics,optical neural nets,recurrent neural nets,shot noise,bibo recurrent networks,markov chain stationary distribution,binary-input/binary-output multistage networks,biological systems,error probability,fixed computing-noise parameter,iterations,nonlinearity threshold levels,optical neural networks,shot-noise-limited performance,weight-recording-noise parameter | Statistical physics,Particle number,Mathematical optimization,Nonlinear system,Markov chain,Stationary distribution,Large deviations theory,Cycles per instruction,Artificial neural network,Shot noise,Mathematics | Journal |
Volume | Issue | ISSN |
7 | 3 | 1045-9227 |
Citations | PageRank | References |
1 | 0.53 | 3 |
Authors | ||
3 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| M. M. Hayat | 1 | 37 | 5.89 |
| Saleh, B.E.A. | 2 | 1 | 0.53 |
| Gubner, J.A. | 3 | 52 | 7.59 |