Name
Title | ||
|---|---|---|
Ensemble Learning in Fixed Expansion Layer Networks for Mitigating Catastrophic Forgetting |
Abstract | ||
|---|---|---|
Catastrophic forgetting is a well-studied attribute of most parameterized supervised learning systems. A variation of this phenomenon, in the context of feedforward neural networks, arises when nonstationary inputs lead to loss of previously learned mappings. The majority of the schemes proposed in the literature for mitigating catastrophic forgetting were not data driven and did not scale well. We introduce the fixed expansion layer (FEL) feedforward neural network, which embeds a sparsely encoding hidden layer to help mitigate forgetting of prior learned representations. In addition, we investigate a novel framework for training ensembles of FEL networks, based on exploiting an information-theoretic measure of diversity between FEL learners, to further control undesired plasticity. The proposed methodology is demonstrated on a basic classification task, clearly emphasizing its advantages over existing techniques. The architecture proposed can be enhanced to address a range of computational intelligence tasks, such as regression problems and system control. |
Year | DOI | Venue |
|---|---|---|
2013 | 10.1109/TNNLS.2013.2264952 | Neural Networks and Learning Systems, IEEE Transactions |
Keywords | Field | DocType |
encoding,feedforward neural nets,learning (artificial intelligence),FEL feedforward neural network,catastrophic forgetting mitigation,computational intelligence,ensemble learning,ensembles training,fixed expansion layer network,information theory measure,nonstationary input,parameterized supervised learning system,sparse encoding hidden layer,Catastrophic forgetting,nonstationary inputs,sparse encoding neural networks | Forgetting,Feedforward neural network,Data-driven,Computational intelligence,Pattern recognition,Computer science,Supervised learning,Artificial intelligence,Ensemble learning,Machine learning,Catastrophic interference,Encoding (memory) | Journal |
Volume | Issue | ISSN |
24 | 10 | 2162-237X |
Citations | PageRank | References |
9 | 0.56 | 6 |
Authors | ||
3 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Robert Coop | 1 | 80 | 6.66 |
| Aaron Mishtal | 2 | 9 | 0.56 |
| Itamar Arel | 3 | 160 | 18.56 |