Name
Abstract | ||
|---|---|---|
Learning rate scheduler has been a critical issue in the deep neural network training. Several schedulers and methods have been proposed, including step decay scheduler, adaptive method, cosine scheduler and cyclical scheduler. This paper proposes a new scheduling method, named hyperbolic-tangent decay (HTD). We run experiments on several benchmarks such as: ResNet, Wide ResNet and DenseNet for CIFAR-10 and CIFAR-100 datasets, LSTM for PAMAP2 dataset, ResNet on ImageNet and Fashion-MNIST datasets. In our experiments, HTD outperforms step decay and cosine scheduler in nearly all cases, while requiring less hyperparameters than step decay, and more flexible than cosine scheduler. Code is available at https://github.com/BIGBALLON/HTD. |
Year | DOI | Venue |
|---|---|---|
2019 | 10.1109/WACV.2019.00052 | 2019 IEEE Winter Conference on Applications of Computer Vision (WACV) |
Keywords | Field | DocType |
Training,Error analysis,Adaptive learning,Computer science,Recurrent neural networks,Light rail systems | Stochastic gradient descent,Trigonometric functions,Hyperparameter,Pattern recognition,Scheduling (computing),Adaptive method,Computer science,Algorithm,Hyperbolic function,Artificial intelligence,Artificial neural network,Residual neural network | Conference |
ISSN | ISBN | Citations |
2472-6737 | 978-1-7281-1975-5 | 1 |
PageRank | References | Authors |
0.35 | 0 | 3 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Bo-Yang Hsueh | 1 | 1 | 0.35 |
| Wei Li | 2 | 436 | 140.67 |
| I-Chen Wu | 3 | 208 | 55.03 |