Name
Abstract | ||
|---|---|---|
Gradient Boosting (GB) learns an additive expansion of simple basis-models. This is accomplished by iteratively fitting an elementary model to the negative gradient of a loss function with respect to the expansion’s values at each training data-point evaluated at each iteration. For the case of squared-error loss function, the negative gradient takes the form of an ordinary residual for a given training data-point. Studies have demonstrated that running GB for hundreds of iterations can lead to overfitting, while a number of authors showed that by adding noise to the training data, generalisation is impaired even with relatively few basis-models. Regularisation is realised through the shrinkage of every newly-added basis-model to the expansion. This paper demonstrates that GB with shrinkage-based regularisation is still prone to overfitting in noisy datasets. We use a transformation based on a sigmoidal function for reducing the influence of extreme values in the residuals of a GB iteration without removing them from the training set. This extension is built on top of shrinkage-based regularisation. Simulations using synthetic, noisy data show that the proposed method slows-down overfitting and reduces the generalisation error of regularised GB. The proposed method is then applied to the inherently noisy domain of financial time-series modelling. Results suggest that for the majority of datasets the method generalises better when compared against standard regularised GB, as well as against a range of other time-series modelling methods. |
Year | DOI | Venue |
|---|---|---|
2017 | 10.1007/s10287-017-0280-y | Comput. Manag. Science |
Keywords | Field | DocType |
Boosting algorithms,Gradient boosting,Stagewise additive modelling,Regularisation,Financial time-series modelling,Financial forecasting,Feedforward neural networks,Noisy data,Ensemble learning | Residual,Feedforward neural network,Mathematical optimization,Extreme value theory,Generalization,Overfitting,Finance,Ensemble learning,Mathematics,Gradient boosting,Sigmoid function | Journal |
Volume | Issue | ISSN |
14 | 3 | 1619-697X |
Citations | PageRank | References |
0 | 0.34 | 30 |
Authors | ||
3 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Alexandros Agapitos | 1 | 211 | 22.88 |
| Anthony Brabazon | 2 | 918 | 98.60 |
| Michael O’Neill | 3 | 212 | 23.41 |