Name
Abstract | ||
|---|---|---|
Penalization procedures often suffer from their dependence on multiplying factors, whose optimal values are either unknown or hard to estimate from data. We propose a completely data-driven calibration algorithm for these parameters in the least-squares regression framework, without assuming a particular shape for the penalty. Our algorithm relies on the concept of minimal penalty, recently introduced by Birgé and Massart (2007) in the context of penalized least squares for Gaussian homoscedastic regression. On the positive side, the minimal penalty can be evaluated from the data themselves, leading to a data-driven estimation of an optimal penalty which can be used in practice; on the negative side, their approach heavily relies on the homoscedastic Gaussian nature of their stochastic framework. The purpose of this paper is twofold: stating a more general heuristics for designing a data-driven penalty (the slope heuristics) and proving that it works for penalized least-squares regression with a random design, even for heteroscedastic non-Gaussian data. For technical reasons, some exact mathematical results will be proved only for regressogram bin-width selection. This is at least a first step towards further results, since the approach and the method that we use are indeed general. |
Year | DOI | Venue |
|---|---|---|
2009 | 10.1145/1577069.1577079 | Journal of Machine Learning Research |
Keywords | Field | DocType |
minimal penalty,model selection by penalization,data-driven calibration,data-driven estimation,non-parametric regression,penalized least-squares regression,optimal penalty,least-squares regression framework,gaussian homoscedastic regression,regressogram,heteroscedastic data,general heuristics,least-squares regression,data-driven penalty,heteroscedastic non-gaussian data,data-driven calibration algorithm,non parametric regression,least square,model selection | Least squares,Heteroscedasticity,Mathematical optimization,Data-driven,Regression,Nonparametric regression,Homoscedasticity,Heuristics,Gaussian,Artificial intelligence,Machine learning,Mathematics | Journal |
Volume | ISSN | Citations |
10, | Journal of Machine Learning Research 10 (2009) 245-279 | 20 |
PageRank | References | Authors |
1.57 | 5 | 2 |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Sylvain Arlot | 1 | 65 | 6.87 |
| Pascal Massart | 2 | 108 | 70.36 |