Name
Abstract | ||
|---|---|---|
We study the power-law asymptotics of learning curves for Gaussian process regression (GPR). When the eigenspectrum of the prior decays with rate $\alpha$ and the eigenexpansion coefficients of the target function decay with rate $\beta$, we show that the generalization error behaves as $\tilde O(n^{\max\{\frac{1}{\alpha}-1, \frac{1-2\beta}{\alpha}\}})$ with high probability over the draw of $n$ input samples. Under similar assumptions, we show that the generalization error of kernel ridge regression (KRR) has the same asymptotics. Infinitely wide neural networks can be related to KRR with respect to the neural tangent kernel (NTK), which in several cases is known to have a power-law spectrum. Hence our methods can be applied to study the generalization error of infinitely wide neural networks. We present toy experiments demonstrating the theory. |
Year | Venue | Keywords |
|---|---|---|
2022 | International Conference on Learning Representations (ICLR) | Gaussian process regression,kernel ridge regression,generalization error,power law,neural tangent kernel |
DocType | Citations | PageRank |
Conference | 0 | 0.34 |
References | Authors | |
0 | 3 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Hui Jin | 1 | 0 | 0.34 |
| Pradeep Kr. Banerjee | 2 | 0 | 0.34 |
| Guido Montufar | 3 | 7 | 5.63 |