Name
Abstract | ||
|---|---|---|
We consider the bias and variance of value function estimation that are caused by using an empirical model instead of the true model. We analyze these bias and variance for Markov processes from a classical (frequentist) statistical point of view, and in a Bayesian setting. Using a second order approximation, we provide explicit expressions for the bias and variance in terms of the transition counts and the reward statistics. We present supporting experiments with artificial Markov chains and with a large transactional database provided by a mail-order catalog firm. |
Year | DOI | Venue |
|---|---|---|
2004 | 10.1145/1015330.1015402 | ICML |
Keywords | Field | DocType |
order approximation,explicit expression,mail-order catalog firm,artificial markov chain,value function estimation,bayesian setting,large transactional database,statistical point,reward statistic,empirical model,true model,variance,value function,markov processes,markov process,second order approximation,reinforcement learning,markov chain,bayesian estimation,bias | Econometrics,Frequentist inference,Bias of an estimator,Artificial intelligence,Pattern recognition,Markov property,Markov model,Markov chain,Variable-order Markov model,Markov kernel,Statistics,Causal Markov condition,Mathematics | Conference |
ISBN | Citations | PageRank |
1-58113-838-5 | 21 | 2.08 |
References | Authors | |
6 | 4 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Shie Mannor | 1 | 3340 | 285.45 |
| Duncan I. Simester | 2 | 260 | 20.45 |
| Peng Sun | 3 | 420 | 26.68 |
| John N. Tsitsiklis | 4 | 5300 | 621.34 |