Name
Abstract | ||
|---|---|---|
Second-order reward models are an important class of models for evaluating the performance of real-life systems in which the reward measure fluctuates according to some underlying noise. These models consist of a Markov chain driving the evolution of the system, and a continuous reward variable representing its performance. Thus far, only models with a finite number of states have been studied. We consider second-order reward models driven by Quasi-birth-and-death processes, a class of block-structured Markov chains with infinitely many states. We derive the expressions for the Laplace-Stieltjes transforms of the accumulated reward and demonstrate how they can be efficiently evaluated. We use our results to analyse a simple example and, in doing so, show that the second-order feature can make a significant difference to the accumulated reward. The inclusion of the second-order feature also creates new difficulties which require the development of new conditions in the analysis. |
Year | DOI | Venue |
|---|---|---|
2012 | 10.1016/j.peva.2012.05.002 | Perform. Eval. |
Keywords | Field | DocType |
second-order markov reward model,finite number,second-order reward model,block-structured markov chain,reward measure,new difficulty,continuous reward variable,second-order feature,important class,markov chain,new condition,brownian motion | Mathematical optimization,Finite set,Expression (mathematics),Computer science,Markov chain,Brownian motion | Journal |
Volume | Issue | ISSN |
69 | 9 | 0166-5316 |
Citations | PageRank | References |
0 | 0.34 | 9 |
Authors | ||
3 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| nigel g bean | 1 | 47 | 10.77 |
| Małgorzata M. O'reilly | 2 | 8 | 2.59 |
| Yong Ren | 3 | 0 | 0.34 |