Title
Economic Analysis of Simulation Selection Problems
Abstract
Ranking and selection procedures are standard methods for selecting the best of a finite number of simulated design alternatives based on a desired level of statistical evidence for correct selection. But the link between statistical significance and financial significance is indirect, and there has been little or no research into it. This paper presents a new approach to the simulation selection problem, one that maximizes the expected net present value of decisions made when using stochastic simulation. We provide a framework for answering these managerial questions: When does a proposed system design, whose performance is unknown, merit the time and money needed to develop a simulation to infer its performance? For how long should the simulation analysis continue before a design is approved or rejected? We frame the simulation selection problem as a “stoppable” version of a Bayesian bandit problem that treats the ability to simulate as a real option prior to project implementation. For a single proposed system, we solve a free boundary problem for a heat equation that approximates the solution to a dynamic program that finds optimal simulation project stopping times and that answers the managerial questions. For multiple proposed systems, we extend previous Bayesian selection procedures to account for discounting and simulation-tool development costs.
Year
DOI
Venue
2009
10.1287/mnsc.1080.0949
Management Science
Keywords
Field
DocType
optimal stopping,managerial question,free boundary problem,correct selection,economics of simulation,previous bayesian selection procedure,simulation selection problems,simulation,bayesian bandit problem,simulation analysis,selection procedure,economic analysis,simulation selection problem,ranking and selection,optimal simulation project,stochastic simulation,system design,heat equation,stopping time,statistical significance,net present value
Stochastic simulation,Dynamic programming,Mathematical optimization,Optimal stopping,Ranking,Discounting,Systems design,Operations research,Stopping time,Mathematics,Bayesian probability
Journal
Volume
Issue
ISSN
55
3
0025-1909
Citations 
PageRank 
References 
45
1.96
6
Authors
2
Name
Order
Citations
PageRank
Stephen E. Chick11127152.40
Noah Gans261366.60