Paper Info
Title
Fast-Tracking Stationary MOMDPs for Adaptive Management Problems.
Abstract
Adaptive management is applied in conservation and natural resource management, and consists of making sequential decisions when the transition matrix is uncertain. Informally described as 'learning by doing', this approach aims to trade off between decisions that help achieve the objective and decisions that will yield a better knowledge of the true transition matrix. When the true transition matrix is assumed to be an element of a finite set of possible matrices, solving a mixed observability Markov decision process (MOMDP) leads to an optimal trade-off but is very computationally demanding. Under the assumption (common in adaptive management) that the true transition matrix is stationary, we propose a polynomial-time algorithm to find a lower bound of the value function. In the corners of the domain of the value function (belief space), this lower bound is provably equal to the optimal value function. We also show that under further assumptions, it is a linear approximation of the optimal value function in a neighborhood around the corners. We evaluate the benefits of our approach by using it to initialize the solvers MO-SARSOP and Perseus on a novel computational sustain ability problem and a recent adaptive management data challenge. Our approach leads to an improved initial value function and translates into significant computational gains for both solvers.
Year
Venue
Field
2017
THIRTY-FIRST AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE
Linear approximation,Mathematical optimization,Stochastic matrix,Matrix (mathematics),Computer science,Upper and lower bounds,Partially observable Markov decision process,Markov decision process,Bellman equation,Artificial intelligence,Adaptive management,Machine learning
DocType
Citations 
PageRank 
Conference
0
0.34
References 
Authors
0
4
Name
Order
Citations
PageRank
Martin Péron100.34
Kai Helge Becker200.34
Peter L. Bartlett354821039.97
Iadine Chades4366.00