Name
Title | ||
|---|---|---|
An Inverse Reinforcement Learning Algorithm for Partially Observable Domains with Application on Healthcare Dialogue Management |
Abstract | ||
|---|---|---|
In this paper, we propose an algorithm for learning a reward model from an expert policy in partially observable Markov decision processes (POMDPs). The problem is formulated as inverse reinforcement learning (IRL) in the POMDP framework. The proposed algorithm then uses the expert trajectories to find an unknown reward model-based on the known POMDP model components. Similar to previous IRL work in Markov Decision Processes (MDPs), our algorithm maximizes the sum of the margin between the expert policy and the intermediate candidate policies. However, in contrast to previous work, the expert and intermediate candidate policy values are approximated using the beliefs recovered from the expert trajectories, specifically by approximating expert belief transitions. We apply our IRL algorithm to a healthcare dialogue POMDP where the POMDP model components are estimated from real dialogues. Our experimental results show that the proposed algorithm is able to learn a reward model that accounts for the expert policy. |
Year | DOI | Venue |
|---|---|---|
2012 | 10.1109/ICMLA.2012.31 | ICMLA |
Keywords | Field | DocType |
Markov processes,belief maintenance,health care,interactive systems,learning (artificial intelligence),IRL,POMDP framework,expert belief transition,expert policy,expert trajectory,healthcare dialogue management,intermediate candidate policy,inverse reinforcement learning algorithm,partially observable Markov decision process,partially observable domain,reward model learning,Inverse reinforcement learning,dialogue management,partially observable Markov decision processes | Belief maintenance,Dialogue management,Markov process,Observable,Markov model,Computer science,Partially observable Markov decision process,Markov decision process,Algorithm,Inverse reinforcement learning,Artificial intelligence,Machine learning | Conference |
Volume | Citations | PageRank |
1 | 3 | 0.43 |
References | Authors | |
5 | 2 | |
Name | Order | Citations | PageRank |
|---|---|---|---|
| Hamid R. Chinaei | 1 | 16 | 2.68 |
| Chaib-draa, Brahim | 2 | 1190 | 113.23 |