Title | ||
---|---|---|
Prediction-Based Multi-Agent Reinforcement Learning in Inherently Non-Stationary Environments. |
Abstract | ||
---|---|---|
Multi-agent reinforcement learning (MARL) is a widely researched technique for decentralised control in complex large-scale autonomous systems. Such systems often operate in environments that are continuously evolving and where agents’ actions are non-deterministic, so called inherently non-stationary environments. When there are inconsistent results for agents acting on such an environment, learning and adapting is challenging. In this article, we propose P-MARL, an approach that integrates prediction and pattern change detection abilities into MARL and thus minimises the effect of non-stationarity in the environment. The environment is modelled as a time-series, with future estimates provided using prediction techniques. Learning is based on the predicted environment behaviour, with agents employing this knowledge to improve their performance in realtime. We illustrate P-MARL’s performance in a real-world smart grid scenario, where the environment is heavily influenced by non-stationary power demand patterns from residential consumers. We evaluate P-MARL in three different situations, where agents’ action decisions are independent, simultaneous, and sequential. Results show that all methods outperform traditional MARL, with sequential P-MARL achieving best results. |
Year | DOI | Venue |
---|---|---|
2017 | 10.1145/3070861 | TAAS |
Keywords | Field | DocType |
Multi-agent systems,reinforcement learning,environment prediction,smart grids | Change detection,Smart grid,Computer science,Multi-agent system,Power demand,Artificial intelligence,Autonomous system (Internet),Error-driven learning,Reinforcement learning,Distributed computing | Journal |
Volume | Issue | ISSN |
12 | 2 | 1556-4665 |
Citations | PageRank | References |
4 | 0.44 | 29 |
Authors | ||
3 |
Name | Order | Citations | PageRank |
---|---|---|---|
Andrei Marinescu | 1 | 21 | 4.29 |
Ivana Dusparic | 2 | 75 | 20.37 |
Siobhán Clarke | 3 | 699 | 87.36 |