标题： 可伸缩的多智能体强化学习用于住宅能源灵活性的分布式控制 显示英文标题

标题： Scalable multi-agent reinforcement learning for distributed control of residential energy flexibility

摘要： 本文提出了一种新的可扩展类型多智能体强化学习协调分布式住宅能源的方法。合作智能体学习控制由电动汽车、空间供暖和柔性负载提供的灵活性，在部分可观测的随机环境中进行操作。 在标准独立 Q 学习方法中，部分可观测情况下代理的协调性能在随机环境中按比例下降。这里，通过从历史数据的离线凸优化中学习以及在奖励信号中隔离总奖励的边际贡献的新颖组合，提高了大规模的稳定性和性能。 使用固定大小的 Q 表，产消者能够评估其对总系统目标的边际影响，而无需与其他方或中央协调器共享个人信息。 案例研究用于评估不同探索来源、奖励定义和多智能体学习框架组合的适用性。结果表明，所提出的策略由于能源进口成本、损耗、配电网络拥堵、电池折旧和温室气体排放的减少，在个体和系统层面创造了价值。 显示英文摘要

摘要： This paper proposes a novel scalable type of multi-agent reinforcement learning-based coordination for distributed residential energy. Cooperating agents learn to control the flexibility offered by electric vehicles, space heating and flexible loads in a partially observable stochastic environment. In the standard independent Q-learning approach, the coordination performance of agents under partial observability drops at scale in stochastic environments. Here, the novel combination of learning from off-line convex optimisations on historical data and isolating marginal contributions to total rewards in reward signals increases stability and performance at scale. Using fixed-size Q-tables, prosumers are able to assess their marginal impact on total system objectives without sharing personal data either with each other or with a central coordinator. Case studies are used to assess the fitness of different combinations of exploration sources, reward definitions, and multi-agent learning frameworks. It is demonstrated that the proposed strategies create value at individual and system levels thanks to reductions in the costs of energy imports, losses, distribution network congestion, battery depreciation and greenhouse gas emissions.