标题： 可扩展的多智能体强化学习用于住宅能源灵活性的分布式控制 显示英文标题

标题： Scalable multi-agent reinforcement learning for distributed control of residential energy flexibility

摘要： 本文提出了一种新型的可扩展的基于多智能体强化学习的协调方法，用于分布式住宅能源。 协作智能体在部分可观测的随机环境中学习控制电动汽车、空间供暖和灵活负荷所提供的灵活性。 在标准的独立Q学习方法中，部分可观测性下智能体的协调性能在随机环境中随着规模扩大而下降。 在这里，从历史数据的离线凸优化中学习与在奖励信号中隔离总奖励的边际贡献的新型结合，提高了大规模下的稳定性和性能。 使用固定大小的Q表，生产消费用户能够在不与其他用户或中央协调器共享个人数据的情况下评估其对总系统目标的边际影响。 案例研究用于评估不同探索源、奖励定义和多智能体学习框架组合的适应性。 结果表明，由于能源进口成本、损耗、配电网络拥堵、电池退化和温室气体排放的减少，所提出的策略在个体和系统层面都创造了价值。 显示英文摘要

摘要： This paper proposes a novel scalable type of multi-agent reinforcement learning-based coordination for distributed residential energy. Cooperating agents learn to control the flexibility offered by electric vehicles, space heating and flexible loads in a partially observable stochastic environment. In the standard independent Q-learning approach, the coordination performance of agents under partial observability drops at scale in stochastic environments. Here, the novel combination of learning from off-line convex optimisations on historical data and isolating marginal contributions to total rewards in reward signals increases stability and performance at scale. Using fixed-size Q-tables, prosumers are able to assess their marginal impact on total system objectives without sharing personal data either with each other or with a central coordinator. Case studies are used to assess the fitness of different combinations of exploration sources, reward definitions, and multi-agent learning frameworks. It is demonstrated that the proposed strategies create value at individual and system levels thanks to reductions in the costs of energy imports, losses, distribution network congestion, battery depreciation and greenhouse gas emissions.