标题： 用于2D射击游戏的强化学习智能体 显示英文标题

标题： Reinforcement Learning Agent for a 2D Shooter Game

摘要： 在复杂游戏环境中的强化学习智能体常常面临稀疏奖励、训练不稳定和样本效率低的问题。 本文提出了一种混合训练方法，将离线模仿学习与在线强化学习相结合，用于二维射击游戏的智能体。 我们实现了一个多头神经网络，分别输出行为克隆和Q学习的结果，通过带有注意力机制的共享特征提取层进行统一。 使用纯深度Q网络的初步实验表现出显著的不稳定性，智能体尽管偶尔表现良好，但经常退化为较差的策略。 为了解决这个问题，我们开发了一种混合方法，首先在基于规则的智能体的演示数据上进行行为克隆，然后过渡到强化学习。 我们的混合方法在对抗基于规则的对手时，始终达到70%以上的胜率，明显优于纯强化学习方法，后者表现出较高的方差和频繁的性能下降。 多头架构使得不同学习模式之间的知识迁移更加有效，同时保持了训练的稳定性。 结果表明，将基于演示的初始化与强化学习优化相结合，为在复杂多智能体环境中开发游戏AI智能体提供了一个稳健的解决方案，其中纯粹的探索证明是不足的。 显示英文摘要

摘要： Reinforcement learning agents in complex game environments often suffer from sparse rewards, training instability, and poor sample efficiency. This paper presents a hybrid training approach that combines offline imitation learning with online reinforcement learning for a 2D shooter game agent. We implement a multi-head neural network with separate outputs for behavioral cloning and Q-learning, unified by shared feature extraction layers with attention mechanisms. Initial experiments using pure deep Q-Networks exhibited significant instability, with agents frequently reverting to poor policies despite occasional good performance. To address this, we developed a hybrid methodology that begins with behavioral cloning on demonstration data from rule-based agents, then transitions to reinforcement learning. Our hybrid approach achieves consistently above 70% win rate against rule-based opponents, substantially outperforming pure reinforcement learning methods which showed high variance and frequent performance degradation. The multi-head architecture enables effective knowledge transfer between learning modes while maintaining training stability. Results demonstrate that combining demonstration-based initialization with reinforcement learning optimization provides a robust solution for developing game AI agents in complex multi-agent environments where pure exploration proves insufficient.