Abstract:
To address insufficient exploration and premature convergence to suboptimal solutions in existing multi-agent reinforcement learning (MARL) methods, an adaptive exploration method based on deep reinforcement learning is proposed. Furthermore, using the same exploration rate for all agents limits their adaptability to dynamic environments and inter-agent interactions in complex tasks. Based on the QMIX framework, the proposed method adjusts agent exploration at two temporal scales. At each time step, the policy entropy is computed, and actions are selected according to a probability distribution derived from the state-action values. At a higher temporal scale, performance and exploration indicators are defined over a time window, and exploration rates are adaptively adjusted according to global team performance and exploration status. Experiments conducted on the 3s5z and 5m_vs_6m tasks in the StarCraft Ⅱ environment show that the proposed method consistently outperforms baseline methods and achieves higher average win rates in both tasks, significantly improving the learning performance of the multi-agent system.