中圖分類號： TP18
文獻標識碼： A
DOI： 10.19358/j.issn.2096-5133.2021.06.008
引用格式： 楊尚彤，王子磊. 聯合隨機性策略的深度強化學習探索方法[J].信息技術與網絡安全，2021，40(6)：43-49.

Efficient exploration with stochastic policy for deep reinforcement learning

Abstract： At present, deep reinforcement learning algorithms have been shown to solve many complex tasks, but how to balance the relationship between exploration and exploitation is still a basic problem. Thus, this paper proposes an efficient exploration strategy combined with stochastic policy for deep reinforcement learning. The main contribution is to use the experience generated by stochastic policies to train deterministic policies, which encourages deterministic strategies to learn to explore while maintaining their own advantages. This takes advantage of the exploration ability of stochastic policies. By combining DDPG(Deep Deterministic Policy Gradient) and the proposed exploration method, the algorithm called stochastic guidance for deterministic policy gradient(SGDPG) is obtained. Finally, the results of the experiment in several complex environments show that SGDPG has higher exploration and sample efficiency than DDPG when faced with deep exploration problems.

Key words : reinforcement learning；deep reinforcement learning；exploration-exploitation dilemma