Deep Q-Network Based on Important Experience Replay

Inhalt:
Although traditional prioritized experience replay effectively improves the learning efficiency of experience data, it also causes problems such as loss of sample diversity and additional consumption of computing resources. The loss of sample diversity easily leads to the difficulty of algorithm convergence, and the consumption of extra computing resources leads to the slow running speed of the algorithm. In this paper, the Deep Q-Network based on important experience replay (DQN-IER) is proposed. First, the dual experience memory structure is designed in the DQN-IER algorithm, which divides the experience data into general experience data and important experience data and stores them separately. Then, general experience and important experience are alternately extracted in the process of algorithm execution, which can effectively improve the learning efficiency of experience data and avoid the problem of sample diversity loss. Finally, the mountain car, cart pole and pendulum control task from the OpenAI Gym were selected as the experimental environment for comparison experiments. The experimental results show that DQN-IER performs better than DQN and DQN-PER, which can effectively reduce the training cost.