Volume 16, Number 6
Knowledge Reuse Degree Asymmetry in Transfer Reinforcement Learning
Authors
Satoru Ikeda 1, Kohei Esaki 1, Hitoshi Kono 2, Shota Chikushi 2, Kaori Watanabe 4, and Hidekazu Suzuki 4, 1 Tokyo Denki University, Japan, 2 Kindai University, Japan, 3 New Technology Foundation, Japan, 4 Tokyo Polytechnic University, Japan
Abstract
In recent years, autonomous systems such as automated driving using machine learning have been developed and are being implemented in society. Therefore, reinforcement learning and transfer learning, which are considered useful for control methods of automated systems, are being studied. transfer learning is a method of reusing knowledge from the source task to the target task, and knowledge means that the policy, action-value function, model, and so on, in the machine learning domain. In the transferring situation, to avoid negative transfer such as over learning, the transfer rate can be adjusted transferring action value amplitude can be set. However, the transfer rates are usually determined by human experiences. The automatic transfer rate adjustment method proposed by Kono et al. has demonstrated improved environmental adaptability, and the function shape and a step size parameter determine the adjustment. However, the agent's environmental adaptation performance has not been achieved to the transfer rate set by humans. In this paper, separate step size parameters for when the transfer rate increases and decreases, a method is proposed that improves upon the automatic transfer rate adjustment method proposed by Kono et al. Experimental results have shown that reinforcement learning and transfer learning are conducted through simulations using a shortest path problem in two dimensions. Experimental results have verified that the transfer learning method improves adaptive performance compared to the method proposed by Kono et al.
Keywords
Reinforcement learning, Transfer learning, Transfer rate, Agent simulation