Abstract: Dynamic fast adaptation is one of the basic capabilities that enables the animals to timely and properly adjust its locomotion reacting to the unpredictable changes. Such capability is also essential for the quadruped robot, when working in the unforseen environment. While reinforcement learning (RL) has achieved a significant progress in locomotion control, rapid adaptation to the model uncertainties remains a challenge. In this paper, we seek to ascertain the control mechanism behind the locomotion RL policy, from which we propose a new RL-based Rapid onLine Adaptive Control (RL2AC) algorithm to complementarily combine the RL policy and the adaptive control together. RL2AC is run at a frequency of 1000Hz without the need for simultaneous training with RL. It presents a strong capability against the external disturbances or the sim-to-real gap, resulting in a robust locomotion, which is achieved through proper torque compensation derived from a novel adaptive controller. Various simulation and experiments have demonstrated the effectiveness of the proposed RL2AC against the heavy load, disturbances acted on one leg, lateral torque, sim-to-real gap and various terrains.
