This paper proposes a complete learning system which allows a drone to fly acrobatic maneuvers. It learns a sensorimotor agent purely in simulation and performs zero-shot sim-to-real transfer. Impressive results are demonstrated on a real drone. This paper is very well written. I was able to fully understand the paper without any difficulties. The paper has a clear technical contribution: preprocessing the raw observations to a feature representation that has smaller sim-to-real gap. This is an important contribution because sim-to-real gap is a universal problem for almost all robotic applications, and is a major obstacle that prevents the use of simulation. This paper presents an effective method, and a different perspective to bridge the gap. The paper showed successful deployment of simulation policy on a real drone and conducted thorough analysis to show the effectiveness of the proposed approach. The only thing that I am not confident about is the difficulty of the problem: acrobatic maneuvers for drones, because this is not my immediate research area. If the problem is indeed hard, and this is the first demonstration of such maneuvers, I would vote for acceptance of the paper.

The authors propose a deep learning-based approach to training acrobatic motor skills of drones. The key idea is to apply the imitation learning technique (more specifically, DAGGER) to the reference motion while abstracting the visual inputs with feature extractors. The feature extractor computes the motion of salient keypoint in the visual inputs, which would provide better state estimation to the robot. Then all the features (visual inputs, IMU, reference trajectories) are asynchronously fed to the policy network. The reference trajectory is obtained by training a “privileged expert”, which has access to all the ground-truth state information, using MPC. The framework trains the agent solely in the simulation (Gazebo) and transfers to the real world. The authors demonstrate a few agile skills on real drones (AscTec Hummingbird), including a Barrel Roll, a Power Loop, a Matty-Loop. I am personally impressed by the results presented by this paper: it seems to be pretty agile, without suffering from the sim-to-real transfer. I’m not too familiar with the state-of-the-art in acrobatic motions of drones, but it seems to be a great contribution to have the first acrobatic motion without additional sensory inputs in the robotic community. In this sense, I’m pretty positive about this submission. On the other hand, the main contribution of the work is the sim-to-real transfer technique using visual points, as claimed by the authors. But their feature tracker (FT) does not improve the results (tracking errors or success rates) in Table I: the drone achieves near-perfect success rates for all four tasks (btw, it seems too obvious that it cannot perform good motor skills without reference motions or IMU). Now Figure 4 provides a bit contradictory results, which are only analyzed in simulation with a specific training/testing setting. Therefore, it is not very clear whether the proposed key contribution, a sim-to-real technique, is crucial in the proposed work. And the architecture of asynchronous policy networks seems to be standard. It extracts features from raw inputs, concatenates them into history, applies convolutional operations, and generates outputs with feedforward networks. It might not be a very significant contribution to the community. The paper itself is very well written and reads smoothly. I would suggest the authors adding the following reference to the paper, which also discusses the sim-to-real of aerial vehicles. Xu, J., Du, T., Foshey, M., Li, B., Zhu, B., Schulz, A. and Matusik, W., 2019. Learning to fly: computational controller design for hybrid UAVs with reinforcement learning. ACM Transactions on Graphics (TOG), 38(4), pp.1-12.