Deep Drone Acrobatics

EliaKaufmann, Antonio Loquercio, Rene Ranftl, Matthias Müller, Vladlen Koltun, Davide Scaramuzza


Performing acrobatic maneuvers with quadrotors is extremely challenging. Acrobatic flight requires high thrust and extreme angular accelerations that push the platform to its physical limits. Professional drone pilots often measure their level of mastery by flying such maneuvers in competitions. In this paper, we propose to learn a sensorimotor policy that enables an autonomous quadrotor to fly extreme acrobatic maneuvers with only onboard sensing and computation. We train the policy entirely in simulation by leveraging demonstrations from an optimal controller that has access to privileged information. We use appropriate abstractions of the visual input to enable transfer to a real quadrotor. We show that the resulting policy can be directly deployed in the physical world without any fine-tuning on real data. Our methodology has several favorable properties: it does not require a human expert to provide demonstrations, it cannot harm the physical system during training, and it can be used to learn maneuvers that are challenging even for the best human pilots. Our approach enables a physical quadrotor to fly maneuvers such as the Power Loop, the Barrel Roll, and the Matty Flip, during which it incurs accelerations of up to 3g.

Live Paper Discussion Information

Start Time End Time
07/15 15:00 UTC 07/15 17:00 UTC

Virtual Conference Presentation

Supplementary Video

Paper Reviews

Review 1

This paper proposes a complete learning system which allows a drone to fly acrobatic maneuvers. It learns a sensorimotor agent purely in simulation and performs zero-shot sim-to-real transfer. Impressive results are demonstrated on a real drone. This paper is very well written. I was able to fully understand the paper without any difficulties. The paper has a clear technical contribution: preprocessing the raw observations to a feature representation that has smaller sim-to-real gap. This is an important contribution because sim-to-real gap is a universal problem for almost all robotic applications, and is a major obstacle that prevents the use of simulation. This paper presents an effective method, and a different perspective to bridge the gap. The paper showed successful deployment of simulation policy on a real drone and conducted thorough analysis to show the effectiveness of the proposed approach. The only thing that I am not confident about is the difficulty of the problem: acrobatic maneuvers for drones, because this is not my immediate research area. If the problem is indeed hard, and this is the first demonstration of such maneuvers, I would vote for acceptance of the paper.

Review 2

The authors propose a deep learning-based approach to training acrobatic motor skills of drones. The key idea is to apply the imitation learning technique (more specifically, DAGGER) to the reference motion while abstracting the visual inputs with feature extractors. The feature extractor computes the motion of salient keypoint in the visual inputs, which would provide better state estimation to the robot. Then all the features (visual inputs, IMU, reference trajectories) are asynchronously fed to the policy network. The reference trajectory is obtained by training a “privileged expert”, which has access to all the ground-truth state information, using MPC. The framework trains the agent solely in the simulation (Gazebo) and transfers to the real world. The authors demonstrate a few agile skills on real drones (AscTec Hummingbird), including a Barrel Roll, a Power Loop, a Matty-Loop. I am personally impressed by the results presented by this paper: it seems to be pretty agile, without suffering from the sim-to-real transfer. I’m not too familiar with the state-of-the-art in acrobatic motions of drones, but it seems to be a great contribution to have the first acrobatic motion without additional sensory inputs in the robotic community. In this sense, I’m pretty positive about this submission. On the other hand, the main contribution of the work is the sim-to-real transfer technique using visual points, as claimed by the authors. But their feature tracker (FT) does not improve the results (tracking errors or success rates) in Table I: the drone achieves near-perfect success rates for all four tasks (btw, it seems too obvious that it cannot perform good motor skills without reference motions or IMU). Now Figure 4 provides a bit contradictory results, which are only analyzed in simulation with a specific training/testing setting. Therefore, it is not very clear whether the proposed key contribution, a sim-to-real technique, is crucial in the proposed work. And the architecture of asynchronous policy networks seems to be standard. It extracts features from raw inputs, concatenates them into history, applies convolutional operations, and generates outputs with feedforward networks. It might not be a very significant contribution to the community. The paper itself is very well written and reads smoothly. I would suggest the authors adding the following reference to the paper, which also discusses the sim-to-real of aerial vehicles. Xu, J., Du, T., Foshey, M., Li, B., Zhu, B., Schulz, A. and Matusik, W., 2019. Learning to fly: computational controller design for hybrid UAVs with reinforcement learning. ACM Transactions on Graphics (TOG), 38(4), pp.1-12.