Aerial Manipulation Using Hybrid Force and Position NMPC Applied to Aerial Writing

Dimos Tzoumanikas (Imperial College London); Felix Graule (ETH Zurich); Qingyue Yan (Imperial College London); Dhruv Shah (Berkeley Artificial Intelligence Research); Marija Popovic (Imperial College London); Stefan Leutenegger (Imperial College London)


Aerial manipulation aims at combining the maneuverability of aerial vehicles with the manipulation capabilities of robotic arms. This, however, comes at the cost of the additional control complexity due to the coupling of the dynamics of the two systems. In this paper we present a Nonlinear Model Predictive Controller (NMPC) specifically designed for Micro Aerial Vehicles (MAVs) equipped with a robotic arm. We formulate a hybrid control model for the combined MAV-arm system which incorporates interaction forces acting on the end effector. We explain the practical implementation of our algorithm and show extensive experimental results of our custom built system performing multiple `aerial-writing' tasks on a whiteboard, revealing accuracy in the order of millimetres.

Live Paper Discussion Information

Start Time End Time
07/15 15:00 UTC 07/15 17:00 UTC

Virtual Conference Presentation

Supplementary Video

Paper Reviews

Review 1

The paper ``Aerial Manipulation Using Hybrid Force and Position NMPC Applied to Aerial Writing'' presents an MAV-arm platform and nonlinear model predictive control approach for writing tasks. While I find the paper interesting, there are a few things that hold back the clarity and rigor of the presentation. My two main critiques are (1) the presentation of the "algorithm" is convoluted, and (2) the experiments fail to offer any performance comparisons. MAIN CONCERNS: - THEORY: The presentation of the theory lacks clarity. Perhaps the most prominent example of this is the authors claim their algorithm is easily-extended to other work, but nowhere in the paper is an algorithm -- For equations, all variables should be introduced and defined before the equation is presented -- How does the framework of this NMPC compare to other methods? - EXPERIMENTS: While the authors present an extremely detailed literature review, there is little tangible analysis between various approaches. A summary comparison of performance characteristics (tracking accuracy, speed, etc) would make the experimental analysis stronger. -- It is unclear from the experiments why this particular delta arm is appropriate for the writing task -- It would be helpful to present some of the error metrics as a percentage on the accuracy of the trajectory. MINOR COMMENTS: - It seems like the related work could be condensed to give more room to technical content - The paper "Nonlinear Model Predictive Control for Aerial Manipulation" (Lunni et al, 2017) seems relevant to this work. Can the authors comment on the differences in approaches?

Review 2

This paper presents an important contribution to the field of aerial manipulation by demonstrating an impressively accurate tracking result for direct force feedback in combined position and force control for an underactuated MAV with an actuated arm. With fast arm dynamics to compensate for error in the underactuated base, tracking of the end effector is significantly improved. Further compliments to the team for achieving this result with mostly low cost and easily available parts. The reviewer sees this work as original, high quality, clear, and very significant to the aerial manipulation community. Title + introduction: “Aerial manipulation” might be a bit strong for the title. Would suggest “Aerial Interaction” or simplifying to “Combined Force and Position NMPC Applied to Aerial Writing”. See comment about hybrid position and force control below... “Millimeter accuracy” should refer to accuracy of 1mm, in this case it is around 1cm, so would be centimeter accuracy. It’s just a name, but should honestly reflect the result. Otherwise, just mention accuracy of about plus/minus 10mm. “In contrast to the second approach, we achieve on par precision while ...” → Introduction section shouldn’t really include results. Also this statement seems to highlight a superior approach, when better performance could be attributed to a nicer hardware implementation, control method (NMPC), or better tuning. Typos: page 2: - an underactuated MAVs → an underactuated MAV page 5, section VI: - a trust stand → a thrust stand - T_{wT} → T_{WT} page 8: - feasible plann → feasible plan - tranformation → transformation Equations: Page 3: - Revisit the formulations of (1b) and (1e). - The line of text after eq (4) should refer to _{T}r_{E_z}, instead of _{C}r_{E_z}. Page 4, section C: - Equations describing {A}r{J} from the geometry of figure 4, the reviewer believes should use only R instead of (R – r). Page 5, Fig 4: - (Front view) Frame F_A should be at the center of the delta structure - (Side view) {A}r{I_1} should be {A}r{J_1} Comments: - There is little discussion on the limitations of an underactuated system in terms of force exertion. The reference (and experimentally measured) forces are very small, particularly for contact inspection applications. There is clearly a relationship between higher force exertion and stability, that is not discussed in this paper. What are the limitations of force control for an underactuated MAV? How are force magnitude, position error, and stability coupled when we push these limits? - The term hybrid force and position control usually refers to Raibert and Craig’s implementation involving a selection function to control force in the constrained direction and motion in the orthogonal directions. Is this relevant here? It seems that this control approach combines both without selection, which would mean that the wall and end effector position must be exactly where expected. Perhaps the author could revisit the terminology and discuss the limitations of this environment model in an unstructured world (the discussion point that the whiteboard is not perfectly flat is already in this direction, and whiteboards are indeed quite flat!). - Experimental tuning of the costs Q is mentioned. The experimental values would be interesting for the research community, and useful for repeating results. Also, what are the effects of varying the prediciton horizon? - All error plots show end effector error above 1cm at some point, so it isn’t exactly sub-centimeter accuracy, but certainly on the order of 1 cm! The text states several times that the error does not go above 10cm, please revisit this. Interestingly, the higher error tends to occur when the system is in free flight, any thoughts on this? - Force trajectory generation is not discussed, but from the results plot seems to be a step function. Would smooth force trajectories give a better result, or is the predictive model element able to handle this very well? - The last paragraph in VI.C. mentions that the control model assumes the position of the end effector can be controlled infinitely fast, meaning that a step response would not be handled well by the MPC formulation. Some comment to address this? Should the MPC be reformulated so these can be reflected in control input constraints? - The statistical evaluation approach with multiple trials for different trajectories is well presented and highly appreciated!

Review 3

The paper discussed aerial manipulation systems of MAVs and proposed a new method to solve the problem of end effector trajectories tracking of a MAV equipped with a manipulator, where the task is to control both the vehicle and the manipulator for "aerial-writing". It introduced a novel formulation for the hybrid system, in which a set of standard Newton-Euler equations are used for modeling the dynamics. In particular, the effect of the external contact force that is introduced by the manipulator and acted on the MAV is modeled in the dynamics, where the forces are approximated via a linear spring model. A nonlinear MPC was used for the trajectory tracking task. The author also talked about the trajectory generation method they used for mapping arbitrary sets of characters to end effector trajectories, where they assume the accelerations are constant. The author(s) conducted a list of experiments and demonstrated the effectiveness of the proposed approach. The proposed approach achieved high accuracy (millimetre-level accuracy) in writing different characters, such as RSS or E=mc^2, on a whiteboard given a perfect state estimation of both the vehicle and the board from a motion capture system. Experiment setups are discussed. Detailed explanations of the experimental results are provided by the author(s). The author pointed out the implementation details, technical difficulties they encountered during the experiments, and limitations of the method. The paper was written in clear and formal English, with a well-organized structure and concise expressions. Overall, the paper contributes to aerial manipulations by combing a novel hybrid dynamical model with nonlinear model predictive control.