Demonstrating Mobile Manipulation in the Wild: A Metrics-Driven Approach


Max Bajracharya
Toyota Research Institute
James Borders
Toyota Research Institute
Richard Cheng
Toyota Research Institute
Dan Helmick
Toyota Research Institute
Lukas Kaul
Toyota Research Institute
Dan Kruse
Toyota Research Institute
John Leichty
Toyota Research Institute
Jeremy Ma
NVIDIA
Carolyn Matl
Toyota Research Institute
Frank Michel
Toyota Research Institute
Chavdar Papazov
Toyota Research Institute
Josh Petersen
Amazon
Krishna Shankar
Apple
Mark Tjersland
Toyota Research Institute
Paper Website

Paper ID 55

Nominated for Best Demo Paper

Session 7. Mobile Manipulation and Locomotion

Demo

Poster Session Wednesday, July 12

Poster 23

Abstract: We present our general-purpose mobile manipulation system consisting of a custom robot platform and key algorithms spanning perception and planning. To extensively test the system in the wild and benchmark its performance, we choose a grocery shopping scenario in an actual, unmodified grocery store. We derive key performance metrics from detailed robot log data collected during six week-long field tests, spread across 18 months. These objective metrics, gained from complex yet repeatable tests, drive the direction of our research efforts and let us continuously improve our system’s performance. We find that thorough end-to-end system-level testing of a complex mobile manipulation system can serve as a reality-check for state-of-the-art methods in robotics. This effectively grounds robotics research efforts in real world needs and challenges, which we deem highly useful for the advancement of the field. To this end, we share our key insights and takeaways to inspire and accelerate similar system-level research projects.