Communicating with Robots Naturally

Organizers: Nakul Gopalan, Rohan Paul, Stefanie Tellex, Jesse Thomason and Lawson L.S. Wong


For robots to be our companions in workplaces and homes, we need them to understand the most common modalities of human communication, such as natural language and gestures. Moreover, to solve problems alongside humans, robots need to understand world models that humans express. These problems are complicated because human language is free-form with partially observable intentions, and most environments are unstructured and partially observable. Fortunately, language can be used both to instruct and to guide robots. In particular, natural language provides a common interface between mental models of the world. By learning and leveraging these models, robots can become more effective partners for humans when operating alongside them in shared environments.