Abstract: In this talk, I will share my journey toward building generalist robot policies—and how it surprisingly mirrors the process of cultivating my backyard garden. I’ll begin by unpacking the key components of robotics foundation models and their promise for generalization across diverse tasks and environments. To ground this discussion, I will introduce StarGen, a taxonomy for evaluating generalization in robot policies, spanning semantic, visual, and behavioral dimensions. Next, I’ll explore what truly enables generalization both algorithmically and in terms of data—drawing an analogy to the delicate process of paintbrush pollination in gardening. I’ll present RT-H: Action Hierarchies using Language Motion, a framework that leverages low-level language abstractions—language motions—to stitch together diverse sources of data. Building on this, I’ll discuss the broader concept of intermediate representations—such as bounding boxes, language-conditioned motions, and trajectory traces—that help bridge the gap between the perceptual strengths of vision-language models and the physical execution required by robots. On the data side, I’ll introduce RoboCrowd, a scalable approach to collecting robotic demonstrations through crowdsourcing and incentive design. I’ll conclude with a look at GeminiRobotics, a system showcasing steerable and interactive capabilities, and discuss its potential to shape the future of human-robot interaction.
Bio: Dorsa Sadigh is an associate professor in Computer Science and an HAI senior fellow at Stanford University. She is also a research scientist at Google DeepMind. Her research interests lie in the intersection of robot learning and human-robot interaction. Specifically, she is interested in developing algorithms for adaptive learning agents that can learn from humans and interact with them. Dorsa received her doctoral degree in Electrical Engineering and Computer Sciences (EECS) from UC Berkeley in 2017, and received her bachelor’s degree in EECS from UC Berkeley in 2012. She is awarded the Sloan Fellowship, PECASE award, NSF CAREER award, ONR Young Investigator award, and MIT TR35.
Abstract: Over the past decade, flying a spacecraft has gone from something only government agencies and large corporations could afford to something universities and small startups can accomplish on a shoestring budget. While the growth of the commercial launch industry is partly responsible for this transformation, an even bigger factor has been advances in electronics and computation that have enabled smaller and cheaper spacecraft. I will discuss my work to advance low-cost, autonomous spacecraft that can accomplish exciting new missions like global wildlife tracking, building a “Martian GPS,” and searching for life in the geysers of Enceladus. These technologies also carry the promise of a future in which access to spaceflight is democratized and widespread. I will highlight a number of recent missions, including KickSat-2, which deployed the four-gram Sprite – the world’s smallest spacecraft; V-R3x, which demonstrated on-orbit mesh networking; PY4, which demonstrated attitude-control and formation-flying capabilities without relying on expensive conventional sensors or actuators; and the upcoming Argus mission, which will demonstrate autonomous vision-based navigation in low-Earth orbit.
Bio: Zac Manchester is an Assistant Professor in The Robotics Institute at Carnegie Mellon where he leads the Robotic Exploration Lab. His research leverages insights from physics, control theory, and optimization to enable robotic systems that can achieve the same level of agility, robustness, and efficiency as humans and animals. His lab develops algorithms for controlling a wide range of autonomous systems from cars merging onto highways to spacecraft landing on Mars. Zac Previously worked at NASA Ames Research Center and received a NASA Early Career Faculty Award in 2018 and a Google Faculty Award in 2020. He has also served as Principal Investigator of four NASA small-satellite missions.