Abstract: It is hard not to be impressed by the competence of animals compared to state-of-the-art robots in real-world tasks. For example, an individual ant is able to emerge for the first time from her nest into an unknown environment, rapidly learn the surrounding cues, traverse hundreds of metres of rough terrain, identify, manipulate and grasp a suitable food item, and carry it directly back to the nest, which (by the way) she has cooperatively constructed. But how do we go beyond inspiration – how can we use our observations of animals (plus much perspiration) to produce useful solutions for robot tasks? My approach is to engage closely in trying to understand the biological system through reverse engineering, with robots serving as physical models to test that understanding. This work has strategically focused on insects, as they exhibit a wide range of robotics-relevant capabilities, achieved with apparently minimal computation. Moreover, surprisingly similar brain structures have been preserved through evolution across the huge variety of body morphologies, modes of locomotion, lifestyles and habitats of insects, suggesting the solutions are highly generalisable. I will describe what we currently know about insect navigation and discuss ongoing work to understand insect grasping and manipulation.
Bio: Barbara Webb obtained a BSc in Psychology at the University of Sydney followed by a PhD in Artificial Intelligence at the University of Edinburgh, where she began her exploration of insect-inspired robots by building a robot cricket (featured in Scientific American). She held faculty positions at the University of Nottingham and University of Stirling before returning to the University of Edinburgh where she is now Professor of Biorobotics in the School of Informatics. She leads the Insect Robotics group, investigating navigation, learning and sensorimotor control, and holds an EPSRC Advanced Career Fellowship to study how insect grasp objects. She has been invited to write reviews describing her pioneering approach to using robots to understand animal behavior for Nature and Science. She was elected a Fellow of the Royal Society of Edinburgh in 2022.
Abstract: Learning-based large models are revolutionizing robotic perception. Surprisingly, even “blind” text-based large models seem able to model scene structures! Multimodal models excel even in zero shot settings, yet are prone to hallucination and scale poorly with large images. I’ll show how VLM hallucination can be reduced with “Retrospective thinking” tokens, and how efficient inference can be performed on large-format imagery with attentive token decoding. I’ll also cover conditional visual prediction via Navigation World Models. Finally, I’ll present 4D representations for robotics, which combine dynamic 3D reconstruction with implicit pointwise correspondence, which offers the promise of scalable robotic learning with world- and task-centric representations.
Bio: Prof. Darrell is on the faculty of the CS and EE Divisions of the EECS Department at UC Berkeley. He founded and co-leads Berkeley’s Berkeley Artificial Intelligence Research (BAIR) lab, the Berkeley DeepDrive (BDD) Industrial Consortia, and the BAIR Commons program. He also was Faculty Director of the PATH research center at UC Berkeley from 2015-2021, and led the Vision group at the UC-affiliated International Computer Science Institute in Berkeley from 2008-2014. Prior to that, Prof. Darrell was on the faculty of the MIT EECS department from 1999-2008, where he directed the Vision Interface Group. He was a member of the research staff at Interval Research Corporation from 1996-1999, and received the S.M., and PhD. degrees from MIT in 1991 and 1996, respectively. He obtained the B.S.E. degree from the University of Pennsylvania in 1988. Darrell’s group develops algorithms for large-scale perceptual learning, including object and activity recognition and detection, for a variety of applications including autonomous vehicles, media search, and multimodal interaction with robots and mobile devices. His areas of interest include computer vision, machine learning, natural language processing, and perception-based human computer interfaces.
Prof. Darrell also co-founded and serves as President of Prompt AI. Darrell is an advisor to several other ventures, including SafelyYou, Nexar, and SuperAnnotate. Previously, Darrell advised Pinterest, Tyzx (acquired by Intel), IQ Engines (acquired by Yahoo), Koozoo, BotSquare/Flutter (acquired by Google), MetaMind (acquired by Salesforce), Trendage, Center Stage, KiwiBot, WaveOne, DeepScale, and Grabango. Darrell has also served as an expert witness for patent litigation relating to computer vision.