STDArm: Transfer Visuomotor Policy From Static Data Training to Dynamic Robot Manipulation


Yifan Duan, Heng Li, Yilong Wu, Wenhao Yu, Xinran Zhang, Yedong Shen, Jianmin Ji, Yanyong Zhang

Paper ID 147

Session 16. Manipulation III

Poster Session (Day 4): Tuesday, June 24, 12:30-2:00 PM

Abstract: Learning visuomotor policy from human demonstrations serves as an effective method for robots to acquire complex tasks. However, data collection on mobile platforms such as drones is extremely challenging, resulting in most research being conducted with robots in stationary conditions for data collection and policy training. Nonetheless, these policies struggle to maintain operational stability when the robot is in motion, primarily due to the low frequency of decision-making, inconsistencies in the input data distribution and differences in the motion characteristics of different robots. To address these issues, we propose a system named STDArm, which effectively transfers stationary-trained action strategies to dynamic states, including the robot’s ego-motion or wobble caused by structural factors. Specifically, we first design an action manager to efficiently store and manage the sequence of actions asynchronously output by the policy network, and to stably output actions at a higher control frequency as needed. Next, we predict the future state of the robotic arm’s base by observing its historical states, thereby developing an end-effector stabilizer to counteract the robot’s movements. Additionally, we incorporate a warm-up step during the robot’s initialization phase to estimate the parameters used in the system online, ensuring tight integration between the stabilizer and policy modules. We conduct comprehensive evaluations of the proposed system on two types of robotic arms, two types of mobile platforms, and three tasks. Experimental results indicate that the STDArm system significantly improves task execution success rates during the robot’s movement.