From Reaction to Anticipation: Proactive Failure Recovery through Agentic Task Graph for Robotic Manipulation


Sheng Xu, Ruixing Jin, Huayi Zhou, Bo Yue, Guanren Qiao, Yunxin Tai, Yueci Deng, Kui Jia, Guiliang Liu

Paper ID 180

Session Planning

Poster session details TBA

Abstract: Recent advances in robotic manipulation remain hindered by the inevitability of task failures, particularly in dynamic and unstructured environments. To handle such failure, existing frameworks typically follow a stepwise detect–reason–recover pipeline, which often incurs high latency and limited robustness due to delayed reasoning and reactive planning. Inspired by the human capability to anticipate and proactively plan for potential failures, we introduce AgentChord, an agentic system that models a manipulation task as a directed, recovery-augmented graph. Prior to execution, this graph is enriched with anticipatory recovery branches that specify context-aware corrective behaviors, enabling immediate and targeted responses when failures occur. During execution, AgentChord operates through a choreography of specialized agents, covering a composer for task structuring, an arranger for execution compilation, and a conductor for recovery orchestration. AgentChord coordinates these agents via low-latency monitors that detect deviations and trigger pre-compiled recoveries without re-planning. Empirical studies on diverse long-horizon bimanual manipulation tasks demonstrate that AgentChord substantially improves success rates and execution efficiency, advancing the reliability and autonomy of real-world robotic systems.