Abstract: Robotic systems employing continuum bodies offer a high degree of dexterity, which provides advantages in terms of accuracy and safety when operating in cluttered environments. However, current methods of describing posture or detecting contact for such continuum structures are focusing on bespoke designs or are limited to a single sensing modality, which could hinder their possibility for scalability and generalization. This study proposes a novel vision-based tactile sensing system, named ConTac, that provides both proprioception and tactile detection for a continuum-emulated arm with soft skin. To realize the mentioned functions, we employ two corresponding deep-learning models trained using simulation data. The models are zero-shot applied to real-world data without fine-tuning. The experimental results show that the system could predict the posture of a skinned robot arm with a mean tip position error of 8.83 mm, while the mean error for touch location was 28.86 mm. We then compared the model performance on two different robot modules, proving the justification of the system. An admittance control strategy is then developed using the shape and contact information, allowing the robot arm to react properly to collisions. The proposed method shows potential in adapting to hyper-redundant or continuum robots, enhancing their perception capabilities and control paradigms.
