Robot

Linguistics

Philosophy

Deep neural networks

Biology

Economics

Computer science

Controller (irrigation)

Point cloud

Position (finance)

Object (grammar)

Agronomy

Feature (linguistics)

Artificial intelligence

Visual servoing

Deep learning

Finance

Artificial neural network

Computer vision

IEEE Robotics and Automation Letters

Due to its high dimensionality, deformable object manipulation is a challenging problem in robotics. In this letter, we present a deep neural network based controller to servo control the position and shape of deformable objects with unknown deformation properties. In particular, a multi-layer neural network is used to map between the robotic end-effector's movement and the object's deformation measurement using an online learning strategy. In addition, we introduce a novel feature to describe deformable objects' deformation used in visual servoing. This feature is directly extracted from the 3-D point cloud rather from the 2-D image as in previous work. In addition, we perform simultaneous tracking and reconstruction for the deformable object to resolve the partial observation problem during the deformable object manipulation. We validate the performance of our algorithm and controller on a set of deformable object manipulation tasks and demonstrate that our method can achieve effective and accurate servo control for general deformable objects with a wide variety of goal settings. Experiment videos are available at https://sites.google.com/view/mso-deep.

3-D Deformable Object Manipulation Using Deep Neural Networks