Physics

Maximum bubble pressure method

Nanotechnology

Human–computer interaction

Interface (matter)

Bubble

GRASP

Parallel computing

Artificial intelligence

Soft robotics

Materials science

Programming language

Computer science

Quantum mechanics

Soft materials

Simulation

Triboelectric effect

Robot

Nature Communications

In this paper, we propose a multimodal flexible sensory interface for interactively teaching soft robots to perform skilled locomotion using bare human hands. First, we develop a flexible bimodal smart skin (FBSS) based on triboelectric nanogenerator and liquid metal sensing that can perform simultaneous tactile and touchless sensing and distinguish these two modes in real time. With the FBSS, soft robots can react on their own to tactile and touchless stimuli. We then propose a distance control method that enabled humans to teach soft robots movements via bare hand-eye coordination. The results showed that participants can effectively teach a self-reacting soft continuum manipulator complex motions in three-dimensional space through a “shifting sensors and teaching” method within just a few minutes. The soft manipulator can repeat the human-taught motions and replay them at different speeds. Finally, we demonstrate that humans can easily teach the soft manipulator to complete specific tasks such as completing a pen-and-paper maze, taking a throat swab, and crossing a barrier to grasp an object. We envision that this user-friendly, non-programmable teaching method based on flexible multimodal sensory interfaces could broadly expand the domains in which humans interact with and utilize soft robots.

Touchless interactive teaching of soft robots through flexible bimodal sensory interfaces