This post is also available in:
עברית (Hebrew)
A research team at MIT has introduced a new method for controlling soft, flexible robots using just a single image—without relying on complex sensors or detailed modeling. The approach, recently published in Nature, could help make soft robotics more practical for use in tight or unpredictable environments.
Soft robots, inspired by biological systems, are better at moving through confined spaces and adapting to uneven terrain than traditional rigid machines. However, this flexibility usually requires carefully designed control systems, often involving multiple sensors, expensive equipment, and robot-specific calibration. These requirements have made soft robots harder to prototype and scale, according to TechXplore.
The new method sidesteps many of these challenges. The researchers trained a deep learning model on around three hours of video footage of robots performing random movements from different angles. This taught the system to understand a robot’s shape and motion capabilities from just one image. This allows the robot to follow commands without needing extensive pre-programmed models or motion capture systems.
The team tested their system on several robotic platforms, including a 3D-printed pneumatic hand, a soft wrist joint, a 16-joint robotic hand, and a low-cost robotic arm. In all cases, the system was able to control movement with a high degree of accuracy—achieving less than three degrees of joint error and under 4 millimeters of fingertip accuracy error. It also adapted to changes in the robot’s surroundings without additional adjustments.
One of the main advantages of this approach is that it removes the need for robot-specific customization. This could make it easier to develop and test new designs without rebuilding control systems each time.
The system currently relies on visual input, so it may not perform as well in tasks that require touch or precise force feedback. However, the researchers suggest that adding tactile or other sensors could expand its capabilities.
Overall, the work offers a more streamlined way to control soft robots, with potential applications in areas like search-and-rescue, inspection, and medical tools.
