Rhoda Blindfolds Its Robot—and Lets Touch Guide the Search

Rhoda AI has demonstrated a robot searching for textured balls with its head and wrist cameras blocked, using touch to guide its movements. The company combines a new tactile sensor with its Direct Video-Action model, extending a system built around predicting visual observations to predict future tactile signals.

Key Takeaways

Show all

• Rhoda’s demonstration uses touch to search for, identify and retrieve textured balls without camera feedback.
• The model predicts future tactile signals, which an inverse dynamics model converts into robot actions.
• The video shows mistakes and recovery, while the technical post does not report a task success rate or trial count.

Finding objects without seeing them

In its October 13 technical post, Rhoda describes training on one smooth ball and one textured ball. It then tests the same model with seven balls, including two unfamiliar textured examples with different diameters, firmness and surface patterns. The company reports that the robot retrieves all three textured balls.

The accompanying video makes the challenge visible: the robot must search the box, judge an object through contact and adjust its grip. It also shows wrong-ball selections and a return to re-scan the box. Sections are labelled as running at normal speed or accelerated to 6× and 12×, rather than presenting the entire sequence as real-time operation.

Predicting what contact should feel like

Rhoda’s narration explains that its model generates future tactile signals and an inverse dynamics model translates those predictions into actions. Fresh measurements then inform subsequent predictions as the robot moves.

That extends the approach behind Rhoda’s Direct Video-Action model. The useful question is whether predicting contact can help a robot continue manipulating an object when visual feedback becomes inadequate—for example, when its own gripper obscures the contact point.

The sensor demonstrations include detecting a feather’s touch after heavy loading and sensing vibrations from textured surfaces. Rhoda says the optical and magnetic sensors it tested did not show a distinguishable response to the feather; this is a comparison with those tested devices, rather than evidence of superiority over every competing sensor.

The blog adds measurements spanning a few millinewtons to more than 40 newtons in a dumbbell test, plus vibration sampling at approximately 500 Hz. Rhoda also integrates the sensors into a handheld Universal Manipulation Interface, recording contact forces during human demonstrations for policy learning.

The demonstration links sensing to action, but it leaves reliability unquantified. Without repeated-trial results, the footage establishes an example of touch-guided manipulation rather than a measured level of performance across everyday tasks.

Other approaches to robotic touch

Rhoda’s work sits alongside other efforts to integrate touch into robot hands. A recent Tesla patent application describes flexible tactile sensors shaped to fit fingers, palms and other curved surfaces, with independently readable contact-sensing regions. The filing focuses on manufacturing methods, including printing conductive patterns and molding layered sensors into three-dimensional shapes.

That application offers a look at sensor design and production, while Rhoda demonstrates sensing linked to robot actions.