Rhoda Tests Whether Scaling Web-Video Pretraining Makes Robots Better at Real Work

Key Takeaways

Show all

• Rhoda is testing the link between video pretraining and useful robot work.
• Its Direct Video-Action system predicts future video, then translates those predictions into movement.
• Video models, sensorized demonstrations and simulation address different parts of the robotics learning problem—and can be combined.

Rhoda AI reports improved industrial manipulation from scaled web-video pretraining. Its September 10 study, shared on X today, tests a central physical-AI premise.

The result: completion with a deadline

Across four model sizes, Rhoda reports 3.7%, 65.0%, 75.3% and 84.7% completion without intervention in under 100 seconds, unpacking bearings and sorting waste. The largest scored 94/111. Larger models also used more compute and video, so this comparison does not isolate parameter count.

A separate fixed-size experiment increased pretraining compute: performance rose from 57.8% to 75.3%, plateauing near the upper budgets with full task data. Differences widened with reduced demonstrations. Better held-out video prediction also correlated with robot performance.

The company’s 200-plus evaluation hours cover one task and setup, with one post-training run per condition. Its deadline is stricter than the customer requires.

How video becomes a robot policy

Rhoda’s approach builds on the Direct Video-Action, or DVA, architecture we covered at its March launch.

The video model predicts what the robot should see next, using its observation history and other inputs. A separate inverse dynamics model works backward from that visual future to the movements needed to produce it. The robot acts, observes the result and repeats the cycle.

Rhoda pretrains its causal video model from scratch on general web video, then adapts it using robot task demonstrations. “Causal” means predictions follow from preceding observations. Its March technical account describes overlapping prediction and execution so the robot can keep moving while the next prediction is computed.

This gives video a direct role in choosing behavior. It also leaves an essential translation problem: predicting a useful future and making a particular machine reach it require different capabilities.

Where the other paths diverge

Our recent UMI feature examined another response to the data problem: designing what people wear or hold while demonstrating skills. Matching the collection device to the robot can make those demonstrations easier to transfer, while recording movement and sensing information that an ordinary video may lack.

These approaches can be complementary. Broad pretraining can supply reusable representations; purpose-built demonstrations can supply precise examples of how to act. The distinction is where a company invests to make the next useful skill cheaper to acquire.

Some of the companies we have covered recently illustrate several combinations:

The table describes emphases, not exclusive categories or a ranking. Physical Intelligence’s use of visual subgoals, for example, shows why “VLA versus world model” can obscure how systems are actually assembled. Likewise, training from scratch says where parameters begin, not what kind of data a model learns from.

For readers tracking this field, the useful comparison is increasingly concrete: what information enters pretraining, what additional demonstrations each task needs, how predictions become controls, and how reliably the resulting robot works. A shared ambition for general-purpose robotics leaves plenty of room for different answers to each question.

Sources and further reading

• Rhoda: Scaling Web-Video Pre-training
• Rhoda: Direct Video-Action technical explanation
• Rhoda’s X post
• Odyssey: Introducing Odyssey-3
• Mimic: mimic-video research
• Generalist: Going Beyond World Models & VLAs
• Genesis AI: GENE-26.5 announcement
• Physical Intelligence: A Steerable Model with Emergent Capabilities
• Skild: Building the General-Purpose Robotic Brain