NVIDIA Releases Flora to Turn Robot Videos Into Training Data

NVIDIA has released Flora, a suite of models that turns robot and human video into labels for training and evaluating robotics AI. The system identifies individual actions, describes them in language and scores how well a task was performed, addressing the work required to turn large video collections into useful learning data.

NVIDIA says Flora already supports its GR00T training pipeline and has processed 200,000 hours of video per day using 512 GB300 GPUs. That figure describes annotation throughput, rather than newly collected robot experience or hours of robot training.

Key Takeaways

Show all

• Flora combines action segmentation, captioning and reward modeling to label human and robot video.
• NVIDIA reports processing 200,000 video hours daily on 512 GB300 GPUs, with much lower annotation costs than its frontier-model comparison.
• Three model releases are available on Hugging Face; the Captioner and Rewarder cards specify noncommercial research and development use.

The GEAR research team introduced the release on October 20. In his announcement, NVIDIA robotics researcher Jim Fan described Flora as a way to scale the data preparation behind robot learning. The project page includes the research paper, benchmark results and examples.

Turning footage into training signals

Collecting a demonstration is only part of preparing robot data. Researchers also need to know where one operation ends and another begins, what was being attempted and whether the execution was useful.

Flora divides that work among three models. Flora-TempSeg splits a recording into individual action intervals. Flora-Captioner describes the intended operation in each interval, using surrounding footage for context. Flora-Rewarder assesses execution, producing scores for an entire attempt or shorter portions of it.

As researcher Yunze Man explains in the launch thread, the pipeline handles footage of human hands, robot grippers and dexterous hands. Timestamped descriptions help researchers filter and balance datasets, while reward annotations support evaluation and reinforcement-learning post-training.

The distinction between intention and outcome matters. A clip of an unsuccessful grasp can still be labeled with the action the robot was attempting. Separately scoring the execution allows a training pipeline to distinguish a useful demonstration from a mistake without treating the mistake as the goal.

The Rewarder model card describes overall success scores from one to five, alongside finer-grained assessments that distinguish normal actions, suboptimal actions and failures. It also supports scoring against a sequence of task criteria.

A cost claim with a specific scope

NVIDIA reports annotation quality competitive with GPT-6 Astra at roughly one-125th of its cost. This is the team’s comparison for robotics annotation workloads, rather than a claim that Flora matches a general-purpose model across all tasks.

The published results are mixed across individual metrics. Flora scores higher on the project’s human-video captioning joint F1 measure and dense-reward macro F1, while Astra leads on the robot-video captioning joint F1 measure. The results support a specialized tool with competitive performance, rather than a universal benchmark win.

For robot developers, the potential benefit is making repeated labeling and evaluation more affordable as datasets grow. The reported daily throughput also relies on a substantial GPU deployment; it should not be read as performance available from a single workstation.

What researchers can download

NVIDIA’s Hugging Face collection contains TempSeg, Captioner and Rewarder. The Captioner and Rewarder cards identify them as derivatives of Qwen3.5-27B and describe Flora as a data engine used for GR00T-2 training.

Although the launch uses “open-sourcing” language, those cards specify NVIDIA’s OneWay Noncommercial License and restrict the intended use to research and development. They explicitly exclude supported commercial or production use.

Flora produces annotations rather than motor commands. Its contribution is to the learning pipeline: helping researchers organize what robots have seen, judge what they have done and select better material for the next round of training.