GPT-6 Is Beating Specialist Models in Robotics Tests

Key Takeaways

Show all

• GPT-6 Astra scored 70.5% on a 200-episode MolmoSpaces evaluation, ahead of the open-source robotics baselines in the published comparison. The result covers a subset, not the full benchmark.
• A separate report records 49 successes in 50 trials on ten selected RoboLab simulation tasks, but much lower success on a more demanding two-arm task set.
• On those RoboDojo tasks, combining Astra with a learned robotics policy raised success from 26% to 48%, suggesting complementary strengths.
• Video-guided robot-arm demonstrations and an iterated kitchen simulation broaden the evidence, while leaving reliability, latency and real-world deployment unresolved.

GPT-6 Astra's robotics results are expanding beyond individual tabletop demonstrations. A new MolmoSpaces evaluation puts the model ahead of the open-source robotics policies in its comparison, while a separate study reports near-perfect performance on selected RoboLab tasks. Alongside them, developers are showing robot arms following human video demonstrations and a simulated robot attempting a multi-step kitchen job.

The strongest new evidence comes with a revealing split: Astra can perform remarkably well when a task rewards identifying objects and reasoning about where to move them. On more demanding manipulation tasks, a dedicated robotics policy can still make it substantially better.

That adds a more concrete basis to the debate sparked by Astra's real-world painting demonstration. General-purpose reasoning is becoming a credible ingredient in robot control. The emerging results also show why the surrounding control software, task selection and evaluation conditions matter.

MolmoSpaces: 70.5% on a 200-episode subset

In a September 14 results thread, MolmoSpaces researcher Omar Rayyan reported that GPT-Astra outperformed every open-source vision-language-action (VLA) and world-action-model (WAM) baseline in his comparison on a subset of the zero-shot MolmoSpaces-v1 benchmark.

His chart puts Astra at 70.5%, compared with 53.0% for Cosmos3, 38.6% for MolmoAct-2, 34.5% for π0.5-OSS and 23.2% for π0-OSS. Among the other general-purpose model APIs, Fable 5.1 scored 37.5%, Gemini 3.8 Flash 30.5%, Opus 5 30.0% and Gemini ER 2 26.0%.

Rayyan described the API-model setup as a closed feedback loop using the harness from RoboCurve's results: models receive external and robot-view RGB images from the DROID setup, together with robot state, and return Cartesian poses. Conversation history is cleared after each of the 200 episodes. He also published execution traces for inspection.

The scope is important. MolmoSpaces is a simulation benchmarking ecosystem, and Rayyan explicitly said API costs prevented him from running the full benchmark for inclusion on its leaderboard. The chart therefore establishes a lead in this reported comparison, not a full-benchmark ranking or a measured success rate in arbitrary physical homes. He also cautioned that reasoning-token counts across providers may not be directly comparable.

Sharing the result, researcher Mahi Shafiullah asked whether robotics was experiencing its own GPT moment. It is an understandable question, especially after the earlier RoboCurve hardware trials: Astra completed 19 of 20 block-in-bowl runs there, but only two of 20 precision puzzle insertions. The new simulation result broadens the evidence without erasing that physical limitation.

RoboLab's 98% result has a revealing companion

A second set of results attracted attention when Max Zhaoshuo Li highlighted Astra's near-perfect RoboLab score. The linked technical report, GPT 6 Astra as an Embodied Policy, lists Jiayi Su, Yixin Zheng, Mi Yan, Li Yi, Zhizheng Zhang and He Wang as authors.

On ten selected RoboLab tasks using a simulated single-arm Franka, the report records 49 successful trials out of 50 for direct Astra control, or 98%. A hybrid of Astra and π0.5 reached 46/50; π0.5 and Cosmos3-Nano-Policy each reached 18/50, and DreamZero reached 17/50. Each task received five trials per policy.

Those tasks largely involve picking and placing with semantic requirements: selecting objects, distinguishing sizes and colors, understanding spatial relationships, stacking blocks in order and reorienting a mug. The authors note that the specialist baselines used DROID-trained weights for zero-shot transfer to these tasks.

The same report's RoboDojo simulation experiment tells a different story. Across ten selected two-arm tasks, direct Astra control succeeded in 13 of 50 trials (26%). Letting π0.5 propose actions and Astra accept or correct them raised that to 24 of 50 (48%). The hybrid followed π0.5 for 85.6% of executed control steps and used Astra corrections for the remaining 14.4%.

The RoboDojo selection emphasized tasks with lower published π0.5 success rates, and included packing, construction, sequence imitation and clothes folding. It was not a representative sweep of the entire benchmark. Public baseline comparisons were recomputed from published task aggregates, rather than rerun with the experiment's seeds.

Together, the two evaluations suggest that the value of a learned action policy depends on how well its motor skills fit the task. They do not establish that general reasoning has made specialist robotics models obsolete.

There is also a timing caveat. The report explicitly warns that simulation lets Astra deliberate while the world waits. Its reported physical durations exclude model-response latency, and the authors identify latency as an unresolved barrier to real-time deployment.

From a human video to robot-arm actions

The demonstrations extend beyond simulation. In a September 9 post, Wenli Xiao reported giving Astra a recording of a human performing a novel task through the Codex app, then asking it to drive a robot arm in the same way. Xiao described success on the first attempt and characterized the behavior as physical in-context learning: using the demonstration supplied in the current interaction to guide execution.

A technical clarification from Tonghe Zhang, linked by Xiao, explains the division of labor. The model calls motion-planning tools and outputs target end-effector positions; the surrounding software handles inverse kinematics, converting those targets into joint motion.

That makes the demonstration significant without implying that Astra directly handles every layer of motor control. It is a reported example of adapting to a video instruction through a robot interface, rather than a repeated-trial measure of how reliably that adaptation works across unfamiliar tasks.

The kitchen demo shows the engineering between attempts

Andre Infante's posts offer a particularly useful view of that interface work. In a September 14 kitchen simulation experiment, he described an initial failure when GPT-6 encountered an obstacle and stopped. After he instructed it to persist, a second attempt failed because unstable simulated contact let a tray pass through the gripper. A follow-up noted that elevator control had also been omitted from the interface.

On September 15, Infante reported approximate success on a snack-tray task after changing the harness and hardware setup. That third attempt followed one canceled over a system-prompt issue and another stopped after the robot dropped orange juice beyond its reach.

The progression shows a model getting further on a multi-step task, while also documenting the human iteration behind the result. It should be read as an evolving demonstration, rather than evidence of reliable unattended kitchen work.

A stronger robotics case, with the control problem still open

These developments arrive after Sam Altman confirmed that OpenAI will “definitely” build a humanoid robot. They make the company's broader robotics ambition easier to understand: a general model that interprets unfamiliar scenes, follows demonstrations and reconsiders failed actions could contribute useful capabilities across different robot bodies.

The current evidence remains narrower than a deployable humanoid. The benchmark subsets measure selected manipulation tasks, the physical demos lack broad repeated-trial results, and the kitchen experiment required intervention between runs. None establishes whole-body reliability or continuous operation in a changing household.

The most useful next test is whether these gains survive broader task coverage, measured wall-clock response times and repeated physical trials with failure recovery included. For now, Astra's growing collection of results strengthens the case for general-purpose models in robotics—and the hybrid results give specialist motor policies a clear role in making that reasoning work.