Serving an Ace: LATENT Framework Teaches Unitree G1 Athletic Tennis Skills

Humanoid athletes are increasingly moving beyond pre-programmed routines toward dynamic, reactive sports. While recent demonstrations like the Unitree Robotics OmniXtreme have showcased "extreme balance" and acrobatics, athletic sports like tennis present a unique set of challenges: sprinting at speeds over 6 m/s, reacting to balls traveling at 15–30 m/s, and striking a ball with millisecond precision.

To bridge this gap, a collaborative team from Tsinghua University, Peking University, and Galbot Inc. has developed LATENT (Learns Athletic humanoid Tennis skills from imperfect human motion data). Deployed on the 29-DoF Unitree G1, the system allows the robot to sustain multi-shot rallies with human players, adapting its posture and swing to diverse incoming ball trajectories.

Learning from Imperfect Data

A major hurdle in humanoid sports is the scarcity of high-quality training data. Collecting precise motion capture of a full-court tennis match is technically difficult and expensive. The LATENT framework bypasses this by learning from "imperfect" motion fragments—primitive skills like forehand strokes, backhand strokes, and crossover steps—rather than complete, realistic match sequences.

By focusing on these primitives, the researchers reduced the required motion capture area by over 17x compared to a full-size court. This approach leverages the "priors" of human movement while acknowledging that the data is both incomplete (missing match context) and imprecise (lacking fine wrist control).

The Latent Action Barrier

To translate these fragments into a cohesive policy, the researchers built a hierarchical controller with a latent action space. This mirrors the architecture of other "System 1" controllers like NVIDIA’s SONIC, which provides a fast, reactive layer for whole-body skills.

However, without constraints, a high-level policy can "exploit" the latent space, leading to jittery or unnatural movements. To solve this, LATENT introduces a Latent Action Barrier (LAB).

• The LAB constrains the robot's exploration during reinforcement learning (RL).
• It uses a Mahalanobis distance-based barrier to keep actions within a distribution that resembles natural human motion.
• This ensures the robot maintains "motion naturalness" even when sprinting to intercept a high-speed ball.

Residual Precision and Sim-to-Real

Because human-to-humanoid retargeting often results in inaccurate wrist positioning, the LATENT policy employs a hybrid control strategy. The high-level planner simultaneously outputs latent body actions and direct "corrective" commands for the right wrist. This residual learning approach shares DNA with Amazon’s ResMimic, which adds a corrective policy on top of general motion tracking to handle precise object manipulation.

The transfer from simulation to the real world was achieved through aggressive domain randomization.

• Dynamics Randomization: The team randomized robot body mass, joint friction, and ball physics, such as air-drag and restitution.
• Observation Noise: To handle real-world latency, the system uses a four-frame sliding window to estimate ball velocity accurately.

This robust pipeline allowed for zero-shot transfer, a feat also recently highlighted in Amazon’s PHP parkour framework.

Real-World Performance

In real-world deployment, the $16,000 Unitree G1 demonstrated striking versatility. The robot successfully returned balls at peak velocities exceeding 15 m/s, using diverse stroke types across different court regions.

Experimental results showed that the inclusion of the Latent Action Barrier and wrist correction was critical. Without wrist correction, success rates dropped significantly, while removing the LAB led to suboptimal, jittery sequences.

While the current system relies on an external optical motion capture system for global state estimation, the researchers suggest that future iterations could incorporate active vision or multi-agent training to achieve performance comparable to professional human players. For now, LATENT serves as a powerful demonstration of how humanoids can master complex, high-dynamic tasks even when the available training data is far from perfect.