Robbyant Open-Sources LingBot-Touch, One Tactile Model for Three Sensor Types

Robbyant, the embodied AI company within Ant Group, has open-sourced LingBot-Touch, a tactile model that reads signals from magnetic, visuotactile and piezoresistive touch sensors and turns them into one shared set of tactile tokens. Code and weights for two model sizes are on GitHub and Hugging Face under the Apache 2.0 license, alongside a technical report on arXiv.

In the paper's real-robot tests, adding the smaller model to four existing robot policies raised their average success rates across 26 tasks by 13.9 to 24.8 percentage points. Feeding the same policies raw sensor data instead produced at most about half that gain.

Why touch data rarely carries over

Robot hands use several kinds of touch sensor, and each produces a different raw signal for the same contact. Magnetic sensors measure how a magnetic field shifts as the fingertip deforms. Visuotactile sensors, such as those on the Sharpa Wave, film a soft skin from the inside. Piezoresistive sensors register a change in electrical resistance under pressure. Taxel counts and layouts differ too.

A model trained on one of those signals usually has to be retrained when the hand or sensor changes. Robbyant's paper argues that this keeps tactile datasets small and siloed, because no single sensor has thousands of hours of recordings behind it.

LingBot-Touch handles the mismatch in two steps. Each sensor's taxel positions are first mapped into a shared "canonical fingertip space", so the model sees positions on a finger rather than indices in a sensor-specific array. Separate tokenizers then handle each sensing principle before a shared Haptic Transformer learns from all of them.

Training uses a joint-embedding predictive architecture, or JEPA: the model hides part of the tactile signal and predicts its representation in latent space instead of reconstructing the raw values. The corpus has no contact labels. Its 4,983 hours (464,431 episodes, about 1.33 billion frames) come from piezoresistive and magnetic gloves worn by human operators doing manipulation tasks, plus teleoperated dual-arm robots fitted with Sharpa Wave hands. Sharpa, which makes the Wave, said on X that the hand contributed to the research as a “dexterous robotic hands partner.”

The release comes in two sizes. LingBot-Touch-Tiny has 5.46 million parameters and LingBot-Touch-Small has 21.76 million.

The real-robot tests

The team ran 26 tasks, 20 on a dual-arm Sharpa Wave setup and six on a dual-arm XHand1 setup. Four policies were tested: Diffusion Policy as an imitation-learning baseline, and π0.5, LingBot-VLA and T-Rex as vision-language-action models.

Each policy was post-trained three ways on the same demonstrations: without touch, with the raw tactile signal added, and with tokens from a frozen LingBot-Touch-Tiny encoder. That meant 200 demonstrations per task, 500 for tasks needing both hands, and 20 consecutive trials per task.

Averaged over all 26 tasks, the LingBot-Touch configuration raised success rates as follows:

• Diffusion Policy: 21.6% to 35.6%
• π0.5: 48.1% to 69.9%
• LingBot-VLA: 48.3% to 73.1%
• T-Rex: 45.4% to 68.6%

Raw tactile input added 7.0, 6.6, 8.2 and 11.2 points to the same four policies, and lowered success in 22 of the 104 task–policy pairs. In the worst cases, LingBot-VLA fell from 28.8% to 5.0% on mahjong sorting, and π0.5 fell from 39.0% to 15.0% on zipper closing. The authors' explanation is that 200 demonstrations are not enough for a policy to learn to read a sparse, high-dimensional sensor from scratch.

The biggest per-task gains came at steps that a camera can't settle. With T-Rex on the Sharpa Wave, cap twisting went from 56.25% without touch to 62.5% with raw signals and 90% with LingBot-Touch. That case turns on whether the cap has broken loose. LingBot-VLA's zipper-closing rate rose from 11% to 55%, and T-Rex's SD-card ejection went from 32% to 92%.

The results are not uniform. LingBot-Touch was the best of the three configurations in 89 of the 104 pairs. It scored below the raw-signal setup in nine pairs, five of them with Diffusion Policy, and below the policy without touch in five.

A sensor the model never saw

XHand1 is a 12-degree-of-freedom hand from Robotera. The pretraining data included magnetic sensing, but not XHand1's specific magnetic sensor model. Its taxel layout was mapped into the canonical fingertip space and the encoder was used as-is.

On XHand1's six tasks, raw tactile input changed mean success by +4.6, −1.7, 0.0 and +2.0 points across Diffusion Policy, π0.5, LingBot-VLA and T-Rex. LingBot-Touch tokens computed from the same signal added 16.3, 24.5, 17.8 and 13.8 points.

The paper also compares the frozen encoder with FTP-1, an earlier tactile encoder built for mixed sensor types, which was given a sensor-specific adapter for each hand. On five probe tasks that read contact properties from the encoder's output, LingBot-Touch-Small outperformed it on both hands. The error in locating the center of a contact was 0.12 mm against FTP-1's 2.19 mm on the Sharpa Wave, and 0.16 mm against 5.89 mm on XHand1.

In a separate five-task test inside LingBot-VLA, swapping in FTP-1's encoder lifted average success from 55.9% to 67.5%. LingBot-Touch took it to 82.2%.

Scaling and stated limits

The team pretrained both sizes on five data budgets from 1,000 to 5,000 hours. Small improved or held steady on every probe task at each step. Tiny tracked the same curve up to about 4,000 hours and then flattened, which the authors read as model capacity, not data, becoming the limit. Real-robot results with LingBot-VLA also improved as pretraining hours grew.

In simulation, reinforcement learning policies using LingBot-Touch features succeeded on 69% of attempts across twelve dexterous tasks. Policies with features from a task-specific CNN reached 46%, and those without touch reached 21%.

The paper names three limitations. Only two hands were tested on real robots. Capacitive sensors are missing from pretraining because the recordings available to the team did not meet its quality bar. And the canonical geometry centers on fingertips, so contact on the palm and along the fingers is underrepresented.

Part of a larger robot brain

Robbyant describes LingBot-Touch as one capability of the general-purpose robot brain it is building, and says the model is now integrated into LingBot-VLA. The tests above deliberately included three policies Robbyant did not build, and the release is pitched at hand makers and robotics teams that want to keep their own policies while changing sensors.

The company's previous releases covered depth perception, vision-language-action and world models. LingBot-VLA 2.0 is also the base model for Robbyant's Embodied AI Challenge, whose 15-team real-robot final runs in Shanghai from November 14 to 16.

Touch has been getting more attention across the industry. In October, Rhoda showed a blindfolded robot using touch to search for objects, and NEURA added touch to its robot AI. A Tesla patent application described flexible touch sensors for robot hands.