GRASP: Global-Local Representation Alignment and Spatial-Physical Grounding for Bimanual Robotic Manipulation
Adding a physics-based training loss and two learned visual queries to a pi0.5 robot policy raised average success on 50 simulated two-arm tasks from 64.1% to 79.2% and lifted a real watermelon-juice task from 56.7% to 83.3%, according to a preprint led by Chinese robot maker COWARobot. The extra training signals are removed at run time, so the deployed robot runs an ordinary joint-space policy.
Can a vision-language-action policy that outputs joint angles directly be trained so that its loss reflects what actually matters on the robot, the position of the gripper tips and of the two arms relative to each other, and so that it looks at the hand and the object rather than the whole scene, without adding any cost when it is deployed?
Policies that output joint angles avoid an inverse-kinematics solver at run time, but their training loss treats every joint error as equal. A small error at the shoulder moves the gripper much further than the same error at the wrist, so a low joint-space loss can still mean a missed grasp. Visual attention is also learned only indirectly from the action loss, so the policy may not focus on where the hand meets the object.
Most VLAs either predict gripper poses and rely on an inverse-kinematics solver, which can produce unreachable poses and jumps near singular configurations, or regress joints with a plain per-dimension loss. Earlier fixes added kinematic penalties to the final actions or added query tokens distilled from visual models, but did not separate scene-level meaning from the local contact region.
GRASP starts from the pretrained pi0.5 model and adds two training-only priors. First, at every noise level of the flow-matching process, the model's implied clean action is passed through differentiable forward kinematics and penalised on gripper tip (TCP) pose and on the pose of one arm relative to the other. Second, a global query group learns to predict future-frame features of a frozen DINOv3 vision model, and a local interaction query is steered by attention-level distillation toward hand-object boxes from Grounding DINO. Both teacher models and the kinematics branch are dropped at inference.
On the RoboTwin 2.0 benchmark (50 tasks, Aloha AgileX arms, 100 episodes per task and condition), the full model reached 79.22% in clean scenes against 64.13% for the fine-tuned pi0.5 baseline (+15.1 points) and 52.84% in randomized scenes against 36.15% (+16.7). The query-only variant did best under randomization at 65.15% (+29.0) but only 68.30% in clean scenes. Mean gripper-tip deviation from the expert path fell from 5.3 mm to 4.6 mm and the maximum from 7.7 mm to 5.7 mm. In clean scenes the two priors are super-additive: geometry alone adds 3.61 points and queries alone 4.17, while together they add 15.09, 7.3 points more than the sum. On a real dual-arm PiPER-X robot (about 150 demonstrations per task, 5 seeds, 30 trials each), average success rose 14.2 points across four long tasks; watermelon juice went from 56.7% to 83.3%. The paper's official-protocol references are RDT 34.5% and pi0 46.4% in clean scenes.
This is a preprint. The full model trails the query-only variant by 12.3 points under visual randomization, so the authors recommend different variants for different conditions, and the balance between losses is set by fixed weights. Only one simulation benchmark and one real platform were tested, per-task real-robot numbers other than watermelon juice appear only in a chart, and the local prior depends on an offline detector. The baseline was fine-tuned per task with 50 demonstrations, which says little about multi-task or cross-robot generalisation.
The method costs nothing at deployment, which makes it attractive to companies fine-tuning pi0.5 or similar open VLAs on their own dual-arm cells: kitchen, retail and light assembly robots where millimetre gripper error decides success. It matters to Physical Intelligence's ecosystem, to arm makers such as AgileX, whose PiPER arms were used in the tests, and to COWARobot itself, which funded the work. Expect such training-time priors to appear in commercial fine-tuning toolkits within a year.
The full text is not republished here because the paper's license does not allow it. Read the original on arXiv.