Relational Spatial Alignment for Vision-Language-Action Models
A London and Oxford team taught an OpenVLA-OFT robot policy 3D spatial sense without damaging what it already knew, lifting average success on the perturbed LIBERO-Plus benchmark to 75.0% from 69.6% for the plain model and 70.3% for the leading rival method, according to a preprint. The 3D module is used only in training, so the robot runs it at no extra inference cost.
Can a vision-language-action model gain 3D spatial awareness from a 3D foundation model without the representation collapse that direct feature copying causes?
The usual shortcut is to force a VLA's internal features to match those of a 3D model such as VGGT point by point. The authors show this acts like a low-rank regulariser: it squeezes the VLA's features into fewer dimensions, so the model gets better at geometry but loses robustness elsewhere, for example when the camera moves.
Explicit methods feed depth maps or point clouds into the VLA, which depends on clean sensors and pre-scanned scenes. Implicit methods such as Spatial Forcing align VLA hidden states directly with 3D model features; this preprint measures a clear drop in effective rank with that approach.
RIGID aligns relationships rather than features. It distils the pair-wise structure of VGGT's attention (which image patches relate to which, within and across camera views) into the VLA using a rotation-invariant, optimal-transport (Sinkhorn) loss. The VLA keeps its own feature space; the 3D module is dropped at inference.
On LIBERO-Plus, averaged over four task suites, RIGID scores 75.0% versus 70.3% for Spatial Forcing, 69.6% for OpenVLA-OFT and 53.6% for pi0. Under camera perturbations it reaches 62.1% against 56.5% for Spatial Forcing and 56.4% for OpenVLA-OFT. On standard LIBERO it ties Spatial Forcing at 98.5% average. Training time rises 8.0% over an attention-KL baseline; inference matches OpenVLA. On a Franka Panda arm in a toy kitchen, with 10 trials on each of six tasks, RIGID had the best average, though the paper reports the real-robot numbers only in a chart.
This is a preprint. Real-robot evidence is small (60 trials per model) and given only graphically. Gains over Spatial Forcing are uneven: on some perturbations, such as language changes, Spatial Forcing scores higher. The authors note they cannot attribute the gains to rank preservation alone. Everything is built on OpenVLA-OFT; transfer to other VLA families is not tested.
Any company fine-tuning open VLAs for factory or warehouse arms faces camera placement that differs from the training data. A training-only fix that adds 3D robustness without slowing the robot is directly usable by teams building on OpenVLA-style models, and the finding that point-wise 3D distillation hurts robustness is a warning for anyone already using it. Time scale: months, since the method is a fine-tuning recipe.
The full text is not republished here because the paper's license does not allow it. Read the original on arXiv.