UniWAM: Unified World-Action Model

How can strong semantic reasoning and generalization from vision-language models be combined with strong physical dynamics understanding from world-action models in a single robot policy?

Vision-language-action models using action-only supervision learn limited world dynamics, while world-action models built on video generation have strong spatiotemporal priors but weak out-of-distribution reasoning and semantic generalization. Existing systems therefore fail to robustly handle complex instructions, perturbations, and long-horizon tasks.

Prior work split into VLA models such as RT-2, OpenVLA, π0/π0.5, and GR00T, which adapt pretrained VLMs with tokenized or flow-matching actions, and world-action models such as DreamZero, Cosmos Policy, Motus, and Dyna-2, which predict future visual states on video-generation backbones. VLAs under-use physical dynamics; WAMs inherit video priors but remain brittle under distribution shifts and complex reasoning.

UniWAM uses a Mixture-of-Transformers architecture with three experts: a Qwen3-VL-2B-Instruct physical reasoner, a Wan2.2-TI2V-5B world generator, and an action predictor, connected by joint multimodal attention. It pretrains on about 4,958 hours of robot data, 5,072 hours of human egocentric data, and 8.293 million VQA pairs. Low-level actions are converted to natural-language physical descriptions for the VLM. Human egocentric videos are cleaned and annotated with an automated VLM pipeline called EgoANT. Post-training adds future-frame visual noise augmentation and history-conditioned flow matching, initializing action denoising from previous actions rather than Gaussian noise.

In simulation, UniWAM attains 99.2% average success on standard LIBERO, 92.6% total success on LIBERO-Plus OOD, 75.14% on RoboTwin 2.0 Clean2Clean, and 68.32% on Clean2Random. On Clean2Random this is the highest among listed methods and drops only 6.82 points from clean to randomized, versus 24.70 for π0.5 and 50.46 for Spatial Forcing. In real-world dual-arm tests on AgileX Piper hardware, it averages 67.5% task success and 82.5% instruction-following across four tasks, beating π0.5 by 13.1 and 21.9 points respectively; long-horizon task progress is 5.0 versus 4.8 for π0.5 and 3.2 for Motus. Scaling experiments fit validation loss with L(D)=0.03205-0.00302 ln(D), R²=0.983, and show task completion rising from 0.35 at 250 robot hours to 0.67 at 5,000 robot hours; human data helps after about 1,000 robot hours.

The paper does not state explicit limitations. Evident limits include real-world evaluation on a single dual-arm platform with only four instruction-following tasks and one long-horizon task, relatively few real-world demos (200 per task), no reported total real-world trial counts, and substantial training compute (80K steps on 8 H100 GPUs for the real-world policy). Generalization to very different robot morphologies and safety-critical deployment remains untested.