Ropedia W0: Scaling Loco-Manipulation World Models across Embodiments with Human Experience
Ropedia W0, a video world model trained on about 10,000 hours of first-person human video and 3,000 hours of robot video, predicts what a walking, manipulating human or robot will see next, and cut hand-position error on unseen human clips to 32.7 pixels from 46.9 for NVIDIA's Cosmos3-Nano. The preprint from NTU, Ropedia and partners argues that cheap human video can stand in for scarce robot data when training simulators for humanoids.
Can a single world model predict future first-person video for both locomotion and manipulation across humans, gripper robots and dexterous humanoids, and does more human video make it better on robots?
World models let robots imagine the outcome of actions before taking them, but most handle a fixed arm with a gripper. Walking robots change viewpoint as they move, and every robot has a different action space, so robot data cannot easily be pooled and remains scarce.
Models such as NVIDIA's Cosmos3, DreamDojo, OSCAR and RynnWorld-Teleop condition video generation on robot actions or text, usually for one embodiment or a narrow set, and many focus on fixed-base manipulation.
Ropedia W0 replaces raw joint commands with a shared visual control signal: camera poses describe locomotion, and skeletons projected into the image describe the hands or grippers, built for robots from their kinematics plus manual keypoint correction. The team assembled Xp-Data from 14 human and 13 robot public datasets plus in-house recordings, trained 5B and 14B-active models, released a humanoid teleoperation benchmark called Xp-Teleop, and distilled the model into a few-step generator for real-time use.
On 200 held-out human first-person clips with no fine-tuning, the 5B model reached 18.90 dB PSNR and 0.2444 LPIPS against 18.66 dB and 0.2753 for Cosmos3-Nano (16B) and 18.36 dB for Cosmos3-Super (64B); hand-position error fell to 32.7 pixels from 46.9, about 30% lower by MEKAWIRE's arithmetic. The 14B-active model cut camera trajectory error to 1.542 cm from 1.931 cm. Cosmos3-Nano still had a better full-length FVD (178.9 against 189.9). The authors report state-of-the-art results on five cross-embodiment benchmarks (DreamDojo, Psi0, omega-HOME, Xp-Teleop and AgiBot), steady gains as human pre-training data grows, and a distilled version that runs real-time, minute-long rollouts for live VR interaction.
This is a preprint. The control interface captures camera and hand or gripper motion but not contact forces or object physics, so the model predicts what things look like, not how they behave under force. Long rollouts beyond about a minute need better memory. Scaling was tested only up to a 14B-active model with a fixed data mix. Video-quality metrics such as PSNR measure appearance; the paper's policy-evaluation and planning demonstrations are early, and a world model that looks right can still mislead a controller on contact-heavy tasks.
Humanoid makers that collect teleoperation data, and simulation vendors such as NVIDIA with Cosmos, are the immediate audience: the result supports buying or recording human first-person video as a cheaper substitute for robot hours. Expect such models in policy testing and data generation pipelines within one to two years, well before they replace physics simulators for contact-rich training.
The full text is not republished here because the paper's license does not allow it. Read the original on arXiv.