ORD-WM: Learning Object-Centric Rigid Dynamics for 3D World Modeling in Robotic Manipulation

Can a robot world model keep objects' shapes intact over many predicted steps, and stay fast enough for real-time planning, if it predicts only how each rigid object moves instead of regenerating the whole scene?

World models let a robot imagine the result of an action before taking it. Video-based and dense 3D point models regenerate the scene at every step, so over long rollouts objects warp, drift or move inconsistently, which makes them unreliable for planning pushes and placements that take dozens of steps.

Video world models predict future frames; point-based models such as PointWorld predict per-point scene flow. Both must learn implicitly that a mug stays a mug, and errors accumulate as each predicted step is fed back in. V-JEPA 2 style planners work in learned latent spaces but also do not impose rigid-body structure.

ORD-WM encodes each object's point cloud with a rotation-equivariant vector-neuron autoencoder trained on signed distance fields, giving a latent code made of 3D vectors. A dynamics network, conditioned on the robot action, predicts each object's rigid motion increment (rotation and translation), which is applied analytically to the latent code. Shape is therefore preserved by construction and only pose changes. A model-predictive controller samples candidate 8-step action sequences, rolls them out in latent space and executes the best first action.

On held-out LIBERO trajectories, error for moving points was 1.58 mm after one step against 10.04 mm for PointWorld fine-tuned on LIBERO (about 6.4 times lower, MEKAWIRE), 7.71 against 79.62 mm after ten steps and 37.60 against 225.68 mm over the full trajectory (83.3% lower; 88.3% lower than the released PointWorld at 321.11 mm, MEKAWIRE). Shape drift measured by Chamfer distance was 0.00 against 3.58 and 19.87 mm. In real pushing with a Franka Research 3 and three RealSense D435 cameras, success (final error under 2 cm) rose from 10% with 64 candidate plans to 70% with 256 and 90% with 1,024, mean final error fell from 3.72 cm to 0.97 cm and executed steps from 367 to 85 (76.8% fewer, MEKAWIRE). Planning per step took 55.4, 115.7 and 269.9 ms.

This is a preprint. Objects are either free or fixed rigid bodies, so hinged, articulated and deformable objects are out of scope. Mass, friction and contact are learned implicitly, and the rigid updates do not prevent objects from passing through each other. The real-robot evidence is pushing with success rates but no trial counts in the table, plus qualitative pick-and-place. Training data is LIBERO plus 1,000 pushing trajectories, and the authors say they have not tested larger or more diverse datasets.