world models for dexterous manipulationExpert
Perceive, Predict, and Plan: a Unified World Action Model for Egocentric Dexterous Manipulation
This preprint builds one video model, P3-WAM, that can read 3D hand motion out of a first-person video, predict what a scene will look like after a given hand movement, and plan both the movement and the resulting video from a text instruction. Training the three skills together made each one better, and it beat specialised models on four public datasets, though robot use so far is two open-loop demonstrations.