World Action Modeling with Progressive Visual Planning
ProWAM is a preprint describing a world action model that predicts sparse visual sub-goals indexed by normalized progress and uses them to condition action chunks, rather than generating dense video. It sets new simulated state-of-the-art on LIBERO-Plus (85.8%) and randomized RoboTwin (75.7%) and reaches 70.0% zero-shot real-world success vs 55.0% for DreamZero. This matters because it offers a more efficient and robust way to transfer video-pretrained visual foresight into long-horizon robot control.
Can a world action model improve long-horizon control by predicting an ordered sequence of sparse visual sub-goals indexed by progress, instead of dense future video or a single final goal?
Long-horizon robotic control requires visual foresight, but existing world action models either generate dense video rollouts (expensive and slow), predict no future visual states (lacking guidance), or predict only a final frame (lacking intermediate progress), and modular hierarchical planners suffer cascading errors.
Full-imagination WAMs such as those in Pai et al. and DreamZero condition actions on dense predicted video clips; zero-imagination WAMs omit future latents; ImageWAM conditions on a single endpoint frame. Hierarchical planners use separate goal-generation modules. These either incur high latency/artifacts, lack intermediate guidance, or propagate errors between stages.
ProWAM uses a Wan2.2-TI2V-5B video backbone with a mixture-of-transformers video expert and lightweight diffusion action expert. It appends k sparse visual sub-goal latents at normalized progress values r∈[0,1], modulates them with progress-conditioned adaptive LayerNorm offsets, couples them to actions via asymmetric attention, and pretrains the video branch on action-free videos before joint fine-tuning on robot demonstrations. At inference it caches sub-goal key-value features from a single video pass and reuses them during action denoising.
On LIBERO-Plus zero-shot, ProWAM reaches 85.8% vs 80.5% for ABot-M0 and 83.1% for Image-WAM. On RoboTwin with scene randomization, it reaches 75.7%, beating the strongest prior WAM (MV-WAM 55.7%) by +20.0 points (+35.9% relative). On RoboCasa365, overall success is 48.1% and Composite-Unseen 18.2%, ranking 4th among listed public methods. In zero-shot real-world Franka experiments across 4 tasks, ProWAM achieves 70.0% vs 55.0% for DreamZero and 50.0% for π0.5. The KV-cache inference uses 18.7 TFLOPs and 313 ms per 16-action chunk on H100, versus 115.5 TFLOPs and 581 ms for naive joint denoising, an 84% compute and 46% latency reduction. In ablations, k=4 sub-goals yields 75.7% OOD vs 73.8% for k=1, and a Near progress schedule r∈[0.1,0.5] yields 75.7% vs 68.4% for Far.
The paper reports no separate limitations section. It shows failure cases where sub-goal geometry is blurred/ambiguous, leading to premature gripper opening and dropped objects, indicating sensitivity to errors in task-relevant geometry. Single-pass sub-goal generation can produce texture artifacts. Real-world evaluation covers only 4 tasks on a single Franka robot, and the number of trials is not reported in the paper. No code release is mentioned.
Robot foundation-model developers and generalist manipulation vendors could use this WAM approach to improve zero-shot and out-of-distribution policies from video pretraining. Product impact is likely 1–3 years for controlled industrial or service robot pilots, provided larger real-robot validation, on-robot inference optimization, and safety/reliability thresholds are met.
The full text is not republished here because the paper's license does not allow it. Read the original on arXiv.