Direct Experience World-Model Optimization: Learning the World Beyond Action Imitation
This arXiv preprint introduces DEWO, a post-deployment learning method for world-action models that learns from observed visual futures around interaction turning points, not just action imitation. Across five simulated dexterous tasks, DEWO improves average success for multiple world-action model variants, and on two real dexterous-hand platforms it lifts success on initially viable spatial cells from 51.0% to 71.7% over two deployment rounds. The work matters because it shows continued predictive learning from deployment experience can make dexterous policies more reliable.
Can continued world modeling from matched successful and failed continuations around interaction turning points improve dexterous control beyond action imitation alone?
Dexterous manipulation policies degrade after small action errors because high-dimensional contact changes push execution outside the training distribution; existing post-deployment updates target behavior through action imitation or value estimation and do not require better predictions of physical interactions, so compounding errors are not corrected by improved world modeling.
Post-deployment policy improvement typically uses direct action supervision from corrections or value-based reinforcement learning such as RECAP and DSRL; world-action model adaptation has also used successful rollouts or teacher-generated futures, and FACT extends video supervision to failed rollouts but without matched divergent continuations or progress-gated guidance. On FastWAM, SFT, RECAP, and DSRL all fall below the initial 80.7% average success, showing these methods can hurt dexterous policies.
DEWO adds a replay exploration stage that restores states from failed rollouts, samples alternative continuations, and localizes interaction turning points where recoverability changes. Matched successful and failed continuations from those contexts supply outcome-conditioned visual prediction with success or failure labels, while only successful continuations provide action targets; a value head estimates progress from video representations and gates classifier-free guidance between base and success-conditioned action predictions at inference. On real robots, matching uses human-guided physical scene reset rather than exact state restoration, and the method is instantiated with FastWAM, FACT, and a Joint mixture-of-transformers built from pretrained Wan2.2 VideoDiT and ActionDiT components.
In simulation across five DexJoCo tasks, DEWO raises FastWAM from 80.7% to 82.3% average success and FACT from 64.5% to 81.2%, while SFT reaches 76.1% for FastWAM and 70.1% for FACT. On π0.5, SFT drops from 69.5% to 63.9%, RECAP reaches 69.6%, and DSRL reaches 64.3%. In formulation ablations, DEWO improves Joint from 81.9% to 86.7%, IDM from 82.1% to 84.0%, and FastWAM from 80.7% to 82.3%; DEWO-S constructed from pretrained components achieves 89.9%. On Water Plant and Fold Glasses, adding failed visual futures to successful action supervision increases success from 86.7% to 90.7% and from 72.0% to 77.3%, and reduces held-out video loss by 32.7% and 17.8% relative to action-only training; critical experience outperforms full trajectories at matched budgets with 90.7% vs 89.3% and 77.3% vs 74.0%; value-gated guidance beats no guidance and continuous guidance with 90.7% vs 86.7%/78.0% and 77.3% vs 73.3%/52.0%. Real-world evaluation on Wuji and Sharpa hands uses 2,160 total trials across eight object-hand pairs and three rounds. Pooled full-grid success rises from 20.6% (148/720) to 28.9% (208/720); on 29 cells with at least one initial success, success rises from 51.0% to 71.7%; off-center cells improve from 40.0% to 65.2%; initially zero cells remain zero; Wuji rises from 18.9% to 26.7%, and Sharpa from 22.2% to 31.1%.
The paper reports that DEWO primarily makes existing spatial generalization more reliable; all 43 initially zero cells in real-world tests remained zero, so it does not expand reach to entirely new positions. Real-world evaluation is limited to four grasp-and-place objects, two hands, ten trials per cell, and human-guided scene resets; simulation covers five DexJoCo tasks. Code release and compute requirements are not reported. The method assumes the ability to restore or reset task-relevant physical configurations and a predefined success label.
Manufacturers of dexterous robot hands and general-purpose manipulation platforms could use this post-deployment adaptation loop to improve policies from field data without requiring new action labels, provided they have world-action model architectures, reliable resets or scene restoration, and per-task success detection. In 1–3 years, robot fleets with vision-based task progress and safe failure recovery could benefit; immediate use is more likely for research prototypes or companies already using world-action models, while deployment in settings where resets are costly or failures unsafe would require further validation.
The full text is not republished here because the paper's license does not allow it. Read the original on arXiv.