ProWorld: Multi-View World Model with Proprioceptive Feedback for Closed-Loop Policy Evaluation

Abstract

Evaluating robot manipulation policies through repeated physical trials is costly. We study whether an action-conditioned world model can serve as a closed-loop test environment whose policy success rates track those on real robots. Beyond realistic video, the model must return the camera views and proprioceptive state needed for the policy’s next action and preserve failed-action outcomes to avoid overestimating policy capability. We introduce ProWorld, a world model that jointly predicts multi-view video and robot state, then feeds both back to a frozen policy for repeated interaction. ProWorld comprises a shared transformer for joint video–state flow prediction and Shared-Time View Encoding to distinguish cameras while preserving temporal synchronization. To reduce interaction hallucinations that turn missed grasps into successful lifts, we augment training with failed interactions collected in high-fidelity simulation, reducing inflated success rates in the generated environment. ProWorld achieves the highest overall TriWorldBench score among the compared methods. Across three policies and four local manipulation tasks, it attains a Pearson correlation of 0.95 and a Spearman correlation of 0.94 between generated and real success rates. These results support joint visual-proprioceptive feedback and failure-aware training as a basis for world-model-based policy evaluation.

12

1 Introduction

Evaluating robot policies through physical trials is costly, both in terms of robot time and the need for repeated scene resets. With the development of action-conditioned video foundation models (Yang et al., 2024; Zhu et al., 2025; Liao et al., 2025; Gao et al., 2026), recent work has extended these models to policy evaluation (Guo et al., 2025; GigaWorld Team et al., 2026; Jeon et al., 2026; Tseng et al., 2025). These approaches use learned world models as closed-loop evaluation environments: initialized from a real observation, the world model predicts the consequences of actions executed by a fixed policy and generates the observations used for subsequent policy decisions. The goal is to estimate policy success rates through generated closed-loop interactions that faithfully reflect real-robot performance.

However, two requirements that are largely overlooked by existing approaches remain critical for reliable policy evaluation. First, the world model should provide the complete observation space required by the policy, including multiple camera views and proprioceptive states such as joint positions. Second, it must faithfully preserve unsuccessful interactions. For example, if a missed grasp is subsequently rendered as a successful lift, the world model produces an interaction hallucination: it predicts a successful outcome that is not supported by the executed action. Such hallucinations can inflate estimated success rates and obscure genuine policy failures. A reliable evaluator therefore requires both complete visual–proprioceptive feedback and faithful modeling of failure outcomes.

We introduce ProWorld, which jointly predicts multi-view videos and robot state from observation history and an action chunk, and then feeds both back to a frozen policy. A shared transformer learns both outputs through joint flow matching. State queries read the visual stream through asymmetric attention, encouraging agreement between predicted state and visible motion. The policy thus receives predicted execution feedback rather than its preceding command as the next state.

To preserve camera identity and synchronization, we develop Shared-Time View Encoding (STVE). Independently encoded camera streams receive shared time coordinates and explicit view coordinates. This avoids pixel-tiling boundaries and separates camera identity from elapsed time. Compatible with pretrained video backbones, STVE improves visual prediction over pixel tiling and time concatenation methods.

To reduce interaction hallucinations, we augment demonstrations and robot-policy rollouts with failed interactions collected in high-fidelity simulation. Successful demonstrations provide limited evidence of what happens when actions fail. For example, a missed grasp may leave the object in place, while an unintended push may knock it over. Simulated failures supply this missing supervision. In our local evaluation, this training reduces hallucinated successes and preserves failed interactions.

We assess policy-outcome agreement and video prediction quality. Across three policies and four local tasks, ProWorld achieves a Pearson correlation of 0.95 and a Spearman correlation of 0.94 with real success rates. We also evaluate two policies on six RoboChallenge tasks. On TriWorldBench, ProWorld scores 53.68, exceeding the strongest compared baseline’s 51.72.

Our contributions are threefold: (1) Joint video–state prediction that supplies visual observations and learned proprioceptive feedback for closed-loop policy interaction; (2) Shared-Time View Encoding, which preserves camera identity and synchronization while improving visual prediction over pixel tiling and time concatenation; and (3) Failure-aware training that uses simulated failures to reduce interaction hallucinations and success-rate inflation.

2 Related Work

Interactive robot world models.

Generative video models have progressed from visually plausible rollouts to action-conditioned simulators (Yang et al., 2024; Zhu et al., 2025; Liao et al., 2025; Gao et al., 2026; Guo et al., 2025; Qiu et al., 2026; GigaWorld Team et al., 2026; NVIDIA and others, 2026). Recent systems explicitly use such simulators to assess policies, including scalable video-model evaluation, RoboWorld, and an interactive simulator for long-horizon robot learning (Tseng et al., 2025; Jeon et al., 2026; Wang et al., 2026). DW0.5 further uses generated successes and failures for policy post-training (Dexmal, 2026). Table 1 summarizes these systems. ProWorld combines learned state feedback with calibration-free long-horizon multi-view rollout.

Multi-view, multimodal, and world–action models.

Multiple cameras expose complementary evidence about contact and motion. Multi-View Masked World Models learns visual representations from multiple viewpoints (Seo et al., 2023), while recent generative systems study joint multi-view prediction and cross-embodiment synthesis (Guo et al., 2025; Yuan et al., 2026a; Li et al., 2026). Multi-view streams can be organized as pixel canvases (NVIDIA and others, 2026; GigaWorld Team et al., 2026) or concatenated frame sequences (NVIDIA et al., 2025; Bai et al., 2025). Geometry-aware rotary embeddings incorporate camera geometry into attention (Xiang et al., 2026); ProWorld instead assigns camera identity a dedicated rotary coordinate alongside shared time, without requiring calibration (Section 3.3). World–action models (WAMs) use predictive world representations to produce executable actions: DreamZero jointly models future video and actions, and Motus unifies world modeling, control, and inverse dynamics (Ye et al., 2026; Bi et al., 2025). ProWorld instead supplies the observations through which an external policy is evaluated, including its proprioceptive feedback.

Long-horizon generation and evaluation.

Diffusion transformers and flow matching provide the modeling substrate for high-capacity video prediction (Peebles and Xie, 2023; Lipman et al., 2023; Team Wan et al., 2025); autoregressive diffusion work studies how to continue generated history without excessive drift (Chen et al., 2024; Huang et al., 2025; Zheng et al., 2026). Policy evaluation poses a separate measurement question. WorldArena and TriWorldBench evaluate the quality of action-conditioned video predictions (Shang et al., 2026a; Shang et al., 2026b; Liu et al., 2026; Xu et al., 2026), while real-world and simulation benchmarks evaluate robot policies through task success under specified evaluation protocols (Yakefu et al., 2025; Chen et al., 2025a; Jiang et al., 2026).

3 Method

3.1 Overview

We treat ProWorld as the environment in a repeated observe–act loop, with only the initial observation supplied by the real environment. As shown in Figure 2, at decision time t, the policy receives synchronized images O_{t}=\{I_{t}^{(v)}\}_{v=1}^{V}, proprioceptive state s_{t}, and a task instruction g. The policy returns an action chunk A_{t}, and ProWorld predicts the resulting camera observations and robot states over H steps:

A_{t}\sim\pi(\cdot\mid O_{\leq t},s_{\leq t},g),\qquad(\widehat{O}_{t+1:t+H},\widehat{s}_{t+1:t+H})\sim p_{\theta}(\cdot\mid O_{\leq t},s_{\leq t},A_{t},g).

Here, O_{\leq t} and s_{\leq t} denote the available observation and state histories; actions and observations are aligned by physical time even when their sampling rates differ. The predicted images and state become the policy’s next input, and the loop repeats.

3.2 Joint Video–State Prediction

ProWorld jointly predicts visual observations and the resulting robot state using a shared transformer, a joint flow objective, and asymmetric attention. State queries read visual dynamics while both losses train the backbone, providing learned execution feedback for the next policy decision.

Representations and conditioning.

A video VAE encodes each view into latent tokens. A lightweight projection embeds the normalized proprioceptive state into the transformer width, and a separate output head predicts its flow field. Text and metadata specify the task, embodiment, camera configuration, and sampling rate. Controls are grouped by their associated latent interval and projected into adaptive layer-normalization parameters. State tokens receive their own noise-level embedding; the control modulation is applied to the video tokens.

For end-effector trajectories, we use a velocity-style representation obtained from smoothed poses at the control rate. Let \widetilde{r}_{k} contain the continuous pose coordinates and g_{k} the gripper state. The control feature is

u_{k}=\left[\frac{\widetilde{r}_{k}-\widetilde{r}_{k-1}}{\Delta t},\;g_{k}\right],\qquad\bar{u}_{k}=\mathcal{N}_{u}(u_{k}),\qquad\bar{s}_{k}=\mathcal{N}_{s}(s_{k}),

where \mathcal{N}_{u} and \mathcal{N}_{s} are embodiment-specific normalizers. For joint-controlled deployments, the interface uses the policy’s joint-command representation and predicts the corresponding measured joint state.

Joint flow matching.

Let x^{v} denote clean video latents and x^{s} the normalized state sequence. For modality m\in\{v,s\}, we interpolate with Gaussian noise \epsilon^{m} at noise level \sigma_{m}:

x^{m}_{\sigma_{m}}=(1-\sigma_{m})x^{m}+\sigma_{m}\epsilon^{m},\qquad d^{m}=\epsilon^{m}-x^{m}.

The model predicts both flow fields in the same action-conditioned forward pass. We optimize

\mathcal{L}=\mathbb{E}\!\left[w_{v}(\sigma_{v})\|f^{v}_{\theta}-d^{v}\|_{\mathrm{mean}}^{2}+\lambda_{s}w_{s}(\sigma_{s})\|f^{s}_{\theta}-d^{s}\|_{\mathrm{mean}}^{2}\right],

with clean context excluded from the prediction loss. Here, \lambda_{s} controls the relative weight of state supervision. We use \lambda_{s}=2 in all experiments with state outputs, except when explicitly varying this weight in Appendix B.3. Sampling integrates from noise to data using the corresponding modality schedules (Lipman et al., 2023). Both losses update the shared transformer.

Attention structure.

Video and state share a backbone but have different visibility. In bidirectional training, context and video tokens attend to each other; state queries read video and state tokens. Video queries do not read the noisy state stream. This makes state prediction a visual readout while retaining joint supervision of the backbone. For autoregressive training, each noisy chunk reads clean preceding chunks and its own noisy tokens, but not the clean target for that chunk or any future chunk. The state stream also reads preceding states. Figure 3 summarizes these masks. During rollout, denoising reads a fixed history cache; the completed chunk refreshes the cache for the next interaction.

3.3 Shared-Time View Encoding

Figure 2 compares three ways to organize multi-view inputs. Pixel tiling, used by Cosmos 3 and GigaWorld-1 (NVIDIA and others, 2026; GigaWorld Team et al., 2026), combines camera images before VAE encoding, reusing the video pipeline but introducing artificial spatial boundaries. Time concatenation, used in Cosmos-Predict2.5’s multi-view implementation (NVIDIA et al., 2025) and ReCamMaster’s source–target conditioning (Bai et al., 2025), independently encodes each camera stream and concatenates the resulting tokens along the temporal sequence dimension.

STVE instead assigns camera identity its own rotary coordinate (Figure 2, right). Each camera is encoded independently, with tokens indexed by (t,h,w,v): synchronized views share time t, (h,w) remain local to each camera, and v identifies the view. Disjoint rotary subspaces encode these coordinates:

\mathcal{R}(t,h,w,v)=\operatorname{diag}\!\left(R_{t}(t),R_{h}(h),R_{w}(w),R_{v}(v)\right),\qquad\widetilde{q}_{i}=\mathcal{R}_{i}q_{i},\quad\widetilde{k}_{i}=\mathcal{R}_{i}k_{i}.

Joint attention thus distinguishes views without spatial seams or view-dependent time offsets, using the same coordinates for history and generated chunks.

The view coordinate fits within the existing attention-head width: MiniMax-H3 uses its unrotated tail, while Wan reserves a small portion of its spatial rotary channels, preserving the temporal allocation. STVE adds neither tokens nor a separate cross-view attention module. Table 5 compares the three structures in our model.

3.4 Failure Data and Closed-Loop Rollout

Successful demonstrations cover intended behavior extensively but offer few examples of policy mistakes. We collect failed trajectories in GenieSim (Yin et al., 2026), a high-fidelity simulator with realistic scenes and physical robot–object interaction, retaining synchronized multi-view observations, commands, and measured states. The observed aftermath of failed actions serves as the prediction target (Figure 4).

Under the same prediction objective, demonstrations supply broad manipulation behavior, robot-policy rollouts capture policy-induced execution, and simulated failures expand unsuccessful interaction coverage. Figure 6 compares generated and physical success rates.

Training follows the four stages in Figure 3. Bidirectional supervised fine-tuning learns action-conditioned video and state prediction. Chunk-wise teacher forcing extends the model to longer rollouts, following an established autoregressive training recipe (Zhao et al., 2026a). Domain adaptation specializes the model to the target robot and observation setup. Following prior work on autoregressive distillation (Zhao et al., 2026b; Zhao et al., 2026a; Yuan et al., 2026b), we distill the model for few-step generation. At evaluation time, a frozen policy repeatedly receives generated images and predicted state in its native units, reacting to estimated execution rather than reusing its preceding command. Appendix A provides action/state specifications and temporal alignment, failure-data collection details, and stage-specific training configurations.

4 Experiments

4.1 Experimental Setup

Training data and implementation.

We pretrain on approximately 6,486 hours of heterogeneous manipulation data, combining large open datasets with a smaller private collection. The mixture includes InternData-A1 (264 h) (Tian et al., 2025), RoboCOIN (312 h) (Wu et al., 2025b), ABC-130K (2,420 h) (Allshire et al., 2026), InternData-M1 (156 h) (Chen et al., 2025b), AgiBotWorld-Beta (2,233 h) (AgiBot-World-Contributors et al., 2025), RealOmni-Open (199 h) (Genrobot, 2026), Galaxea Open-World (119 h) (Jiang et al., 2025), and RoboMIND (60 h) (Wu et al., 2025a), together with 71 hours of GenieSim data containing diverse failure trajectories and 652 hours of private data. We use MiniMax-H3 (MiniMax, 2026) and Wan (Team Wan et al., 2025) as video diffusion backbones, with details on their use and training settings provided in Appendix A.1.

Evaluation settings.

The local closed-loop study uses three policies on four manipulation tasks, with Ctrl-World and Cosmos 3 Nano as alternative rollout environments. A second study evaluates two policies on six challenging RoboChallenge tasks (Yakefu et al., 2025), covering bimanual handling, placement, sweeping, stacking, and knot tying (Figure 4). For the representation ablations, we initialize models from a pretrained backbone and train them on AgiBotWorld-Alpha (AgiBot-World-Contributors et al., 2025). We evaluate action-conditioned video prediction on TriWorldBench (Liu et al., 2026), comprising 500 three-view trajectories (Table 2), and on WorldArena, comprising 1,000 single-view trajectories.

Evaluation protocol.

A frozen policy starts from a real observation, then repeatedly predicts actions from generated images and state. We compare generated and physical success rates using Pearson and Spearman correlations, and measure paired video quality using PSNR, SSIM, and LPIPS (Zhang et al., 2018). For video–state consistency error (VSC), we train DreamGen’s inverse-dynamics model (IDM) (Jang et al., 2025) on AgiBotWorld-Alpha and subtract its reference-video position error (approximately 9.1 mm) from the discrepancy between video-inferred and predicted trajectories. VSC is reported in millimeters, alongside motion-direction cosine similarity (Dir.) and gripper accuracy (Grip.). Evaluation details appear in Appendices A.5 and B.1.

4.2 Closed-Loop Policy Evaluation

ProWorld’s generated success rates closely track real outcomes across 12 local policy–task pairs (Figure 6), with Pearson r=0.95 and Spearman \rho=0.94. Both correlations exceed those of Ctrl-World (0.33/0.26) and Cosmos 3 Nano (0.67/0.54), indicating stronger agreement in success rates and outcome ordering.

Failure data improves Pearson/Spearman correlation from 0.93/0.91 to 0.95/0.94. The fitted relation changes from y=0.76x+18.41 to y=0.82x+10.53, reducing overestimation at low real success rates. This supports transferring failure supervision across embodiments.

RoboChallenge tasks often require over half a minute of stable closed-loop execution, with coherent observations throughout object manipulation. ProWorld supports these sustained interactions and preserves success-rate variation across policies and tasks (Figure 4d), extending evaluation beyond short demonstrations of a single successful motion.

4.3 Video Prediction and Multi-View Quality

On TriWorldBench (Table 2), ProWorld achieves the highest overall score, with gains in cross-view agreement, task following, and motion. Head and wrist observations provide complementary constraints on robot motion and scene interaction.

On the single-view WorldArena benchmark, ProWorld performs especially well on trajectory accuracy, photometric consistency, and semantic alignment (Table 3). Together, these results indicate that motion, persistent scene appearance, and task content survive the generated transition. Appendix C reports detailed results for both benchmarks and explains the exclusion of frame-rate-sensitive WorldArena motion-quality metrics and EWMScore.

What changes during interaction?

Figure 7 compares two tasks from matched initial scenes. Cooking requires retaining the pot and its graspable parts; drawer manipulation requires consistent gripper contact across wrist and workspace views. Baselines exhibit object disappearance, blurred structure, and inconsistent views. ProWorld preserves these details, giving the policy stable visual context for its next decision.

Without failure data, errors concentrate around contact: handles, grasp points, and gripper–object positions. Failure examples teach the model how scenes evolve when contact is absent or incomplete. In Figure 8, an empty lift leaves the banana behind, and a one-sided grasp produces a one-sided fold. These distinctions lead to different closed-loop task outcomes.

4.4 Ablations

Further ablations of action conditioning, visual context, noise schedules, and training resolution are provided in Appendix B.

Action representation and joint state learning.

In Table 4, velocity-style control improves visual prediction over absolute-state conditioning by directly relating the commanded change to visible motion. Adding state supervision improves the shared visual representation and supplies proprioceptive feedback. Motion-based conditioning with asymmetric attention gives the strongest combination: state queries read the evolving video, while state loss trains the backbone to retain robot-motion information.

Compared with full attention between noisy streams, asymmetric attention lowers VSC and improves motion direction with little change in gripper accuracy, thereby strengthening agreement between predicted continuous motion and the generated video.

Multi-view encoding.

Table 5 compares the three structures in Figure 2. Time concatenation improves all video metrics over pixel tiling. Replacing its view-dependent temporal offsets with STVE further raises PSNR from 21.56 to 22.60 and SSIM from 0.809 to 0.829, and lowers LPIPS from 0.1266 to 0.1069. State results are mixed: STVE improves directional agreement and matches gripper accuracy, but time concatenation has lower VSC. Thus, STVE benefits visual prediction rather than uniformly improving all outputs.

5 Conclusion and Limitations

ProWorld evaluates robot policies through closed-loop interaction, combining joint visual–proprioceptive feedback, STVE for camera identity and synchronization, and simulated failures to reduce interaction hallucinations and success-rate inflation. Strong correlation with real policy success and competitive video prediction support complete observation feedback and faithful failure outcomes as foundations for policy evaluation.

Object deformation and drift over exceptionally long rollouts remain limitations. Improving geometry persistence and rollout stability could reduce accumulated evaluation errors.

References

1. AgiBot-World-Contributors et al. (2025) AgiBot-World-Contributors, Q. Bu, J. Cai, L. Chen, X. Cui, Y. Ding, et al. AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems. arXiv preprint arXiv:2503.06669.
2. Allshire et al. (2026) A. Allshire, H. G. Singh, R. Singh, A. Rashid, H. Choi, et al. Scalable Behavior Cloning with Open Data, Training, and Evaluation. arXiv preprint arXiv:2606.27375.
3. Bai et al. (2025) J. Bai, M. Xia, X. Fu, X. Wang, L. Mu, J. Cao, Z. Liu, H. Hu, X. Bai, P. Wan, and D. Zhang ReCamMaster: Camera-Controlled Generative Rendering from A Single Video. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14834–14844.
4. Bi et al. (2025) H. Bi, H. Tan, S. Xie, Z. Wang, S. Huang, et al. Motus: A Unified Latent Action World Model. arXiv preprint arXiv:2512.13030.
5. Chen et al. (2024) B. Chen, D. M. Monsó, Y. Du, M. Simchowitz, R. Tedrake, and V. Sitzmann Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion. In Advances in Neural Information Processing Systems, Vol. 37. External Links: Document
6. Chen et al. (2025a) T. Chen, Z. Chen, B. Chen, Z. Cai, Y. Liu, et al. RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation. arXiv preprint arXiv:2506.18088.
7. Chen et al. (2025b) X. Chen, Y. Chen, Y. Fu, N. Gao, J. Jia, et al. InternVLA-M1: A Spatially Guided Vision-Language-Action Framework for Generalist Robot Policy. arXiv preprint arXiv:2510.13778.
8. Dexmal (2026) Dexmal DW0.5: A Closed-Loop World Model for VLA. Note: Official technical article
9. Gao et al. (2026) S. Gao, W. Liang, K. Zheng, A. Malik, S. Ye, S. Yu, et al. DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos. arXiv preprint arXiv:2602.06949.
10. Genrobot (2026) Genrobot RealOmni-Open Dataset. Note: Official dataset release
11. GigaWorld Team et al. (2026) GigaWorld Team, A. Ma, B. Wang, B. Li, C. Ni, G. Li, et al. GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation. arXiv preprint arXiv:2607.02642.
12. Guo et al. (2025) Y. Guo, L. X. Shi, J. Chen, and C. Finn Ctrl-World: A Controllable Generative World Model for Robot Manipulation. arXiv preprint arXiv:2510.10125.
13. Huang et al. (2025) X. Huang, Z. Li, G. He, M. Zhou, and E. Shechtman Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion. In Advances in Neural Information Processing Systems, Vol. 38. External Links: Document
14. Jang et al. (2025) J. Jang, S. Ye, Z. Lin, J. Xiang, J. Bjorck, et al. DreamGen: Unlocking Generalization in Robot Learning through Video World Models. arXiv preprint arXiv:2505.12705.
15. Jeon et al. (2026) B. Jeon, S. Ye, J. Doo, S. Kim, M. Seo, H. Son, and K. Lee RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation. arXiv preprint arXiv:2607.01060.
16. Jiang et al. (2025) T. Jiang, T. Yuan, Y. Liu, C. Lu, J. Cui, et al. Galaxea Open-World Dataset and G0 Dual-System VLA Model. arXiv preprint arXiv:2509.00576.
17. Jiang et al. (2026) T. Jiang, X. Tan, S. Wheeler, L. Sun, T. W. Ayalew, and M. Walter What Are We Actually Benchmarking in Robot Manipulation?. arXiv preprint arXiv:2606.04233.
18. Li et al. (2026) X. Li, J. Guo, Q. Li, L. Qian, H. Lai, et al. Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Model. arXiv preprint arXiv:2607.11643.
19. Liao et al. (2025) Y. Liao, P. Zhou, S. Huang, D. Yang, S. Chen, Y. Jiang, et al. Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulation. arXiv preprint arXiv:2508.05635.
20. Lipman et al. (2023) Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le Flow Matching for Generative Modeling. In The Eleventh International Conference on Learning Representations,
21. Liu et al. (2026) X. Liu, H. Wang, R. Li, D. Yu, R. Wan, R. Zhang, S. Tao, X. Yang, S. Zhang, Z. Zhang, J. Zhang, and S. Ma TriWorldBench: A Tri-View Consistency Perspective on Embodied World Models. arXiv preprint arXiv:2609.26314.
22. MiniMax (2026) MiniMax MiniMax H3: An Open Model Breaking the Boundaries Between Tasks and Modalities. Note: Technical announcement
23. NVIDIA et al. (2025) NVIDIA, A. Ali, J. Bai, M. Bala, Y. Balaji, A. Blakeman, et al. World Simulation with Video Foundation Models for Physical AI. arXiv preprint arXiv:2511.00062.
24. NVIDIA et al. (2026) NVIDIA et al. Cosmos 3: Omnimodal World Models for Physical AI. arXiv preprint arXiv:2606.02800.
25. Peebles and Xie (2023) W. Peebles and S. Xie Scalable Diffusion Models with Transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4195–4205.
26. Qiu et al. (2026) B. Qiu, L. Chen, Y. Liao, N. Wang, L. Wang, J. Luo, et al. GE-Sim 2.0: A Roadmap Towards Comprehensive Closed-loop Video World Simulators for Robotic Manipulation. arXiv preprint arXiv:2605.27491.
27. Seo et al. (2023) Y. Seo, J. Kim, S. James, K. Lee, J. Shin, and P. Abbeel Multi-View Masked World Models for Visual Robotic Manipulation. In Proceedings of the 40th International Conference on Machine Learning, Vol. 202, pp. 30613–30632.
28. Shang et al. (2026a) Y. Shang, Z. Li, Y. Ma, W. Su, X. Jin, Z. Wang, et al. WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Models. arXiv preprint arXiv:2602.08971.
29. Shang et al. (2026b) Y. Shang, Y. Tang, Y. Ma, Z. Li, L. Jin, W. Su, et al. WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platform. arXiv preprint arXiv:2605.17912.
30. Team Wan et al. (2025) Team Wan, A. Wang, B. Ai, B. Wen, C. Mao, C. Xie, et al. Wan: Open and Advanced Large-Scale Video Generative Models. arXiv preprint arXiv:2503.20314.
31. Tian et al. (2025) Y. Tian, Y. Yang, Y. Xie, Z. Cai, X. Shi, et al. InternData-A1: Pioneering High-Fidelity Synthetic Data for Pre-training Generalist Policy. arXiv preprint arXiv:2511.16651.
32. Tseng et al. (2025) W. Tseng, J. Gu, Q. Zhang, H. Mao, M. Liu, F. Shkurti, and L. Yen-Chen Scalable Policy Evaluation with Video World Models. arXiv preprint arXiv:2511.11520.
33. Wang et al. (2026) Y. Wang, R. Syed, F. Wu, M. Zhang, A. Onol, J. Barreiros, H. Nayyeri, T. Dear, H. Zhang, and Y. Li Interactive World Simulator for Robot Policy Training and Evaluation. arXiv preprint arXiv:2603.08546.
34. Wu et al. (2025a) K. Wu, C. Hou, J. Liu, Z. Che, X. Ju, et al. RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulation. In Proceedings of Robotics: Science and Systems, External Links: Document
35. Wu et al. (2025b) S. Wu, X. Liu, S. Xie, P. Wang, X. Li, et al. RoboCOIN: An Open-Sourced Bimanual Robotic Data COllection for INtegrated Manipulation. arXiv preprint arXiv:2511.17441.
36. Xiang et al. (2026) C. Xiang, J. Liu, J. Zhang, X. Yang, Z. Fang, et al. Geometry-Aware Rotary Position Embedding for Consistent Video World Model. arXiv preprint arXiv:2602.07854.
37. Xu et al. (2026) X. Xu, Z. Lin, K. He, Y. Feng, X. Mao, Y. Yin, K. Zhang, and Y. Ge WorldMark: A Unified Benchmark Suite for Interactive Video World Models. arXiv preprint arXiv:2604.21686.
38. Yakefu et al. (2025) A. Yakefu, B. Xie, C. Xu, E. Zhang, E. Zhou, F. Jia, et al. RoboChallenge: Large-scale Real-robot Evaluation of Embodied Policies. arXiv preprint arXiv:2510.17950.
39. Yang et al. (2024) S. Yang, Y. Du, K. Ghasemipour, J. Tompson, L. Kaelbling, D. Schuurmans, and P. Abbeel Learning Interactive Real-World Simulators. In The Twelfth International Conference on Learning Representations,
40. Ye et al. (2026) S. Ye, Y. Ge, K. Zheng, S. Gao, S. Yu, et al. World Action Models are Zero-shot Policies. arXiv preprint arXiv:2602.15922.
41. Yin et al. (2026) C. Yin, D. Huang, D. Yang, J. Wang, N. Zhao, C. Xu, et al. Genie Sim 3.0 : A High-Fidelity Comprehensive Simulation Platform for Humanoid Robot. arXiv preprint arXiv:2601.02078.
42. Yuan et al. (2026a) G. Yuan, Q. Qiao, J. Zhang, and D. Xu AdaWorldPolicy: World-Model-Driven Diffusion Policy with Online Adaptive Learning for Robotic Manipulation. arXiv preprint arXiv:2602.20057.
43. Yuan et al. (2026b) S. Yuan, Y. Yin, Z. Li, X. Huang, X. Yang, and L. Yuan Helios: Real Real-Time Long Video Generation Model. arXiv preprint arXiv:2603.04379.
44. Zhang et al. (2018) R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 586–595.
45. Zhao et al. (2026a) M. Zhao, H. Zhu, B. Yan, Z. Zhou, Y. Chen, H. Wang, W. Sun, Z. Fang, Z. Xun, Z. Li, K. Zheng, G. He, X. Yang, C. Li, F. Bao, and J. Zhu minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Models. arXiv preprint arXiv:2605.30263.
46. Zhao et al. (2026b) M. Zhao, H. Zhu, K. Zheng, Z. Zhou, B. Yan, X. Li, X. Yang, C. Li, and J. Zhu Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation. arXiv preprint arXiv:2605.15141.
47. Zheng et al. (2026) K. Zheng, G. He, M. Zhao, J. Zhang, H. Chen, J. Chen, et al. Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Models. arXiv preprint arXiv:2606.25473.
48. Zhu et al. (2025) F. Zhu, H. Wu, S. Guo, Y. Liu, C. Cheang, and T. Kong IRASim: A Fine-Grained World Model for Robot Manipulation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9834–9844.

Appendix

Appendix A Training and Interface Details

A.1 Training Details

Backbones and initial fine-tuning.

We instantiate ProWorld with two pretrained video backbones: Wan2.1-14B and MiniMax-H3. We perform full-parameter fine-tuning on Wan2.1-14B and low-rank adaptation (LoRA) on MiniMax-H3, using learning rates of 2\times 10^{-5} and 10^{-4}, respectively. Both models are initially fine-tuned on our proposed dataset for 36,000 steps with a batch size of 64.

Wan-based training and experiments.

Following this initial fine-tuning, we continue training the Wan-based model using the standard Helios Stage 1 procedure with an autoregressive attention mask. The ablation studies in the main text and the RoboChallenge experiments use this Wan-based model.

MiniMax-H3-based training and experiments.

For MiniMax-H3, we follow the standard minWM pipeline for long-video generation training and distillation after the initial fine-tuning. The additional ablation studies in Appendix B use the MiniMax-H3-based model.

A.2 Action/State Definitions and Temporal Alignment

Definitions and layouts.

The control feature u_{k} conditions prediction, whereas s_{k} is the absolute proprioceptive target. End-effector data use 32-dimensional vectors: each arm has Cartesian position (3), a 6D rotation representation (6), and gripper opening (1), occupying entries 0–9 and 10–19; unused entries are zero. Continuous pose coordinates are median-filtered and Savitzky–Golay smoothed before differencing at the control rate. Translation features have units of length per second; rotation features are 6D-coordinate rates, not angular velocities. Grippers retain raw absolute openings. State targets remain the original absolute poses and openings. These trajectory-derived controls are motion proxies. For joint data with command logs, absolute commanded and measured positions remain separate. The RoboTwin adapter, for example, places each arm’s six joints and gripper in entries 0–6 and 16–22, with zeros elsewhere.

Normalization.

\mathcal{N}_{u} and \mathcal{N}_{s} follow the dataset/embodiment schema. Scalar coordinates use 1st/99th-percentile affine scaling; absolute 6D rotations retain their bounded representation. Velocity features use separate statistics, with continuous coordinates clipped to [-5,5] and grippers to [-1,1] after normalization. Joint data use joint-layout statistics. State predictions are mapped back to native units before policy feedback; clipping is not invertible.

Temporal alignment.

Synchronized views share source-frame indices. For clip start b, source rate f_{s}, video rate f_{v}, and control rate f_{u}, the binary loader samples

i_{j}^{v}=b+\operatorname{round}(jf_{s}/f_{v}),\qquad i_{k}^{u}=b+\operatorname{round}(kf_{s}/f_{u}).

The initial image uses j=0; target controls start at k=1. Each state target is read at its control’s source index. At equal video/control rates, (u_{k},s_{k}) aligns with target image O_{k}. For k>1, the derived control describes the interval ending at that sample. The first sampled velocity of each continuous coordinate is zero; the gripper retains its absolute value. Across rates, alignment follows physical time rather than equal array positions. History uses the same sampling grid, with control indices before the episode start clamped to its initial sample.

A.3 Metadata Prompt Format

Fields and serialization.

The metadata prompt describes the data source, embodiment, and observation setup in a compact JSON object. Its fields have a fixed order: dataset identifies the dataset group or individual data subset; robot names the embodiment; fps gives the control sampling rate in Hz; deploy specifies the domain (e.g., real or simulation); and views lists camera names in the same order as the input video streams. Thus, fps corresponds to f_{u} in Appendix A.2, and the camera list is reordered whenever the views are shuffled.

Example and optional task text.

For AgiBotWorld-Alpha at 10 Hz, group-level metadata take the following form (line breaks are added for readability):

{"dataset":"agibot/agibot_alpha","robot":"agibot_alpha", "fps":10,"deploy":"real", "views":["head_color","hand_left_color","hand_right_color"]}

The loader serializes this object without extra whitespace. Group-level naming permits samples with the same metadata to share a prompt; subset-level naming retains the individual dataset identifier while removing a trailing train/evaluation split name. If task conditioning is enabled and a nonempty instruction is available, a final task string is appended to the object. Otherwise, that field is omitted. Numeric actions and proprioceptive targets are supplied through the separate interface in Appendix A.2, rather than inserted into this JSON object.

A.4 Failure Data Collection

Perturbed physical replay.

We generate missed-grasp examples from successful GenieSim trajectories. Each replay restores the recorded scene and initial robot–object poses, then tracks the recorded joint trajectory with a Cartesian offset applied to the active arm. A damped least-squares Jacobian update maps the offset to joint corrections, ramped in before gripper closure and maintained during lifting. The target object remains a dynamic rigid body, allowing contact and gravity to determine the outcome; non-target objects remain kinematic.

Failure selection and recording.

We label a perturbed replay as a missed grasp only when the target’s maximum vertical displacement from its first replay frame is below a threshold adjusted during collection and bounded by 5 cm. Replays reaching this threshold are excluded from the failure set. Each replay saves synchronized camera videos, the perturbed joint reference trajectory, and per-frame target and end-effector poses. A manifest links each variant to its successful source and records the requested offset, achieved offset at closure, and lift-based label.

A.5 Closed-Loop Evaluation Configuration

Adaptation and policy prerequisites.

Before closed-loop evaluation, we fine-tune ProWorld for the target embodiment and tasks using trajectories collected on the corresponding real-robot platform. GenieSim failure trajectories constitute 10% of the fine-tuning mixture. Evaluation additionally requires locally deployable robot-policy weights and real-robot success rates for the same policies and tasks.

Closed-loop protocol.

Evaluation recordings provide only the initial frames for world-model rollouts; subsequent recorded video and actions are never supplied to either ProWorld or the policy. After initialization, the policy acts on observations generated by ProWorld, which predicts the next multi-view observations and proprioceptive state for continued policy feedback. Thus, all subsequent actions are chosen online by the policy. Below, one interaction round denotes one policy query followed by the corresponding world-model transition.

RoboChallenge.

We use the Table30v2 dataset for adaptation and evaluate \pi_{0.5} and DM0.5, selected for their publicly released fine-tuned policy weights and nontrivial success rates reported on the official test platform. We use these official rates as the real-robot reference and collect 120 evaluation trajectories from the RoboChallenge website across the two policies and six tasks. These recordings supply initial frames under the protocol above. Most tasks require more than 80 interaction rounds to complete; success in the world-model rollouts is judged manually.

Cobot.

For our local Cobot evaluation, we collect 200 real-robot trajectories across three policies and four tasks. These recordings serve only to measure real-robot policy success rates and provide initial frames for world-model evaluation. Cobot tasks require approximately 25 policy–world-model interaction rounds. A locally deployed vision–language model (VLM) scores the generated rollouts to determine task success.

Appendix B Additional Ablation Studies

We examine action conditioning, the state-loss weight, visual context, noise schedules, and training resolution. The action-conditioning, visual-context, noise-schedule, and training-resolution ablations use MiniMax-H3 on AgiBotWorld-Alpha, with 108 evaluation clips per setting. Slight differences in the starting configurations make absolute scores across ablations not directly comparable.

B.1 Trajectory-Quality Evaluation Protocol

Inputs and alignment.

The AgiBotWorld-Alpha ablations assess video–state agreement and state prediction separately. For video–state agreement, a fixed inverse dynamics model (IDM) reads only the head-camera video at 10 FPS, resized to 224\times 224. We remove the initial conditioning frame to align video frames with saved state predictions. The IDM takes pairs of frames 16 steps apart and predicts 16 motion steps; overlapping predictions are averaged and denormalized. Cumulative motion is anchored at the first state of the trajectory being compared, while gripper outputs remain absolute.

Video–state consistency (VSC).

Let E(V,S) denote the mean Euclidean distance, in millimeters, between the IDM-reconstructed end-effector positions from video V and the positions in state sequence S. We average over time and both arms within each clip, then equally over clips. With generated video \hat{V}, predicted states \hat{S}, and recorded references (V,S), we report

\mathrm{VSC}=E(\hat{V},\hat{S})-E(V,S).

Thus, VSC measures excess video–state disagreement relative to the IDM’s reference-video error; lower is better. This reference error is an empirical calibration, not a guaranteed lower bound. VSC covers the head view only and does not measure task success.

State prediction (Dir. and Grip.).

Dir. is the motion-direction cosine similarity between predicted and ground-truth end-effector trajectories. Grip. is the accuracy of predicted gripper open/closed states against ground truth. Both are higher-is-better measures computed from predicted versus reference states, independently of the IDM. State metrics are omitted for models without state supervision.

B.2 Action Conditioning and State Supervision

Direct AdaLN provides a strong interface between actions and visual dynamics (Table 6). Replacing it with velocity tokens reduces PSNR by 3.48 dB and increases LPIPS from 0.1698 to 0.2786. Action guidance recovers part of this gap, but direct modulation remains substantially stronger. State supervision also benefits visual prediction: removing it costs 1.36 dB and degrades both structural and perceptual quality.

B.3 State-Loss Weight

The state-loss weight controls how strongly proprioceptive supervision contributes to joint video–state learning. Figure 9 varies \lambda_{s}, the coefficient of the state term in the main text’s joint flow-matching objective, over \{0,0.05,0.1,0.5,1,2,5,15,100\}. The figure uses \lambda as shorthand for \lambda_{s}; \lambda_{s}=0 removes state supervision, and state-related metrics are omitted for this setting.

A moderate state-loss weight provides a favorable balance between visual quality and video–state agreement. Among the tested values, \lambda_{s}=2 yields the lowest LPIPS and VSC and the highest gripper accuracy, while PSNR and SSIM are slightly higher at \lambda_{s}=1. Motion-direction similarity does not peak at \lambda_{s}=2, so this choice does not optimize every metric individually. Larger weights, particularly 15 and 100, degrade visual quality and increase VSC relative to 2. We therefore adopt \lambda_{s}=2 for all experiments with state outputs, except for this explicit weight sweep.

B.4 Visual Context and Noise Scheduling

Initial scene descriptions and noise scheduling improve different aspects of prediction (Table 7). Descriptions derived from synchronized initial views reduce video–state consistency error from 9.71 to 7.93 mm, with little change in visual quality. Increasing the shared video/state flow shift to 12 gives the strongest visual scores, improving PSNR by 0.43 dB and reducing LPIPS to 0.1472. The lower shift of 5 preserves better motion direction and video–state agreement, revealing a trade-off between visual fidelity and motion accuracy.

B.5 Coordinating Video and State Denoising

Separate noise schedules let video and state prediction use different flow shifts while sharing the same denoising progress. Table 8 compares a shared shift of 12 with separate video/state shifts of 12/5, using either aligned or independently sampled progress. Aligned progress gives the lowest VSC and highest motion-direction similarity, while maintaining the visual quality of the shared schedule. Independent sampling raises VSC from 7.83 to 8.91 mm. Coordinating denoising progress therefore matters even when the two modalities use different noise levels.

B.6 Mixed-Resolution Training

Mixed-resolution training substantially improves 480p prediction (Table 9). Compared with training exclusively at 480p, it raises PSNR by 2.11 dB, lowers VSC by 5.29 mm, and improves motion-direction similarity from 0.247 to 0.403. At 288p, however, the dedicated model retains better visual quality and motion accuracy. Resolution mixing thus offers a practical route to higher-resolution generation, with a trade-off in lower-resolution performance.

Appendix C Detailed Benchmark Results

Tables C and C provide the complete results for the retained WorldArena metrics and all 19 TriWorldBench submetrics, respectively. Scores are reported on a 0–100 scale; higher is better. Bold and underlined values indicate the best and second-best results among the compared models.

Frame-rate sensitivity in WorldArena.

WorldArena’s motion-quality metrics are strongly affected by frame rate: lowering the evaluation frame rate can substantially increase these scores without a corresponding improvement in generated video quality. We therefore omit motion quality and the aggregate EWMScore from our reported WorldArena tables, including the main-paper comparison, rather than select a lower frame rate solely to improve benchmark scores.

Appendix D Additional Qualitative Observation

Cross-view recovery.

We observe that ProWorld can use the other two camera views to recover content in a target view. Figure D shows qualitative examples alongside the corresponding ground-truth frames. The recovered views capture the broad scene layout and object appearance, although fine details can differ. We report this as an additional qualitative observation of the model’s cross-view prediction capability.