FlowVLA: Enabling World Dynamics Understanding through Pluggable 3D Flow Learning

Abstract

Recent advances in 3D reconstruction enable the extraction of temporally consistent 3D motion from RGB observations; however, current Vision-Language-Action (VLA) models still predominantly rely on 2D image-text priors. Therefore, they do not accurately represent the fundamental dynamics of the 3D world associated with physical actions. This over-reliance on static 2D visual cues limits the capacity of VLA models in achieving robust robot manipulation. To address these limitations, we introduce FlowVLA, a reconstruction-guided, pluggable 3D flow learning framework aimed at empowering VLA models with an inherent understanding of action-relevant dynamics. Instead of requiring explicit 3D sensor inputs or altering the inference-time architecture, our method is designed to utilize temporally aligned 3D flow only during training. Our method allows the model to learn the dynamic evolution of the physical scene while preserving the computational efficiency of conventional 2D VLA backbones. To bridge scene-level dynamics with robotic manipulation, we propose a modular 3D flow learning framework defined by a dual-level flow supervision design. Specifically, point-level flow guides the model to forecast the future displacements of task-critical 3D points, while TCP-level flow aggregates these motions to align with the robot’s Tool-Center-Point (TCP) trajectories. This dual-level flow supervision encourages the model to focus on semantically meaningful dynamic regions, effectively coupling scene-level motion with precise control signals. Extensive evaluations across simulation and real-world experiments demonstrate that FlowVLA consistently improves manipulation performance, zero-shot robustness, and optimizes data efficiency. FlowVLA adds only lightweight auxiliary heads during training and discards them at inference. Therefore, FlowVLA introduces no additional 3D sensor inputs, architectural modifications, or inference-time computation.

Keywords: Motion-relevant 3D Dynamics, 3D Scene Understanding, Training-only Supervision

1 INTRODUCTION

Reliable robotic manipulation should consider interaction within a dynamic, three-dimensional physical world. This dynamic interaction is addressed through the emergence of VLA models as a strong paradigm, transferring the rich semantic knowledge of pre-trained LLMs and VLMs to physical robot controls [4, 38, 3, 12, 21]. By integrating large-scale multimodal backbones with an action decoder, recent VLAs have achieved remarkable performance across diverse robotic tasks, including complex scene reasoning [33, 34], cross-embodiment generalization [18, 12, 24], and language-instructed manipulation [14, 3, 25].

Despite their impressive capabilities, most existing VLA models remain constrained by their dependence on multimodal representations derived from 2D image-text pretraining. Although these 2D representations provide rich semantic priors, they are not inherently designed to capture action-relevant 3D dynamics required for real-world robot manipulation. Consequently, action policies often overfit to static visual cues—such as backgrounds, static camera viewpoints, or absolute global geometry—rather than prioritizing action-relevant 3D interactions and dynamics. To mitigate these limitations, recent approaches [19, 16, 15, 35] have explored 3D-aware VLAs that integrate explicit 3D signals, including depth maps or point clouds. Although these approaches improve physical manipulation by leveraging spatial understanding, they also present new challenges. In their approaches, auxiliary 3D sensors are required, and costly retraining on paired 3D-action datasets is demanded. More crucially, these approaches emphasize the extraction of static 3D geometry from single observations; as a result, they still overlook the dynamic evolution of the scene that is directly tied to physical manipulation.

To address these challenges, we propose FlowVLA, a reconstruction-guided, pluggable 3D flow learning framework that injects action-relevant 3D motion cues into VLA representations. Our core insight is to leverage 3D flow exclusively as a privileged signal during training to inject manipulation-relevant 3D dynamics directly into the model’s latent representations. Instead of providing static snapshots, 3D flow provides direct supervision on how the physical environment moves and evolves under robot actions. Therefore, we employ 3D flow not as an input modality, but as a training-only module, while preserving the architecture of VLAs during inference. To achieve this, we derive temporally aligned 3D point trajectories from consecutive frames [22]. By computing the positional differences between adjacent frames, we formulate 3D flow that highlights regions of real physical movement within the scene. To effectively bring this spatial motion into the learning process, we establish a dual-level scene-to-control 3D flow learning framework comprising scene-level and control-level modules. The scene-level module captures dense 3D scene dynamics by supervising the future displacements of motion-salient 3D points, while the control-level module distills these dense motions into TCP-aligned flow that reflects the robot end-effector trajectory. By jointly optimizing these two modules, our framework bridges holistic environmental changes in 3D space with control-relevant robot motion.

We comprehensively validate FlowVLA through extensive evaluations on standard benchmarks, such as LIBERO and LIBERO-Plus, and real-world robot experiments. Empirical results demonstrate that our 3D sensor-free framework improves diverse VLA models with zero inference overhead. By improving task success rates in both simulated and real-world environments, the proposed approach highlights the potential of FlowVLA as a practical framework for 3D-aware policy learning.

2 RELATED WORK

2.1 Vision-Language-Action Models

VLAs integrate visual observations, proprioceptive states, and language instructions to generate robot actions. The viability of language-model-based sequence modeling for robot control was established by early studies [1, 20, 8], leading to RT-1 [4]. RT-2 [38] further showed that the semantic knowledge acquired from internet-scale pretraining can transfer effectively to robot manipulation. Recent advancements have concentrated on scaling, generalization, and expressive action generation. Octo [25] and OpenVLA [14] utilize large-scale robot datasets [18, 27] to improve adaptation to novel environments. Other works extend VLA models with continuous action generation via flow matching or diffusion [3, 12, 29], internal reasoning mechanisms [33, 34], and world modeling [26, 5]. Even with these advances, the current VLA paradigm still relies on representations inherited from 2D image-text pretraining. This makes existing models highly effective at semantic grounding, but leaves action-relevant 3D dynamics insufficiently modeled [35, 11, 6].

2.2 3D-Aware VLA Models

To improve spatial grounding in VLA models, recent works have incorporated explicit 3D information into their pipelines. Models such as LLaVA-3D [37], SpatialVLA [19], and DM0 [32] exploit 3D representations to enhance spatial understanding, whereas 3DVLA [35] injects 3D features into the VLM embedding space for improved reasoning capabilities. It is demonstrated by these methods that useful physical grounding for robot manipulation can be provided by 3D structures. However, incorporating 3D information into 2D-pretrained backbones poses significant challenges. Dedicated alignment mechanisms are required due to the domain gap between pixel-based and geometric representations [15, 36, 16, 7]. Furthermore, high-quality 3D-action paired datasets are remarkably scarce compared to large-scale 2D corpora, making the computational cost of retraining prohibitively expensive [35, 31]. Several recent methods therefore explore more efficient ways to integrate 3D information. PointVLA [15] introduces lightweight modules for injecting 3D information without retraining the entire backbone, while GeoVLA [23] employs a parallel 3D embedding network to preserve multimodal alignment. BridgeVLA [16] instead projects 3D point clouds into multiple 2D views so that they can be processed within a unified representation space. Despite their effectiveness, these approaches typically rely on 3D sensor-derived inputs or 3D representations, and primarily emphasize static 3D geometry while providing limited supervision on how motion-salient regions physically evolve under robot actions. In contrast, FlowVLA uses reconstruction-derived 3D flow as a training-only privileged signal, encouraging existing 2D VLA policies to encode action-relevant 3D motion cues without the need for auxiliary 3D sensors or additional inference-time computation.

3 METHOD

As shown in Fig. 1, we present FlowVLA, a reconstruction-guided 3D flow learning framework that injects action-relevant 3D dynamics into VLA models through training-only supervision. Temporally aligned 3D point trajectories are derived from RGB sequences by our FlowVLA using a feed-forward dynamic reconstruction model [22]. Then, it converts 3D point trajectories into future 3D flow targets. These targets supervise the VLA with a dual-level scene-to-control objective: the action latent is encouraged by point-level flow to encode dense 3D scene motion, while motion cues are distilled into a control-oriented representation aligned with end-effector displacement by TCP-level flow.

3.1 Preliminaries

VDPM [22].

We adopt VDPM, a model intended to recover temporally consistent 3D point trajectories from RGB observations, to obtain a 3D motion signal without any other sensors. The core advantage of VDPM is its ability to consistently track points defined in a reference frame over time. Specifically, given an observation o_{t_{\mathrm{ref}}} at reference time step t_{\mathrm{ref}}, VDPM represents the 3D location of each point on the reference pixel grid u\in\{0,\dots,H-1\}\times\{0,\dots,W-1\} at an arbitrary time step t^{\prime} as P_{t_{\mathrm{ref}}}(t^{\prime},\pi)\in\mathbb{R}^{H\times W\times 3}, where \pi denotes the coordinate system in which the 3D points are expressed. Thus, P_{t_{\mathrm{ref}}}(t^{\prime},\pi)(u) is the 3D position of the point corresponding to pixel u in the reference frame, evaluated at time step t^{\prime}. For fixed t_{\mathrm{ref}} and \pi, the 3D trajectory of a reference point over a sequence is defined as \{P_{t_{\mathrm{ref}}}(t^{\prime},\pi)(u)\}_{t^{\prime}=0}^{T-1}.

Coordinate Alignment.

We set the first frame as the reference frame, i.e., t_{\mathrm{ref}}=0, and define the corresponding reference camera coordinate as \pi_{0}. VDPM initially generates reference-anchored point trajectories within this reference camera coordinate, denoted as P_{0}(t^{\prime},\pi_{0})(u). However, robot motion, including TCP displacement, is defined in the robot base coordinate. Before constructing flow targets, all 3D points are expressed in the robot base coordinate \pi_{\mathrm{base}} to ensure that the reconstruction-derived point trajectories are directly comparable with robot motion. For each task, we compute a rigid transform T_{\mathrm{base}\leftarrow 0}=[R_{\mathrm{base}\leftarrow 0}\mid\mathbf{t}_{\mathrm{base}\leftarrow 0}]\in SE(3) from the reference camera frame to the robot base frame. Subsequently, we proceed to transform each reference-anchored 3D point as follows:

\displaystyle P_{0}(t^{\prime},\pi_{\mathrm{base}})(u)=R_{\mathrm{base}\leftarrow 0}P_{0}(t^{\prime},\pi_{0})(u)+\mathbf{t}_{\mathrm{base}\leftarrow 0}.

For a trajectory of length N, we define the 3D point flow at time t^{\prime} in the robot base coordinate as:

\displaystyle\Delta P_{0}(t^{\prime},u)=P_{0}(t^{\prime}+1,\pi_{\mathrm{base}})(u)-P_{0}(t^{\prime},\pi_{\mathrm{base}})(u),\quad t^{\prime}=0,\dots,N-2.

Here, \Delta P_{0}(t^{\prime},u)\in\mathbb{R}^{3} is referred to as the 3D displacement of reference point u from t^{\prime} to t^{\prime}+1 in the robot base coordinate. This alignment assures that the resulting 3D point flows are defined in the same coordinate system as the TCP displacements used in the TCP-level objective.

3.2 Reconstruction-Guided 3D Flow Construction

Given visual observations o_{t-L+1:t}, a language instruction l, and proprioceptive states s_{t-L+1:t}, where L denotes the historical observation length and k denotes the future prediction horizon, the VLA backbone generates action latent representations z_{t:t+k-1} as follows:

\displaystyle z_{t:t+k-1} \\ \displaystyle=f_{\theta}\!\left(f_{v}(o_{t-L+1:t}),f_{l}(l),f_{s}(s_{t-L+1:t})\right), \\ \displaystyle a_{t:t+k-1} \\ \displaystyle=p_{\theta}\!\left(z_{t:t+k-1}\right),

where f_{\theta} and p_{\theta} denote the VLA backbone and action policy, respectively, and f_{v}, f_{l}, and f_{s} act as modality-specific encoders. While rich semantic priors are provided by these 2D-centric representations, they often struggle to capture the underlying 3D dynamics essential for complex manipulation. The reference-anchored 3D flow defined in Eq. 2 is utilized as a reconstruction-derived, training-only supervision signal to bridge this gap.

Magnitude-based Selection.

Since not all points contribute equally to action-relevant dynamics, we prioritize the selection of dynamically informative points according to their motion magnitude. This filtering directs the model’s attention towards dynamically informative regions rather than static backgrounds or reconstruction artifacts. Specifically, for each point u on the reference-image pixel grid, we compute its instantaneous flow magnitude at time \tau within an observation window of length L+k using the formula m_{\tau}(u)=\left\|\Delta P_{0}(\tau,u)\right\|_{2}, where \tau ranges from t-L+1 to t+k-1. The average flow magnitude over the sequence window is calculated as follows:

\displaystyle\bar{m}_{t}(u)=\frac{1}{L+k-1}\sum_{\tau=t-L+1}^{t+k-1}m_{\tau}(u).

By selecting the top-K points that exhibit the highest \bar{m}_{t}(u) values, we construct a keypoint set \mathcal{U}_{t}=\operatorname{TopK}\big(\{\bar{m}_{t}(u)\}_{u},K\big), which identifies regions with dominant 3D motion. For each selected point u\in\mathcal{U}_{t}, the future 3D flow target at prediction step j is defined as:

\displaystyle\Delta P_{0}^{(j)}(t,u):=\Delta P_{0}(t+j,u)=P_{0}(t+j+1,\pi_{\mathrm{base}})(u)-P_{0}(t+j,\pi_{\mathrm{base}})(u),

where j\in\{0,\dots,k-1\} and t+k\leq N-1. These selected points and their corresponding future displacements serve as ground-truth supervision signals for the dual-level flow learning objective, guiding the VLA action latent to encode the 3D evolution of the workspace.

3.3 Dual-Level 3D Flow Learning

Point-Level Flow.

The point-level flow branch facilitates the action latent representation in effectively capturing the future motion of dynamically informative 3D points. With the selected keypoint set \mathcal{U}_{t} and the future flow targets \Delta P_{0}^{(j)}(t,u) from Sec. 3.2, the action latent token z_{t+j} is first mapped to a higher-dimensional space and broadcast across the K=|\mathcal{U}_{t}| points as follows:

\displaystyle\mathbf{h}_{t}^{(j)}=\operatorname{Broadcast}\big(g_{\phi}(z_{t+j}),K\big)\in\mathbb{R}^{K\times d_{h}},\quad j\in\{0,\dots,k-1\}.

Here, g_{\phi} is a 2-layer MLP and d_{h} represents the latent dimension. To provide spatial context during training, we collect the 3D coordinates of the selected points at time step t+j as:

\displaystyle\mathbf{P}_{t}^{(j)}=\big[P_{0}(t+j,\pi_{\mathrm{base}})(u)\big]_{u\in\mathcal{U}_{t}}\in\mathbb{R}^{K\times 3}.

We fuse the broadcast action feature with the selected 3D coordinates to obtain \mathbf{F}_{t}^{(j)}=q_{\phi}([\mathbf{h}_{t}^{(j)};\mathbf{P}_{t}^{(j)}]), where q_{\phi} is a point-wise feature extractor. The flow head h_{\mathrm{flow}} predicts the future displacement of each selected point as \Delta\hat{\mathbf{P}}_{0}^{(j)}(t)=h_{\mathrm{flow}}(\mathbf{F}_{t}^{(j)})\in\mathbb{R}^{K\times 3}. These point-wise predictions are supervised by the point-level flow loss:

\displaystyle\mathcal{L}_{\mathrm{point}}=\frac{1}{kK}\sum_{j=0}^{k-1}\sum_{u\in\mathcal{U}_{t}}\rho\!\left(\Delta\hat{P}_{0}^{(j)}(t,u),\Delta P_{0}^{(j)}(t,u)\right),

where \rho(\cdot,\cdot) denotes the Smooth L1 loss. This objective encourages the action latent to reflect scene-level 3D dynamics by predicting the future motion of selected points.

TCP-Level Flow.

While point-level supervision grounds the model in scene dynamics, not all motions are equally relevant to the control task. We aggregate the predicted point flows into a TCP flow estimate to focus on control-relevant dynamics. Importance weights \mathbf{r}_{t}^{(j)} are predicted through a softmax-normalized weighting head h_{\mathrm{weight}} using the fused representation \mathbf{F}_{t}^{(j)}:

\displaystyle\mathbf{r}_{t}^{(j)}=\operatorname{Softmax}_{u}\!\left(h_{\mathrm{weight}}\!\left(\mathbf{F}_{t}^{(j)}\right)\right)\in\mathbb{R}^{K}.

The predicted TCP-level flow is then computed as the weighted sum of point-wise displacements:

\displaystyle\Delta\hat{\mathbf{x}}_{\mathrm{tcp}}^{(j)}(t)=\sum_{u\in\mathcal{U}_{t}}\mathbf{r}_{t}^{(j)}(u)\,\Delta\hat{P}_{0}^{(j)}(t,u)\in\mathbb{R}^{3}.

The TCP-level flow loss is then defined as:

\displaystyle\mathcal{L}_{\mathrm{tcp}}=\frac{1}{k}\sum_{j=0}^{k-1}\rho\!\left(\Delta\hat{\mathbf{x}}_{\mathrm{tcp}}^{(j)}(t),\Delta\mathbf{x}_{\mathrm{tcp}}^{(j)}(t)\right),

where \Delta\mathbf{x}_{\mathrm{tcp}}^{(j)}(t)=\mathbf{x}_{\mathrm{tcp}}(t+j+1)-\mathbf{x}_{\mathrm{tcp}}(t+j) is the ground-truth TCP flow. This scene-to-control distillation encourages the action latent to capture control-relevant dynamics in addition to scene-level motion. The final objective combines the action loss with the auxiliary flow losses:

\displaystyle\mathcal{L}_{\mathrm{total}}=\mathcal{L}_{\mathrm{action}}+\lambda_{\mathrm{point}}\mathcal{L}_{\mathrm{point}}+\lambda_{\mathrm{tcp}}\mathcal{L}_{\mathrm{tcp}},

where \lambda_{\mathrm{point}} and \lambda_{\mathrm{tcp}} are the loss weights. Critically, all auxiliary heads and 3D geometric inputs are discarded at inference. The policy operates through its original 2D-based pathway, benefiting from 3D-aware representation with zero inference-time overhead.

4 SIMULATION EXPERIMENTS

Settings.

The proposed 3D flow supervision is evaluated across heterogeneous policy designs to confirm the generalization ability of FlowVLA. We instantiate FlowVLA on two representative VLAs with substantially different action-generation policies: Seer [26] and \pi_{0.5} [12]. Seer adopts a two-layer MLP-based policy that reconstructs future observations and predicts actions, whereas \pi_{0.5} employs a PaliGemma-based [2] VLM backbone with a flow-matching action policy. For \pi_{0.5}, we follow the official fine-tuning recipe and perform LoRA adaptation [10]. For both backbones, we attach 3D flow objectives only during training while preserving their model architectures and inference pathways. Further details on the simulation settings and datasets are provided in Appendix A.

4.1 Simulation Results

LIBERO.

Tab. 1 summarizes the results on the LIBERO benchmark. Our scene-to-control 3D flow supervision yields consistent improvements on recent VLA models, with particularly large gains in long-horizon manipulation. It is notable that FlowVLA achieves particularly large gains on LIBERO-Long, which requires sustained spatial-temporal reasoning over extended horizons. On Seer [26], our method improves the success rate from 80.4% to 87.8%, a substantial 7.4%p increase. Even on the state-of-the-art \pi_{0.5}, FlowVLA improves the success rate from 92.4% to 94.8%. Importantly, these performance gains are achieved with zero inference-time overhead, as the auxiliary flow heads are entirely decoupled after training.

LIBERO-Plus.

We conduct an additional evaluation of zero-shot robustness on LIBERO-Plus, and report the results in Tab. 2. The average success rate is improved from 36.1% to 41.1% on Seer [26] model. The performance improvements are particularly noticeable for perturbation types such as Camera, Robot, Light, Noise, and Layout. This suggests that reconstruction-derived 3D flow supervision guides the policy to focus on action-relevant 3D dynamics, rather than overfitting to superficial visual appearances. Notably, we also observe a modest gain under the Language perturbation, indicating that the proposed method enhances dynamic motion grounding without compromising the semantic priors acquired from image-text pretraining.

4.2 Ablation Studies

Component Analysis.

We verify the contribution of each design choice through a component ablation study performed on LIBERO-Long (Tab. 3). ❶ Coordinate alignment: Aligning 3D point trajectories across different camera coordinate systems improves performance, indicating that robot-aligned flow targets are important for consistent supervision. ❷ Reference-frame alignment: Using reference-anchored, temporally trackable 3D points further improves performance, supporting their effectiveness for defining reliable 3D flow targets. ❸ Magnitude-based selection: Replacing magnitude-based point selection with random-K selection reduces performance, indicating that directing supervision towards dynamic, motion-salient regions is important for efficient action-relevant learning. ❹ Dual-level flow supervision: Removing either TCP-level flow or point-level flow degrades performance, demonstrating that FlowVLA benefits from both scene-level motion grounding and end-effector-aligned control supervision. Additional qualitative visualizations of the magnitude-based point selection and TCP-level aggregation weights are provided in Appendix C.

Data Efficiency and Computational Cost.

In Fig. 4.1, we analyze FlowVLA in terms of data efficiency and computational cost. FlowVLA consistently outperforms the baseline across all data regimes when trained with 10%, 25%, and 50% of the full dataset. This suggests that utilizing training-only 3D flow supervision provides a useful inductive bias under limited data, allowing for less reliance on superficial visual correlations and better capture of control-relevant scene dynamics by the policy. It is also evident that the computational overhead is kept at a manageable level: the proposed flow heads increase the total number of parameters by only 0.34% and add no inference-time computation. The only cost occurs during training, where total training time increases by approximately 1.26\times. This trade-off is favorable given the consistent gains across evaluated tasks and backbones.

Effect of Supervision Signal.

We compare different supervision targets under the same training and evaluation protocol. Tab. 4 shows that dynamic 3D flow supervision outperforms both 2D optical flow and static 3D point supervision. In contrast to 2D optical flow, 3D flow captures motion in physical space instead of image-plane displacement. This allows it to align more closely with scene-level dynamics and end-effector motion. Unlike static 3D point supervision, 3D flow emphasizes state change rather than static geometry. This supports our motivation for using dynamic 3D flow, which captures temporally varying scene dynamics more directly tied to action. For a fair comparison, all supervision baselines are implemented with the same auxiliary-head capacity and training protocol as FlowVLA.

5 REAL-WORLD EXPERIMENTS

Settings.

We construct a real-world deployment setup with a UR5e platform equipped with third-person and wrist-view cameras. We evaluate four long-horizon manipulation tasks—Cup & Plush, Fruit, Stack Cups, and Cabinet—and three more challenging tasks that require stronger 3D spatial understanding—Hang Cup, Basketball, and Ring Insertion. Zero-shot generalization is also evaluated under viewpoint and height variations. Further details on the setup are provided in Appendix B.

Real-world Results.

To validate real-world manipulation performance, we evaluate FlowVLA on four long-horizon base tasks and three 3D-aware tasks. In Tab. 5, FlowVLA consistently improves baseline performance across the base tasks; notably, on Stack Cups, FlowVLA fails only once out of 15 trials, demonstrating reliable multi-step manipulation. The performance gain becomes more pronounced in the 3D-aware tasks that require precise spatial reasoning and action-relevant 3D dynamics. FlowVLA improves Basketball by approximately 20%p over the baseline and substantially boosts Hang Cup from 46.7% to 86.7%. These results indicate that scene-to-control 3D flow supervision effectively enhances control-relevant 3D scene understanding in real-world manipulation.

Results under Test-Time Variations.

We further evaluate zero-shot robustness under test-time variations. For Basketball, we adjust the angle and height of the third-person camera; the initial ring height is modified for Ring Insertion. As shown in Tab. 6, the baseline fails under viewpoint variation with 0% success, whereas FlowVLA achieves 46.7%. Under height variation in Ring Insertion, FlowVLA improves the success rate from 46.7% to 73.3%, with a reduction in gripper collisions and inaccurate insertions. These results demonstrate that FlowVLA improves both in-distribution performance and zero-shot robustness against real-world spatial variations. A detailed failure case analysis is provided in Appendix C.3.

6 CONCLUSION

We present FlowVLA, a reconstruction-guided 3D flow learning framework that injects action-relevant 3D dynamics into existing VLA models through training-only supervision. By deriving temporally aligned 3D point trajectories from RGB observations into future 3D flow targets, FlowVLA couples scene-level point motion with control-relevant TCP displacement through dual-level flow supervision. Experiments on LIBERO, LIBERO-Plus, and real-world tasks show consistent improvements in performance and robustness across different VLA backbones. FlowVLA notably operates without auxiliary 3D sensors, architectural modifications, or inference-time computation, as all 3D flow modules are used only during training. These results suggest that proposed 3D flow supervision serves as an effective approach to impart action-relevant 3D dynamics to 2D VLA models.

Limitations.

FlowVLA requires an offline reconstruction step to generate 3D point trajectories for training. Although this does not affect inference-time efficiency, the quality of the auxiliary supervision depends on the reconstruction model. In addition, while the visualization in Appendix C.2 suggests that magnitude-based point selection tends to capture dynamically active regions, it does not explicitly guarantee task relevance. This suggests future extensions with semantic or contact-aware point selection.

Acknowledgments

This research was supported by the National Research Foundation of Korea (NRF), Electronics and Telecommunications Research Institute (ETRI), and Institute of Information & Communications Technology Planning & Evaluation (IITP), funded by the Korean government [26CS1100, Development of Proprietary Physical AI-based Small-scale Computers and Integrated Soft Suits] and Korean government (MSIT) (RS-2026-25518808 and RS-2026-25617480).

References

1. [1] M. Ahn, A. Brohan, N. Brown, Y. Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakrishnan, K. Hausman, et al. (2022) Do as i can, not as i say: grounding language in robotic affordances. arXiv preprint arXiv:2204.01691.
2. [2] L. Beyer, A. Steiner, A. S. Pinto, A. Kolesnikov, X. Wang, D. Salz, M. Neumann, I. Alabdulmohsin, M. Tschannen, E. Bugliarello, et al. (2024) Paligemma: a versatile 3b vlm for transfer. arXiv preprint arXiv:2407.07726.
3. [3] K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al. (2024) pi_{0}: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164.
4. [4] A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, J. Hsu, et al. (2022) Rt-1: robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06817.
5. [5] J. Cen, C. Yu, H. Yuan, Y. Jiang, S. Huang, J. Guo, X. Li, Y. Song, H. Luo, F. Wang, et al. (2025) Worldvla: towards autoregressive action world model. arXiv preprint arXiv:2506.21539.
6. [6] S. Chen, R. Garcia, C. Schmid, and I. Laptev (2023) Polarnet: 3d point clouds for language-guided robotic manipulation. arXiv preprint arXiv:2309.15596.
7. [7] Z. Chen, L. Jing, Y. Li, and B. Li (2023) Bridging the domain gap: self-supervised 3d scene understanding with foundation models. Advances in Neural Information Processing Systems 36, pp. 79226–79239.
8. [8] D. Driess, F. Xia, M. S. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu, et al. (2023) Palm-e: an embodied multimodal language model. arXiv preprint arXiv:2303.03378.
9. [9] S. Fei, S. Wang, J. Shi, Z. Dai, J. Cai, P. Qian, L. Ji, X. He, S. Zhang, Z. Fei, et al. (2025) Libero-plus: in-depth robustness analysis of vision-language-action models. arXiv preprint arXiv:2510.13626.
10. [10] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen, et al. (2022) Lora: low-rank adaptation of large language models.. Iclr 1 (2), pp. 3.
11. [11] J. Huang, S. Yong, X. Ma, X. Linghu, P. Li, Y. Wang, Q. Li, S. Zhu, B. Jia, and S. Huang (2023) An embodied generalist agent in 3d world. arXiv preprint arXiv:2311.12871.
12. [12] P. Intelligence, K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, et al. (2025) pi_{0.5}: A vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054.
13. [13] N. Karaev, I. Rocco, B. Graham, N. Neverova, A. Vedaldi, and C. Rupprecht (2024) Cotracker: it is better to track together. In European conference on computer vision, pp. 18–35.
14. [14] M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, et al. (2024) Openvla: an open-source vision-language-action model, 2024. URL https://arxiv. org/abs/2406.09246 1 (2), pp. 4.
15. [15] C. Li, J. Wen, Y. Peng, Y. Peng, and Y. Zhu (2026) Pointvla: injecting the 3d world into vision-language-action models. IEEE Robotics and Automation Letters 11 (3), pp. 2506–2513.
16. [16] P. Li, Y. Chen, H. Wu, X. Ma, X. Wu, Y. Huang, L. Wang, T. Kong, and T. Tan (2025) Bridgevla: input-output alignment for efficient 3d manipulation learning with vision-language models. arXiv preprint arXiv:2506.07961.
17. [17] B. Liu, Y. Zhu, C. Gao, Y. Feng, Q. Liu, Y. Zhu, and P. Stone (2023) Libero: benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems 36, pp. 44776–44791.
18. [18] A. O’Neill, A. Rehman, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jain, et al. (2024) Open x-embodiment: robotic learning datasets and rt-x models: open x-embodiment collaboration 0. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 6892–6903.
19. [19] D. Qu, H. Song, Q. Chen, Y. Yao, X. Ye, Y. Ding, Z. Wang, J. Gu, B. Zhao, D. Wang, et al. (2025) Spatialvla: exploring spatial representations for visual-language-action model. arXiv preprint arXiv:2501.15830.
20. [20] S. Reed, K. Zolna, E. Parisotto, S. G. Colmenarejo, A. Novikov, G. Barth-Maron, M. Gimenez, Y. Sulsky, J. Kay, J. T. Springenberg, et al. (2022) A generalist agent. arXiv preprint arXiv:2205.06175.
21. [21] M. Shukor, D. Aubakirova, F. Capuano, P. Kooijmans, S. Palma, A. Zouitine, M. Aractingi, C. Pascal, M. Russi, A. Marafioti, et al. (2025) Smolvla: a vision-language-action model for affordable and efficient robotics. arXiv preprint arXiv:2506.01844.
22. [22] E. Sucar, E. Insafutdinov, Z. Lai, and A. Vedaldi (2026) V-dpm: 4d video reconstruction with dynamic point maps. arXiv preprint arXiv:2601.09499.
23. [23] L. Sun, B. Xie, Y. Liu, H. Shi, T. Wang, and J. Cao (2025) Geovla: empowering 3d representations in vision-language-action models. arXiv preprint arXiv:2508.09071.
24. [24] G. R. Team, S. Abeyruwan, J. Ainslie, J. Alayrac, M. G. Arenas, T. Armstrong, A. Balakrishna, R. Baruch, M. Bauza, M. Blokzijl, et al. (2025) Gemini robotics: bringing ai into the physical world. arXiv preprint arXiv:2503.20020.
25. [25] O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu, et al. (2024) Octo: an open-source generalist robot policy. arXiv preprint arXiv:2405.12213.
26. [26] Y. Tian, S. Yang, J. Zeng, P. Wang, D. Lin, H. Dong, and J. Pang (2024) Predictive inverse dynamics models are scalable learners for robotic manipulation. arXiv preprint arXiv:2412.15109.
27. [27] H. R. Walke, K. Black, T. Z. Zhao, Q. Vuong, C. Zheng, P. Hansen-Estruch, A. W. He, V. Myers, M. J. Kim, M. Du, et al. (2023) Bridgedata v2: a dataset for robot learning at scale. In Conference on Robot Learning, pp. 1723–1736.
28. [28] J. Wang, M. Chen, N. Karaev, A. Vedaldi, C. Rupprecht, and D. Novotny (2025) Vggt: visual geometry grounded transformer. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 5294–5306.
29. [29] J. Wen, Y. Zhu, M. Zhu, Z. Tang, J. Li, Z. Zhou, X. Liu, C. Shen, Y. Peng, and F. Feng (2025) Diffusionvla: scaling robot foundation models via unified diffusion and autoregression. In Forty-second International Conference on Machine Learning,
30. [30] Y. Xiao, J. Wang, N. Xue, N. Karaev, Y. Makarov, B. Kang, X. Zhu, H. Bao, Y. Shen, and X. Zhou (2025) Spatialtrackerv2: 3d point tracking made easy. arXiv preprint arXiv:2507.12462.
31. [31] R. Xu, X. Wang, T. Wang, Y. Chen, J. Pang, and D. Lin (2024) Pointllm: empowering large language models to understand point clouds. In European Conference on Computer Vision, pp. 131–147.
32. [32] E. Yu, H. Lv, J. Sun, K. Lin, R. Zhang, Y. Shi, Y. Chen, Z. Chen, Z. Zhang, F. Jia, et al. (2026) DM0: an embodied-native vision-language-action model towards physical ai. arXiv preprint arXiv:2602.14974.
33. [33] M. Zawalski, W. Chen, K. Pertsch, O. Mees, C. Finn, and S. Levine (2024) Robotic control via embodied chain-of-thought reasoning. arXiv preprint arXiv:2407.08693.
34. [34] Q. Zhao, Y. Lu, M. J. Kim, Z. Fu, Z. Zhang, Y. Wu, Z. Li, Q. Ma, S. Han, C. Finn, et al. (2025) Cot-vla: visual chain-of-thought reasoning for vision-language-action models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 1702–1713.
35. [35] H. Zhen, X. Qiu, P. Chen, J. Yang, X. Yan, Y. Du, Y. Hong, and C. Gan (2024) 3d-vla: a 3d vision-language-action generative world model. arXiv preprint arXiv:2403.09631.
36. [36] Z. Zhou, P. Wang, Z. Liang, H. Bai, and R. Zhang (2025) Cross-modal 3d representation with multi-view images and point clouds. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 3728–3739.
37. [37] C. Zhu, T. Wang, W. Zhang, J. Pang, and X. Liu (2024) Llava-3d: a simple yet effective pathway to empowering lmms with 3d-awareness. arXiv preprint arXiv:2409.18125.
38. [38] B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, et al. (2023) Rt-2: vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pp. 2165–2183.

Appendix A SIMULATION ENVIRONMENTS

A.1 Datasets

LIBERO.

LIBERO [17] is a standardized benchmark for robot manipulation. It consists of 4 task suites—LIBERO-Spatial, LIBERO-Object, LIBERO-Goal, and LIBERO-Long—where each suite contains 10 tasks and 50 expert demonstrations per task. These suites are designed to evaluate complementary aspects of manipulation capability, including spatial reasoning, generalization to unseen object instances, language-conditioned manipulation, and long-horizon planning.

LIBERO-Plus.

LIBERO-Plus [9] extends LIBERO to provide a more fine-grained evaluation of robustness under distribution shifts. It introduces 7 perturbation dimensions and contains 10,030 test episodes organized across 5 difficulty levels. These perturbations include changes in camera viewpoint, robot initialization, language instructions, lighting, background, layout, and sensor noise, thereby enabling a detailed analysis of zero-shot robustness beyond standard in-distribution evaluation.

A.2 Ablation Study

Magnitude-based Top-K Selection.

We ablate the number of selected motion-salient 3D points under the same training setting by varying K\in\{1024,2048,3072\}. As shown in Tab. 7, K=2048 achieves the best performance. A smaller value, K=1024, may not provide sufficient coverage of scene-level motion, limiting the supervision of diverse motion-relevant dynamics. Conversely, a larger value, K=3072, can include less informative regions such as static backgrounds or reconstruction noise, which weakens the quality of the flow supervision. Nevertheless, FlowVLA maintains strong performance across all tested values of K, indicating that the method is not highly sensitive to this hyperparameter. Thus, K=2048 offers the best trade-off between capturing meaningful 3D motion and filtering out irrelevant points.

Effect of Reconstruction Model for 3D Flow Supervision.

To examine the dependency of FlowVLA on the reconstruction model, we replace VDPM [22] with VGGT [28] and another temporally consistent dynamics tracker, SpatialTrackerV2 [30], for generating 3D flow supervision while keeping the policy architecture, training protocol, and evaluation setting unchanged. All 3D flow supervision settings improve over the baseline, indicating that FlowVLA is not specific to VDPM. However, VGGT performs lower than VDPM- and SpatialTrackerV2-based supervision. We attribute this gap to the fact that VDPM and SpatialTrackerV2 are explicitly designed for temporally consistent dynamic point trajectories, which provide cleaner motion targets for action-relevant 3D flow learning.

Sensitivity to Noisy 3D Flow Supervision.

To evaluate the robustness of FlowVLA to imperfect reconstruction targets, we conduct a noise sensitivity analysis on LIBERO-Long. After selecting K motion-salient sparse points, we randomly corrupt a fixed percentage of selected point trajectories by adding zero-mean Gaussian noise to their 3D coordinates. As shown in Tab. 9, performance decreases as the corruption ratio increases, from 87.8% without noise to 84.7% and 84.0% under 5% and 20% corruption, respectively. Nevertheless, FlowVLA still outperforms the baseline of 80.4% under both settings, indicating that the proposed supervision is not overly sensitive to moderate reconstruction noise. These results also confirm that the quality of reconstructed 3D trajectories remains an important factor for effective flow supervision.

Offline Reconstruction Cost.

For Seer [26] experiments on LIBERO-Long, VDPM [22] preprocessing takes approximately 4 wall-clock hours to generate 3D flow supervision, compared to approximately 13 wall-clock hours for policy training excluding this preprocessing. This offline cost is about 30% of the policy training time, is incurred only once before training, and the generated supervision targets are cached for subsequent experiments. Since the 3D flow supervision is training-only, it does not introduce any additional inference-time computation.

Appendix B REAL-WORLD ENVIRONMENTS

B.1 Real-World Setups

As illustrated in Fig. 5, our real-world experimental setup consists of a single UR5e robotic arm equipped with two RealSense D435i cameras. The real-world evaluation comprises four long-horizon base tasks and three 3D-aware task scenarios. To assess the zero-shot generalization capability of VLA models, we additionally evaluate the models under viewpoint and height variations. For the Basketball task, we build the task environment using a Bambu Lab P1S 3D printer. For collecting training demonstrations, we perform eye-to-hand camera calibration to obtain calibrated third-person observations and normalize all coordinates with respect to the robot base frame.

B.2 Datasets

Base Tasks.

We define four long-horizon manipulation tasks as base tasks. (a) Fruit requires the robot to place a green apple into a pink bowl and an orange into a green bowl. (b) Cup & Plush requires the robot to place a white cup on top of an upside-down pink cup, and then put a blue penguin plush toy into the white cup. (c) Stack Cups requires the robot to stack cups of different sizes in a specified order. (d) Cabinet requires the robot to open the cabinet, swap the object inside with the object outside, and then close the cabinet.

3D-Aware Tasks.

We further design three 3D-aware tasks to assess whether the VLA model can capture action-relevant 3D dynamics and spatial relationships beyond conventional pick-and-place skills. (a) Basketball requires the robot to grasp a basketball and place it into a basketball hoop. (b) Hang Cup requires the robot to grasp a cup and accurately hang its handle on a cup holder. Unlike the base tasks, these scenarios require fine-grained 3D understanding beyond basic pick-and-place manipulation. (c) Ring insertion requires the robot to pick up two rings and precisely insert them onto a vertical pillar by aligning the ring holes with the pillar. We additionally evaluate zero-shot robustness under height and viewpoint variations, where the model is tested under shifted object heights and camera viewpoints without additional fine-tuning.

B.3 Further Analysis

Component Analysis.

We further conduct a component analysis under the viewpoint variation setting. Removing coordinate alignment eliminates the robot-centric alignment of the 3D targets, while removing magnitude-based selection replaces motion-salient point selection with random sampling. We also ablate the point-level and TCP-level losses, which supervise scene-level motion and control-relevant motion, respectively. As shown in Tab. 11, all components contribute to robustness under viewpoint variation. In particular, removing magnitude-based selection or the TCP-level loss reduces the success counts from 7/15 to 0/15 and 2/15, respectively.

Real-World Deployment with \pi_{0.5}.

We further evaluate FlowVLA with the strong baseline [12] on real-world 3D-aware tasks. As shown in Tab. 12, FlowVLA improves the success rate on both Basketball and Hang Cup, from 10/15 to 13/15 and from 9/15 to 14/15, respectively. We additionally evaluate robustness to a dynamic visual environment by introducing a moving background during deployment in the Basketball task. Under this test-time dynamic-background variation, FlowVLA improves the success rate from 5/15 to 8/15, demonstrating improved robustness to dynamic scene changes beyond the static training environment.

Appendix C Qualitative Results

C.1 3D Point Composition Analysis

Fig. 8 illustrates how our motion-based selection and TCP aggregation change the composition of 3D points in LIBERO-Spatial. Here, object points include robot components and scene objects, whereas non-object points correspond to the background. (Panel-❶) shows the 3D points labeled in the MuJoCo simulator, comprising 50k points. (Panel-❷) shows that the proportion of 3D object points, such as grippers and robot arms, becomes substantially higher than that of non-object points, such as the background, after applying our first-stage motion-based selection. We also demonstrate that the second selection stage in the TCP branch further filters out non-object 3D points during the attention operation, as shown in (Panel-❸). These results illustrate that motion-based selection suppresses largely static background regions, while TCP aggregation further emphasizes points associated with robot-object interactions.

C.2 VDPM Visualization

Fig. 9 and Fig. 10 show qualitative visualizations of VDPM [22] 3D point trajectories in LIBERO simulation and real-world scenarios. For visual clarity, we display all reconstructed points in this analysis, whereas FlowVLA uses only the top-K dynamically informative points selected by the flow magnitude during training. The visualization confirms that VDPM can recover temporally consistent 3D point motion from RGB observation sequences. The resulting trajectories highlight motion-salient physical changes, including object displacement and interaction regions around the robot end-effector. These results provide qualitative evidence that reconstruction-derived 3D point trajectories can serve as a useful training-only supervision signal for FlowVLA.

Selected Point Visualization.

We provide a qualitative analysis of the proposed magnitude-based point selection by visualizing the selected K points used for training-time flow supervision. As shown in Fig. 11, the selected points consistently appear around regions undergoing physical motion, such as the TCP, the falling object (the juice in the first frame), and the object interacting with the TCP (the basket in the sixth frame). In contrast, points on static backgrounds and reconstruction-induced noisy regions are largely filtered out. These suggest that motion magnitude serves as a simple yet effective criterion for extracting motion-salient 3D points from reconstruction-derived trajectories, which often coincide with action-relevant regions. Importantly, this selection does not require semantic masks, object annotations, or contact labels, making it a scalable mechanism for constructing 3D dynamics supervision from RGB-only demonstrations.

TCP Trajectory Visualization.

We also visualize the predicted TCP trajectories to assess whether FlowVLA can convert scene-level point motion into control-relevant dynamics. Specifically, we compare the TCP trajectories predicted by the TCP-level aggregation module with the ground-truth TCP trajectories from robot execution. As shown in Fig. 12, the predicted trajectories closely follow the ground-truth trajectories, exhibiting similar motion directions. This qualitative agreement supports our scene-to-control distillation design: the model learns to aggregate dense 3D point displacements into an end-effector-level motion representation. Therefore, the auxiliary TCP-level objective provides a direct bridge between reconstruction-derived scene dynamics and the robot control trajectory.

TCP-Level Aggregation Weight Visualization.

We further visualize the learned aggregation weights of the TCP-level branch to examine how FlowVLA transfers scene-level 3D motion cues into control-relevant dynamics. As shown in Fig. 13, magnitude-based selection identifies K motion-salient 3D points from reconstruction-derived trajectories, and we visualize the top-512 points with the largest TCP-level aggregation weights. The TCP-level aggregation module further refines these scene-level motion cues by assigning larger weights to points around the robot TCP and object–robot interaction region. This indicates that the TCP-level branch learns to distill motion-salient 3D point displacements into end-effector-relevant flow, rather than treating all selected points uniformly. This visualization further supports our scene-to-control distillation design by showing how the TCP-level branch extracts control-relevant motion cues from reconstruction-derived scene dynamics.

C.3 Failure Case Visualization

Fig. 14, Fig. 15, and Fig. 16 visualize representative failure cases in the real-world experiments. With limited demonstration data, the baseline frequently stops execution prematurely or places objects at incorrect locations under variations in the initial object configuration. These failures become more severe in 3D-aware tasks and viewpoint variation, often leading to collisions or emergency stops. In the height variation setting, the baseline also struggles to recognize shifted grasp positions, resulting in missed grasps or collisions with the table surface. FlowVLA substantially mitigates these failure modes. It shows no premature stopping behavior and performs precise manipulation in most 3D-aware scenarios, except for extreme cases where the target object lies near the camera boundary. FlowVLA also achieves more accurate grasping under viewpoint and height variations, significantly reducing collisions and insertion failures. These qualitative results suggest that training-only 3D flow supervision improves the 3D spatial awareness of VLA policies, even with limited demonstration.