Vision-Predictor-Action: Towards Efficient Embodied AI via Neuro-Symbolic JEPA Predictors

Abstract

Vision-Language-Action (VLA) models commonly generate robot actions token by token through Large Language Model (LLM) decoders, so their inference latency grows with the length of the action sequence and limits the achievable closed-loop control frequency. We present the Goal-Conditioned Vision-Predictor-Action (VPA) framework, which replaces the autoregressive decoder with a lightweight latent control loop. A shared vision encoder and a frozen text encoder embed observations, goal images, and instructions; a neuro-symbolic selector chooses a discrete skill; a Joint-Embedding Predictive Architecture (JEPA) predictor forecasts the next latent state; and a flow-matching solver generates an entire action chunk in a few integration steps, so that the number of sequential network evaluations per decision step does not depend on the action horizon. We analyze the inference complexity of this design, its regularization against representation collapse, and the assumptions under which latent sub-goal thresholding can limit error accumulation across task phases. As an initial proof of concept, we implemented VPA in PyTorch and evaluated it on the LIBERO-10 long-horizon manipulation suite. The trainable model has fewer than 20M parameters, and each released checkpoint is under 75 MB, excluding a frozen 63.4M-parameter CLIP text encoder; a decision step of a single-camera reference configuration takes approximately 10\text{--}14 ms on an Apple M4; and, trained only on the LIBERO-10 demonstrations in under three hours on a single GPU, a policy observing the scene through two cameras succeeded in 40.0\% of 200 evaluation episodes. These results come from a single training run and remain below previously reported success rates on this suite, but they suggest that the approach is workable. Code and model weights are available at https://github.com/ylee-sling/Vision-Predictor-Action and https://huggingface.co/ylee-sling/vpa-libero10, and the preprint is archived at https://doi.org/10.5281/zenodo.22960920.

Keywords Embodied AI \cdot Neuro-Symbolic AI \cdot Joint-Embedding Predictive Architecture \cdot Flow Matching \cdot Robotic Manipulation

1 Introduction

Recent advances in embodied artificial intelligence have been largely driven by Vision-Language-Action (VLA) models, which extend web-scale pre-trained backbones to physical robotic manipulation [1, 2]. By mapping visual observations and natural language instructions into discrete token spaces, these systems exhibit strong semantic generalization across diverse tasks [3]. However, many current VLA architectures emit actions through autoregressive Large Language Model (LLM) decoders [4]. When continuous actions are discretized into tokens, generating an H-step action chunk of dimension d_{a} requires N=q\cdot H\cdot d_{a} tokens, where q denotes the number of tokens per scalar action component, and hence \Theta(N) sequential forward passes through a multi-billion-parameter Transformer per execution cycle (Section 5.1). For RT-2, for example, real-time inference rates of about 1\text{--}5 Hz have been reported [1], which restricts the ability of autoregressive VLAs to execute reactive, dynamic maneuvers or to compensate for unexpected physical perturbations in real time.

To mitigate these latency constraints, hybrid approaches such as NS-VLA [4] introduce discrete neuro-symbolic skill pointers, while VLA-JEPA [5] incorporates joint-embedding predictive architectures (JEPA) [6, 7] to model latent world dynamics [8, 9]. Despite these improvements, such systems remain coupled to large causal vision-language backbones (e.g., Qwen3-VL-2B) that consume substantial computational resources, and their high-level plans are expressed as predefined action symbols (e.g., pick, place_on) that constitute the interface between reasoning and control. Concurrently, non-autoregressive trajectory generators based on Action Chunking with Transformers (ACT) [10], Diffusion Policies [11], and Conditional Flow-Matching (CFM) [12, 13, 14] have demonstrated high-frequency motor execution (30\text{--}100 Hz). These methods, however, typically either operate as reactive visual policies without forward-predictive dynamics or explicit sub-goal tracking, or, when language-conditioned as in \pi_{0} [14], retain a large vision-language backbone for instruction processing.

In this work, we present the Goal-Conditioned Vision-Predictor-Action (VPA) framework, which replaces autoregressive language decoding with a non-autoregressive latent control loop. As summarized in Figure 1, our reference implementation, trained only on the LIBERO-10 demonstrations, has fewer than 20M trainable parameters (a checkpoint of under 75 MB), runs a decision step in approximately 10\text{--}14 ms on an Apple M4 for a single-camera reference configuration, and reaches a promising initial success rate of 40.0\% on LIBERO-10. By substituting the LLM decoder—the dominant source of sequential latency in token-based VLA models—with a lightweight JEPA predictor, VPA reduces both the parameter count and the sequential depth of the per-step computation. Rather than communicating through text tokens, the predictor supplies forecasted state-transition latents directly to the downstream action solver, which generates continuous and potentially multi-modal action distributions within the skill selected by a discrete primitive. VPA thus retains a symbolic vocabulary for high-level skill selection, but confines it to the choice of which skill is executed, while the motor trajectory itself is synthesized in a continuous space. We note that this design does not remove the symbolic bottleneck entirely: the set of high-level behaviors that VPA can compose is bounded by its primitive dictionary, and zero-shot generalization to skills outside this dictionary is not addressed by the present formulation. As illustrated in Figure 2, VPA decouples high-level task logic from low-level continuous motor control across a four-stage pipeline:

1. Latent Multimodal Goal Encoding (E_{\psi} and Text Encoder): A non-generative, self-supervised vision encoder [15, 16] maps sensory frames into a compact latent space \mathcal{Z}\subset\mathbb{R}^{d}, in which Euclidean distances \|z_{t}-z_{g}\|_{2} are intended to serve as proxies for physical task progress, an idea related to the latent shaping potential of NS-VLA [4], which uses negative Euclidean distances to learned primitive prototypes. Concurrently, a lightweight, frozen text encoder (e.g., CLIP [17]) embeds the user’s natural language instruction x into a dense vector c_{\text{text}}, injecting the overarching task intent without requiring an LLM decoder.
2. Neuro-Symbolic Primitive Selector (\pi_{\phi}^{h}): A non-autoregressive classifier selects the active primitive token u_{t}\in\mathcal{U} from a fixed dictionary of N_{u} atomic skills, conditioned on the current latent z_{t}, the milestone latent z_{g}^{(m_{t})}, and the text embedding c_{\text{text}}. This multimodal conditioning allows the selector to disambiguate task intent (e.g., “move the cup” versus “throw it away”). Milestone progression is governed by a symbolic transition rule [18] based on the latent threshold test \|z_{t}-z_{g}^{(m_{t})}\|_{2}<\tau, where \tau is scaled by the running latent variance margin maintained by the VICReg objective rather than fixed a priori (Section 4.2).
3. JEPA Predictor (P_{\omega}): A lightweight predictive dynamics model forecasts the next latent state together with a scalar uncertainty estimate, (\hat{z}_{t+1},\sigma_{t+1})=P_{\omega}(z_{t},u_{t}), and is regularized against representation collapse via Variance-Invariance-Covariance Regularization (VICReg) [19]. The predictor is trained with conditioning on discrete primitive tokens and subsequently frozen, so that the downstream solver is trained against a stationary conditioning distribution rather than against latent inputs that drift as the representation evolves.
4. Flow-Matching Action Solver (\pi_{\theta}^{l}): A conditional vector field v_{\theta} synthesizes a continuous action chunk A_{t}\in\mathbb{R}^{H\times d_{a}} jointly over all H steps using K\in\{1,2,3\} numerical ODE integration steps [12], of which a prefix of length H_{t}\leq H, adapted to the predictor’s uncertainty \sigma_{t+1}, is executed.

Implementation and Initial Validation. We implemented VPA in PyTorch and evaluated it as an initial proof of concept on LIBERO-10, the long-horizon suite of the LIBERO manipulation benchmark [20]. The implemented equations (Eqs. 7–15, 18, and 19) were each verified against independent double-precision reference computations, and the sequence of network evaluations in every decision step was audited at runtime to confirm the sequential depth of K+3 stated in Proposition 5.1. The trainable modules E_{\psi}, \pi_{\phi}^{h}, P_{\omega}, and v_{\theta} together contain fewer than 20M parameters, so that each released checkpoint occupies under 75 MB; the frozen CLIP text encoder contributes a further 63.4M parameters, is downloaded separately, and lies outside the per-step dependency chain, because c_{\text{text}} is computed once per episode. Trained only on the LIBERO-10 demonstrations, with a randomly initialized vision encoder, K=2, and H=16, and in under three hours on a single NVIDIA L4 GPU, the trained policy succeeded in 40.0\% of 200 evaluation episodes (20 per task; 95\% Wilson interval 33.5\text{--}46.9\%) when the observation I_{t} comprised a third-person and a wrist-mounted camera view, compared with 10.5\% (7.0\text{--}15.5\%) with the third-person view alone. Using two views changes only the input of E_{\psi} and leaves all subsequent equations unchanged (Section 6). On an Apple M4 at batch size one, a complete decision step of the single-camera reference configuration took 10.5\text{--}14.2 ms for K\in\{1,2,3\} and H\leq 64.

We wish to state the scope of these results plainly. They derive from a single training run per setting and 20 trials per task, so the success rates carry uncertainties of roughly \pm 7 percentage points. They remain below the success rates previously reported on this suite for existing policies, including diffusion-based policies trained without large-scale pre-training [2], and they do not constitute a controlled comparison with other methods. Latency was measured for the network pipeline in isolation; closed-loop control frequency on physical hardware, robustness to dynamic perturbations, and the encoder regularity constants of Section 5.3 were not evaluated and remain future work (Section 6.5). The experiments also used a single goal image per task and primitive labels derived heuristically from the gripper command, so the multi-milestone sequencing mechanism was not exercised. We therefore regard the present results as initial evidence that the latent control loop can be trained on standard demonstration data and executed at low latency on commodity hardware, rather than as evidence of competitive task performance.

Open-Source Release. To support reproduction and further investigation, we release the PyTorch reference implementation, including the training, evaluation, and latency-benchmarking code, together with the trained LIBERO-10 checkpoints; the links are given in Section 1.1.

Summary of Contributions:

• Horizon-Independent Sequential Depth: We show that replacing autoregressive token generation with flow-matching action chunking reduces the number of sequential network evaluations per control cycle from \Theta(N), with N=q\cdot H\cdot d_{a}, to K+3=\mathcal{O}(K), a quantity independent of the action horizon H. The arithmetic cost of each solver evaluation nevertheless grows with the chunk dimensionality H\cdot d_{a}; wall-clock latency is therefore approximately horizon-independent only while this computation fits within the available hardware parallelism. Latency measurements of the reference implementation were consistent with this analysis: on an Apple M4, the encoder, selector, and predictor times remained approximately constant across H, and the time of one solver evaluation, t_{l}(H), remained within 10\% of its value at H=8 up to H=64 before rising by 23\text{--}27\% at H=128. For H\leq 64, the summed network time of Eq. 17 was 6.2\text{--}9.5 ms, and the complete decision step, including non-network operations and timing synchronization, took 10.5\text{--}14.2 ms.
• Anti-Collapse Representation Analysis: We characterize the conditions under which the variance hinge penalty v(Z) and the covariance decorrelation penalty c(Z) exclude the constant (fully collapsed) representation from the global minimizers of the joint-embedding objective, without relying on pixel-reconstruction decoders. We further use the realized variance margin to calibrate the latent completion threshold \tau, so that \tau remains commensurate with the scale of the latent space throughout training.
• Conditional Error Containment Across Task Phases: We make explicit the encoder regularity conditions under which discrete sub-goal thresholding \|z_{t}-z_{g}^{(m_{t})}\|_{2}<\tau limits the physical deviation at each phase transition. These are an upper Lipschitz constant L_{E}, which can be enforced with respect to the input image via spectral normalization [21] and governs whether the transition is triggered as the physical state approaches the milestone, and a lower (co-Lipschitz) constant \mu_{E} with respect to task-relevant physical state, which bounds the physical deviation at transition by \tau/\mu_{E}. Because standard Vision Transformers are not globally Lipschitz [22], and because the lower constant depends on the relation between images and physical state and cannot be enforced through architectural constraints alone, we present this error containment as an empirical expectation to be verified rather than as a formal guarantee; the LIBERO-10 experiments reported in this work do not yet estimate L_{E} or \mu_{E}.
• Open-Source Reference Implementation and Proof of Concept: We provide a PyTorch implementation of the complete pipeline, verified equation by equation against independent reference computations, and report preliminary closed-loop results on LIBERO-10, together with ablations over the camera views and the executed horizon H_{t} (Section 6). The code and trained checkpoints are publicly available (Section 1.1).

1.1 Code and Model Availability

The reference implementation, the trained checkpoints, and the archived preprint are publicly available:

• GitHub: https://github.com/ylee-sling/Vision-Predictor-Action PyTorch source code for the four stages of the pipeline, the two-stage training and closed-loop LIBERO evaluation scripts, the latency benchmark of Section 5.1, per-module self-tests that compare each implemented equation with an independent reference computation, and the LIBERO-10 evaluation results of Section 6. Evaluation videos for three trials per task and model are attached to the repository’s release page.
• Hugging Face: https://huggingface.co/ylee-sling/vpa-libero10 The one-camera and two-camera LIBERO-10 checkpoints reported in Section 6, in PyTorch and safetensors formats, together with their model configurations.
• Zenodo: https://doi.org/10.5281/zenodo.22960920 The archived record of this preprint, from which later versions are also accessible.

The code and the checkpoints are released under the GNU General Public License, version 3 (GPL-3.0-only); the checkpoints were trained on the LIBERO datasets [20], whose terms also apply. The preprint is distributed under the Creative Commons Attribution 4.0 International license (CC BY 4.0). The models were trained on Linux with an NVIDIA L4 GPU (Section 6.1). The self-tests and latency benchmarks were run on macOS 15 with an Apple M4 processor (MPS backend), Python 3.13, and PyTorch 2.14.1; the multi-camera (V=2) self-tests and the single-camera regression test were also run on CPU on the Linux training instance, with the CUDA build of PyTorch 2.14.1. We would be grateful for reports of errors or of attempts to reproduce the results.

2 Related Work

2.1 Vision-Language-Action Models

Recent advancements in embodied artificial intelligence have been largely driven by Vision-Language-Action (VLA) models, such as RT-2 [1], OpenVLA [2], and Instruct2Act [3]. These architectures extend web-scale pre-trained backbones to map visual observations and natural language instructions into multi-modal action spaces. However, VLAs that emit discretized action tokens through an autoregressive decoder incur an inherent sequential bottleneck: generating a multi-step action chunk requires N sequential forward passes through a multi-billion-parameter Transformer decoder, where N grows linearly with the action horizon, resulting in \Theta(N) sequential decoding steps per execution cycle. For RT-2, real-time inference rates of about 1\text{--}3 Hz for the 55B-parameter model and about 5 Hz for its 5B-parameter variant have been reported [1]; such rates reduce the effectiveness of autoregressive VLAs for real-time dynamic manipulation and for compensating physical perturbations. Generalist policies that replace token decoding with continuous generative action heads, such as Octo [23] and \pi_{0} [14], partially alleviate this bottleneck; we discuss these approaches in Section 2.4.

2.2 Neuro-Symbolic Robotics

To combine high-level semantic reasoning with continuous motor execution, neuro-symbolic robotics incorporates formal logic and discrete task structures. Classical approaches utilize Linear Temporal Logic (LTL) and automata to enforce reactive mission planning and safety constraints [18]. More recently, hybrid frameworks like NS-VLA [4] introduce discrete skill pointers to decompose complex manipulation tasks into atomic primitives. Nevertheless, existing neuro-symbolic VLA architectures continue to rely on causal Vision-Language Models (VLMs) or autoregressive text parsers for instruction processing and high-level logic; consequently, they inherit the sequential inference latency bottleneck, which limits high-frequency closed-loop execution. Concurrent work has also begun to examine neuro-symbolic formulations of JEPA, including approaches that couple knowledge and rule embeddings with joint-embedding prediction [24] and hybrid symbolic-neural latent predictors [25]. Our framework builds upon the principles of Neuro-Symbolic Joint-Embedding Predictive Architecture (NS-JEPA) [26], replacing autoregressive language parsers with a non-autoregressive primitive selector coupled directly to a latent predictive world model. We emphasize that the symbolic component of VPA is deliberately minimal: its transition rule (Section 4.2) realizes a monotone sequential automaton over a fixed primitive dictionary rather than a general temporal-logic specification, and it therefore inherits the expressivity limits of that dictionary.

2.3 Predictive World Models and Joint-Embedding Architectures

In contrast to generative world models that reconstruct high-dimensional visual pixels, Joint-Embedding Predictive Architectures (JEPAs) [6, 7] forecast future environment dynamics directly within abstract latent spaces. Because they dispense with pixel-level reconstruction decoders, JEPAs are not required to allocate model capacity to stochastic visual nuisances such as dynamic lighting and background shifts. Recent works, including LeWorldModel [8] and hierarchical latent planning frameworks [9], construct stable, non-collapsing latent spaces for world modeling and planning. The regularizers employed in this line of work range from variance–covariance penalties such as VICReg [19] to distribution-matching objectives that encourage isotropic Gaussian embeddings, as in LeJEPA [27]. At scale, V-JEPA 2 [28] shows that an action-conditioned latent predictor, post-trained on a modest amount of robot interaction data, can support zero-shot manipulation through planning in representation space, albeit at the cost of iterative optimization over action sequences at inference time.

In robotic control, joint-embedding mechanics have been applied both as predictive dynamics models and as visual feature extractors. VLA-JEPA [5] introduces a unified pretraining framework with an online, time-causal latent world model to forecast future states. Conversely, JEPA-VLA [29] argues that standard visual backbones lack temporal policy priors and employs frozen video-predictive JEPA features as plug-and-play visual representations inside existing VLA architectures via gated cross-attention. Additionally, ACT-JEPA [30] employs latent prediction to regularize continuous imitation learning policies. While these approaches enhance visual representations or latent forward modeling, they either lack explicit, deterministic state-machine mechanisms for sub-goal verification or remain coupled to latency-heavy, text-conditioned causal language backbones.

2.4 Diffusion and Flow-Matching Policies

To enable high-frequency continuous control, non-autoregressive trajectory generation methods have emerged as an effective alternative. Action Chunking with Transformers (ACT) [10] predicts multi-step continuous action trajectories A_{t}\in\mathbb{R}^{H\times d_{a}} in a single forward pass, which reduces the effective number of decisions per episode and has been shown to mitigate compounding errors in imitation learning. Building upon this, Diffusion Policies [11] and Conditional Flow Matching (CFM) [12, 13] formulate action-chunk generation as iterative denoising or as transport along a learned ODE vector field, respectively, enabling expressive multi-modal trajectory sampling; straight or nearly straight transport paths further permit accurate integration in few steps [31]. When combined with action chunking, these continuous solvers support real-time execution rates (30\text{--}100 Hz). However, they typically function either as reactive visual policies that lack long-horizon predictive dynamics, symbolic verification, or explicit sub-goal state tracking, or as action heads attached to generalist transformer backbones [23], which in the case of \pi_{0} [14] is a pre-trained vision-language model with billions of parameters. VPA adopts a CFM solver for low-level generation but conditions it on forecasted latent transitions and a lightweight text embedding, rather than on the hidden states of a large language backbone.

3 Problem Formulation

We formulate visual-goal robotic manipulation as a Goal-Conditioned Markov Decision Process (G-MDP) defined by the tuple \mathcal{M}=(\mathcal{S},\mathcal{G},\mathcal{A},\mathcal{P},\mathcal{R},\gamma_{\text{disc}}), where \mathcal{S} represents the continuous state space, \mathcal{G} denotes the multimodal goal space, \mathcal{A} is the continuous action-chunk space, \mathcal{P}:\mathcal{S}\times\mathcal{A}\rightarrow\Delta(\mathcal{S}) represents the stochastic chunk-level transition dynamics, \mathcal{R} is the goal-conditioned (milestone-dependent) reward function, and \gamma_{\text{disc}}\in[0,1) is the discount factor. We write \gamma_{\text{disc}} to distinguish the discount factor from the VICReg variance margin \gamma introduced in Section 5.2.

3.1 State, Observation, and Goal Spaces

At control step t, the system state s_{t}\in\mathcal{S} pairs raw visual sensory inputs with low-level kinematic feedback:

s_{t}=\left(I_{t},S_{t}\right)\in\mathcal{S}

where I_{t}\in\mathbb{R}^{H_{\text{img}}\times W_{\text{img}}\times C} denotes the raw RGB observation frame with height H_{\text{img}}, width W_{\text{img}}, and C channels, while S_{t}\in\mathbb{R}^{d_{s}} represents the instantaneous robot proprioceptive state (e.g., joint positions, joint velocities, and end-effector pose).

The pair s_{t} is an observation of an underlying physical configuration that is not directly accessible. We denote by y_{t}\in\mathcal{Y}\subset\mathbb{R}^{d_{y}} the task-relevant physical state (e.g., the robot joint configuration and the poses of manipulated objects) and model image formation as I_{t}=\mathcal{I}(y_{t},\eta_{t}), where \mathcal{I} is the image-formation (rendering) map and \eta_{t} collects task-irrelevant nuisance factors such as illumination, background content, and sensor noise. Following common practice, we treat s_{t} as the agent’s state, implicitly assuming that it is approximately Markovian for the control task; the distinction between y_{t} and I_{t} is made explicit only in the analysis of Section 5.3.

To form a Goal-Conditioned VPA framework, the overarching task intent is specified by natural language alongside visual targets. The multimodal goal space \mathcal{G} is defined by a tuple pairing a user language instruction x with target goal frames I_{g}:

g=(x,I_{g})\in\mathcal{G}

The language string x provides the semantic intent of the task. For multi-stage compositional tasks, the visual component I_{g} is an ordered sequence of M visual milestone goals (I_{g}^{(1)},I_{g}^{(2)},\dots,I_{g}^{(M)}), with I_{g}^{(m)}\in\mathbb{R}^{H_{\text{img}}\times W_{\text{img}}\times C}; single-stage tasks correspond to M=1. Each milestone frame is an observation of a target physical configuration, I_{g}^{(m)}=\mathcal{I}(y_{g}^{(m)},\eta_{g}^{(m)}), and its latent embedding is denoted z_{g}^{(m)}=E_{\psi}(I_{g}^{(m)}).

3.2 Action Space and Action Chunking

To account for sensorimotor latency and to reduce single-step compounding execution drift, the controller operates over continuous multi-step action chunks rather than instantaneous atomic commands. Let \mathcal{A}_{0}\subseteq\mathbb{R}^{d_{a}} denote the per-step action space, where d_{a} is the dimensionality of the robot’s action space (e.g., 7-DoF joint command vectors alongside 1-DoF parallel-gripper actuation). At step t, the controller selects an open-loop action chunk over horizon H:

A_{t}=(a_{t},a_{t+1},\dots,a_{t+H-1})\in\mathcal{A}=\mathcal{A}_{0}^{H}\subseteq\mathbb{R}^{H\times d_{a}}

As detailed in Section 4.4, the solver always generates a chunk of length H, of which an uncertainty-dependent prefix of length H_{t}\leq H is executed before the inference pipeline is invoked again. We refer to the control steps at which the pipeline is invoked as decision steps; the primitive u_{t}, the milestone pointer m_{t}, and the conditioning vector e_{t} are evaluated at decision steps and held fixed during chunk execution.

3.3 Transition Dynamics and Latent Metric Reward

The ground-truth physical dynamics of the environment govern state transitions at the chunk level according to:

s_{t+H}\sim\mathcal{P}\left(\cdot\mid s_{t},A_{t}\right)

where \mathcal{P} is induced by composing the per-step dynamics over the actions of the chunk (or over its executed prefix of length H_{t}).

Rather than predicting high-dimensional pixel transitions p(I_{t+H}\mid I_{t},A_{t}) directly, the dynamics are modeled in a compact latent space \mathcal{Z}\subset\mathbb{R}^{d} obtained via the visual encoder E_{\psi}. Within \mathcal{Z}, the latent transition dynamics are modeled by the forward predictor P_{\omega} under the active discrete primitive token u_{t}\in\mathcal{U}:

z_{t}=E_{\psi}(I_{t}),\qquad(\hat{z}_{t+1},\sigma_{t+1})=P_{\omega}(z_{t},u_{t})

where \sigma_{t+1}\geq 0 is a scalar predictive uncertainty (Section 4.3). The predictor operates at a fixed temporal stride of \nu\in\mathbb{N} control steps, and we assume \nu\leq H_{\min}, where H_{\min} is the minimum executed chunk length (Section 4.4), so that the forecast horizon falls within every executed chunk. For notational brevity, we write z_{t+1} and \hat{z}_{t+1} for the actual and forecast latents z_{t+\nu} and \hat{z}_{t+\nu} throughout. Because P_{\omega} is conditioned on the primitive u_{t} rather than on the action chunk A_{t}, it models the expected latent consequence of executing a skill, as represented in the demonstration data, rather than the consequence of a specific motor trajectory; the motor-level realization is delegated to the solver of Section 4.4.

The sparse goal-reaching objective is approximated by a dense latent proxy reward based on the Euclidean distance to the active milestone:

\mathcal{R}(s_{t},g;m_{t})=-\left\|E_{\psi}(I_{t})-E_{\psi}\big(I_{g}^{(m_{t})}\big)\right\|_{2}=-\left\|z_{t}-z_{g}^{(m_{t})}\right\|_{2}

where m_{t}\in\{1,\dots,M\} is the active milestone pointer. Since \mathcal{R} depends on m_{t}, the pointer acts as a discrete component of an augmented state (s_{t},m_{t}). In the present formulation, \mathcal{R} serves as an analysis and monitoring signal rather than as an optimization target, as all components of VPA are trained by self-supervised latent prediction or by imitation of demonstrations (Section 4). Whether this proxy is monotonically related to physical task progress depends on the regularity of E_{\psi} with respect to y_{t}, which we examine in Section 5.3.

Milestone m_{t} is regarded as completed when the latent distance falls below a completion threshold \tau>0, such that \|z_{t}-z_{g}^{(m_{t})}\|_{2}<\tau. Because the scale of \mathcal{Z} is not fixed by the joint-embedding objective, \tau is calibrated relative to the running latent variance margin rather than set to an absolute constant (Section 4.2). The task is complete when this condition is satisfied for the final milestone m_{t}=M.

4 The Goal-Conditioned VPA Architecture

Current Vision-Language-Action (VLA) architectures that decode actions autoregressively are constrained by a sequential computing bottleneck on control frequency. To overcome this, we introduce the Goal-Conditioned Vision-Predictor-Action (VPA) framework. The architecture bypasses large language model (LLM) backbones and formulates robotic manipulation as a closed-loop latent prediction problem.

The architecture decouples high-level task logic from low-level continuous motor control through a four-stage pipeline: (1) a latent multimodal goal encoder that projects raw observations and text into a compact latent space; (2) a non-autoregressive neuro-symbolic selector that discretizes task primitives; (3) a collapse-regularized latent predictor that forecasts state transitions; and (4) a continuous flow-matching solver that outputs multi-step action trajectories. The number of sequential network evaluations per decision step in this pipeline does not depend on the action horizon (Section 5.1), which keeps the per-step computation short, as reactive control requires, while retaining semantic task specification. The complete data flow is illustrated in Figure 3.

4.1 Latent Multimodal Goal Encoding

The perception module in the VPA framework is designed to retain task-relevant spatial and physical configurations while suppressing high-frequency visual nuisance factors \eta_{t} (e.g., lighting variations, reflective glare, or dynamic backgrounds). Let I_{t}\in\mathbb{R}^{H_{\text{img}}\times W_{\text{img}}\times C} denote the current observation frame at timestep t, and let I_{g}^{(m)}\in\mathbb{R}^{H_{\text{img}}\times W_{\text{img}}\times C} denote the visual goal frame of milestone m\in\{1,\dots,M\}.

We employ a shared vision encoder E_{\psi} (e.g., a Vision Transformer backbone [15]), which operates as a Siamese network to independently map the high-dimensional image frames into a compact, continuous latent embedding space \mathcal{Z}\subset\mathbb{R}^{d}:

z_{t}=E_{\psi}(I_{t}),\qquad z_{g}^{(m)}=E_{\psi}\big(I_{g}^{(m)}\big),\quad m=1,\dots,M

where z_{t} is intended to encode the current configuration of the robot end-effector and scene objects, and z_{g}^{(m)} the target configuration of milestone m. The milestone embeddings depend only on the task specification and can be computed once per episode.

To capture the overarching task intent without relying on a computationally heavy LLM, we introduce a lightweight, frozen text encoder (e.g., CLIP [17] or a distilled RoBERTa [32]). This encoder embeds the user’s natural language instruction x into a dense text vector c_{\text{text}}\in\mathbb{R}^{d_{c}}, which is likewise computed once per episode.

Unlike standard autoencoders or generative models that minimize a pixel-level reconstruction loss \mathcal{L}_{\text{MSE}}(I_{t},\hat{I}_{t}), E_{\psi} is optimized via a joint-embedding predictive objective (Section 4.3). Because this objective does not require the representation to retain information that is unpredictable from one frame to the next, it encourages, but does not guarantee, invariance to the nuisance factors \eta_{t} and a focus on predictable physical quantities such as kinematics, spatial geometry, and relative object distances. The latent distance \|z_{t}-z_{g}^{(m)}\|_{2} is accordingly intended to serve as a proxy for physical task progress; the extent to which it does so is governed by the regularity of the composite map y\mapsto E_{\psi}(\mathcal{I}(y,\eta)) from physical state to latent, which we analyze in Section 5.3. In the base formulation, no explicit Lipschitz constraint is imposed on E_{\psi}; the consequences of this choice, and the role of optional spectral normalization, are discussed in the same section. The text embedding c_{\text{text}} provides the semantic conditioning for the discrete primitive transitions and the continuous action generation modeled in subsequent stages.

4.2 Neuro-Symbolic Primitive Selector (\pi_{\phi}^{h})

A fundamental limitation of autoregressive VLA models is their reliance on computationally heavy text generation to dictate sequential task logic. The VPA architecture instead formulates high-level reasoning as a discrete, latent-conditioned classification problem governed by a symbolic finite-state machine. The per-decision cost of this stage is a single classifier evaluation, independent of the number of milestones M.

Let \mathcal{U}=\{u^{(1)},u^{(2)},\dots,u^{(N_{u})}\} define a pre-determined dictionary of N_{u} executable atomic skills, or “primitives” (e.g., reach, grasp, push, place). To handle multi-stage tasks, the overarching visual goal is specified as a sequence of milestones, represented in the latent space as (z_{g}^{(1)},z_{g}^{(2)},\dots,z_{g}^{(M)}) (Eq. 7).

We introduce a discrete, monotonically non-decreasing milestone pointer m_{t}\in\{1,2,\dots,M\} that tracks the active sub-goal at decision step t. The neuro-symbolic primitive selector is a lightweight classifier \pi_{\phi}^{h} (parameterized by a Multi-Layer Perceptron) that maps the current state embedding, the active milestone embedding, and the text embedding c_{\text{text}} directly to a categorical primitive token u_{t}:

u_{t}=\argmax_{u\in\mathcal{U}}\;\pi_{\phi}^{h}\big(u\mid z_{t},z_{g}^{(m_{t})},c_{\text{text}}\big)

The selector is trained by minimizing the cross-entropy between \pi_{\phi}^{h} and the primitive labels provided by the demonstration data (Figure 4). By incorporating c_{\text{text}} as a conditioning variable, the selector can disambiguate task intent (e.g., whether the instruction is to “move the cup” or to “throw the cup away”) without generating text tokens. It acts as a reactive policy that selects the skill expected to satisfy both the semantic instruction and the geometric goal by reducing the latent discrepancy between z_{t} and z_{g}^{(m_{t})}.

The progression of the task is governed by a symbolic transition rule that tests sub-goal completion via an \ell_{2}-norm threshold in \mathcal{Z}. A threshold \tau fixed in absolute units, however, is ill-suited to a jointly trained latent space: the variance hinge of the VICReg objective (Eq. 18) penalizes per-dimension standard deviations below the margin \gamma but does not bound them from above, so the overall scale of \mathcal{Z} can drift during training, and a static \tau would become relatively looser or stricter as training proceeds. We therefore scale \tau with a running estimate of the realized latent variance margin. Let Z^{(n)}\in\mathbb{R}^{B\times d} denote the batch of online-encoder latents at training iteration n (Section 5.2). The milestone pointer is updated at each decision step, and the threshold is calibrated during training, according to:

\displaystyle m_{t+1} \\ \displaystyle=\begin{cases}m_{t}+1,&\text{if }\|z_{t}-z_{g}^{(m_{t})}\|_{2}<\tau\text{ and }m_{t}<M,\\
m_{t},&\text{otherwise},\end{cases} \\ \displaystyle\bar{\gamma}_{n} \\ \displaystyle=(1-\alpha_{\gamma})\,\bar{\gamma}_{n-1}+\alpha_{\gamma}\sqrt{\frac{1}{d}\sum_{j=1}^{d}\Var\big(Z^{(n)}_{\cdot,j}\big)+\epsilon}, \\ \displaystyle\tau_{n} \\ \displaystyle=\kappa\sqrt{2d}\;\bar{\gamma}_{n},

where \alpha_{\gamma}\in(0,1] is an exponential-moving-average rate, \epsilon is the numerical constant of Eq. 18, and \kappa\in(0,1) is a dimensionless tolerance. The scaling in Eq. 9c has a direct interpretation: for two independent latents z_{a},z_{b} drawn from the batch distribution, \mathbb{E}\|z_{a}-z_{b}\|_{2}^{2}=2\operatorname{tr}\big(\mathrm{Cov}(Z)\big), so that \sqrt{2d}\,\bar{\gamma}_{n} approximates (up to \epsilon) the root-mean-square distance between independently drawn states, and \kappa expresses the completion tolerance as a fraction of this distance. Moreover, whenever the variance hinge is inactive on the current batch, every per-dimension term satisfies \Var(Z_{\cdot,j})+\epsilon\geq\gamma^{2}, so the instantaneous estimate in Eq. 9b is at least \gamma. Because \bar{\gamma}_{n} is a moving average, the threshold is bounded below by \kappa\sqrt{2d}\,\gamma only if the hinge was inactive on every batch that contributed to \bar{\gamma}_{n} and \bar{\gamma}_{0}\geq\gamma; under these conditions, the VICReg margin anchors the scale of \tau. Whenever the transition rule is evaluated during training (e.g., to segment demonstrations into milestone phases), \tau=\tau_{n} is used. Once E_{\psi} is frozen (Section 4.3), \bar{\gamma} is fixed to its final value \bar{\gamma}^{\star}, and deployment uses the constant threshold \tau=\tau^{\star}=\kappa\sqrt{2d}\,\bar{\gamma}^{\star}. When the test in Eq. 9a is satisfied for m_{t}=M, the task is declared complete and the episode terminates.

This neuro-symbolic formulation yields two structural properties. First, the selector output is restricted to the finite set \mathcal{U}, which precludes the generation of non-executable or out-of-vocabulary commands. Second, the threshold test turns sub-task completion into an explicit, checkable predicate on the latent state. This predicate does not by itself guarantee that the physical preconditions of the next primitive are satisfied, which depends on the regularity of E_{\psi} (Section 5.3), nor does it guarantee progress: if a milestone becomes unreachable, the pointer remains at m_{t} indefinitely. A per-milestone time-out that triggers primitive re-selection can be used to detect such stalls; its design is deferred to the evaluation phase (Section 6.5).

4.3 Latent Predictor (P_{\omega})

To generate kinematically stable and physically grounded continuous actions, the control system anticipates the consequences of its decisions. While generative world models achieve this by predicting future pixel observations \hat{I}_{t+1}, pixel-level reconstruction can be computationally demanding for high-frequency control loops and may allocate model capacity toward stochastic visual nuisances (e.g., dynamic shadows or moving backgrounds) rather than relevant physical dynamics.

To address this, the VPA framework introduces a predictor P_{\omega}, parameterized as a multi-layer perceptron or causal transformer, that operates within the low-dimensional latent space \mathcal{Z}. Given the current latent state z_{t} and the selected active primitive u_{t}, the predictor forecasts both the expected future latent state \hat{z}_{t+1} (i.e., \nu control steps ahead; Section 3.3) and an associated predictive uncertainty score \sigma_{t+1}:

(\hat{z}_{t+1},\sigma_{t+1})=P_{\omega}(z_{t},u_{t})

The uncertainty score is intended to reflect epistemic uncertainty of the latent dynamics model and is used to modulate the downstream execution horizon. Because the prediction loss below does not supervise \sigma_{t+1}, we obtain it from the disagreement of a small ensemble [33]: P_{\omega} comprises N_{e} prediction heads \{P_{\omega}^{(i)}\}_{i=1}^{N_{e}} with independent initializations, \hat{z}_{t+1} is the ensemble mean of the head outputs \hat{z}_{t+1}^{(i)}, and

\sigma_{t+1}=\frac{1}{\sqrt{d}\,\bar{\gamma}^{\star}}\left(\frac{1}{N_{e}}\sum_{i=1}^{N_{e}}\big\|\hat{z}_{t+1}^{(i)}-\hat{z}_{t+1}\big\|_{2}^{2}\right)^{1/2},

where normalization by the frozen latent scale \bar{\gamma}^{\star} (Eq. 9b) renders \sigma_{t+1} dimensionless, so that the sensitivity hyperparameter in Eq. 15 does not depend on the scale of \mathcal{Z}.

A known challenge in joint-embedding architectures lacking a pixel-level decoder is the vulnerability to representation collapse, in which the encoder and predictor output a constant vector to minimize prediction error. To mitigate this, P_{\omega} and the encoder E_{\psi} are jointly optimized using a non-collapsing prediction objective based on Variance-Invariance-Covariance Regularization (VICReg) [19]. The objective consists of a latent \ell_{2} prediction loss, which plays the role of the invariance term, coupled with a distributional regularizer:

\mathcal{L}_{\text{JEPA}}=\mathbb{E}_{t}\left[\big\|\hat{z}_{t+1}-\operatorname{sg}\big(E_{\bar{\psi}}(I_{t+1})\big)\big\|_{2}^{2}\right]+\mathcal{L}_{\text{anti-collapse}}(Z),\qquad\mathcal{L}_{\text{anti-collapse}}(Z)=\lambda_{v}v(Z)+\lambda_{c}c(Z)

where z_{t+1}=E_{\bar{\psi}}(I_{t+1}) is the target embedding produced by a momentum (exponential-moving-average) copy E_{\bar{\psi}} of the encoder, \operatorname{sg}(\cdot) denotes the stop-gradient operator, and, with an ensemble, the prediction term is applied to each head \hat{z}_{t+1}^{(i)}. The regularizer is evaluated on the batch Z of online-encoder latents z_{t}=E_{\psi}(I_{t}), since the momentum encoder receives no gradients; the variance and covariance terms v(Z) and c(Z) are defined in Section 5.2. These terms encourage the variance of each latent dimension to remain above the margin \gamma and reduce the off-diagonal covariance to decorrelate the latent variables.

This regularized formulation encourages the prediction error \|\hat{z}_{t+1}-z_{t+1}\|_{2}^{2} to capture task-relevant physical state evolutions (e.g., object displacement, gripper state changes) associated with the primitive u_{t}. Consequently, \hat{z}_{t+1} provides the downstream continuous action solver with predictive physical context, substituting a forward-simulated latent state for the autoregressive reasoning of large language models.

Training proceeds in stages. In the representation-learning stage, E_{\psi} and P_{\omega} are trained jointly with Eq. 11, with P_{\omega} conditioned on the discrete primitive labels u_{t} provided by the demonstration data. At the end of this stage, both E_{\psi} and P_{\omega} are frozen, which fixes the latent space, its scale \bar{\gamma}^{\star}, and the deployment threshold \tau^{\star}. Freezing provides the downstream solver with a stationary conditioning distribution rather than latent inputs that drift as the representation evolves, which simplifies the learning problem for the motor control module; it does not by itself guarantee the semantic consistency of \mathcal{Z}, which depends on the quality of the learned representation.

4.4 Continuous Action Chunking via Flow-Matching (\pi_{\theta}^{l})

While the higher-level modules (Sections 4.2 and 4.3) operate in the semantic and latent predictive spaces, the low-level solver \pi_{\theta}^{l} grounds these representations into high-frequency motor commands. To manage compounding execution errors and account for sensorimotor latency, we formulate \pi_{\theta}^{l} as an action chunking policy.

At each decision step t, rather than predicting a single atomic action, the solver synthesizes an H-step continuous action trajectory chunk A_{t}=(a_{t},a_{t+1},\dots,a_{t+H-1})\in\mathbb{R}^{H\times d_{a}}, where d_{a} denotes the dimensionality of the robot’s control space (e.g., 7-DoF joint velocities and 1-DoF gripper state) and H\equiv H_{\max} is the fixed length of the generated chunk.

To bridge the latent dynamics with physical embodiment and task intent, we construct a unified conditioning vector e_{t} that encapsulates intent, current state, predicted future, and real-time kinematics:

e_{t}=\Concat\big(\Embed(u_{t}),\,z_{t},\,\hat{z}_{t+1},\,S_{t},\,c_{\text{text}}\big)\in\mathbb{R}^{d_{e}}

where \Embed:\mathcal{U}\rightarrow\mathbb{R}^{d_{u}} is a learned embedding of the discrete primitive, z_{t},\hat{z}_{t+1}\in\mathbb{R}^{d} are the current and predicted latents, S_{t}\in\mathbb{R}^{d_{s}} is the instantaneous proprioceptive state of the robot, c_{\text{text}}\in\mathbb{R}^{d_{c}} is the text embedding, and d_{e}=d_{u}+2d+d_{s}+d_{c}. The continuous components enter Eq. 12 after the fixed standardization described in Section 4.5.

Generating multimodal trajectory distributions p(A_{t}\mid e_{t}) often relies on Denoising Diffusion Probabilistic Models (DDPMs) [34, 11]. However, the iterative sampling procedure of DDPMs, which typically requires many denoising steps, lengthens the sequential chain of network evaluations per decision step and can therefore conflict with the control rates of 30 Hz or more that motivate this architecture.

We therefore parameterize \pi_{\theta}^{l} using a Conditional Flow-Matching (CFM) vector field v_{\theta}:\mathbb{R}^{H\times d_{a}}\times[0,1]\times\mathbb{R}^{d_{e}}\rightarrow\mathbb{R}^{H\times d_{a}}, which transports a simple base distribution (a standard Gaussian) to the action-chunk distribution via an Ordinary Differential Equation (ODE) [12]. Let \rho\in[0,1] denote the continuous integration time of the flow, distinct from the discrete control step t and from the latent completion threshold \tau. We define a linear interpolation path between Gaussian noise \xi\sim\mathcal{N}(0,I), \xi\in\mathbb{R}^{H\times d_{a}}, and the ground-truth action chunk A_{t}:

A_{t}^{(\rho)}=\rho A_{t}+(1-\rho)\,\xi

The neural vector field v_{\theta}(A_{t}^{(\rho)},\rho\mid e_{t}) is trained to regress the time-derivative of this path, i.e., the target velocity A_{t}-\xi, using the Flow-Matching objective:

\mathcal{L}_{\text{solver}}=\mathbb{E}_{t,\,\xi,\,\rho\sim\mathcal{U}[0,1]}\left[\big\|v_{\theta}\big(A_{t}^{(\rho)},\rho\mid e_{t}\big)-(A_{t}-\xi)\big\|_{2}^{2}\right]

To mitigate the risk of physical errors caused by inaccurate latent forecasting, the system employs an uncertainty-aware execution strategy. Rather than always executing the full generated chunk, the executed horizon is scaled based on the predictor’s uncertainty. We define the executed horizon H_{t} at decision step t as:

H_{t}=\max\left(H_{\min},\,\left\lfloor H_{\max}\cdot\exp\big(-\beta\bar{\sigma}_{t+1}\big)\right\rfloor\right)

where \beta>0 is a sensitivity hyperparameter, H_{\min}\leq H_{\max}=H bound the executed chunk length, and \bar{\sigma}_{t+1} is a smoothed uncertainty signal. Using the raw score \sigma_{t+1} directly would allow high-frequency fluctuations in the uncertainty estimate to toggle the controller between long and short horizons on consecutive decision steps, which can induce jitter at the motor level. We therefore apply an asymmetric (fast-attack, slow-release) filter, \bar{\sigma}_{t+1}=\max\{\sigma_{t+1},\,(1-\alpha_{\sigma})\bar{\sigma}_{t}+\alpha_{\sigma}\sigma_{t+1}\} with \alpha_{\sigma}\in(0,1], under which the horizon contracts immediately when uncertainty rises but expands only gradually as it decays. When the environment dynamics are predictable (low \bar{\sigma}_{t+1}), a long prefix of the chunk is executed; during high-uncertainty events, H_{t} contracts toward H_{\min}, so that the control loop relies more frequently on fresh sensory feedback I_{t} rather than on the latent prediction.

During closed-loop execution, a chunk \hat{A}_{t}\in\mathbb{R}^{H\times d_{a}} is sampled by drawing \xi\sim\mathcal{N}(0,I) and integrating the learned vector field from \rho=0 to \rho=1 with K steps of a numerical ODE solver; with the Euler method, A^{(\rho_{k+1})}=A^{(\rho_{k})}+K^{-1}v_{\theta}(A^{(\rho_{k})},\rho_{k}\mid e_{t}) for \rho_{k}=k/K, k=0,\dots,K-1, and A^{(0)}=\xi. The first H_{t} actions of \hat{A}_{t} are then executed. Although the conditional paths in Eq. 13 are straight, the learned marginal flow can be curved, particularly for strongly multimodal action distributions; few-step integration is therefore accurate only to the extent that the learned flow is close to straight, a property that rectification procedures explicitly promote [31]. In practice, a small number of steps K\in\{1,2,3\} is commonly used. Each decision step thus requires K sequential evaluations of v_{\theta}, each of which processes the full H\times d_{a} chunk in parallel: the number of sequential evaluations is independent of H, whereas the arithmetic cost of each evaluation grows with H\cdot d_{a} (Section 5.1). In the reference implementation, a complete decision step took 10.5\text{--}14.2 ms on an Apple M4 for H\leq 64 (Section 5.1), which is short relative to the open-loop execution of a chunk at control rates of 30\text{--}100 Hz. Overlapping the computation of the next chunk with the execution of the current one, which such rates would ultimately require on robotic hardware, was not evaluated in this work (Section 6.5).

4.5 Representation Alignment and Solver Warm-Starting

Before deploying end-to-end continuous action chunking, we adopt an initial alignment phase that matches the representation used by the frozen JEPA modules to the input expected by the continuous Flow-Matching Solver \pi_{\theta}^{l}.

Because the latent space is learned independently of the action data, its coordinates may have arbitrary offsets and a scale \bar{\gamma}^{\star} that differs substantially from that of the proprioceptive and text inputs. Conditioning the vector field on unnormalized latents can produce large gradients in the early iterations of Eq. 14. After freezing, we therefore standardize the latent components of e_{t} with fixed statistics, \tilde{z}=(z-\bar{\mu}_{Z})/\bar{\gamma}^{\star}, applied to both z_{t} and \hat{z}_{t+1}, where \bar{\mu}_{Z} is the mean latent over the training data; the proprioceptive state S_{t} is standardized analogously using dataset statistics. Because the same scale \bar{\gamma}^{\star} also determines \tau^{\star} and normalizes \sigma_{t+1}, the completion threshold, the uncertainty signal, and the solver inputs are expressed in mutually consistent units.

Utilizing principles of Behavior Cloning, the solver is then warm-started on demonstration datasets: conditioning vectors e_{t} are computed by the frozen encoder and predictor along demonstration trajectories and paired with the corresponding action chunks, and v_{\theta} is trained with Eq. 14. This alignment phase is intended to prevent the low-level solver from misinterpreting the latent transitions before any subsequent fine-tuning of the vector field. By establishing a shared representation early in training, the action decoder is aligned with the stationary latent space of the frozen modules, which is expected to facilitate a smoother and more reliable optimization of the flow-matching objective.

5 Theoretical Analysis and System Properties

This section analyzes the runtime complexity of the VPA pipeline, the properties of its regularized latent representation, and the conditions under which its discrete sub-goal mechanism limits error accumulation across task phases. We distinguish between statements that follow directly from the definitions of the architecture and its objectives (Propositions 5.1 and 5.2) and statements that additionally rely on regularity properties of the learned encoder (Proposition 5.3). Because the latter properties cannot be enforced by the training objective or by architectural constraints alone, the corresponding conclusions are presented as conditional results and as empirical expectations to be verified in the evaluation of Section 6.5.

5.1 Computational Complexity and Latency

A primary limitation of contemporary VLA models is their reliance on autoregressive token generation for continuous control. Let H be the length of the required action chunk and d_{a} the action dimension. Token-based VLAs discretize the continuous action space and generate N=q\cdot H\cdot d_{a} tokens per inference cycle, where q is the number of tokens per scalar action component.

For an LLM-based policy with parameters W_{\text{LLM}}, generating N tokens requires N forward passes, each of which depends on the token produced by its predecessor and therefore cannot be executed in parallel. With key-value caching, the i-th pass costs \Theta(|W_{\text{LLM}}|+L_{i}\,d_{\text{model}}) arithmetic operations, where L_{i} is the attention context length at step i and d_{\text{model}} the hidden width. Moreover, at batch size one, each pass must read all decoder parameters from memory; for models whose parameters exceed on-chip memory, the wall-clock latency therefore satisfies

T_{\text{VLA}}\;=\;\sum_{i=1}^{N}t_{\text{pass}}(L_{i})\;\geq\;\frac{N\cdot|W_{\text{LLM}}|}{\mathrm{BW}},\qquad N=q\cdot H\cdot d_{a},

where t_{\text{pass}}(L_{i}) is the wall-clock time of the i-th pass and \mathrm{BW} denotes the effective memory bandwidth, measured in parameters per unit time. The latency of autoregressive action decoding thus grows at least linearly in N, and hence in H. For RT-2, real-time inference rates of about 1\text{--}3 Hz for the 55B-parameter model and about 5 Hz for its 5B-parameter variant have been reported [1]; rates in this range restrict reactive, high-speed physical adjustments.

To analyze the VPA pipeline, we distinguish two cost measures for a single decision step: the sequential depth D, defined as the length of the longest chain of network evaluations in which each evaluation requires the output of its predecessor, and the total arithmetic work W. Wall-clock latency is lower-bounded by the sequential depth times the minimum per-evaluation latency, and approaches the work divided by the available parallelism when the hardware is saturated.

Proposition 5.1 (Sequential depth and work of a VPA decision step).

Consider one decision step of VPA in which the solver generates a chunk of length H with K integration steps, and assume that the milestone embeddings z_{g}^{(m)} and the text embedding c_{\text{text}} have been computed once per episode. Then:

1. the sequential depth is D_{\text{VPA}}=K+3, which is independent of H and d_{a}, whereas autoregressive decoding of N=q\cdot H\cdot d_{a} tokens has sequential depth D_{\text{VLA}}=N=\Theta(H\cdot d_{a});
2. the work is W_{\text{VPA}}(H)=C_{E}+C_{h}+C_{P}+K\cdot C_{l}(H), where C_{E}, C_{h}, and C_{P} are the costs of the encoder, the selector, and the predictor (including all N_{e} ensemble heads), none of which depends on H, and C_{l}(H) is the cost of one evaluation of v_{\theta}. Since v_{\theta} outputs an element of \mathbb{R}^{H\times d_{a}}, C_{l}(H)=\Omega(H\cdot d_{a}); if v_{\theta} applies self-attention across the H chunk positions, C_{l}(H) additionally contains a term quadratic in H.

Proof.

(i) Within one decision step, the data dependencies form the chain I_{t}\to z_{t}\to u_{t}\to(\hat{z}_{t+1},\sigma_{t+1})\to e_{t}\to A^{(\rho_{1})}\to\cdots\to A^{(\rho_{K})}: the selector requires z_{t} (Eq. 8), the predictor requires u_{t} (Eq. 10), and each Euler step requires the previous iterate (Section 4.4). This chain contains one evaluation each of E_{\psi}, \pi_{\phi}^{h}, and P_{\omega}, and K evaluations of v_{\theta}; the ensemble heads of P_{\omega} are mutually independent and may be evaluated in parallel. No evaluation in the chain is repeated as a function of H. For autoregressive decoding, token i is an input to pass i+1, so all N passes lie on a single dependency chain. (ii) The expression for W_{\text{VPA}} follows by summing the costs along the chain. Each evaluation of v_{\theta} writes H\cdot d_{a} output entries, which requires \Omega(H\cdot d_{a}) operations. ∎

Proposition 5.1 makes precise the sense in which the VPA solver removes the dependence on the action horizon. The wall-clock latency of a decision step is

T_{\text{VPA}}=\underbrace{t_{E}+t_{h}+t_{P}}_{\text{independent of }H}+\;K\cdot t_{l}(H),

where t_{E}, t_{h}, t_{P}, and t_{l}(H) denote the wall-clock times of the encoder, the selector, the predictor, and one solver evaluation, respectively. Because K\ll N and the sequential dependency on output generation is removed, the number of sequential network evaluations drops from \Theta(H\cdot d_{a}) to K+3=\mathcal{O}(K). This does not imply that the per-cycle cost is \mathcal{O}(1) in H: by part (ii) of Proposition 5.1, the arithmetic work of each solver evaluation grows at least linearly with H\cdot d_{a}, and t_{l}(H) is approximately constant in H only while a single evaluation of v_{\theta} fits within the available hardware parallelism; beyond that regime, t_{l}(H) grows with C_{l}(H). The reduction in sequential depth is shared with other non-autoregressive action heads (Section 2.4); the distinguishing latency property of VPA is that all networks on the dependency chain are lightweight relative to a vision-language backbone. The absolute values of the terms in Eq. 17 depend on the sizes of E_{\psi}, \pi_{\phi}^{h}, P_{\omega}, and v_{\theta} and on the hardware, and therefore do not follow from Proposition 5.1. We measured them for the reference implementation, as described below.

Measured latency on an Apple M4.

We timed the four terms of Eq. 17 inside the complete decision step of the reference implementation (Section 1.1), sweeping K\in\{1,2,3\} and H\in\{8,16,32,64,128\}. The measurements used a Mac mini with an Apple M4 processor (10-core CPU, 32 GB memory), macOS 15.3.1, and PyTorch 2.14.1 with the MPS backend, at batch size one and in float32 arithmetic. Each stage was timed with forward hooks, with device synchronization before every clock read, and each value in Table 1 is the median of 100 runs after 10 warm-up runs. Because these timings do not depend on the values of the weights, randomly initialized weights were used. The benchmarked configuration uses the default, untuned module sizes of the implementation with a single camera view: E_{\psi} is a Vision Transformer with 224\times 224 input, patch size 16, width 384, depth 6, and latent dimension d=256 (11.12M parameters); \pi_{\phi}^{h} is a multilayer perceptron with two hidden layers of width 512 over N_{u}=8 primitives (0.79M); P_{\omega} comprises N_{e}=5 ensemble heads with two hidden layers of width 512 (2.79M); and v_{\theta} is a Transformer of width 256 and depth 4 with d_{a}=8 (3.55M at H=8 and 3.58M at H=128, since only its positional embedding grows with H). The networks on the dependency chain thus total approximately 18.3M parameters. The frozen text encoder is not on this chain, because c_{\text{text}} is computed once per episode, and is excluded from the measurements.

Observations.

The measurements are consistent with Proposition 5.1 in four respects. First, the implementation audits the order of network evaluations in every decision step and raises an error if it differs from one evaluation each of E_{\psi}, \pi_{\phi}^{h}, and P_{\omega} followed by K evaluations of v_{\theta}; this audit passed in all configurations of the sweep, so the sequential depth was K+3 throughout, as stated in part (i). Second, across all values of K and H, t_{E}, t_{h}, and t_{P} remained within 3.70\text{--}3.87 ms, 0.33\text{--}0.34 ms, and 0.69\text{--}0.72 ms, respectively, consistent with their independence of H in Eq. 17. Third, t_{l}(H) remained within 10\% of its value at H=8 up to H=64 and rose by 23\text{--}27\% at H=128. This is in line with part (ii): on this hardware, one evaluation of v_{\theta} appears to fit within the available parallelism up to approximately H=64, beyond which its growing work becomes visible in the wall-clock time. Fourth, each additional integration step lengthened the complete decision step by approximately one solver evaluation (e.g., 10.6, 12.1, and 13.7 ms for K=1,2,3 at H=16). For H\leq 64, the summed network time of Eq. 17 was 6.2\text{--}9.5 ms, and the complete decision step took 10.5\text{--}14.2 ms (at most 14.8 ms at H=128). At K=2 and H=16, the setting used for the LIBERO-10 experiments (Section 6), the complete decision step took 12.1 ms. Executed back to back, these step times correspond to approximately 70\text{--}95 decision steps per second.

Scope of the measurements.

We regard these results as initial empirical support for the sequential-depth analysis rather than as a characterization of deployed performance, for four reasons. (i) The benchmarked configuration uses a single 224\times 224 camera view, whereas the LIBERO-10 models of Section 6 use 128\times 128 frames and, in the two-camera setting, two views per observation. Because both views are processed as one batch within a single evaluation of E_{\psi}, the sequential depth of the two-camera model remains K+3, but its latency is not reported here. (ii) The complete decision-step times include the synchronization introduced by the per-stage timing and may therefore somewhat overstate the time of an uninstrumented step; conversely, they exclude camera capture and actuation, and thus do not measure closed-loop control frequency. (iii) Table 1 reports a single sweep on a single machine. In a repeated sweep, the median difference per timing value was 1.5\%, 55 of the 60 per-stage values agreed within 5\%, and the largest difference was 19\%; differences of this size should be regarded as run-to-run variation. (iv) We did not run an autoregressive VLA on the same hardware. The contrast with the rates of about 1\text{--}5 Hz reported for RT-2 [1] is therefore indicative only, given differences in hardware, model scale, and measurement protocol, and should not be read as a measured speed-up. Measurements on robotic deployment hardware and in closed-loop operation remain future work (Section 6.5).

5.2 Representation Regularization and Anti-Collapse Properties

In predictive architectures trained with a mean-squared-error objective and no pixel-level decoder, the encoder and predictor admit a degenerate solution: representation collapse. The vision encoder E_{\psi} and predictor P_{\omega} can drive the prediction term \mathbb{E}\big[\|\hat{z}_{t+1}-z_{t+1}\|_{2}^{2}\big] to zero by mapping all observations to an identical constant vector \zeta\in\mathbb{R}^{d}.

To exclude this degenerate solution, VPA incorporates the variance and covariance terms of VICReg [19]. Given a batch of B online-encoder latents Z\in\mathbb{R}^{B\times d} evaluated during training, the anti-collapse loss acts through two penalties.

First, to counteract collapse to a point mass, a hinge loss is applied to the standard deviation of each latent dimension j\in\{1,\dots,d\} with target margin \gamma>0:

v(Z)=\frac{1}{d}\sum_{j=1}^{d}\max\left(0,\gamma-\sqrt{\Var(Z_{\cdot,j})+\epsilon}\right)

where \epsilon>0 is a small numerical constant, with \epsilon<\gamma^{2}, that prevents unbounded gradients at zero variance, and \Var is computed with the same estimator as C(Z) below. The term v(Z) vanishes if and only if \Var(Z_{\cdot,j})\geq\gamma^{2}-\epsilon for every j. It is a penalty rather than a hard constraint: at an optimum of the joint objective at which v(Z)>0, some dimensions may remain below the margin. Furthermore, since v(Z) is a batch statistic, it does not imply that E_{\psi} is injective on the state space; physically distinct states may still be mapped to nearby or identical latents. The hinge also does not bound the per-dimension standard deviations from above, which is why the completion threshold is calibrated to the realized margin \bar{\gamma} rather than to \gamma alone (Eq. 9c).

Second, to counteract dimensional collapse, in which several latent dimensions encode the same variable, we penalize the off-diagonal entries of the empirical covariance matrix C(Z)\in\mathbb{R}^{d\times d}:

c(Z)=\frac{1}{d}\sum_{i\neq j}\left([C(Z)]_{i,j}\right)^{2}

where C(Z)=\frac{1}{B-1}\sum_{b=1}^{B}(Z_{b,\cdot}-\bar{Z})(Z_{b,\cdot}-\bar{Z})^{\top} and \bar{Z} is the batch mean. Equation 19 penalizes linear correlations between latent dimensions; it does not penalize nonlinear dependencies, so decorrelation should not be equated with statistical independence.

The regularization term is \mathcal{L}_{\text{anti-collapse}}(Z)=\lambda_{v}v(Z)+\lambda_{c}c(Z) (Eq. 11). The following proposition characterizes what this regularizer does and does not exclude. For tractability, we analyze the idealized objective in which the momentum target encoder coincides with the online encoder, E_{\bar{\psi}}=E_{\psi}, as at a fixed point of the moving-average update, and write \mathcal{L}(\psi,\omega)=\mathcal{L}_{\text{pred}}(\psi,\omega)+\lambda_{v}v(Z)+\lambda_{c}c(Z), where \mathcal{L}_{\text{pred}} denotes the prediction term of Eq. 11.

Proposition 5.2 (Anti-collapse properties).

Assume \lambda_{v}>0 and \epsilon<\gamma^{2}.

1. Every constant encoder E_{\psi}\equiv\zeta, paired with an optimal predictor, attains the objective value \mathcal{L}_{\text{const}}=\lambda_{v}\big(\gamma-\sqrt{\epsilon}\big). Consequently, if the model class contains parameters (\psi^{\prime},\omega^{\prime}) with \mathcal{L}(\psi^{\prime},\omega^{\prime})<\lambda_{v}(\gamma-\sqrt{\epsilon}), no constant encoder is a global minimizer.
2. If a batch Z satisfies both v(Z)=0 and c(Z)=0, then C(Z) is diagonal with all diagonal entries at least \gamma^{2}-\epsilon>0; hence C(Z) is positive definite and the batch latents span all d dimensions of \mathcal{Z}. This can occur only if B>d.

Proof.

(i) For a constant encoder, every column of Z is constant, so \Var(Z_{\cdot,j})=0 for all j and C(Z)=0. Each hinge term therefore equals \max(0,\gamma-\sqrt{\epsilon})=\gamma-\sqrt{\epsilon}, giving v(Z)=\gamma-\sqrt{\epsilon}, while c(Z)=0. A predictor that outputs \zeta attains \mathcal{L}_{\text{pred}}=0, which is optimal since \mathcal{L}_{\text{pred}}\geq 0. Hence \mathcal{L}_{\text{const}}=\lambda_{v}(\gamma-\sqrt{\epsilon}), and any parameters with a strictly smaller objective value exclude constant encoders from the set of global minimizers. (ii) c(Z)=0 forces all off-diagonal entries of C(Z) to vanish, and v(Z)=0 forces [C(Z)]_{j,j}=\Var(Z_{\cdot,j})\geq\gamma^{2}-\epsilon for all j. A diagonal matrix with positive diagonal entries is positive definite and has rank d; since \operatorname{rank}C(Z)\leq B-1, this requires B>d. ∎

Part (i) of Proposition 5.2 refines a frequently stated but imprecise claim: the positivity of \lambda_{v} alone does not exclude collapse. Collapse is excluded from the global minimizers only if some non-degenerate representation achieves an objective value below the collapse level \lambda_{v}(\gamma-\sqrt{\epsilon}), a condition that is more readily satisfied for larger \lambda_{v}. Part (ii) shows that, at points where both penalties vanish, total and dimensional collapse are excluded on the training batch. Part (ii) does not apply to the batch size used in Section 6: with B=64 and d=256, \operatorname{rank}C(Z)\leq 63<d, so v(Z) and c(Z) cannot vanish together on a training batch. Neither part implies that latent distances faithfully reflect physical distances: a non-collapsed representation may still be insensitive to task-relevant differences or sensitive to nuisance factors. Non-collapse is therefore necessary but not sufficient for \|z_{t}-z_{g}^{(m)}\|_{2} to serve as a proxy for task progress; the additional conditions are examined in Section 5.3.

5.3 Error Accumulation and Conditional Phase-Transition Bounds

In step-by-step imitation learning (Behavioral Cloning), policies are subject to compounding covariate shift: if the learned policy errs with probability \epsilon_{\pi} per step under the demonstrator’s state distribution, its expected total cost over a horizon of T steps (with per-step costs in [0,1]) exceeds that of the demonstrator by at most \mathcal{O}(T^{2}\epsilon_{\pi}), and this quadratic dependence is tight in the worst case [35]. VPA addresses error accumulation through two mechanisms: temporal action chunking within each task phase, and discrete sub-goal thresholding between phases.

Action chunking.

The Flow-Matching solver \pi_{\theta}^{l} generates an H-step action chunk A_{t}, and executing it open-loop reduces the number of decisions over a horizon of T steps from T to T/H. Applying the same reduction at the chunk level, treating each chunk as a single decision whose cost is the sum of H per-step costs and whose error probability under the demonstrator’s distribution is \epsilon_{H}, yields

J(\hat{\pi})-J(\pi^{\ast})\;\leq\;\Big(\frac{T}{H}\Big)^{2}\cdot H\cdot\epsilon_{H}\;=\;\frac{T^{2}}{H}\,\epsilon_{H},

where J(\hat{\pi}) and J(\pi^{\ast}) are the expected total costs of the learned and demonstrator policies. This bound improves on T^{2}\epsilon_{\pi} only if \epsilon_{H} grows sublinearly in H. Since a chunk contains H actions, a union bound yields only \epsilon_{H}\leq H\epsilon_{\pi}, under which the chunk-level bound coincides with the step-level bound. The benefit of chunking is therefore an empirical property, consistent with prior findings on action chunking [10], rather than a guarantee. Open-loop execution moreover forgoes sensory feedback within each chunk, a trade-off that the adaptive horizon of Eq. 15 is intended to manage.

Sub-goal thresholding.

The milestone pointer m_{t} advances only when \|z_{t}-z_{g}^{(m_{t})}\|_{2}<\tau (Eq. 9a). Whether this latent condition constrains the physical state depends on the regularity of the composite map from task-relevant physical state to latent, y\mapsto E_{\psi}(\mathcal{I}(y,\eta)) (Section 3.1). We state two conditions on this map, with a slack \delta_{\eta}\geq 0 that accounts for the latent displacement caused by variation of the nuisance factors \eta within the deployment set \mathcal{N}; without loss of generality, the same slack is used in both.

Assumption 5.1 (Upper regularity).

There exist L_{E}<\infty and \delta_{\eta}\geq 0 such that, for all y,y^{\prime}\in\mathcal{Y} and all \eta,\eta^{\prime}\in\mathcal{N},

\big\|E_{\psi}(\mathcal{I}(y,\eta))-E_{\psi}(\mathcal{I}(y^{\prime},\eta^{\prime}))\big\|_{2}\;\leq\;L_{E}\|y-y^{\prime}\|_{2}+\delta_{\eta}.

Assumption 5.2 (Lower (co-Lipschitz) regularity).

There exist \mu_{E}>0 and \delta_{\eta}\geq 0 such that, for all y,y^{\prime}\in\mathcal{Y} and all \eta,\eta^{\prime}\in\mathcal{N},

\big\|E_{\psi}(\mathcal{I}(y,\eta))-E_{\psi}(\mathcal{I}(y^{\prime},\eta^{\prime}))\big\|_{2}\;\geq\;\mu_{E}\|y-y^{\prime}\|_{2}-\delta_{\eta}.

Proposition 5.3 (Conditional phase-transition bounds).

Let y_{g}^{(m)} denote the target physical configuration of milestone m, observed as I_{g}^{(m)}=\mathcal{I}(y_{g}^{(m)},\eta_{g}^{(m)}) with \eta_{g}^{(m)}\in\mathcal{N}, and let z_{g}^{(m)}=E_{\psi}(I_{g}^{(m)}).

1. Soundness. Under Assumption 5.2, a satisfied threshold test implies physical proximity to the milestone: \|z_{t}-z_{g}^{(m_{t})}\|_{2}<\tau\;\Longrightarrow\;\|y_{t}-y_{g}^{(m_{t})}\|_{2}<\frac{\tau+\delta_{\eta}}{\mu_{E}}. (20)
2. Completeness. Under Assumption 5.1 and \tau>\delta_{\eta}, sufficient physical proximity triggers the transition: \|y_{t}-y_{g}^{(m_{t})}\|_{2}<\frac{\tau-\delta_{\eta}}{L_{E}}\;\Longrightarrow\;\|z_{t}-z_{g}^{(m_{t})}\|_{2}<\tau.
3. Phase-onset reset. Under Assumption 5.2, let t_{k} be the decision step at which the pointer advances from k to k+1, and let r_{k+1}=\|y_{t_{k}}-y_{g}^{(k)}\|_{2} denote the physical residual with respect to milestone k at the onset of phase k+1. Then r_{k+1}<(\tau+\delta_{\eta})/\mu_{E}, irrespective of the trajectory executed during phases 1,\dots,k.

Proof.

(i) Applying Assumption 5.2 with (y,\eta)=(y_{t},\eta_{t}) and (y^{\prime},\eta^{\prime})=(y_{g}^{(m_{t})},\eta_{g}^{(m_{t})}) gives \mu_{E}\|y_{t}-y_{g}^{(m_{t})}\|_{2}-\delta_{\eta}\leq\|z_{t}-z_{g}^{(m_{t})}\|_{2}<\tau; rearranging yields Eq. 20. (ii) Applying Assumption 5.1 to the same pair gives \|z_{t}-z_{g}^{(m_{t})}\|_{2}\leq L_{E}\|y_{t}-y_{g}^{(m_{t})}\|_{2}+\delta_{\eta}<(\tau-\delta_{\eta})+\delta_{\eta}=\tau. (iii) At t_{k}, the threshold test is satisfied for milestone k by the definition of Eq. 9a; part (i) then gives the bound, which depends only on \tau, \delta_{\eta}, and \mu_{E}. ∎

Remark 5.1 (Direction of the Lipschitz inequality).

An upper bound of the form \|z_{a}-z_{b}\|_{2}\leq L_{E}\|y_{a}-y_{b}\|_{2} (Assumption 5.1 with \delta_{\eta}=0) implies only \|y_{t}-y_{g}^{(m_{t})}\|_{2}\geq\|z_{t}-z_{g}^{(m_{t})}\|_{2}/L_{E}, which is a lower bound on the physical deviation. An upper Lipschitz condition alone therefore cannot bound the physical error at a transition from above; that conclusion requires the lower regularity of Assumption 5.2. The two assumptions play complementary roles: Assumption 5.2 makes the threshold test sound (a transition implies physical proximity), whereas Assumption 5.1 makes it complete (physical proximity triggers a transition).

Consequences and their limits.

Under Assumption 5.2, part (iii) of Proposition 5.3 implies \max_{k}r_{k}<(\tau^{\star}+\delta_{\eta})/\mu_{E} over all phases that are entered: the physical residual with which each primitive begins is bounded uniformly and independently of the length M of the milestone sequence, so that deviation accumulated in one phase is not propagated beyond this bound into the next. Three qualifications apply. First, the bound concerns phase onsets only; deviations within a phase are governed by the low-level policy and are subject to the chunk-level analysis above. Second, the bound is conditional on each transition occurring: by part (ii), a transition is guaranteed only if the physical state enters the ball of radius (\tau^{\star}-\delta_{\eta})/L_{E} around the milestone, which requires \tau^{\star}>\delta_{\eta}, that is, a threshold exceeding the latent displacement induced by nuisance variation; otherwise the pointer may stall (Section 4.2). Third, \tau^{\star} and \delta_{\eta} jointly determine a trade-off: increasing \kappa in Eq. 9c raises \tau^{\star}, which improves completeness under nuisance variation but loosens the soundness bound of Eq. 20.

Why these properties are expectations rather than guarantees.

The constants L_{E}, \mu_{E}, and \delta_{\eta} are properties of the learned encoder composed with image formation, and none of them is controlled by the objective of Eq. 11. Regarding Assumption 5.1, standard dot-product self-attention is not globally Lipschitz [22], so a Vision Transformer encoder possesses no global Lipschitz bound with respect to its input unless explicit constraints are imposed. On a bounded image domain a finite constant exists, but it may be very large and is neither estimated nor controlled during training. Spectral normalization of the linear layers [21], combined with a Lipschitz-continuous attention variant [22], can bound the pixel-space Lipschitz constant L_{\psi} of E_{\psi}. Even then, Assumption 5.1 concerns the composition with the image-formation map \mathcal{I}, whose local sensitivity to y can be very large at occlusion and contact events, so that L_{E} is bounded by L_{\psi} times the Lipschitz constant of \mathcal{I} only where the latter is finite. Spectral normalization is not part of the base formulation; its effect on representation quality and on the threshold mechanism is a subject for empirical evaluation. Regarding Assumption 5.2, a lower bound cannot be obtained from architectural constraints of this kind at all. The encoder is designed to be non-injective with respect to pixels, being intended to be invariant to \eta, so the lower bound must hold with respect to y alone. It therefore depends on the data and on the learned representation, and it is violated wherever task-relevant differences are not visible in the image, for example under occlusion or object symmetry. The variance term of Eq. 18 promotes only batch-level spread and does not imply Assumption 5.2.

Accordingly, we do not claim that VPA guarantees non-compounding error. We instead state the expectation that, to the extent that the learned encoder satisfies Assumptions 5.1 and 5.2 with favorable constants on the deployment distribution, discrete sub-goal thresholding limits the propagation of physical error across task phases in the manner described by Proposition 5.3. This expectation is directly testable. In simulation, where the physical state y_{t} is available, the constants \mu_{E}, L_{E}, and \delta_{\eta} can be estimated from the lower and upper envelopes of \|z-z^{\prime}\|_{2} as a function of \|y-y^{\prime}\|_{2} over sampled state pairs and nuisance conditions, and the phase-onset residuals r_{k} can be measured directly (Section 6.5).

6 Empirical Evaluation

This section reports a first empirical evaluation of the reference implementation on the LIBERO-10 benchmark [20]. The experiments are intended as a proof of concept: they test whether the pipeline of Section 4 can be trained on standard demonstration data and executed in closed loop, and they measure the effect of two design choices, the camera configuration and the executed horizon H_{t}. Each configuration was trained once and evaluated on 200 episodes, so that success rates carry 95\% intervals of roughly \pm 7 percentage points, and no baselines were evaluated under the same protocol. The results should therefore be read as preliminary. The trained checkpoints, the evaluation summaries, and the code used to produce them are publicly available (Section 1.1).

6.1 LIBERO-10 Implementation

Benchmark and data.

LIBERO-10 is the long-horizon suite of LIBERO [20]: ten tabletop manipulation tasks in kitchen, living-room, and study scenes, each specified by a natural-language instruction, most of which combine two sub-tasks (e.g., placing two objects in a basket, or placing an object in a drawer and closing it). We trained on the demonstrations provided with the suite, holding out two demonstrations per task (20 in total) for validation. Frames were rotated by 180^{\circ}, because LIBERO renders them upside down, and used at their stored resolution of 128\times 128 pixels. The proprioceptive state S_{t} concatenated the end-effector position and orientation, the gripper state, and the joint states (d_{s}=15), and the action space was LIBERO’s six-dimensional end-effector displacement with a gripper command (d_{a}=7). We verified on the dataset files that each recorded frame follows the action with the same index, so that each training pair associates the observation (I_{t},S_{t}) with the chunk of the H subsequent actions.

Primitive labels and milestones.

The formulation of Section 4.2 assumes primitive labels supplied with the demonstrations, which LIBERO does not provide. As a stand-in, we derived N_{u}=4 primitives from the recorded gripper command with a window of 10 steps: grasp for the 10 steps after the gripper closes, carry while it remains closed thereafter, release for the 10 steps after it opens, and reach otherwise. This heuristic is not part of the method and was chosen for simplicity; annotated or learned segmentations may behave differently. Each task was specified by a single milestone (M=1): during training, the goal image I_{g}^{(1)} was the final frame of the same demonstration, and at evaluation, trial k of each task used the final frame of demonstration k\bmod 50 in that task’s LIBERO demonstration file, so that the 20 trials used demonstrations 0\text{--}19, most of which were also training demonstrations. With M=1, the milestone pointer of Eq. 9a never advances between phases, so the multi-phase sequencing mechanism and the phase-transition analysis of Section 5.3 were not exercised by these experiments. We further note that the goal image provides the policy with information about the target arrangement that language-conditioned policies evaluated on LIBERO do not receive.

Training.

Following Section 4.3, training proceeded in two stages of 50{,}000 steps each with batch size 64. In the first stage, E_{\psi} and P_{\omega} were trained from random initialization with Eq. 11, after which both were frozen, which fixed \bar{\gamma}^{\star} and \tau^{\star}. In the second stage, every frame was encoded once by the frozen encoder, the conditioning inputs were standardized as described in Section 4.5, and the selector and the solver were trained with the cross-entropy loss of Section 4.2 and with Eq. 14, respectively, using the demonstration primitive labels as inputs to \Embed(u_{t}) and P_{\omega}. The two-camera model differs from the one-camera model only in the input of E_{\psi}: both views are passed through the same Vision Transformer body, and a single linear head maps their concatenated class tokens to z_{t}, so that Eqs. 8–15 are unchanged. The trainable modules of both models contain fewer than 20M parameters; the frozen CLIP text encoder (63.4M parameters) is evaluated once per episode. All settings are listed in Table 2; they are default values of the implementation and were not tuned. Both models were trained on a Google Cloud g2-standard-4 instance with a single NVIDIA L4 GPU (24 GB), four vCPUs, and 16 GB of memory. Training took under 2 hours of wall-clock time for the one-camera model and under 3 hours for the two-camera model. Most of this time was spent in the first stage; the second stage, which operates on cached latents, took well under an hour.

Evaluation protocol.

Each model was evaluated in closed loop in the LIBERO simulator on 20 trials per task (200 episodes per configuration), starting from LIBERO’s fixed initial states with evaluation seed 0. After each reset, five zero-action steps were executed before control started. An episode was counted as a success if LIBERO’s task-completion predicate was satisfied within 600 control steps. Although Section 4.2 specifies that an episode terminates once the test of Eq. 9a holds for m_{t}=M, this completion flag did not end episodes in our experiments, and success was judged only by LIBERO’s predicate. Unless stated otherwise, the executed horizon followed Eq. 15 with \beta=1, which in practice yielded H_{t}=15 of the H=16 generated actions (Section 6.3). We report success rates with 95\% Wilson score intervals. Because each configuration was trained with a single seed, these intervals reflect evaluation variability only, not variability across training runs.

6.2 Task Success Rates (One vs. Two Cameras)

Table 3 reports success rates for the one-camera model, which observes only the third-person view, as in the single-frame formulation of Section 4.1, and for the two-camera model, which additionally observes the wrist-mounted view. The one-camera model succeeded in 10.5\% of episodes (21/200; 95\% interval 7.0\text{--}15.5\%) and failed in all trials of six of the ten tasks. Adding the wrist view raised the success rate to 40.0\% (80/200; 33.5\text{--}46.9\%). The two aggregate intervals do not overlap, and the two-camera model achieved a higher rate on each of the ten tasks. With 20 trials per task, however, individual per-task rates carry intervals of up to about \pm 20 percentage points, and we do not draw conclusions about particular tasks.

The size of this effect suggests that, in this configuration (128\times 128 frames and an encoder trained from random initialization on the demonstrations alone), the third-person view did not provide the policy with sufficient information on its own. One plausible explanation, which we have not tested directly, is that the wrist view resolves the relation between the gripper and nearby objects more clearly at this resolution. The result also indicates that the single-frame observation assumed in Section 4.1 was insufficient in this setting, and that treating I_{t} as a set of views, which changes only the input of E_{\psi}, is a useful extension of the formulation.

Dependence on the third-person view.

Replacing the third-person view of the two-camera model with a black image at every decision step, while leaving the goal image unchanged, reduced its success rate to 0\% (0/200; upper 95\% bound 1.9\%). Because a black frame lies outside the training distribution, this test shows that the policy relies on the third-person view, but it does not cleanly measure how much task-relevant information that view contributes.

Training length.

Evaluating the two-camera model at intermediate checkpoints after 40k, 45k, and 50k second-stage steps gave 32.5\% (26.4\text{--}39.3\%), 36.0\% (29.7\text{--}42.9\%), and 40.0\%, respectively. The differences between adjacent checkpoints are within the statistical uncertainty, but the trend suggests that the policy had not converged when training stopped.

Context.

Previously reported success rates on LIBERO-10 are higher than those reported here, including for policies trained without large-scale pre-training [2]. Our protocol differs from the standard one in the number of trials and training seeds and, as noted in Section 6.1, in the goal image supplied to the policy, so we do not make a direct comparison. We regard the two-camera result as evidence that the pipeline can learn non-trivial long-horizon behavior from demonstrations, not as evidence of competitive performance.

6.3 Execution Horizon Ablation

The executed horizon H_{t} determines how many of the H=16 generated actions are executed open loop before the next decision step. Using the final two-camera checkpoint, we compared fixed executed horizons of 2, 4, and 8 actions with Eq. 15 at two sensitivities: \beta=1, for which the filtered uncertainty \bar{\sigma}_{t+1} remained small enough that H_{t}=15, and \beta=40.4, chosen so that the 90th percentile of the per-step uncertainty \sigma_{t+1} observed in the \beta=1 run gives H_{t}=H_{\min}, for which H_{t} varied with \bar{\sigma}_{t+1} and was usually much shorter than the chunk. All variants used the same checkpoint and the same 200 initial states. Table 4 shows the results.

Success increased monotonically with the number of executed actions, from 16.0\% with H_{t}=2 to 40.0\% with H_{t}=15. The uncertainty-adaptive setting (\beta=40.4) reached 30.5\%, comparable to a fixed horizon of 8 actions and below the near-full-chunk setting. Differences between adjacent rows lie within the statistical uncertainty, but the overall trend is consistent.

These results do not support a benefit of uncertainty-adaptive execution in this setting. The best result was obtained when Eq. 15 executed almost the entire chunk, which effectively disabled its adaptivity, whereas the configuration in which H_{t} responded to \bar{\sigma}_{t+1} performed worse. A likely contributing factor is that each new chunk is generated from fresh noise \xi, so that frequent replanning can switch between different, individually valid realizations of the same motion; executing a longer prefix reduces the number of such switches. We have not verified this explanation. It also remains open whether a better-calibrated uncertainty estimate, a different form of Eq. 15, or a mechanism that keeps consecutive chunks consistent would change this outcome, and whether adaptive horizons become beneficial in the dynamic settings that motivated them (Section 6.5).

6.4 Latency Benchmarks

Section 5.1 reports per-stage latency measurements of the reference implementation on an Apple M4 (Table 1); we summarize them here in the context of the LIBERO-10 experiments. For K\in\{1,2,3\} and H\leq 64, a complete decision step took 10.5\text{--}14.2 ms at batch size one, of which 6.2\text{--}9.5 ms was spent in the four networks on the dependency chain. At K=2 and H=16, the values used in the LIBERO-10 experiments, the complete step took 12.1 ms. The sequential depth was K+3 in all measured configurations, the encoder, selector, and predictor times did not change appreciably with H, and the time of one solver evaluation remained within 10\% of its value at H=8 up to H=64, consistent with Proposition 5.1.

Two points limit how these figures relate to the experiments above. First, the latency benchmark used the default module sizes with a single 224\times 224 view, whereas the LIBERO-10 models used 128\times 128 frames, two views in the two-camera case, and the settings of Table 2. We have not separately benchmarked the released checkpoints, although the released benchmark script accepts the configuration of a trained model. Second, in the LIBERO-10 experiments a decision step was followed by the open-loop execution of H_{t}=15 actions, so the networks were evaluated once per 15 control steps, and the simulated episodes did not run in real time. The measured latency therefore indicates the computational headroom of the pipeline rather than a constraint that was active in these experiments. Measurements on robotic deployment hardware and under real-time closed-loop operation remain future work.

6.5 Future Work

The experiments above establish that the pipeline can be trained and executed in a standard benchmark setting, but they leave open several questions that bear directly on the claims of this paper. First, the LIBERO-10 results should be strengthened by following the standard protocol, with several training seeds, 50 trials per task, all LIBERO suites, and baselines evaluated under identical conditions. Initializing E_{\psi} from a pre-trained self-supervised encoder [16], which the implementation already supports, and replacing the gripper-derived primitive labels with annotated or learned segmentations are natural next steps. Second, multi-milestone tasks (M>1) are required to exercise the sequencing mechanism of Section 4.2, including the per-milestone time-out deferred there. Third, the properties that motivated the design, namely reactive control under dynamic perturbations and the conditional analysis of Section 5, require environments that support such perturbations and expose the privileged physical state y_{t}.

Section 5 establishes two kinds of properties: properties that follow from the architecture and its objectives (Propositions 5.1 and 5.2), and properties that hold only under regularity assumptions on the learned encoder (Proposition 5.3). The latency measurements of Section 5.1 provide initial support for the former on one hardware platform; the latter remain untested. We therefore plan an evaluation that benchmarks high-frequency reactive control and visual robustness, areas in which current autoregressive Vision-Language-Action (VLA) models often experience difficulties, together with targeted measurements of the theoretical quantities introduced in Section 5.

6.5.1 Simulation Environments

To benchmark the system reproducibly prior to physical hardware deployment, we plan to use established GPU-accelerated simulation platforms:

• NVIDIA Isaac Sim [36]: This platform is selected for its PhysX physics engine and ray-traced rendering. It is suited to testing robustness against visual distribution shifts, such as dynamic lighting or reflections, and to assessing whether the regularized JEPA representations remain informative without pixel reconstruction.
• ManiSkill3 [37]: This simulator is selected for its standardized manipulation tasks and parallelized tensor-based API. It allows for the rapid evaluation of continuous action chunking and neuro-symbolic transitions across numerous episodic rollouts.

Both platforms expose the ground-truth physical state y_{t} (robot configuration and object poses), which is unavailable on physical hardware and is required for the measurements of Section 6.5.3.

6.5.2 Planned Benchmark Tasks

The planned evaluation focuses on dynamic, non-stationary environments in which the low per-step latency measured in Section 5.1 may translate into a measurable advantage over autoregressive VLAs, for which inference rates of about 1\text{--}5 Hz have been reported for RT-2 [1].

1. High-Speed Reactive Pick-and-Place: Unlike static tabletop manipulation, this task involves moving targets, such as items on a moving conveyor belt or objects physically perturbed mid-grasp. Autoregressive models often experience action lag in these scenarios, executing stale trajectories based on outdated observations. The Flow-Matching solver (\pi_{\theta}^{l}) will be evaluated on its ability to update action chunks at >50 Hz to track and grasp moving targets. In light of Section 6.3, this task will also test whether frequent replanning can be made beneficial when the target moves.

2. Reactive Obstacle Avoidance: In this task, the robot is instructed to reach a visual goal, but dynamic obstacles are introduced into the workspace mid-trajectory. This evaluates the neuro-symbolic primitive selector (\pi_{\phi}^{h}) and the discrete sub-goal thresholding (\|z_{t}-z_{g}^{(m_{t})}\|_{2}<\tau). If an obstacle blocks a trajectory, the system is expected to halt milestone progression or switch primitives to mitigate error accumulation; the frequency and duration of such halts also serve to characterize pointer stalls (Section 4.2).

3. Visual Nuisance Generalization: To test the nuisance robustness motivated in Sections 4.1 and 5.2, and to estimate the nuisance slack \delta_{\eta} of Section 5.3, the agent will be trained in a visually sterile environment and tested in environments with visual distractors, such as changing background textures, moving camera perspectives, and variable illumination. The metric of interest is the zero-shot success rate decay compared to pixel-reconstruction baselines.

4. Language Intent Disambiguation: To evaluate the system’s semantic comprehension, this task presents environments in which visual goals are ambiguous without language context (e.g., distinguishing between an instruction to move a cup and an instruction to throw it away). This benchmark tests whether the text embedding c_{\text{text}} produced by the lightweight frozen encoder provides sufficient conditioning for the primitive selector and the flow-matching solver, which are the modules that receive it (Eqs. 8 and 12), to resolve the overarching task intent without a large language model.

6.5.3 Verification of Theoretical Properties and Ablations

Beyond task-level performance, we plan the following measurements, each of which targets a specific statement or design choice of Sections 4 and 5:

1. Latency on deployment hardware (Proposition 5.1): Extend the measurements of Section 5.1 to robotic deployment hardware and to trained configurations, including the two-camera encoder, by measuring T_{\text{VPA}} and its components t_{E}, t_{h}, t_{P}, and t_{l}(H) (Eq. 17) as H and K are varied, together with the end-to-end closed-loop control frequency.
2. Encoder regularity (Assumptions 5.1 and 5.2): Using the privileged state y_{t}, sample state pairs under controlled nuisance variation and estimate \hat{\mu}_{E}, \hat{L}_{E}, and \hat{\delta}_{\eta} from the lower and upper envelopes of \|z-z^{\prime}\|_{2} as a function of \|y-y^{\prime}\|_{2} (e.g., via quantile regression), with and without spectral normalization of the encoder.
3. Phase-onset residuals (Proposition 5.3): On tasks with M>1, record the residual r_{k} at every milestone transition and compare its empirical distribution with the conditional bound (\tau^{\star}+\hat{\delta}_{\eta})/\hat{\mu}_{E}, together with the rates of premature transitions, missed transitions, and stalls under a per-milestone time-out.
4. Threshold calibration (Eq. 9): Compare a fixed absolute threshold with the variance-margin-scaled threshold of Eq. 9c across training checkpoints, and sweep the tolerance \kappa to trace the trade-off between soundness and completeness described in Section 5.3.
5. Horizon stability (Eq. 15): Building on Section 6.3, compare the raw uncertainty score \sigma_{t+1} with the filtered signal \bar{\sigma}_{t+1} in terms of the frequency of horizon switches and the magnitude of action discontinuities at chunk boundaries, and examine whether keeping consecutive chunks consistent changes the effect of the executed horizon on task success.

6.5.4 Baselines and Evaluation Metrics

The VPA architecture will be benchmarked against autoregressive VLA models (e.g., RT-2 [1]), generalist policies with continuous action heads (e.g., Octo [23] and \pi_{0} [14]), and diffusion-based behavior cloning policies [34, 11], under identical observations, training data, and evaluation protocols. Including continuous-head baselines isolates the contribution of the lightweight latent pipeline from that of non-autoregressive action generation, since the latter reduction in sequential depth is shared by these methods (Section 5.1). The primary evaluation metrics will include:

1. Task Success Rate (%): Measured across static and dynamic environments, with several training seeds.
2. End-to-End Inference Latency (ms): Wall-clock time from image capture to motor command generation, reported together with its per-module breakdown and the deployment hardware.
3. Control Frequency (Hz): The maximum stable closed-loop execution rate.
4. Representation Health: The variance and covariance penalties v(Z) and c(Z), together with the effective rank of C(Z) [38], evaluated for the frozen visual encoder under visual shift.
5. Phase-Transition Fidelity: The distribution of phase-onset residuals r_{k} and the rates of premature transitions, missed transitions, and stalls.
6. Execution Smoothness: The frequency of changes in the executed horizon H_{t} and the magnitude of action discontinuities at chunk boundaries.

7 Conclusion

The pursuit of generalized embodied intelligence has recently been supported by the scaling of autoregressive Vision-Language-Action (VLA) models. However, the sequential token generation on which many of these models rely makes them challenging to deploy for the high-frequency, reactive demands of physical robotic control.

In this paper, we introduced the Goal-Conditioned Vision-Predictor-Action (VPA) framework, proposing a shift from text-conditioned autoregressive decoding to latent-predictive continuous control, and implemented it as an open-source reference implementation. By decoupling high-level task logic from low-level motor execution and utilizing a lightweight frozen text encoder to capture user instructions, the VPA framework aims to retain semantic task comprehension without relying on computationally heavy Large Language Models (LLMs).

Our analysis establishes three results at different levels of strength. First, formulating multi-step action chunking through Conditional Flow-Matching (CFM) reduces the number of sequential network evaluations per decision step from \Theta(H\cdot d_{a}) for autoregressive token decoding to K+3, a quantity independent of the action horizon H; the arithmetic cost of each solver evaluation nevertheless still scales with the chunk dimensionality H\cdot d_{a}, so the per-cycle cost is not \mathcal{O}(1) in H. Second, the VICReg variance and covariance penalties exclude fully collapsed representations from the global minimizers of the joint-embedding objective whenever a non-degenerate representation attains a lower objective value, and they exclude dimensional collapse on batches where both penalties vanish; the realized variance margin further provides a principled scale for calibrating the milestone completion threshold \tau. Third, discrete sub-goal thresholding bounds the physical residual at each phase transition by (\tau+\delta_{\eta})/\mu_{E}, but only under a co-Lipschitz condition on the encoder with respect to task-relevant physical state. Because Vision Transformers are not globally Lipschitz without explicit constraints, and because this lower bound cannot be enforced through architectural constraints alone, we regard the containment of error across task phases as an empirical expectation rather than a formal guarantee.

Initial empirical validation.

The reference implementation provides a first proof of concept for the framework (Section 6). The trainable modules contain fewer than 20M parameters, and the frozen CLIP text encoder (63.4M parameters) is evaluated only once per episode, outside the per-step dependency chain. Each model trained in under three hours on a single NVIDIA L4 GPU. On an Apple M4, a complete decision step of the single-camera reference configuration took 10.5\text{--}14.2 ms for K\in\{1,2,3\} and H\leq 64, and the per-stage timings were consistent with the horizon-independent sequential depth of Proposition 5.1 (Section 5.1). On LIBERO-10, with a vision encoder trained from random initialization on the demonstrations alone, the policy succeeded in 40.0\% of 200 evaluation episodes (95\% interval 33.5\text{--}46.9\%) when observing a third-person and a wrist-mounted camera, compared with 10.5\% with the third-person camera alone. In an ablation of the executed horizon, executing nearly the full action chunk before replanning gave the best results. We regard these results as promising but preliminary: they indicate that a latent control loop without a language-model decoder can be trained from demonstrations to complete long-horizon manipulation tasks in a meaningful fraction of trials, while its per-step computation remains small enough to run at low latency on commodity hardware.

Limitations.

The repertoire of high-level behaviors is bounded by the fixed primitive dictionary \mathcal{U}, and zero-shot synthesis of skills outside \mathcal{U} is not addressed by the present formulation. The empirical results come from a single training run per configuration with 20 trials per task. They were not compared with baselines under the same protocol, and they remain below success rates previously reported on this suite. The experiments used a single goal image per task (M=1) and primitive labels derived heuristically from the gripper command, so the multi-milestone sequencing mechanism and the conditional analysis of Proposition 5.3 were not exercised; the encoder constants \mu_{E}, L_{E}, and \delta_{\eta} and the phase-onset residuals r_{k} have not yet been measured. The uncertainty-adaptive execution horizon of Eq. 15 did not improve task success in our ablation (Section 6.3). Latency was measured for a single-camera reference configuration on one hardware platform, not for the released checkpoints or on robotic deployment hardware, and closed-loop control frequency under real-time operation was not measured. Finally, the single-frame observation of the original formulation proved insufficient in our experiments, and the reported success rate depends on extending I_{t} to two camera views.

We therefore view VPA as an analytically characterized architecture with an initial working implementation, whose theoretical properties are stated together with the assumptions they require and whose empirical support is, so far, limited to the setting reported here. The next steps are to evaluate the framework under the standard LIBERO protocol with baselines, on multi-milestone tasks, and in the dynamic simulation environments outlined in Section 6.5. We expect the explicit separation between unconditional and conditional properties to provide a sound basis for this work, and we hope that the released code and checkpoints (Section 1.1) will help others reproduce, test, and improve upon these results.

References

1. [1] A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, X. Chen, K. Choromanski, T. Ding, D. Driess, A. Dubey, C. Finn, et al. (2023) Rt-2: vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818.
2. [2] M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, et al. (2024) Openvla: an open-source vision-language-action model. arXiv preprint arXiv:2406.09246.
3. [3] S. Huang, Z. Jiang, H. Dong, Y. Qiao, P. Gao, and H. Li (2023) Instruct2act: mapping multi-modality instructions to robotic actions with large language model. arXiv preprint arXiv:2305.11176.
4. [4] Z. Zhu, S. Wu, S. Zhao, Z. Zhao, J. Zhang, S. Li, Y. Wang, A. T. Luu, X. Zhou, F. Li, and H. Luo (2026) NS-VLA: towards neuro-symbolic vision-language-action models. Note: arXiv preprint arXiv:2603.09542 External Links: 2603.09542
5. [5] J. Sun, W. Zhang, Z. Qi, S. Ren, Z. Liu, H. Zhu, G. Sun, X. Jin, and Z. Chen (2026) Vla-jepa: enhancing vision-language-action model with latent world model. arXiv preprint arXiv:2602.10098.
6. [6] Y. LeCun (2022) A path towards autonomous machine intelligence. Note: OpenReview preprint, version 0.9.2, 2022-06-27
7. [7] M. Assran, Q. Duval, I. Misra, P. Bojanowski, P. Vincent, M. Rabbat, Y. LeCun, and N. Ballas (2023) Self-supervised learning from images with a joint-embedding predictive architecture. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15619–15629.
8. [8] L. Maes, Q. L. Lidec, D. Scieur, Y. LeCun, and R. Balestriero (2026) Leworldmodel: stable end-to-end joint-embedding predictive architecture from pixels. arXiv preprint arXiv:2603.19312.
9. [9] W. Zhang, B. Terver, A. Zholus, S. Chitnis, H. Sutaria, M. Assran, R. Balestriero, A. Bar, A. Bardes, Y. LeCun, et al. (2026) Hierarchical planning with latent world models. arXiv preprint arXiv:2604.03208.
10. [10] T. Z. Zhao, V. Kumar, S. Levine, and C. Finn (2023) Learning fine-grained bimanual manipulation with low-cost hardware. arXiv preprint arXiv:2304.13705.
11. [11] C. Chi, Z. Xu, S. Feng, E. Cousineau, Y. Du, B. Burchfiel, R. Tedrake, and S. Song (2025) Diffusion policy: visuomotor policy learning via action diffusion. The International Journal of Robotics Research 44 (10-11), pp. 1684–1704.
12. [12] Y. Lipman, R. T. Chen, H. Ben-Hamu, M. Nickel, and M. Le (2022) Flow matching for generative modeling. arXiv preprint arXiv:2210.02747.
13. [13] M. S. Albergo and E. Vanden-Eijnden (2022) Building normalizing flows with stochastic interpolants. arXiv preprint arXiv:2209.15571.
14. [14] K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al. (2024) \pi_{0}: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164.
15. [15] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. (2020) An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929.
16. [16] M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, et al. (2023) Dinov2: learning robust visual features without supervision. arXiv preprint arXiv:2304.07193.
17. [17] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. (2021) Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748–8763.
18. [18] H. Kress-Gazit, G. E. Fainekos, and G. J. Pappas (2009) Temporal-logic-based reactive mission and motion planning. IEEE transactions on robotics 25 (6), pp. 1370–1381.
19. [19] A. Bardes, J. Ponce, and Y. LeCun (2021) Vicreg: variance-invariance-covariance regularization for self-supervised learning. arXiv preprint arXiv:2105.04906.
20. [20] B. Liu, Y. Zhu, C. Gao, Y. Feng, Q. Liu, Y. Zhu, and P. Stone (2023) LIBERO: benchmarking knowledge transfer for lifelong robot learning. In Advances in Neural Information Processing Systems, Vol. 36.
21. [21] T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida (2018) Spectral normalization for generative adversarial networks. arXiv preprint arXiv:1802.05957.
22. [22] H. Kim, G. Papamakarios, and A. Mnih (2021) The lipschitz constant of self-attention. In International Conference on Machine Learning, pp. 5562–5571.
23. [23] O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu, et al. (2024) Octo: an open-source generalist robot policy. arXiv preprint arXiv:2405.12213.
24. [24] Y. Huang and H. Raza (2026) Knowledge, rules and their embeddings: two paths towards neuro-symbolic jepa. arXiv preprint arXiv:2603.13265.
25. [25] Y. Huang (2026) SJEPA: learning elegant latent dynamics with hybrid symbolic-neural predictors. arXiv preprint arXiv:2608.04060.
26. [26] Y. Lee (2026) NS-JEPA: Towards Efficient Embodied AI via Neuro-Symbolic Joint-Embedding Predictive Architecture. Note: Zenodo, https://doi.org/10.5281/zenodo.22493915
27. [27] R. Balestriero and Y. LeCun (2025) Lejepa: provable and scalable self-supervised learning without the heuristics. arXiv preprint arXiv:2511.08544.
28. [28] M. Assran, A. Bardes, D. Fan, Q. Garrido, R. Howes, M. Komeili, M. Muckley, A. Rizvi, C. Roberts, K. Sinha, A. Zholus, S. Arnaud, A. Gejji, A. Martin, F. Robert Hogan, D. Dugas, P. Bojanowski, V. Khalidov, P. Labatut, F. Massa, M. Szafraniec, K. Krishnakumar, Y. Li, X. Ma, S. Chandar, F. Meier, Y. LeCun, M. Rabbat, and N. Ballas (2025) V-jepa 2: self-supervised video models enable understanding, prediction and planning. arXiv preprint arXiv:2506.09985.
29. [29] S. Miao, N. Feng, J. Wu, Y. Lin, X. He, D. Li, and M. Long (2026) Jepa-vla: video predictive embedding is needed for vla models. arXiv preprint arXiv:2602.11832.
30. [30] A. Vujinović and A. Kovačević (2026) ACT-jepa: novel joint-embedding predictive architecture for efficient policy representation learning. IEEE Access.
31. [31] X. Liu, C. Gong, and Q. Liu (2022) Flow straight and fast: learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003.
32. [32] Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov (2019) Roberta: a robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
33. [33] B. Lakshminarayanan, A. Pritzel, and C. Blundell (2017) Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems 30.
34. [34] J. Ho, A. Jain, and P. Abbeel (2020) Denoising diffusion probabilistic models. Advances in neural information processing systems 33, pp. 6840–6851.
35. [35] S. Ross and D. Bagnell (2010) Efficient reductions for imitation learning. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp. 661–668.
36. [36] NVIDIA (2026) Isaac Sim. Note: https://developer.nvidia.com/isaac/simAccessed: 2026-10-08
37. [37] S. Tao, F. Xiang, A. Shukla, Y. Qin, X. Hinrichsen, X. Yuan, C. Bao, X. Lin, Y. Liu, T. Chan, et al. (2024) Maniskill3: gpu parallelized robotics simulation and rendering for generalizable embodied ai. arXiv preprint arXiv:2410.00425.
38. [38] O. Roy and M. Vetterli (2007) The effective rank: a measure of effective dimensionality. In 2007 15th European signal processing conference, pp. 606–610.