EqC: Equilibrium Energy Control for Introspective Lifelong Robot Learning

Abstract

Long-term robot autonomy depends on more than effective action generation: a policy must recognize when its behavior is no longer sufficiently supported and use assistance to improve. We introduce Equilibrium Energy Control (EqC), a framework for introspective lifelong robot learning built around an explicit conditional energy landscape. EqC casts generative action refinement as equilibrium-seeking optimization, while the geometry of the refinement process reveals whether the generated behavior remains consistent with the learned landscape. This competence signal enables Energy-Gated DAgger, where persistent energy-consistency violations trigger expert takeover and the resulting recovery trajectories are aggregated for subsequent policy updates. EqC integrates naturally into both Diffusion Transformer policies and flow-matching-based vision-language-action (VLA) policies. Across simulation and real-world bimanual manipulation, EqC matches or improves control performance and robustness while enabling timely intervention near the policy’s competence boundary. A real-world study further shows how these interventions progressively expand the range of situations handled autonomously, providing a practical path from static generative controllers toward self-improving robot systems.

0

Figure 1: Overview of Equilibrium Energy Control (EqC). Top: EqC generates actions through energy-based refinement, while the geometry of the refinement path reveals when autonomous execution is weakly supported. Energy-Gated DAgger uses this signal to trigger expert takeover and subsequent policy improvement. Bottom: In real-world deployment, persistent U_{\Delta E} alarms prompt timely assistance before failure, progressively converting assisted interventions into fully autonomous execution through lifelong learning.

1 Introduction

Robots deployed in open-ended environments must manage their autonomy as operating conditions change (Farid et al., 2024; Filos et al., 2020). A policy may perform reliably when the current situation is well represented by prior experience, yet become unreliable as deployment approaches the limits of that experience. This transition defines the policy’s context-dependent competence boundary. An introspective robot recognizes its proximity to this boundary and regulates autonomous execution before uncertainty develops into failure. Such awareness is also central to lifelong robot learning: assistance requested near the competence boundary protects deployment and reveals where corrective experience is most valuable. Modern generative robot policies, however, are commonly deployed as static action generators that provide little indication of whether their generated behavior remains reliable in the current context (Chi et al., 2025; Black et al., 2024). Competence awareness can instead turn these boundary cases into targeted opportunities for learning rather than avoidable failures.

We introduce Equilibrium Energy Control (EqC), an equilibrium energy formulation inspired by Equilibrium Matching (Wang and Du, 2025) that equips generative robot policies with an explicit conditional energy landscape. Given visual observations, language instructions, and robot state, EqC optimizes an action chunk on this landscape and executes a short prefix under receding-horizon control. The optimization trajectory also reveals whether refinement follows the energy-progress relation learned from expert demonstrations, providing an introspective view of the policy’s competence during execution. Energy-Gated DAgger uses the resulting path-wise energy-consistency signal to trigger expert intervention and aggregate recovery trajectories near the policy’s current competence boundary. This condition-dependent energy formulation therefore supports action generation, online competence monitoring, and targeted data acquisition, while integrating naturally with both Diffusion Transformer policies (Peebles and Xie, 2023) and flow-matching-based vision-language-action (VLA) policies (Black et al., 2024; Physical Intelligence et al., 2025).

We evaluate EqC on simulation benchmarks including LIBERO (Liu et al., 2023), RoboTwin 2.0 (Chen et al., 2025b), VLABench (Zhang et al., 2025), and LIBERO-Plus (Fei et al., 2025), together with real-world bimanual manipulation. EqC matches or improves control performance and robustness across visual and task variation, while its geometry-derived competence signal enables online detection of failure-prone execution. In real-world lifelong learning experiments, the resulting requests for assistance concentrate recovery data near the policy’s competence boundary, and subsequent updates convert previously assisted situations into autonomous capabilities. This process progressively enlarges the range of situations the robot can handle independently, which we refer to as autonomous competence expansion. Code is available on the GitHub page. Video is provided in the supplementary material.

Our contributions are summarized as follows:

• Equilibrium energy control. EqC formulates action generation as equilibrium-seeking optimization under an explicit conditional energy function. The formulation integrates with both Diffusion Transformer and flow-matching-based VLA policies, providing a common energy-based control mechanism across distinct policy families.
• Introspection from energy geometry. EqC characterizes finite energy progress across the action-refinement trajectory and measures its agreement with the learned equilibrium relation. The resulting signal detects when autonomous execution approaches the policy’s competence boundary and provides an online criterion for timely assistance.
• Energy-Gated DAgger for lifelong robot learning. Persistent energy-consistency violations trigger expert recovery, and the resulting trajectories are aggregated to update the policy around its current competence boundary. Real-world deployment shows how this closed loop converts assisted situations into autonomous capabilities and progressively expands robot competence.

2 Related Work

Generative models for robotics.

Generative modeling has driven substantial progress in robot learning through diffusion-based policies (Janner et al., 2022; Ajay et al., 2022; Chi et al., 2025; Huang et al., 2024; Huang et al., 2025), world models (Zhang and Du, 2026; Chen et al., 2025a; Yuan et al., 2026), and vision-language-action policies (Wang et al., 2026). Within VLA policies, iterative action generation is commonly instantiated with diffusion, as in RDT-1B (Liu et al., 2025), or with flow-matching action experts, as in \pi_{0} and \pi_{0.5} (Lipman et al., 2022; Black et al., 2024; Physical Intelligence et al., 2025). Recent analysis relates the effectiveness of these generative policies to noise-covered supervision and iterative computation (Pan et al., 2026). EqC builds on this structure by realizing action refinement as optimization over an explicit conditional energy landscape.

Energy-based models for robotics.

Energy-based models represent compatibility through a scalar function over candidate configurations, turning inference into optimization on a learned landscape (LeCun et al., 2006). In robot learning, observation–action energy functions have been learned for imitation and used to optimize actions at inference time (Florence et al., 2022), with subsequent work improving their contrastive objectives, interpolation behavior, and scaling to transformer policies (Singh et al., 2023; Davies et al., 2025). Recent energy-based VLA work uses terminal action energy as a confidence measure for visual anomaly detection, while leaving its use for intervention-driven real-world data aggregation unexplored (Naghiyev et al., ). Equilibrium Matching (EqM) (Wang and Du, 2025) learns stationary generative dynamics with an implicit energy structure, and GeCO (Zhang et al., 2026) carries this implicit-energy formulation into robot control through iterative refinement and adaptive computation. EqC makes the conditional energy explicit in the deployed control field, allowing finite progress along the optimization trajectory to support both action generation and introspection.

Failure detection and competence-aware autonomy.

Failure detection and uncertainty estimation provide useful signals for deciding when autonomous execution should yield to supervision (Farid et al., 2024; Filos et al., 2020). DAgger establishes the iterative aggregation of on-policy expert corrections (Ross et al., 2011), while robot-gated variants use predicted action error, ensemble disagreement, state novelty, and learned risk to decide when those corrections are needed (Zhang and Cho, 2016; Menda et al., 2019; Hoque et al., 2021). This principle also appears in competence-aware autonomy, where a robot adjusts its level of autonomy according to its estimated ability in the current situation (Basich et al., 2020). For generative manipulation policies, diffusion training loss has also served as a failure signal for interactive data collection (Lee et al., 2025). EqC instead reads introspective information from the consistency of energy progress across the full refinement path and uses persistent violations to gate recovery data collection.

3 EqC for Introspective Lifelong Robot Learning

3.1 Overview

EqC organizes robot control and lifelong improvement around the condition-dependent energy function E_{\theta}(x,z). Its conditional energy landscape refines actions through energy gradients, while finite energy changes along the same refinement trajectory provide an introspective signal of policy competence. Energy-Gated DAgger uses this signal to request expert intervention and aggregate targeted recovery experience. We first formulate energy-based action refinement, then derive its introspective readout and use that signal for lifelong learning.

3.2 Equilibrium Energy Control

We consider a robot policy that predicts a continuous action chunk a\in\mathbb{R}^{D} from a condition z, which may include images, a language instruction, robot proprioception, and recent observation history. Demonstrations are drawn from an unknown conditional distribution p_{\mathrm{data}}(a\mid z). We parameterize the bridge from noise to data by t\in[0,1], with t=0 at noise and t=1 at data, and sample t uniformly during training. For \varepsilon\sim\mathcal{N}(0,I), the bridge displacement, intermediate state, and remaining schedule integral are

d:=a-\varepsilon,\qquad x_{t}:=\varepsilon+td,\qquad C(t):=\int_{t}^{1}c(s)\,\mathrm{d}s.

A nonnegative schedule c(t) controls the magnitude of the target field c(t)d.

EqC represents action quality with a differentiable energy E_{\theta}:\mathbb{R}^{D}\times\mathcal{Z}\to\mathbb{R}. Its gradient g_{\theta}(x,z)=\nabla_{x}E_{\theta}(x,z) is a stationary control field: it depends on the current action and condition, but not on the bridge coordinate. At x_{t}, the demonstrated displacement supplies the desired direction, and the energy is trained with the gradient-matching objective

\mathcal{L}_{\mathrm{grad}}=\mathbb{E}_{(a,z),\varepsilon,t}\left[\frac{1}{D}\left\|\nabla_{x}E_{\theta}(x_{t},z)-c(t)d\right\|_{2}^{2}\right].

The bridge coordinate is used only to sample training targets; the deployed field depends on the current action and condition alone. At inference, EqC initializes x^{(0)}\sim\mathcal{N}(0,I) and applies x^{(k+1)}=x^{(k)}+\eta_{k}\nabla_{x}E_{\theta}(x^{(k)},z) for k=0,\ldots,K-1, yielding \hat{x}=x^{(K)}. The controller executes a prefix and replans from the next observation. Standard smoothness conditions give monotonic energy progress for sufficiently stable step sizes (Appendix C.4).

Architecture compatibility.

The framework requires only a differentiable, per-sample scalar over an action chunk. It may be produced by a dedicated energy head or by a differentiable scalarization of action-conditioned features. In our implementations, a stationary backbone representation f_{\theta} defines E_{\theta}(x,z)=\langle x,f_{\theta}(x,z)\rangle over active action dimensions. The DiT model is conditioned on RGB observations, language, and robot state. The VLA model retains the \pi_{0} multimodal prefix, action expert, action representation, and padding convention. Both expose the same interface x\mapsto E_{\theta}(x,z)\mapsto\nabla_{x}E_{\theta}(x,z), so EqC changes the action objective and its optimization through an interface shared across observation encoders and policy scales.

3.3 Introspection from Energy Geometry

The gradient-matching target also determines finite energy progress along a demonstration bridge. For x_{s}=\varepsilon+sd, integrating \frac{d}{ds}E_{\theta}(x_{s},z)=\nabla_{x}E_{\theta}(x_{s},z)^{\top}d=c(s)\|d\|_{2}^{2} gives

E_{\theta}(a,z)-E_{\theta}(x_{t},z)=\|d\|_{2}^{2}C(t).

Violation of this relation along a realized refinement trajectory reveals disagreement between local optimization and the energy geometry learned from demonstrations.

At deployment, we treat \hat{x}=x^{(K)} as the approximate equilibrium and use it as a self-anchor. Let v=\hat{x}-x^{(0)} and D_{v}=\max(\|v\|_{2}^{2},\delta). Projection onto the noise-to-equilibrium chord assigns each iterate a progress coordinate \tau_{k}, after which we compare its normalized energy-to-go with the bridge reference:

\displaystyle\tau_{k} \\ \displaystyle:=\operatorname{clip}\!\left(\frac{\langle x^{(k)}-x^{(0)},v\rangle}{D_{v}},0,1\right), \\ \displaystyle r_{k}^{\Delta E} \\ \displaystyle:=\frac{E_{\theta}(\hat{x},z)-E_{\theta}(x^{(k)},z)}{D_{v}}-C(\tau_{k}), \\ \displaystyle U_{\Delta E}(z) \\ \displaystyle:=\frac{1}{K}\sum_{k=0}^{K-1}\left|r_{k}^{\Delta E}(z)\right|.

The score is small when realized energy progress follows the bridge relation and increases when that relation is violated. Absolute aggregation prevents cancellation across refinement states, while averaging makes the scale comparable across different values of K. It requires neither an expert action nor an auxiliary uncertainty model at deployment. Appendix C.5 relates the residual to field error and off-chord deviation.

3.4 Energy-Gated DAgger for Lifelong Robot Learning

We use EqC’s energy-consistency score to gate a DAgger-style data aggregation loop (Ross et al., 2011), an instantiation we call Energy-Gated DAgger. Before each action prefix, the controller computes U_{\Delta E}(z_{j}); m consecutive threshold crossings trigger expert takeover, and the recovery trajectory is retained for the next policy update. The gate therefore responds to persistent path-wise inconsistency rather than raw energy magnitude.

This deployment loop concentrates supervision on states where the current controller lacks support: the energy used to generate behavior also determines when autonomy should yield, focusing new experience near the policy’s competence boundary.

4 Why EqC Fits Generative Robot Policies

A favorable generation regime.

Image synthesis must cover diverse pixel realizations left unspecified by a class or text prompt. In control, rich visual, language, and proprioceptive conditions narrow the action choice, success requires one competent action, and receding-horizon execution replans after a short prefix. Action outputs are also much smaller than images. These properties make conditional action generation a natural setting for a structured optimizer. This view agrees with the finding that noise-covered supervised iteration explains more of generative-control performance than distinct action modes on common benchmarks (Pan et al., 2026), and suggests why the explicit-energy cost observed for EqM image generation (Wang and Du, 2025) need not transfer directly to control.

Approximation and optimization.

Appendix C.3 compares EqC with matched stationary and time-conditioned regressors and shows

R_{\mathrm{EqC}}^{\Theta}-R_{\mathrm{tc}}^{\star}\leq\left(\sigma_{\mathrm{act}}+\sigma_{\mathrm{scale}}+\delta_{\Theta}\right)^{2}.

The terms measure residual action ambiguity, uncertainty about bridge progress after removing time, and finite energy class approximation. When they are small, explicit energy has limited regression cost. It then exposes a property not provided by an unconstrained field: sufficiently small gradient-ascent steps provably increase the scalar optimized at inference. The same path calculus relates U_{\Delta E} to mean absolute integrated field error and trajectory deviation. Full definitions and proofs are given in Appendix C.

5 Experiments

The experiments evaluate the three capabilities developed by EqC. We first study its effectiveness as a generative controller across DiT and VLA policies in simulation and real-world manipulation. We then test path-wise energy consistency as an introspective signal for online failure detection. Finally, we close the loop with Energy-Gated DAgger to evaluate lifelong improvement on a physical robot. Complete benchmark and implementation protocols, failure-detection analyses, real-robot details, and ablations are provided in Appendices A, B, D, and G, respectively.

5.1 Simulated and Real-World Benchmarks

5.1.1 Setup

We evaluate action generation and competence awareness on LIBERO (Liu et al., 2023), RoboTwin 2.0 (Chen et al., 2025b), VLABench (Zhang et al., 2025), and LIBERO-Plus (Fei et al., 2025). All methods use each benchmark’s binary success criterion. The DiT experiments share one conditioning stack, action interface, and demonstration set across objectives and are built on a modular diffusion-policy implementation (Dong et al., 2024). The VLA experiments use the same flow-matching VLA backbone within each comparison. EqC always refines a noise-initialized action chunk and executes a prefix under receding-horizon control. Appendix A gives the benchmark definitions, rollout budgets, and implementation details.

5.1.2 Diffusion Transformer

We adapt EqC to a DiT architecture and evaluate it on the four LIBERO suites. Table 1 follows the benchmark comparison reported by GeCO (Zhang et al., 2026) and places EqC’s multi-seed mean alongside those published baselines. EqC achieves the highest aggregate result, with 94.0\% average success across training seeds. Appendix A.4 reports the complete EqC statistics together with our independent multi-seed reproductions of the matched rectified-flow and GeCO baselines; Appendix A.2 describes the protocol.

5.1.3 Vision-Language-Action Policies

We integrate EqC into a VLA backbone by adapting the \pi_{0} policy while preserving its multimodal architecture. EqC replaces flow matching with the energy objective E_{\theta}(x,o)=\langle x,v_{\theta}(x,o,0)\rangle, evaluated at the fixed model time t_{\mathrm{model}}=0. Training matches energy gradients along interpolated action trajectories and masks inactive padded dimensions; inference refines actions iteratively with the learned energy field. We evaluate the resulting VLA policies on RoboTwin 2.0, VLABench, and LIBERO-Plus, with benchmark protocols in Appendix A.2; supplementary LIBERO results are provided in Appendix A.6.

RoboTwin 2.0.

RoboTwin 2.0 evaluates bimanual manipulation under structured randomization of clutter, lighting, background, tabletop height, and language instructions. Under its clean Easy and domain-randomized Hard conditions, Table 2 shows that EqC raises the \pi_{0} averages from 49/18 to 63/33, compared with 50/28 for \pi_{0}+GeCO. The improvement is particularly pronounced on Adjust Bottle and Click Alarmclock, including under the more challenging Hard condition.

VLABench.

We evaluate on VLABench across its five standardized tracks (T1–T4 and T6), covering object generalization, commonsense reasoning, semantics, and appearance shifts. As shown in Table 1, EqC consistently enhances generalization for both flow-matching backbones, boosting \pi_{0} from 29.4\% to 36.7\% and \pi_{0.5} from 24.5\% to 27.1\%.

LIBERO-Plus.

LIBERO-Plus tests robustness under seven controlled perturbation families. Table 2 shows that EqC raises overall success from 53.6\% to 65.8\%. The most prominent improvements occur under camera viewpoint (15.8\%\to 61.7\%) and robot initialization shifts (6.6\%\to 39.0\%), demonstrating superior out-of-distribution robustness.

5.1.4 Real-World Experiments

We evaluate EqC on a bimanual workstation with two Galaxea A1Z manipulators using the Galaxea G0.5(Liu et al., 2026) flow-matching based VLA policy, with 20 independent trials per task under randomized object placements. Flower Insertion (Figure 2, top) requires the robot to grasp and reorient a flexible stem before inserting it through a narrow vase opening; Towel Folding (middle) requires coordinated deformable-object manipulation through lateral folding, regrasping, and a final vertical fold. Debris Sweeping (bottom) evaluates multi-stage bimanual tool coordination, requiring the robot to sweep scattered paper scraps into a handheld dustpan using a brush and deposit them cleanly into a waste bin without spillage. Appendix D provides task definitions, success criteria, and training settings.

Table 3 shows that EqC raises average success from 63.3% to 91.7%. Success improves from 45.0% to 80.0% on Flower Insertion, from 80.0% to 95.0% on Towel Folding, and reaches 100.0% on Debris Sweeping, while introducing moderate inference latency overhead.

5.2 Competence Awareness

5.2.1 Setup

EqC’s energy-consistency signal measures whether inferred energy progress remains aligned with the relation induced by expert demonstrations, enabling the policy to perform introspective competence assessment at inference time. We evaluate this property as online failure detection: successful rollouts are negative examples and failed rollouts are positive examples. An alarm requires the signal to exceed a fixed threshold for m consecutive replanning steps, and detection timing is measured by the fraction of the rollout remaining when an alarm is raised in a failed trial. Appendix B.1 formalizes the competence-aware task definition, while Appendix B.2 provides the simulation failure-detection protocol, including the rollout partition, Receiver Operating Characteristic (ROC) curve construction, and threshold calibration.

5.2.2 Simulated Rollout Analysis

In simulation, we evaluate online competence awareness on LIBERO and LIBERO-Plus using three configurations: Diff-DAgger’s flow-loss detector on \pi_{0} (Lee et al., 2025), the same detector on the EqC backbone, and the full energy-consistency signal U_{\Delta E}. Threshold-dependent metrics use a disjoint split: 10% of the rollouts select and freeze (m,\tau^{*}), and the remaining 90% are used only for evaluation. Under this protocol, the calibration subset selects (m,\tau^{*}) while constraining the FPR to approximately 10\%; the pair remains frozen for held-out evaluation, while Area Under the Receiver Operating Characteristic (AUROC) summarizes the full score ranking. Table 3 shows that EqC’s path-wise energy-consistency signal attains the highest AUROC, TPR, accuracy, and saved inference time, together with the lowest FPR, among Diff-DAgger and EqC + Diff-DAgger baseline; its AUROC reaches 0.9244, compared with 0.8430 for Diff-DAgger. Appendix B.5 details the adapted Diff-DAgger baseline, and Appendix B.6 reports calibration variants, window and Monte Carlo sensitivity, saved inference time, and inference latency.

5.2.3 Real-World Rollout Analysis

We next use Hierarchical Block Stacking as a real-world failure-detection testbed and collect 50 rollouts: 25 successful executions and 25 failed executions. Figure 3(a) shows that the signal remains low for successful executions and rises early for failures, whereas the Diff-DAgger trajectories in Figure 3(b) overlap through most of the horizon. The EqC signal therefore preserves a more useful distinction between supported and unsupported execution under physical sensing and control noise; Appendix D provides the task details.

5.3 Energy-Gated DAgger for Lifelong Learning

5.3.1 Experimental Setup and Protocol

Real-world robot data are costly, and deployment conditions cannot be fully anticipated. We therefore test whether EqC recognizes its competence boundary and uses intervention data to expand it across two bimanual tasks on the Galaxea platform: (1) Hierarchical Block Stacking, requiring the robot to construct a freestanding tower across 3!=6 color permutations under tight contact constraints; and (2) Garment Folding, requiring sequential multi-stage folding of deformable fabric. Both policies start from limited seed demonstrations and follow Algorithm 1 (\gamma=0.2), where persistent U_{\Delta E} violations prompt expert recovery. The aggregated trajectories are used for subsequent policy fine-tuning (Appendix D).

5.3.2 Progressive Performance

Figure 4 (top) tracks competence acquisition over Energy-Gated DAgger deployment rounds across Hierarchical Block Stacking and Garment Folding. We evaluate against an automated Galaxea G0.5 + Diff-DAgger baseline and a skilled human-in-the-loop teleoperator. Under identical data and training budgets, Energy-Gated DAgger substantially outperforms all baselines across three independent experimental runs. On Hierarchical Block Stacking, success rises from an initial 0.0\% to an average of 96.7% after incorporating 30 targeted recovery episodes, outperforming Galaxea G0.5 + Diff-DAgger (80.0\%) and human-triggered intervention (58.3\%). On Garment Folding, the policy progresses from 30.0\% to an average of 98.3% autonomous execution in just 2 rounds (12 targeted recovery episodes), surpassing Galaxea G0.5 + Diff-DAgger (81.7\%) and skilled human intervention (55.0\%). Full round-by-round statistics across all individual runs are reported in Table 17 (Appendix D.4). These results confirm that EqC’s path-wise energy-consistency signal identifies weakly supported states more reliably than Diff-DAgger’s signal or manual monitoring, yielding targeted recovery data that systematically expands autonomous competence. As qualitatively shown in Figure 4 (bottom), the detection of execution anomalies trigger timely U_{\Delta E} alarms to prompt expert recovery in earlier rounds, progressively enabling robust autonomous execution under perturbed setups. Furthermore, under sequential cross-task shift (Hierarchical Cup Stacking, Appendix F), EqC detects task boundary transitions, mitigates catastrophic forgetting, and restores 100.0% dual-task autonomy.

6 Limitations and Scope

Our study focuses on continuous action-chunk policies with stationary refinement schedules. Future work should extend EqC to hybrid, memory-based, and hierarchical policies; deepen the analysis of energy geometry, calibration, and repeated data aggregation; and test how competence thresholds and intervention budgets transfer across embodiments and longer deployments.

7 Conclusion

We presented Equilibrium Energy Control (EqC), an energy-based framework connecting generative robot control with introspective competence awareness. Through an explicit conditional energy landscape, EqC refines actions toward equilibrium and uses path-wise energy consistency to assess whether behavior remains supported; Energy-Gated DAgger turns this introspective signal into targeted expert recovery and policy updates. Across simulation and physical robots, EqC matches or improves control performance and robustness. Real-world lifelong learning experiments further show that repeated recovery and updates expand autonomous competence. Its compatibility with diverse robot-manipulation architectures also offers a path toward energy-based control and introspection in locomotion, navigation, and autonomous driving. Embedding competence awareness into generative control may enable introspective, self-evolving embodied agents whose autonomy grows through lifelong learning while retaining a mechanism for safe intervention.

AI use statement

Generative AI tools were used to improve the readability of the manuscript, create or modify scientific figures, and edit code. The authors carefully reviewed all AI-assisted text, figures, and code, checked them against the underlying research and experimental results, and take full responsibility for the final content of the paper, including any AI-assisted material.

Reproducibility statement

We provide the EqC formulation and theoretical derivations in the main text and Appendix C. The appendices document model and training configurations, benchmark protocols, evaluation splits, failure-detection calibration, real-robot tasks, and baseline implementations (Appendices A, B, and D). The code for the EqC implementations and evaluation procedures is available at the anonymous repository linked in the Introduction and Appendix I.

References

1. Ajay et al. (2022) A. Ajay, Y. Du, A. Gupta, J. Tenenbaum, T. Jaakkola, and P. Agrawal Is conditional generative modeling all you need for decision-making?. arXiv preprint arXiv:2211.15657.
2. Basich et al. (2020) C. Basich, J. Svegliato, K. H. Wray, S. Witwicki, J. Biswas, and S. Zilberstein Learning to optimize autonomy in competence-aware systems. arXiv preprint arXiv:2003.07745.
3. Black et al. (2024) K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al. \pi_{0}: A Vision-Language-Action Flow Model for General Robot Control. arXiv preprint arXiv:2410.24164.
4. Chen et al. (2025a) B. Chen, T. Zhang, H. Geng, C. Zhang, P. Li, K. Song, W. T. Freeman, J. Malik, P. Abbeel, R. Tedrake, et al. Large video planner enables generalizable robot control. arXiv preprint arXiv:2512.15840.
5. Chen et al. (2025b) T. Chen, Z. Chen, B. Chen, Z. Cai, Y. Liu, Z. Li, Q. Liang, X. Lin, Y. Ge, Z. Gu, et al. Robotwin 2.0: a scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation. arXiv preprint arXiv:2506.18088.
6. Chi et al. (2025) C. Chi, Z. Xu, S. Feng, E. Cousineau, Y. Du, B. Burchfiel, R. Tedrake, and S. Song Diffusion policy: visuomotor policy learning via action diffusion. The International Journal of Robotics Research 44 (10-11), pp. 1684–1704.
7. Davies et al. (2025) T. Davies, Y. Huang, A. Gladstone, Y. Liu, X. Chen, H. Ji, H. Liu, and L. Hu EBT-Policy: energy unlocks emergent physical reasoning capabilities. arXiv preprint arXiv:2510.27545.
8. Dong et al. (2024) Z. Dong, Y. Yuan, J. Hao, F. Ni, Y. Ma, P. Li, and Y. Zheng Cleandiffuser: an easy-to-use modularized library for diffusion models in decision making. Advances in Neural Information Processing Systems 37, pp. 86899–86926.
9. Farid et al. (2024) A. Farid, S. Veer, D. Pachisia, and A. Majumdar Task-driven detection of distribution shifts with statistical guarantees for robot learning. IEEE Transactions on Robotics 41, pp. 926–945.
10. Fei et al. (2025) S. Fei, S. Wang, J. Shi, Z. Dai, J. Cai, P. Qian, L. Ji, X. He, S. Zhang, Z. Fei, et al. Libero-plus: in-depth robustness analysis of vision-language-action models. arXiv preprint arXiv:2510.13626.
11. Filos et al. (2020) A. Filos, P. Tigas, R. McAllister, N. Rhinehart, S. Levine, and Y. Gal Can autonomous vehicles identify, recover from, and adapt to distribution shifts?. In International Conference on Machine Learning, pp. 3145–3153.
12. Florence et al. (2022) P. Florence, C. Lynch, A. Zeng, O. A. Ramirez, A. Wahid, L. Downs, A. Wong, J. Lee, I. Mordatch, and J. Tompson Implicit behavioral cloning. In Conference on robot learning, pp. 158–168.
13. Hoque et al. (2021) R. Hoque, A. Balakrishna, E. Novoseller, A. Wilcox, D. S. Brown, and K. Goldberg Thriftydagger: budget-aware novelty and risk gating for interactive imitation learning. arXiv preprint arXiv:2109.08273.
14. Huang et al. (2025) R. Huang, H. Balim, H. Yang, and Y. Du Flexible locomotion learning with diffusion model predictive control. arXiv preprint arXiv:2510.04234.
15. Huang et al. (2024) X. Huang, Y. Chi, R. Wang, Z. Li, X. B. Peng, S. Shao, B. Nikolic, and K. Sreenath Diffuseloco: real-time legged locomotion control with diffusion from offline datasets. arXiv preprint arXiv:2404.19264.
16. Janner et al. (2022) M. Janner, Y. Du, J. B. Tenenbaum, and S. Levine Planning with diffusion for flexible behavior synthesis. arXiv preprint arXiv:2205.09991.
17. LeCun et al. (2006) Y. LeCun, S. Chopra, R. Hadsell, M. Ranzato, F. Huang, et al. A tutorial on energy-based learning. Predicting structured data 1 (0).
18. Lee et al. (2025) S. Lee, X. Kang, and Y. Kuo Diff-dagger: uncertainty estimation with diffusion policy for robotic manipulation. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pp. 4845–4852.
19. Lipman et al. (2022) Y. Lipman, R. T. Chen, H. Ben-Hamu, M. Nickel, and M. Le Flow matching for generative modeling. arXiv preprint arXiv:2210.02747.
20. Liu et al. (2023) B. Liu, Y. Zhu, C. Gao, Y. Feng, Q. Liu, Y. Zhu, and P. Stone Libero: benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems 36, pp. 44776–44791.
21. Liu et al. (2025) S. Liu, L. Wu, B. Li, H. Tan, H. Chen, Z. Wang, K. Xu, H. Su, and J. Zhu Rdt-1b: a diffusion foundation model for bimanual manipulation. In International Conference on Learning Representations, Vol. 2025, pp. 29982–30009.
22. Liu et al. (2026) Y. Liu, Z. Dong, B. Ye, T. Yuan, T. Jiang, A. Yang, S. Cao, H. Liu, Y. Sun, Z. Guo, et al. G0. 5: one autoregressive stream for robot reasoning and action. arXiv preprint arXiv:2608.11739.
23. Menda et al. (2019) K. Menda, K. Driggs-Campbell, and M. J. Kochenderfer Ensembledagger: a bayesian approach to safe imitation learning. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5041–5048.
24. [24] J. Naghiyev, K. Khvastow, A. Gladstone, R. Römer, H. Ji, and A. P. Schoellig Energy-based action heads know when they don’t know. In ICRA 2026 Workshop: From Data to Decisions: VLA Pipelines for Real Robots,
25. Pan et al. (2026) C. Pan, G. Anantharaman, N. Huang, C. Jin, D. Pfrommer, C. Yuan, F. Permenter, G. Qu, N. Boffi, G. Shi, et al. Much ado about noising: dispelling the myths of generative robotic control. In International Conference on Learning Representations, Vol. 2026, pp. 90575–90614.
26. Peebles and Xie (2023) W. Peebles and S. Xie Scalable diffusion models with transformers. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4172–4182.
27. Pertsch et al. (2025) K. Pertsch, K. Stachowicz, B. Ichter, D. Driess, S. Nair, Q. Vuong, O. Mees, C. Finn, and S. Levine Fast: efficient action tokenization for vision-language-action models. arXiv preprint arXiv:2501.09747.
28. Physical Intelligence et al. (2025) Physical Intelligence, K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, et al. \pi_{0.5}: A Vision-Language-Action Model with Open-World Generalization. arXiv preprint arXiv:2504.16054.
29. Ross et al. (2011) S. Ross, G. Gordon, and D. Bagnell A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp. 627–635.
30. Singh et al. (2023) S. Singh, S. Tu, and V. Sindhwani Revisiting energy based models as policies: ranking noise contrastive estimation and interpolating energy models. arXiv preprint arXiv:2309.05803.
31. Wang et al. (2026) Q. Wang, M. Li, J. Guan, J. Ye, S. Xie, Y. Liu, J. Chen, Z. Liang, J. Zhang, X. Hu, et al. Qwen-vla: unifying vision-language-action modeling across tasks, environments, and robot embodiments. arXiv preprint arXiv:2605.30280.
32. Wang and Du (2025) R. Wang and Y. Du Equilibrium matching: generative modeling with implicit energy-based models. arXiv preprint arXiv:2510.02300.
33. Yuan et al. (2026) T. Yuan, Z. Dong, Y. Liu, and H. Zhao Fast-wam: do world action models need test-time future imagination?. arXiv preprint arXiv:2603.16666.
34. Zhang and Cho (2016) J. Zhang and K. Cho Query-efficient imitation learning for end-to-end autonomous driving. arXiv preprint arXiv:1605.06450.
35. Zhang et al. (2025) S. Zhang, Z. Xu, P. Liu, X. Yu, Y. Li, Q. Gao, Z. Fei, Z. Yin, Z. Wu, Y. Jiang, et al. Vlabench: a large-scale benchmark for language-conditioned robotics manipulation with long-horizon reasoning tasks. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 11142–11152.
36. Zhang and Du (2026) X. Zhang and Y. Du World action planner: generalizable decision-making with action-conditioned world models. arXiv preprint arXiv:2607.27599.
37. Zhang et al. (2026) Z. Zhang, R. Huang, Y. Liu, S. Zhu, L. Mou, and H. Zhao Generative control as optimization: time unconditional flow matching for adaptive and robust robotic control. arXiv preprint arXiv:2603.17834.

Appendix A Evaluation Details

A.1 Implementation Protocol

Both implementations use the convention t=0 for noise and t=1 for data, and query the stationary backbone at model time zero. The scalar energy is reduced independently for each sample over active action coordinates, E_{\theta}(x,z)=\sum_{i\in\mathcal{A}}x_{i}f_{\theta,i}(x,z); the VLA mask excludes padded embodiment dimensions. Training differentiates this scalar with respect to the complete action chunk and retains the higher-order graph needed to update \theta. Inference performs gradient ascent and detaches the action between refinement steps. The reported models use c(t)=\min\{4,20(1-t)\}, with C(t) evaluated analytically; the analysis allows any schedule satisfying the stated integrability conditions.

A.2 Benchmarks

LIBERO.

We evaluate the DiT policy on the LIBERO simulation benchmark, which contains four task suites—Goal, Spatial, Object, and Long—with ten tasks per suite. These suites evaluate different aspects of language-conditioned robot manipulation, including changes in task goals, spatial configurations, object categories, and long-horizon compositional tasks. We use the shared DiT architecture and data pipeline described in Section 5.

RoboTwin 2.0.

We evaluate the VLA policy on RoboTwin 2.0, a benchmark for bimanual robot manipulation under both clean and domain-randomized conditions. We consider five representative tasks: Adjust Bottle, Beat Block Hammer, Blocks Ranking (Size), Click Alarmclock, and Click Bell. Each task is evaluated under the Easy and Hard settings. The Easy setting uses clean environments, whereas the Hard setting introduces additional domain randomization. We report task-wise success rates and average performance across the five tasks for both settings. All results are averaged over 100 evaluation episodes per task and setting using the benchmark’s success criteria.

VLABench.

We follow the current official VLA evaluation protocol and report the five tracks with released standardized episode configurations: in-distribution task learning (T1), cross-category and object generalization (T2), commonsense understanding (T3), semantic-instruction understanding (T4), and unseen background or table textures (T6). Each task is evaluated over 50 episodes, and we report track-wise success rates and their average. Cross-task skill transfer (T5) is part of the broader VLABench design, but the official protocol defines it as an open setting in which users choose their own training and evaluation tasks; it does not provide a fixed task split or standardized episode configuration. We therefore omit T5 and use the publicly reproducible five-track configuration rather than introducing a study-specific cross-task split.

LIBERO-Plus.

We evaluate robustness under controlled distribution shifts using LIBERO-Plus, which extends the original LIBERO tasks with perturbations that preserve the underlying task semantics. We consider seven perturbation families: camera viewpoint, robot initialization, language, illumination, background, sensor noise, and object layout. We report success rates for each perturbation family and the corresponding robustness aggregate.

A.3 DiT Details

For LIBERO experiments with the Diffusion Transformer (DiT), we encode visual observations using a pretrained DINOv2-base (dinov2-base) model and language instructions using T5-base (t5-base), with an observation horizon of T_{o}=1. The policy predicts continuous action chunks with horizon T_{a}=16 and D_{a}=7 control dimensions (D=112), executing the first 8 steps under receding-horizon control.

Training.

We train all DiT models for 70k steps with a batch size of 256 in bf16-mixed precision. Training uses the AdamW optimizer with cosine learning-rate decay, a peak learning rate of 1\times 10^{-4}, 5,000 linear warmup steps, and weight decay of 1\times 10^{-4}.

A.4 Multi-Seed DiT Results

Table 5 reports the LIBERO results obtained across multiple EqC training seeds. The final row gives the mean and sample standard deviation for each suite. Unlike the published baseline values retained in the main comparison, Table 6 reports our independent multi-seed reproductions of the matched rectified-flow and GeCO objectives alongside EqC, without listing individual baseline runs.

A.5 VLA Training Details

We fine-tune \pi_{0} and \pi_{0.5} VLAs from OpenPI base checkpoints on LeRobot-format datasets using the AdamW optimizer with gradient clipping (clip_gradient_norm=1.0) and a cosine learning-rate schedule (10k warmup steps and a peak learning rate of 5\times 10^{-5}). Models are trained for 50k steps on LIBERO and 30k steps on RoboTwin 2.0 and VLABench.

RoboTwin 2.0.

We fine-tune a task-specific \pi_{0} model for each of the five bimanual tasks (batch size 32, 30k steps). Inputs comprise three RGB camera views (cam_high, cam_left_wrist, cam_right_wrist) and robot proprioception, predicting action chunks of horizon H=50 with 14 active control dimensions (D=700).

LIBERO and LIBERO-Plus.

For LIBERO, we fine-tune \pi_{0} on the physical-intelligence/libero dataset for 50k steps with batch size 32. For LIBERO-Plus, we directly evaluate the LIBERO-trained checkpoint in a zero-shot manner across all seven perturbation suites without any benchmark-specific adaptation.

VLABench.

For VLABench, models are fine-tuned for 30k steps on vlabench/vlabench_primitive_ft_lerobot across five evaluation tracks, using a batch size of 256 for \pi_{0} and 32 for \pi_{0.5}.

A.6 Supplementary LIBERO Results for VLA Policies

We additionally evaluate EqC with VLA policies (\pi_{0}) on LIBERO to examine whether the formulation transfers to large multimodal action models. Table 7 reports the full comparison across the four LIBERO task suites.

EqC achieves a 94.3% aggregate success rate with the VLA policy, together with strong results on Spatial, Object, and Long. Combined with the DiT experiments, these results demonstrate the applicability of the equilibrium-energy formulation across both policy families. Sections 5.2 and 5.3 further evaluate the introspective competence signal enabled by the VLA instantiation and its use for lifelong learning.

Appendix B Competence Awareness Details

B.1 Task Definition: Online Failure Detection

Our goal is online failure detection rather than detecting visual novelty or distribution shift. In robot manipulation, environmental shifts do not necessarily imply execution failure: an adaptable policy may handle unfamiliar scenes autonomously, where takeover is unnecessary and disruptive. Conversely, executions under nominal setups frequently fail due to contact slip, occlusions, or compounding control error. Intervention should therefore depend on whether the policy is actually failing the task, rather than whether the current observation looks unfamiliar.

Ground-truth labels are determined strictly by task outcomes: successful rollouts are labeled negative (autonomous completion, no intervention needed), while failed rollouts are labeled positive (intervention required). This formulation avoids penalizing successful generalization: standard novelty detectors flag unfamiliar yet competent executions as anomalies, prompting unnecessary human takeover. The reported metrics evaluate the ability to anticipate and catch execution failures rather than measuring observation density. Labels remain tied to final task success rather than the scene configuration.

B.2 Simulation Failure Detection Protocol

We evaluate competence-aware failure detection using three methodological configurations:

1. EqC (ours): Our primary approach quantifies policy competence using the energy-consistency signal U_{\Delta E} derived from the EqC-adapted \pi_{0} policy. As introduced in Section 3.3, this score projects each post-update action onto the noise-to-equilibrium chord using the floored denominator D_{v}=\max(\|x^{(K)}-x^{(0)}\|_{2}^{2},10^{-12}) and averages the absolute energy-consistency residuals according to equation 4.
2. Diff-DAgger (baseline): The standard baseline using the original \pi_{0} policy paired with Diff-DAgger’s multi-pass flow-loss detector (Lee et al., 2025). Implementation details are provided in Section B.5.
3. EqC + Diff-DAgger (hybrid): A hybrid ablation variant that deploys our EqC-adapted \pi_{0} policy for action generation while computing the failure-detection signal using Diff-DAgger’s re-noising and denoising flow-loss scoring mechanism.

All three methods are evaluated on the LIBERO and LIBERO-Plus benchmark suites using 50k-step training checkpoints. A failure alarm is triggered when the detection signal exceeds a predefined threshold for m consecutive replanning steps.

The statistical unit of the analysis is a complete rollout rather than an individual replanning step. Before sampling either split, we stratify the evaluation pool across the 8 dataset variants (the original LIBERO and the 7 LIBERO-Plus perturbation families). Within every stratum, rollouts are assigned proportionally to a 10\% calibration split and a 90\% test split, ensuring that neither split is dominated by any particular task or perturbation type. A rollout is treated as an indivisible unit: its initial-state instance, outcome label, and complete sequence of replanning signals are assigned strictly to one split. No rollout or trajectory segment is shared across calibration and testing, and all compared detectors evaluate on the exact same split membership. Further details regarding the composition of the calibration split are provided in Section B.4.

Policy checkpoints are fixed before detector calibration and remain unchanged throughout the analysis. The calibration split is used only to select the temporal window m and operating threshold \tau^{*}; it is not used for checkpoint selection or detector adaptation. Once selected, (m,\tau^{*}) is frozen and applied once to the held-out test rollouts. Test outcomes and signal trajectories therefore cannot influence the alarm rule. By preserving the task, perturbation, and outcome composition across the two splits, the protocol evaluates rollout-level failure discrimination without confounding the comparison through a different benchmark mixture.

Each rollout produces a time-varying sequence of failure-detection signals. For a fixed window size m and threshold \tau, a rollout raises an alarm if its signal exceeds \tau for any sequence of m consecutive replanning steps; otherwise, no alarm is raised. We then compute TPR and FPR from rollout-level success and failure labels and construct the ROC curve by sweeping \tau. To examine whether performance remains stable across calibration procedures rather than being tied to a particular threshold-selection rule, we evaluate two calibration strategies on the calibration split:

1. FPR-Constrained Calibration: As reported in the main text (Section 5.2.2), all candidate methods are first evaluated on the 10\% calibration split. By performing a fine-grained sweep over candidate thresholds logged from these rollouts, we fix a given window size m and select the optimal threshold \tau^{*} that maximizes TPR while strictly constraining the FPR to approximately 10\%. This calibrated parameter pair (m,\tau^{*}) is then frozen and applied to evaluate the remaining 90\% test split.
2. Youden’s J Index Calibration: Alternatively, for a given sliding window size m, the optimal threshold \tau^{*} is selected on the 10\% calibration split by maximizing Youden’s J statistic (J=\text{TPR}-\text{FPR}). The calibrated threshold pair (m,\tau^{*}) is subsequently deployed on the 90\% test split to trigger failure alarms.

Across both FPR-constrained calibration and Youden’s J index calibration in Table 9, EqC consistently achieves the highest AUROC and delivers the most favorable overall trade-off among TPR, FPR, and Accuracy across the vast majority of window sizes m. This broader advantage indicates that our competence signal provides robust failure detection without depending on a specific threshold-selection rule.

AUROC is independent of the operating threshold \tau, so we report a single global ranking statistic obtained by sweeping \tau over all labeled rollouts rather than a separate AUROC for each calibration rule. This descriptive quantity is not used to select (m,\tau^{*}) or to compute the threshold-dependent operating metrics, which retain the calibration/test separation above. Regarding the time-saved metric, when an alarm is raised at elapsed horizon h_{\mathrm{alarm}} in an episode capped at H_{\max}, the percentage of saved inference time is calculated as 100(1-h_{\mathrm{alarm}}/H_{\max})\%.

B.3 Real-World Failure Detection Protocol

The real-world testbed evaluates the intermediate Round 3 policy checkpoint fine-tuned on \mathcal{D}_{3} (36 cumulative demonstrations) on Hierarchical Block Stacking. Across nominal block configurations , we collect 50 rollouts comprising 25 successful and 25 failed executions, with binary ground-truth labels assigned strictly from the final task outcome. To reduce short-timescale measurement noise, each detection signal is smoothed with a three-step moving average: starting from the third replanning step, the current value is averaged with the two preceding values. An alarm is raised when the smoothed signal remains above its threshold for m consecutive replanning steps. Appendix D provides the platform, task, and rollout-collection details.

B.4 Composition of the Calibration Split

The shared 10\% calibration split comprises 1{,}203 rollouts in total. To ensure balanced coverage across environmental conditions, we perform stratified sampling across the 8 dataset variants (the original LIBERO and 7 LIBERO-Plus perturbation types). Within each dataset variant, rollouts are selected purely at random and shared identically across all evaluated methods (though ground-truth outcome labels depend on each policy’s execution). As detailed in Table 8, because rollouts are randomly sampled within each perturbation family, the distribution across the four suites naturally yields a nearly uniform breakdown (\approx 25\% per suite) without requiring explicit suite-level stratification.

B.5 Diff-DAgger Baseline Implementation Adapted for \pi_{0}

To adapt Diff-DAgger as a failure-detection baseline for the \pi_{0} backbone, we use \pi_{0}’s conditional flow-matching formulation. The detector evaluates whether an action sequence generated by \pi_{0} is consistent with the flow-matching vector field learned during training. Because this inference-only procedure does not modify the original \pi_{0} training objective, it preserves the \pi_{0} architecture, including the PaliGemma vision-language backbone, action expert, observation preprocessing, and action projection head. The detector is attached directly to the original Euler sampler.

Given an observation o, \pi_{0} first generates a predicted action sequence \hat{a} through its standard Euler sampler:

\hat{a}=S_{\theta}(o,\epsilon_{0}),

where \epsilon_{0}\sim\mathcal{N}(0,I) is an initial Gaussian noise sample. The generated action sequence \hat{a} is then treated as a pseudo-label, so the detector requires no expert demonstrations or additional supervision at test time.

For each Monte Carlo sample q\in\{1,\dots,M\}, we independently sample a flow-matching timestep t_{q} and Gaussian noise \epsilon_{q}:

t_{q}\sim\operatorname{Beta}(1.5,1),\qquad\epsilon_{q}\sim\mathcal{N}(0,I).

Following \pi_{0}’s standard training formulation, we construct a perturbed version of the generated action x_{t_{q}}:

x_{t_{q}}=t_{q}\epsilon_{q}+(1-t_{q})\hat{a},

with the corresponding target velocity defined as

u_{t_{q}}=\epsilon_{q}-\hat{a}.

The \pi_{0} model is then queried using the original observation o, the perturbed action x_{t_{q}}, and the sampled timestep t_{q}, producing the predicted velocity vector:

v_{\theta}(x_{t_{q}},t_{q},o).

The Diff-DAgger score for the q-th Monte Carlo sample is the normalized flow-matching error

\ell_{q}=\frac{1}{HD_{\mathrm{act}}}\sum_{h=1}^{H}\sum_{d=1}^{D_{\mathrm{act}}}m_{d}\left[v_{\theta}(x_{t_{q}},t_{q},o)_{h,d}-u_{t_{q},h,d}\right]^{2},

where H represents the action horizon, D_{\mathrm{act}} denotes the number of active action dimensions, and m_{d}\in\{0,1\} is an action-space mask. This mask is essential for \pi_{0} as its action representation includes zero-padded dimensions, which are excluded from both the pseudo-label construction and loss evaluation.

The final per-query failure score is computed as the averaged flow loss over M Monte Carlo evaluations:

\mathcal{L}_{\mathrm{flow}}=\frac{1}{M}\sum_{q=1}^{M}\ell_{q}.

We reserve K for the number of EqC refinement iterates and use M for the number of Diff-DAgger Monte Carlo samples; the default EqC and Diff-DAgger configurations use K=10 and M=10, respectively.

B.6 Detailed Results

As presented in Table 9 and Table 10, we evaluate failure detection performance and early intervention capability across sliding window sizes m\in[3,13] under both calibration protocols. Note that AUROC reflects the threshold-independent detection capacity evaluated globally across all dataset rollouts (and is thus independent of split partitions and identical across both subtables for any given m), whereas TPR, FPR, Accuracy, and Time Saved reflect operational performance on the 90\% test split using operating thresholds calibrated from the 10\% calibration split.

A comprehensive comparative analysis highlights the clear advantages of EqC across window sizes, operating metrics, and thresholding strategies:

• Dominance in Theoretical Detection Capacity (AUROC): Across all evaluated window sizes m\in[3,13], EqC consistently achieves the highest AUROC values among all compared methods, ascending monotonically from 0.8877 (m=3) to 0.9300 (m=13). It maintains a decisive lead over both the baseline Diff-DAgger (0.8296\text{--}0.8576) and the hybrid variant EqC + Diff-DAgger (0.8778\text{--}0.8978), confirming that energy-derived competence signals provide an inherently stronger and more reliable failure detection capability.
• Superiority under FPR-Constrained Calibration: Under strict false-alarm control (Table 9(a), target \text{FPR}\approx 10\%), EqC significantly outperforms Diff-DAgger in both TPR and overall Accuracy across almost all m. For example, at m=13, EqC reaches 78.9\% TPR and 87.4\% Accuracy, compared to Diff-DAgger’s 63.8\% TPR and 83.1\% Accuracy. Furthermore, EqC maintains a lead over the hybrid variant across the majority of window sizes, demonstrating higher TPR.
• Superiority under Youden’s J Calibration: When optimizing the decision threshold via Youden’s J index (J=\text{TPR}-\text{FPR}, Table 9(b)), EqC consistently achieves higher J statistic values, yielding a distinctly superior trade-off between TPR and FPR control. Under this objective, EqC attains higher peak TPR (reaching up to 87.5\% at m=12) while securing the highest overall classification Accuracy across the vast majority of window sizes (peaking at 86.4\% at m=11). This confirms that EqC’s operational superiority stems from a fundamentally better signal trade-off profile rather than an artifact of a specific calibration rule.
• Responsiveness in Early Intervention (Time Saved): Beyond accurately classifying failures, a practical failure-detection system must intervene timely to allow trajectory recovery. Table 10 reports the execution time saved specifically on failed rollouts within the 90\% test split. Across both calibration protocols, EqC achieves the highest overall time saved across almost all window sizes m, consistently outperforming the baseline Diff-DAgger and surpassing the hybrid variant (EqC + Diff-DAgger) in the vast majority of cases. While the hybrid model also improves upon pure Diff-DAgger—further confirming the value of incorporating energy-based information—EqC provides the most responsive signal overall, timely avoiding rollouts from reaching an irrecoverable failure state.

Overall, these results confirm that EqC provides a robust, scalable, and highly responsive failure-detection signal regardless of the choice of threshold calibration, window size, or performance metric.

To further establish the theoretical upper bound on detection performance independent of split partitioning, Table 11 presents an evaluation conducted directly on the entire dataset (without calibration/test splitting). In this setup, AUROC is calculated by sweeping the threshold globally, while TPR, FPR, and Accuracy are reported using the theoretical optimal threshold determined by maximizing Youden’s J index over the full dataset. Across all evaluated sliding-window sizes m\in[3,13], EqC consistently achieves the highest AUROC and the best optimal TPR/FPR trade-off compared to the Diff-DAgger baseline (M=10) and the hybrid variant. Furthermore, increasing Diff-DAgger’s sample count from M=10 to M\in\{16,32,64,128\} fails to close this performance gap, indicating that merely scaling Monte Carlo dropout samples does not yield a fundamentally better failure-detection signal.

Beyond detection accuracy, Table 12 provides a detailed breakdown of the theoretical maximum inference time saved across various environmental perturbation suites in LIBERO and LIBERO-Plus. Standalone EqC achieves the highest overall time saved, outperforming both the baseline Diff-DAgger and the hybrid EqC + Diff-DAgger variant across the majority of evaluation setups and representative window sizes m\in\{4,11,13\}, reaching up to 54.0\% in weighted average time saved. Notably, even when scaling Diff-DAgger’s sample budget to M\in\{16,32\}, its performance in time saved does not improve, demonstrating that simply increasing trajectory sampling density cannot enhance its failure detection capability. While the hybrid model also demonstrates notable gains over pure Diff-DAgger—further underscoring the benefit of energy-derived features—standalone EqC provides the most effective and responsive signal overall for timely failure intervention.

Finally, we evaluate the computational efficiency of EqC by benchmarking per-replan inference latency, encompassing both action candidate generation and failure-score computation at each step. As shown in Table 13, standalone EqC exhibits an inference latency comparable to that of low-budget Diff-DAgger variants. Crucially, as demonstrated in our earlier evaluations, EqC achieves significantly stronger and more reliable failure-detection performance than Diff-DAgger. While increasing Diff-DAgger’s sample budget M monotonically inflates its inference latency, it yields no significant improvement in failure detection capability, rendering larger sample budgets computationally inefficient for real-time introspection. Furthermore, EqC + Diff-DAgger incurs a substantial latency overhead compared to either standalone EqC or Diff-DAgger, proving impractical for physical deployment. Consequently, EqC delivers the optimal balance, offering superior failure detection while maintaining the computational responsiveness required for real-time robotic control.

B.7 Extended Studies

To evaluate the fundamental advantage of tracking temporal trajectory dynamics over static point-wise metrics, we compare our path-wise energy-consistency EqC signal against two alternative single-point energy-derived baselines: (1) Terminal Energy: the absolute energy value at the final step of the EqC action optimization; and (2) Last-step Gradient Norm: the gradient norm of the energy evaluated at the final optimization step.

To benchmark these signals, performance is evaluated across sliding-window sizes m\in[3,13], where AUROC is computed globally in a threshold-independent manner, while decision thresholds for TPR, FPR, and Accuracy are calibrated on a 10\% split—using both FPR-constrained and Youden’s J Index regimes—and evaluated on the remaining 90\% test split. As shown in Table 14, comparing these signals reveals a clear progression: scalar Terminal Energy offers weak detection capability (AUROC 0.5237–0.6489) because uncalibrated absolute energy values vary significantly across different environmental states, whereas Last-step Gradient Norm improves performance substantially (AUROC \sim 0.87) by measuring local gradient stationarity at the final optimization step. Building on this, EqC achieves the highest threshold-independent discriminability across all m, reaching a peak AUROC of 0.9300. Under FPR-constrained calibration, EqC strictly dominates both baselines across all m, yielding higher TPR and Accuracy (up to 87.6\%). Under Youden’s J Index, EqC achieves the optimal trade-off between TPR and FPR across all sliding window sizes m, while recovering higher peak Accuracy as m grows. Overall, these results confirm that tracking path-wise temporal energy dynamics along the optimization trajectory provides a fundamentally richer and more reliable introspection signal than static, single-point evaluations.

Appendix C Theory: Assumptions and Guarantees

This appendix establishes the theoretical properties underlying Equilibrium Energy Control (EqC). Section C.1 introduces the bridge construction, the energy-based control field, and the gradient-matching objective. Section C.2 relates local gradient matching to finite-path energy consistency. Section C.3 characterizes when the time-dependent bridge target can be represented by a stationary energy field. Section C.4 establishes monotone energy ascent during inference. Section C.5 shows that the deployment-time consistency score measures integrated field mismatch along the realized refinement trajectory.

C.1 Problem Setup

Let \mathcal{Z} denote the condition space and let

(A,Z)\sim p_{\mathrm{data}}

denote an action–condition pair, where A\in\mathbb{R}^{D} is an expert action chunk and Z\in\mathcal{Z} contains the corresponding observations, language instruction, robot state, or other policy inputs. Let

\varepsilon\sim\mathcal{N}(0,I_{D})

be independent of (A,Z). For a realization (a,z,\varepsilon), define

d:=a-\varepsilon,\qquad x_{t}:=\varepsilon+td,\qquad t\in[0,1].

Thus, x_{0}=\varepsilon and x_{1}=a.

Let

c:[0,1]\to\mathbb{R}_{\geq 0}

be a continuous schedule satisfying c(1)=0, and define the remaining schedule integral

C(t):=\int_{t}^{1}c(s)\,ds.

EqC represents action quality by a differentiable scalar energy

E_{\theta}:\mathbb{R}^{D}\times\mathcal{Z}\to\mathbb{R}.

The induced stationary control field is

g_{\theta}(x,z):=\nabla_{x}E_{\theta}(x,z).

The field is stationary in the sense that it depends on the current action state x and condition z, but not explicitly on the bridge coordinate t.

At the bridge point x_{t}, the target field is

c(t)d.

The gradient-matching objective is

\mathcal{L}_{\mathrm{grad}}(\theta):=\mathbb{E}_{(A,Z),\varepsilon,T}\left[\frac{1}{D}\left\|\nabla_{x}E_{\theta}(X_{T},Z)-c(T)(A-\varepsilon)\right\|_{2}^{2}\right],

where

T\sim\mathrm{Uniform}[0,1],\qquad X_{T}:=\varepsilon+T(A-\varepsilon).

The corresponding finite-path target follows from integrating the target field along the bridge:

\int_{t}^{1}(c(s)d)^{\top}d\,ds=\|d\|_{2}^{2}C(t).

This relation is induced by the gradient target and will provide the reference for the deployment-time consistency score.

At inference time, EqC initializes

x^{(0)}\sim\mathcal{N}(0,I_{D})

and applies gradient ascent:

x^{(k+1)}=x^{(k)}+\eta_{k}\nabla_{x}E_{\theta}(x^{(k)},z),\qquad k=0,\ldots,K-1.

The final action prediction is

\hat{x}:=x^{(K)}.

C.2 From Local Gradient Matching to Path Consistency

Fix a bridge (x_{s})_{s\in[0,1]} defined by equation 13. Define the local gradient-matching error

e_{\theta}(s):=\nabla_{x}E_{\theta}(x_{s},z)-c(s)d.

The following result relates finite energy inconsistency to local gradient-matching error accumulated along the bridge.

Lemma 1 (Path-consistency identity).

Suppose that E_{\theta}(\cdot,z) is continuously differentiable on the bridge

\{x_{s}:s\in[t,1]\}.

Then, for every t\in[0,1],

E_{\theta}(a,z)-E_{\theta}(x_{t},z)-\|d\|_{2}^{2}C(t)=\int_{t}^{1}e_{\theta}(s)^{\top}d\,ds.

Proof.

Since

x_{s}=\varepsilon+sd,

we have

\frac{dx_{s}}{ds}=d.

Applying the chain rule,

\frac{d}{ds}E_{\theta}(x_{s},z)=\nabla_{x}E_{\theta}(x_{s},z)^{\top}d.

By equation 18,

\nabla_{x}E_{\theta}(x_{s},z)=c(s)d+e_{\theta}(s).

Substituting this identity into equation 20 gives

\frac{d}{ds}E_{\theta}(x_{s},z)=c(s)\|d\|_{2}^{2}+e_{\theta}(s)^{\top}d.

Integrating from t to 1 yields

\displaystyle E_{\theta}(x_{1},z)-E_{\theta}(x_{t},z) \\ \displaystyle=\int_{t}^{1}c(s)\|d\|_{2}^{2}\,ds+\int_{t}^{1}e_{\theta}(s)^{\top}d\,ds \\ \displaystyle=\|d\|_{2}^{2}C(t)+\int_{t}^{1}e_{\theta}(s)^{\top}d\,ds.

Since x_{1}=a, rearranging proves equation 19. ∎

Corollary 1 (Accumulated path-consistency error).

Under the assumptions of Lemma 1,

\left|E_{\theta}(a,z)-E_{\theta}(x_{t},z)-\|d\|_{2}^{2}C(t)\right|\leq\|d\|_{2}\sqrt{1-t}\left(\int_{t}^{1}\|e_{\theta}(s)\|_{2}^{2}\,ds\right)^{1/2}.

Proof.

By Lemma 1,

\left|E_{\theta}(a,z)-E_{\theta}(x_{t},z)-\|d\|_{2}^{2}C(t)\right|=\left|\int_{t}^{1}e_{\theta}(s)^{\top}d\,ds\right|.

Cauchy–Schwarz first gives

|e_{\theta}(s)^{\top}d|\leq\|e_{\theta}(s)\|_{2}\|d\|_{2}.

Therefore,

\displaystyle\left|\int_{t}^{1}e_{\theta}(s)^{\top}d\,ds\right| \\ \displaystyle\leq\|d\|_{2}\int_{t}^{1}\|e_{\theta}(s)\|_{2}\,ds \\ \displaystyle\leq\|d\|_{2}\left(\int_{t}^{1}1\,ds\right)^{1/2}\left(\int_{t}^{1}\|e_{\theta}(s)\|_{2}^{2}\,ds\right)^{1/2} \\ \displaystyle=\|d\|_{2}\sqrt{1-t}\left(\int_{t}^{1}\|e_{\theta}(s)\|_{2}^{2}\,ds\right)^{1/2}.

∎

Lemma 1 shows that the consistency residual is the integrated component of the local field error in the bridge direction. Consequently, reducing the gradient-matching error also controls the finite energy relation underlying the deployment-time consistency test.

C.3 Stationary Representation of the Bridge Target

The bridge target c(T)(A-\varepsilon) depends explicitly on T, whereas the deployed EqC field \nabla_{x}E_{\theta}(X,Z) is stationary. We now characterize the approximation cost associated with this restriction.

Let

X:=\varepsilon+T(A-\varepsilon),\qquad Y:=c(T)(A-\varepsilon).

For T<1, define

\alpha(T):=\frac{c(T)}{1-T}.

The value of \alpha(1) may be assigned arbitrarily because \mathbb{P}(T=1)=0.

Since

A-X=A-\varepsilon-T(A-\varepsilon)=(1-T)(A-\varepsilon),

the target satisfies

Y=\alpha(T)(A-X)\qquad\text{almost surely}.

Assume

\mathbb{E}\|A\|_{2}^{2}<\infty,

and define the conditional mean action

a_{\star}(Z):=\mathbb{E}[A\mid Z].

Define the radius of the current state relative to this conditional reference by

R_{X}:=\|X-a_{\star}(Z)\|_{2}.

Definition 1 (Condition-dependent radial scale).

A condition-dependent radial scale is a measurable function

\lambda:\mathcal{Z}\times[0,\infty)\to[0,\infty),

written

\lambda_{z}(r):=\lambda(z,r),

such that r\mapsto\lambda_{z}(r) is continuous for p_{Z}-almost every z.

The random variable \lambda_{Z}(R_{X}) represents a stationary approximation of the bridge coefficient \alpha(T) using only the current state X and condition Z.

For a radial scale \lambda, define the per-dimension action-dispersion and scale-recovery errors by

\displaystyle\sigma_{\mathrm{act}}^{2} \\ \displaystyle:=\frac{1}{D}\mathbb{E}\left[\alpha(T)^{2}\|A-a_{\star}(Z)\|_{2}^{2}\right], \\ \displaystyle\sigma_{\mathrm{scale}}^{2}(\lambda) \\ \displaystyle:=\frac{1}{D}\mathbb{E}\left[\bigl(\alpha(T)-\lambda_{Z}(R_{X})\bigr)^{2}R_{X}^{2}\right].

The first quantity measures the residual variation of the expert action after conditioning on Z. Since T is independent of (A,Z),

D\sigma_{\mathrm{act}}^{2}=\mathbb{E}[\alpha(T)^{2}]\,\mathbb{E}\|A-a_{\star}(Z)\|_{2}^{2}

whenever both factors are finite. The second quantity measures the error incurred when \alpha(T) is represented as a function of (R_{X},Z).

Lemma 2 (Optimal measurable radial scale).

Among all measurable functions of (R_{X},Z), the weighted error

\mathbb{E}\left[\bigl(\alpha(T)-\lambda_{Z}(R_{X})\bigr)^{2}R_{X}^{2}\right]

is minimized, for R_{X}>0, by any version of

\lambda_{z}^{\star}(r):=\mathbb{E}\left[\alpha(T)\mid R_{X}=r,Z=z\right].

At r=0, the value of \lambda_{z}^{\star}(0) does not affect the weighted objective.

Proof.

Condition on (R_{X},Z)=(r,z). For any scalar \ell,

\displaystyle\mathbb{E}\left[\bigl(\alpha(T)-\ell\bigr)^{2}R_{X}^{2}\mid R_{X}=r,Z=z\right] \\ \displaystyle\hskip 56.9055pt=r^{2}\mathbb{E}\left[\bigl(\alpha(T)-\ell\bigr)^{2}\mid R_{X}=r,Z=z\right].

For r>0, multiplication by r^{2} does not change the minimizer. The conditional squared error is minimized by the conditional mean

\ell=\mathbb{E}\left[\alpha(T)\mid R_{X}=r,Z=z\right].

For r=0, the weighted conditional objective is identically zero for every \ell. ∎

If the conditional expectation in equation 29 has a continuous version, it satisfies Definition 1. Otherwise, any continuous approximation may be used in the bound below.

Given a radial scale \lambda, define

E_{\lambda}(x,z):=-\int_{0}^{\|x-a_{\star}(z)\|_{2}}s\lambda_{z}(s)\,ds.

Lemma 3 (Gradient of the radial witness).

If r\mapsto\lambda_{z}(r) is continuous, then E_{\lambda}(\cdot,z) is continuously differentiable and

\nabla_{x}E_{\lambda}(x,z)=\lambda_{z}(\|x-a_{\star}(z)\|_{2})\bigl(a_{\star}(z)-x\bigr).

Proof.

Fix z and write

r(x):=\|x-a_{\star}(z)\|_{2}.

For r(x)>0, the chain rule gives

\displaystyle\nabla_{x}E_{\lambda}(x,z) \\ \displaystyle=-r(x)\lambda_{z}(r(x))\nabla_{x}r(x) \\ \displaystyle=-r(x)\lambda_{z}(r(x))\frac{x-a_{\star}(z)}{r(x)} \\ \displaystyle=\lambda_{z}(r(x))(a_{\star}(z)-x).

It remains to consider x=a_{\star}(z). By continuity, \lambda_{z} is bounded on a neighborhood of zero. Hence, for sufficiently small h, there exists M<\infty such that

\displaystyle\left|E_{\lambda}(a_{\star}(z)+h,z)-E_{\lambda}(a_{\star}(z),z)\right| \\ \displaystyle=\left|\int_{0}^{\|h\|_{2}}s\lambda_{z}(s)\,ds\right| \\ \displaystyle\leq\frac{M}{2}\|h\|_{2}^{2} \\ \displaystyle=o(\|h\|_{2}).

Therefore,

\nabla_{x}E_{\lambda}(a_{\star}(z),z)=0,

which agrees with equation 31. Continuity of the gradient follows from continuity of \lambda_{z}. ∎

The field

g_{\lambda}(x,z):=\nabla_{x}E_{\lambda}(x,z)

is a stationary conservative field. We use it as an explicit witness for the approximation capacity of the EqC energy class.

For a measurable predictor

h:\mathbb{R}^{D}\times[0,1]\times\mathcal{Z}\to\mathbb{R}^{D},

define its normalized squared risk by

R(h):=\frac{1}{D}\mathbb{E}\left[\|Y-h(X,T,Z)\|_{2}^{2}\right].

The optimal time-conditioned regressor is

m_{\mathrm{tc}}(X,T,Z):=\mathbb{E}[Y\mid X,T,Z],

with risk

R_{\mathrm{tc}}^{\star}:=R(m_{\mathrm{tc}}).

A stationary predictor depends only on (X,Z). Define the optimal stationary risk by

R_{\mathrm{st}}^{\star}:=\inf_{h:\,h=h(X,Z)}\frac{1}{D}\mathbb{E}\left[\|Y-h(X,Z)\|_{2}^{2}\right].

For the EqC energy class

\mathcal{E}_{\Theta}:=\{E_{\theta}:\theta\in\Theta\},

define

R_{\mathrm{EqC}}^{\Theta}:=\inf_{\theta\in\Theta}\frac{1}{D}\mathbb{E}\left[\|Y-\nabla_{x}E_{\theta}(X,Z)\|_{2}^{2}\right].

Finally, define the approximation error of the EqC class relative to the radial witness:

\delta_{\Theta}(\lambda):=\inf_{\theta\in\Theta}\left(\frac{1}{D}\mathbb{E}\left[\left\|\nabla_{x}E_{\theta}(X,Z)-g_{\lambda}(X,Z)\right\|_{2}^{2}\right]\right)^{1/2}.

Proposition 1 (Stationary energy approximation).

Suppose that

\sigma_{\mathrm{act}}<\infty,\qquad\sigma_{\mathrm{scale}}(\lambda)<\infty,\qquad\delta_{\Theta}(\lambda)<\infty.

Then

0\leq R_{\mathrm{EqC}}^{\Theta}-R_{\mathrm{st}}^{\star}\leq R_{\mathrm{EqC}}^{\Theta}-R_{\mathrm{tc}}^{\star}\leq\left(\sigma_{\mathrm{act}}+\sigma_{\mathrm{scale}}(\lambda)+\delta_{\Theta}(\lambda)\right)^{2}.

Proof.

For a square-integrable random vector W, define

\|W\|_{L_{D}^{2}}:=\left(\frac{1}{D}\mathbb{E}\|W\|_{2}^{2}\right)^{1/2}.

The orthogonality principle for conditional expectations gives, for every measurable h(X,T,Z),

R(h)-R_{\mathrm{tc}}^{\star}=\|h-m_{\mathrm{tc}}\|_{L_{D}^{2}}^{2}.

Indeed,

\displaystyle\|Y-h\|_{L_{D}^{2}}^{2} \\ \displaystyle=\|Y-m_{\mathrm{tc}}+m_{\mathrm{tc}}-h\|_{L_{D}^{2}}^{2} \\ \displaystyle=\|Y-m_{\mathrm{tc}}\|_{L_{D}^{2}}^{2}+\|m_{\mathrm{tc}}-h\|_{L_{D}^{2}}^{2},

because

\mathbb{E}\left[(Y-m_{\mathrm{tc}})^{\top}(m_{\mathrm{tc}}-h)\right]=0.

Since g_{\lambda}(X,Z) is measurable with respect to (X,T,Z),

\displaystyle\|m_{\mathrm{tc}}-g_{\lambda}\|_{L_{D}^{2}} \\ \displaystyle=\left\|\mathbb{E}\left[Y-g_{\lambda}(X,Z)\mid X,T,Z\right]\right\|_{L_{D}^{2}} \\ \displaystyle\leq\|Y-g_{\lambda}(X,Z)\|_{L_{D}^{2}},

where the inequality follows from conditional Jensen.

Using equation 23 and equation 31,

\displaystyle Y-g_{\lambda}(X,Z) \\ \displaystyle=\alpha(T)(A-X)-\lambda_{Z}(R_{X})(a_{\star}(Z)-X) \\ \displaystyle=\alpha(T)(A-a_{\star}(Z))+\bigl(\alpha(T)-\lambda_{Z}(R_{X})\bigr)(a_{\star}(Z)-X).

Applying Minkowski’s inequality,

\displaystyle\|Y-g_{\lambda}(X,Z)\|_{L_{D}^{2}} \\ \displaystyle\leq\left\|\alpha(T)(A-a_{\star}(Z))\right\|_{L_{D}^{2}} \\ \displaystyle\quad+\left\|\bigl(\alpha(T)-\lambda_{Z}(R_{X})\bigr)(a_{\star}(Z)-X)\right\|_{L_{D}^{2}} \\ \displaystyle=\sigma_{\mathrm{act}}+\sigma_{\mathrm{scale}}(\lambda),

where

\|a_{\star}(Z)-X\|_{2}=R_{X}.

Together with equation 40, this gives

\|m_{\mathrm{tc}}-g_{\lambda}\|_{L_{D}^{2}}\leq\sigma_{\mathrm{act}}+\sigma_{\mathrm{scale}}(\lambda).

For any \theta\in\Theta, Minkowski’s inequality and equation 42 imply

\displaystyle\|m_{\mathrm{tc}}-\nabla_{x}E_{\theta}\|_{L_{D}^{2}} \\ \displaystyle\leq\|m_{\mathrm{tc}}-g_{\lambda}\|_{L_{D}^{2}}+\|g_{\lambda}-\nabla_{x}E_{\theta}\|_{L_{D}^{2}} \\ \displaystyle\leq\sigma_{\mathrm{act}}+\sigma_{\mathrm{scale}}(\lambda)+\|g_{\lambda}-\nabla_{x}E_{\theta}\|_{L_{D}^{2}}.

Taking the infimum over \theta gives

\inf_{\theta\in\Theta}\|m_{\mathrm{tc}}-\nabla_{x}E_{\theta}\|_{L_{D}^{2}}\leq\sigma_{\mathrm{act}}+\sigma_{\mathrm{scale}}(\lambda)+\delta_{\Theta}(\lambda).

By equation 39,

R_{\mathrm{EqC}}^{\Theta}-R_{\mathrm{tc}}^{\star}=\inf_{\theta\in\Theta}\|m_{\mathrm{tc}}-\nabla_{x}E_{\theta}\|_{L_{D}^{2}}^{2}.

Therefore,

R_{\mathrm{EqC}}^{\Theta}-R_{\mathrm{tc}}^{\star}\leq\left(\sigma_{\mathrm{act}}+\sigma_{\mathrm{scale}}(\lambda)+\delta_{\Theta}(\lambda)\right)^{2}.

Every EqC gradient \nabla_{x}E_{\theta}(X,Z) is a stationary predictor. Hence,

R_{\mathrm{st}}^{\star}\leq R_{\mathrm{EqC}}^{\Theta}.

Moreover, every stationary predictor is contained in the class of time-conditioned predictors, so

R_{\mathrm{tc}}^{\star}\leq R_{\mathrm{st}}^{\star}.

Combining these inequalities completes the proof. ∎

Proposition 1 separates the stationary approximation error into three terms: conditional action dispersion, recovery of the bridge scale from the current state, and approximation by the chosen neural energy class.

Corollary 2 (Exact stationary representation).

Suppose that the following conditions hold:

\displaystyle A=a_{\star}(Z) \\ \displaystyle\text{almost surely}, \\ \displaystyle\operatorname{Var}\!\left(\alpha(T)\mid R_{X},Z\right)=0 \\ \displaystyle\text{almost surely}.

Let

\lambda_{z}(r):=\mathbb{E}\left[\alpha(T)\mid R_{X}=r,Z=z\right]

be a continuous version of the conditional expectation. If there exists \theta_{\star}\in\Theta such that

\nabla_{x}E_{\theta_{\star}}(x,z)=\lambda_{z}(\|x-a_{\star}(z)\|_{2})\bigl(a_{\star}(z)-x\bigr)

on the support of (X,Z), then

\nabla_{x}E_{\theta_{\star}}(X,Z)=Y\qquad\text{almost surely}.

Consequently,

R_{\mathrm{EqC}}^{\Theta}=R_{\mathrm{st}}^{\star}=R_{\mathrm{tc}}^{\star}=0.

Proof.

Equation equation 44 implies that \alpha(T) is measurable with respect to (R_{X},Z). Therefore,

\alpha(T)=\mathbb{E}\left[\alpha(T)\mid R_{X},Z\right]=\lambda_{Z}(R_{X})\qquad\text{almost surely}.

Using equation 23 and equation 43,

\displaystyle Y \\ \displaystyle=\alpha(T)(A-X) \\ \displaystyle=\lambda_{Z}(R_{X})(a_{\star}(Z)-X).

By equation 45,

\lambda_{Z}(R_{X})(a_{\star}(Z)-X)=\nabla_{x}E_{\theta_{\star}}(X,Z).

Hence,

Y=\nabla_{x}E_{\theta_{\star}}(X,Z)

almost surely. The corresponding squared risk is zero, which proves equation 47. ∎

In the concentrated case A=a_{\star}(Z),

R_{X}=(1-T)\|\varepsilon-a_{\star}(Z)\|_{2}.

Thus, whenever (R_{X},Z) determines \alpha(T), a stationary energy can represent a nonconstant bridge schedule without receiving T explicitly.

C.4 Monotone Energy Ascent

We next establish the optimization property of the inference update equation 17.

Assumption 1 (Local smoothness).

Fix z. There exists L>0 such that, for every k\in\{0,\ldots,K-1\},

\left\|\nabla_{x}E_{\theta}(x,z)-\nabla_{x}E_{\theta}(y,z)\right\|_{2}\leq L\|x-y\|_{2}

for all x and y on the segment

[x^{(k)},x^{(k+1)}]:=\left\{x^{(k)}+s(x^{(k+1)}-x^{(k)}):s\in[0,1]\right\}.

Theorem 1 (Monotone energy ascent).

Suppose Assumption 1 holds and EqC uses the exact gradient-ascent update

x^{(k+1)}=x^{(k)}+\eta_{k}\nabla_{x}E_{\theta}(x^{(k)},z)

with

0<\eta_{k}<\frac{2}{L}.

Then

E_{\theta}(x^{(k+1)},z)-E_{\theta}(x^{(k)},z)\geq w_{k}\left\|\nabla_{x}E_{\theta}(x^{(k)},z)\right\|_{2}^{2},

where

w_{k}:=\eta_{k}\left(1-\frac{L\eta_{k}}{2}\right)>0.

Therefore,

E_{\theta}(x^{(k+1)},z)\geq E_{\theta}(x^{(k)},z),

with strict inequality whenever

\nabla_{x}E_{\theta}(x^{(k)},z)\neq 0.

Proof.

Let

g_{k}:=\nabla_{x}E_{\theta}(x^{(k)},z).

The update is

x^{(k+1)}=x^{(k)}+\eta_{k}g_{k}.

Define the segment

q_{k}(s):=x^{(k)}+s\eta_{k}g_{k},\qquad s\in[0,1].

By the fundamental theorem of calculus,

\displaystyle E_{\theta}(x^{(k+1)},z)-E_{\theta}(x^{(k)},z) \\ \displaystyle=\int_{0}^{1}\nabla_{x}E_{\theta}(q_{k}(s),z)^{\top}(\eta_{k}g_{k})\,ds \\ \displaystyle=\eta_{k}\|g_{k}\|_{2}^{2}+\eta_{k}\int_{0}^{1}\left(\nabla_{x}E_{\theta}(q_{k}(s),z)-g_{k}\right)^{\top}g_{k}\,ds.

By Cauchy–Schwarz and Assumption 1,

\displaystyle\left(\nabla_{x}E_{\theta}(q_{k}(s),z)-g_{k}\right)^{\top}g_{k} \\ \displaystyle\geq-\left\|\nabla_{x}E_{\theta}(q_{k}(s),z)-g_{k}\right\|_{2}\|g_{k}\|_{2} \\ \displaystyle\geq-L\|q_{k}(s)-x^{(k)}\|_{2}\|g_{k}\|_{2} \\ \displaystyle=-Ls\eta_{k}\|g_{k}\|_{2}^{2}.

Substituting this bound into equation 53,

\displaystyle E_{\theta}(x^{(k+1)},z)-E_{\theta}(x^{(k)},z) \\ \displaystyle\geq\eta_{k}\|g_{k}\|_{2}^{2}-L\eta_{k}^{2}\|g_{k}\|_{2}^{2}\int_{0}^{1}s\,ds \\ \displaystyle=\eta_{k}\left(1-\frac{L\eta_{k}}{2}\right)\|g_{k}\|_{2}^{2} \\ \displaystyle=w_{k}\|g_{k}\|_{2}^{2}.

The step-size condition equation 50 implies w_{k}>0, proving the result. ∎

Corollary 3 (Finite-step stationarity).

Under the assumptions of Theorem 1,

\min_{0\leq k<K}\left\|\nabla_{x}E_{\theta}(x^{(k)},z)\right\|_{2}^{2}\leq\frac{E_{\theta}(x^{(K)},z)-E_{\theta}(x^{(0)},z)}{\sum_{k=0}^{K-1}w_{k}}.

If

E_{\theta}(x^{(k)},z)\leq E_{\sup}\qquad\text{for }k=0,\ldots,K,

then

\min_{0\leq k<K}\left\|\nabla_{x}E_{\theta}(x^{(k)},z)\right\|_{2}^{2}\leq\frac{E_{\sup}-E_{\theta}(x^{(0)},z)}{\sum_{k=0}^{K-1}w_{k}}.

Proof.

Summing equation 51 over k=0,\ldots,K-1 gives

\displaystyle E_{\theta}(x^{(K)},z)-E_{\theta}(x^{(0)},z) \\ \displaystyle=\sum_{k=0}^{K-1}\left(E_{\theta}(x^{(k+1)},z)-E_{\theta}(x^{(k)},z)\right) \\ \displaystyle\geq\sum_{k=0}^{K-1}w_{k}\left\|\nabla_{x}E_{\theta}(x^{(k)},z)\right\|_{2}^{2} \\ \displaystyle\geq\left(\min_{0\leq k<K}\left\|\nabla_{x}E_{\theta}(x^{(k)},z)\right\|_{2}^{2}\right)\sum_{k=0}^{K-1}w_{k}.

Since each w_{k}>0, division by \sum_{k=0}^{K-1}w_{k} proves equation 54. The bound equation 55 follows from

E_{\theta}(x^{(K)},z)\leq E_{\sup}.

∎

Theorem 1 establishes that every EqC refinement step increases the same scalar energy under the stated smoothness and step-size conditions. Corollary 3 quantifies the resulting approach to a first-order stationary point.

C.5 Energy Consistency as a Path-Wise Mismatch Certificate

Let

\hat{x}:=x^{(K)},\qquad v:=\hat{x}-x^{(0)}.

For a fixed normalization constant \delta>0, define

D_{v}:=\max(\|v\|_{2}^{2},\delta),\qquad\gamma_{v}:=\frac{\|v\|_{2}^{2}}{D_{v}}.

For u\in\mathbb{R}, define

\operatorname{clip}(u,0,1):=\min\{1,\max\{0,u\}\}.

The projected progress coordinate of iterate x^{(k)} is

\tau_{k}:=\operatorname{clip}\left(\frac{\langle x^{(k)}-x^{(0)},v\rangle}{D_{v}},0,1\right).

The corresponding point on the chord from x^{(0)} to \hat{x} is

\bar{x}^{(k)}:=x^{(0)}+\tau_{k}v,

and the off-chord displacement is

h_{k}:=x^{(k)}-\bar{x}^{(k)}.

Define the field mismatch along the endpoint chord by

e_{v}(s):=\nabla_{x}E_{\theta}(x^{(0)}+sv,z)-c(s)v,\qquad s\in[0,1].

The deployment-time energy residual is

r_{k}^{\Delta E}:=\frac{E_{\theta}(\hat{x},z)-E_{\theta}(x^{(k)},z)}{D_{v}}-C(\tau_{k}),

and the energy-consistency score is

U_{\Delta E}:=\frac{1}{K}\sum_{k=1}^{K}|r_{k}^{\Delta E}|.

Theorem 2 (Energy-consistency decomposition).

Suppose that E_{\theta}(\cdot,z) is continuously differentiable on the chord

\{x^{(0)}+sv:s\in[0,1]\}.

Then, for each k\in\{1,\ldots,K\},

r_{k}^{\Delta E}=\frac{1}{D_{v}}\int_{\tau_{k}}^{1}e_{v}(s)^{\top}v\,ds+\frac{E_{\theta}(\bar{x}^{(k)},z)-E_{\theta}(x^{(k)},z)}{D_{v}}+(\gamma_{v}-1)C(\tau_{k}).

Proof.

Define the chord

q(s):=x^{(0)}+sv.

Since

q(\tau_{k})=\bar{x}^{(k)},\qquad q(1)=\hat{x},\qquad\frac{dq(s)}{ds}=v,

the fundamental theorem of calculus gives

\displaystyle E_{\theta}(\hat{x},z)-E_{\theta}(\bar{x}^{(k)},z) \\ \displaystyle=\int_{\tau_{k}}^{1}\nabla_{x}E_{\theta}(q(s),z)^{\top}v\,ds \\ \displaystyle=\int_{\tau_{k}}^{1}\bigl(c(s)v+e_{v}(s)\bigr)^{\top}v\,ds \\ \displaystyle=\|v\|_{2}^{2}\int_{\tau_{k}}^{1}c(s)\,ds+\int_{\tau_{k}}^{1}e_{v}(s)^{\top}v\,ds \\ \displaystyle=\|v\|_{2}^{2}C(\tau_{k})+\int_{\tau_{k}}^{1}e_{v}(s)^{\top}v\,ds.

Adding

E_{\theta}(\bar{x}^{(k)},z)-E_{\theta}(x^{(k)},z)

to both sides gives

\displaystyle E_{\theta}(\hat{x},z)-E_{\theta}(x^{(k)},z) \\ \displaystyle=\|v\|_{2}^{2}C(\tau_{k})+\int_{\tau_{k}}^{1}e_{v}(s)^{\top}v\,ds \\ \displaystyle\quad+E_{\theta}(\bar{x}^{(k)},z)-E_{\theta}(x^{(k)},z).

Dividing by D_{v} and subtracting C(\tau_{k}) yields

\displaystyle r_{k}^{\Delta E} \\ \displaystyle=\frac{1}{D_{v}}\int_{\tau_{k}}^{1}e_{v}(s)^{\top}v\,ds \\ \displaystyle\quad+\frac{E_{\theta}(\bar{x}^{(k)},z)-E_{\theta}(x^{(k)},z)}{D_{v}} \\ \displaystyle\quad+\left(\frac{\|v\|_{2}^{2}}{D_{v}}-1\right)C(\tau_{k}).

Using

\gamma_{v}=\frac{\|v\|_{2}^{2}}{D_{v}}

proves equation 63. ∎

The three terms in equation 63 have distinct origins. The first is the integrated field mismatch along the endpoint chord. The second measures the energy discrepancy caused by deviation of the realized iterate from that chord. The third is the correction introduced by the normalization floor.

Define the mean integrated chord mismatch

A_{K}:=\frac{1}{KD_{v}}\sum_{k=1}^{K}\left|\int_{\tau_{k}}^{1}e_{v}(s)^{\top}v\,ds\right|.

Assumption 2 (Bounded field on off-chord segments).

There exists G<\infty such that

\|\nabla_{x}E_{\theta}(y,z)\|_{2}\leq G

for every y on every segment

[x^{(k)},\bar{x}^{(k)}],\qquad k=1,\ldots,K.

Under Assumption 2, define

B_{K}:=\frac{G}{KD_{v}}\sum_{k=1}^{K}\|h_{k}\|_{2}+\frac{|\gamma_{v}-1|}{K}\sum_{k=1}^{K}|C(\tau_{k})|.

Proposition 2 (Bounds on the energy-consistency score).

Under Assumption 2,

[A_{K}-B_{K}]_{+}\leq U_{\Delta E}\leq A_{K}+B_{K},

where

[q]_{+}:=\max(q,0).

Moreover,

A_{K}\leq\frac{\|v\|_{2}}{KD_{v}}\int_{0}^{1}N_{K}(s)\|e_{v}(s)\|_{2}\,ds,

where

N_{K}(s):=\sum_{k=1}^{K}\mathbf{1}\{\tau_{k}\leq s\}.

Proof.

For each k, write

\displaystyle a_{k} \\ \displaystyle:=\frac{1}{D_{v}}\int_{\tau_{k}}^{1}e_{v}(s)^{\top}v\,ds, \\ \displaystyle b_{k} \\ \displaystyle:=\frac{E_{\theta}(\bar{x}^{(k)},z)-E_{\theta}(x^{(k)},z)}{D_{v}}, \\ \displaystyle q_{k} \\ \displaystyle:=(\gamma_{v}-1)C(\tau_{k}).

By Theorem 2,

r_{k}^{\Delta E}=a_{k}+b_{k}+q_{k}.

The triangle and reverse-triangle inequalities give

|a_{k}|-|b_{k}|-|q_{k}|\leq|r_{k}^{\Delta E}|\leq|a_{k}|+|b_{k}|+|q_{k}|.

By the fundamental theorem of calculus on the segment from x^{(k)} to \bar{x}^{(k)},

\displaystyle\left|E_{\theta}(\bar{x}^{(k)},z)-E_{\theta}(x^{(k)},z)\right| \\ \displaystyle\quad\leq\sup_{y\in[x^{(k)},\bar{x}^{(k)}]}\|\nabla_{x}E_{\theta}(y,z)\|_{2}\,\|\bar{x}^{(k)}-x^{(k)}\|_{2} \\ \displaystyle\quad\leq G\|h_{k}\|_{2}.

Therefore,

|b_{k}|\leq\frac{G}{D_{v}}\|h_{k}\|_{2},\qquad|q_{k}|=|\gamma_{v}-1|\,|C(\tau_{k})|.

Averaging equation 69 over k=1,\ldots,K gives

A_{K}-B_{K}\leq U_{\Delta E}\leq A_{K}+B_{K}.

Since U_{\Delta E}\geq 0, the lower bound can be written as [A_{K}-B_{K}]_{+}.

For the second result, Cauchy–Schwarz implies

|e_{v}(s)^{\top}v|\leq\|e_{v}(s)\|_{2}\|v\|_{2}.

Consequently,

\displaystyle A_{K} \\ \displaystyle\leq\frac{\|v\|_{2}}{KD_{v}}\sum_{k=1}^{K}\int_{\tau_{k}}^{1}\|e_{v}(s)\|_{2}\,ds \\ \displaystyle=\frac{\|v\|_{2}}{KD_{v}}\int_{0}^{1}\left(\sum_{k=1}^{K}\mathbf{1}\{\tau_{k}\leq s\}\right)\|e_{v}(s)\|_{2}\,ds \\ \displaystyle=\frac{\|v\|_{2}}{KD_{v}}\int_{0}^{1}N_{K}(s)\|e_{v}(s)\|_{2}\,ds.

∎

Proposition 2 shows that the observed score U_{\Delta E} is controlled by the integrated chord-wise field error, up to explicit corrections for off-chord motion and normalization.

Corollary 4 (Exact on-chord characterization).

Suppose that the normalization floor is inactive,

D_{v}=\|v\|_{2}^{2},

and every refinement iterate lies on the endpoint chord:

x^{(k)}=\bar{x}^{(k)},\qquad k=1,\ldots,K.

Then

U_{\Delta E}=\frac{1}{K\|v\|_{2}^{2}}\sum_{k=1}^{K}\left|\int_{\tau_{k}}^{1}e_{v}(s)^{\top}v\,ds\right|.

Furthermore,

\frac{1}{K\|v\|_{2}^{2}}\left|\int_{0}^{1}N_{K}(s)e_{v}(s)^{\top}v\,ds\right|\leq U_{\Delta E}\leq\frac{1}{K\|v\|_{2}}\int_{0}^{1}N_{K}(s)\|e_{v}(s)\|_{2}\,ds.

Proof.

If D_{v}=\|v\|_{2}^{2}, then

\gamma_{v}=1.

If x^{(k)}=\bar{x}^{(k)}, then

E_{\theta}(\bar{x}^{(k)},z)-E_{\theta}(x^{(k)},z)=0.

The decomposition in equation 63 therefore reduces to

r_{k}^{\Delta E}=\frac{1}{\|v\|_{2}^{2}}\int_{\tau_{k}}^{1}e_{v}(s)^{\top}v\,ds.

Taking absolute values and averaging proves equation 72.

For the lower bound, observe that

\displaystyle\int_{0}^{1}N_{K}(s)e_{v}(s)^{\top}v\,ds \\ \displaystyle=\int_{0}^{1}\sum_{k=1}^{K}\mathbf{1}\{\tau_{k}\leq s\}e_{v}(s)^{\top}v\,ds \\ \displaystyle=\sum_{k=1}^{K}\int_{\tau_{k}}^{1}e_{v}(s)^{\top}v\,ds.

Hence, by the triangle inequality,

\left|\int_{0}^{1}N_{K}(s)e_{v}(s)^{\top}v\,ds\right|\leq\sum_{k=1}^{K}\left|\int_{\tau_{k}}^{1}e_{v}(s)^{\top}v\,ds\right|.

Dividing by K\|v\|_{2}^{2} proves the lower bound.

For the upper bound,

\displaystyle U_{\Delta E} \\ \displaystyle=\frac{1}{K\|v\|_{2}^{2}}\sum_{k=1}^{K}\left|\int_{\tau_{k}}^{1}e_{v}(s)^{\top}v\,ds\right| \\ \displaystyle\leq\frac{1}{K\|v\|_{2}}\sum_{k=1}^{K}\int_{\tau_{k}}^{1}\|e_{v}(s)\|_{2}\,ds \\ \displaystyle=\frac{1}{K\|v\|_{2}}\int_{0}^{1}N_{K}(s)\|e_{v}(s)\|_{2}\,ds.

∎

Corollary 4 provides the exact interpretation of the EqC score in the ideal geometric regime: U_{\Delta E} is the average normalized magnitude of the remaining integrated longitudinal field error along the refinement path.

C.6 Scope of the auxiliary analysis

• The empirical findings of Pan et al. (2026) motivate the conditional-concentration regime; they do not prove that \sigma_{\mathrm{act}} or \sigma_{\mathrm{scale}} is small for EqC. These quantities require direct diagnostics on the selected datasets and trained policies.
• Proposition 1 is an existence upper bound based on a single-basin witness. Low conditional dispersion is sufficient, not necessary: a nonlinear energy may represent several action basins. Its weighted moment conditions also cover schedules for which c(T)/(1-T) grows near the endpoint.
• Theorem 1 concerns optimization of the learned scalar only. Neither stationarity nor increasing energy is a correctness guarantee, and a generic stationary field can have a separately established Lyapunov function.
• Theorem 2 decomposes the mean absolute self-consistency residual. Absolute averaging prevents cancellation across refinement iterates, but opposite field errors can still cancel within an individual remaining-work integral. The normalization floor matters for short chords, and a wrong but internally coherent trajectory can have a small score.

Appendix D Real-Robot Experimental Setup and Evaluation Protocol

D.1 Platform and Hardware Architecture

We conduct all physical experiments on a bimanual manipulation workstation comprising two lightweight, force-controlled, 6-DOF Galaxea A1Z robotic arms. The policies are fine-tuned from the Galaxea G0.5 VLA model. Each A1Z manipulator weighs 5\,\text{kg} and has an operating reach radius of 626.5\,\text{mm}, an operating height of 722\,\text{mm}, and a rated continuous payload of 3\,\text{kg} (with a peak payload of 5\,\text{kg} at full extension).

Both manipulators are equipped with parallel-jaw grippers, yielding 14 active control degrees of freedom (twelve arm joints and two gripper channels). Visual inputs are captured simultaneously by three synchronized RGB cameras: an overhead global camera and two wrist-mounted cameras (left and right). Policy inputs also incorporate 14-dimensional bimanual proprioception (joint angles and gripper states). The physical control frequency is set to 15\,\text{Hz}, and the receding-horizon action chunk size is set to T_{a}=32.

D.2 Task Specifications and Success Criteria

Benchmark 1: Flower Insertion.

The Flower Insertion task (Section 5.1.4) evaluates bimanual coordination, 3D spatial alignment, and contact-rich insertion. To successfully complete the task, the dual-arm system must:

1. Approach and grasp the flexible stem from the workspace surface;
2. Lift and reorient the stem in 3D space to align its longitudinal axis with the vertical entry axis of the vase opening;
3. Smoothly feed the stem through the aperture until fully seated, dynamically correcting minor contact misalignments at the rim without tipping the vase over.

A trial is scored as a success if the stem is fully inserted and the gripper safely releases without disturbing the upright posture of the vase.

Benchmark 2: Towel Folding.

The Towel Folding task evaluates bimanual spatial coordination, deformable object manipulation, and sequential multi-stage execution. The robot must perform a two-stage quarter-fold (folding both laterally and longitudinally):

1. First Fold (Horizontal): Simultaneously approach and pinch two adjacent corners/edges of the unfolded towel using both grippers, lift and fold the towel in half laterally, and lay it down smoothly;
2. Reorientation: Rotate the laterally folded towel using both arms to reorient it for the longitudinal fold;
3. Second Fold (Vertical) and Release: Reposition both arms to regrasp the newly formed folded edges, lift the towel, and fold it in half longitudinally to form a compact four-layer square; then smooth the top layer down and cleanly release both grippers without disturbing the folded creases or dragging the fabric.

A trial is scored as a success if and only if: (1) both the lateral and longitudinal folds are successfully executed, (2) the final folded fabric is reduced to approximately a quarter of its original footprint with aligned edges, and (3) the towel remains stably and neatly folded without unfolding or flipping after gripper release.

Benchmark 3: Debris Sweeping.

The Debris Sweeping task evaluates multi-stage bimanual tool coordination and granular object collection. The dual-arm robot must coordinate a hand brush (or mini broom) and a dustpan to gather scattered paper scraps from the tabletop and dump them into a designated waste bin:

1. Dual-Tool Grasping and Positioning: The left manipulator picks up the brush from the holder tray, while the right manipulator grasps the dustpan and places its front edge flat against the table surface near the perimeter of the debris cluster;
2. Contact-Rich Sweeping: The left arm coordinates with the right arm to sweep scattered paper scraps cleanly into the cavity of the dustpan using multiple sweeping strokes, maintaining proper surface contact without crushing or scattering the debris beyond the reachable workspace;
3. Transport and Dumping: The right arm lifts the loaded dustpan, transfers it over the rim of the desktop waste bin, tilts it to dump the collected scraps into the container, and returns to a neutral position.

A trial is scored as a success if and only if: (1) all of the scattered paper scraps are cleanly collected and transferred into the waste bin, (2) neither the brush nor the dustpan is dropped or detached from the grippers, and (3) the waste bin remains standing upright without being tipped over during the dumping motion.

Benchmark 4: Hierarchical Block Stacking.

The Hierarchical Block Stacking task serves as the testbed for both online competence awareness (Section 5.2.3) and closed-loop lifelong learning (Section 5.3), evaluating multi-stage sequential execution, precise contact and balance, and tolerance to compounding errors. The task involves three color-coded blocks (green, blue, and red). In the nominal collinear configuration, the three blocks are placed collinearly on the tabletop in each of the 3!=6 possible color permutations (e.g., green-blue-red, blue-green-red, and red-blue-green). The green block serves as a stationary base, remaining at its initial position while the dual-arm system identifies the scene layout and manipulates the other blocks. In the disordered configuration, the blocks are initialized at arbitrary, non-collinear 2D positions with unaligned orientations. The robot must perform the following stages:

1. Base Identification and Blue Block Placement: The dual-arm system identifies the scene layout, grasps the blue block, and balances it on the green base;
2. Red Block Placement: The robot then grasps the red block and balances it on the blue block to construct a freestanding three-tier tower;
3. Release and Stabilization: Both grippers release the assembly and clear the scene, leaving the tower upright.

A trial is scored as a success if and only if: (1) the blue block is successfully balanced on the green base, (2) the red block is successfully balanced on the blue block, and (3) the final three-tier assembly (green \to blue \to red) remains upright and stable for at least 3 seconds after both grippers release and clear the scene.

Benchmark 5: Garment Folding.

The Garment Folding task evaluates long-horizon bimanual deformable-object manipulation. Starting from an unfolded garment laid flat on the table, the dual-arm system must:

1. Sleeve Folding: Sequentially grasp, lift, and fold the left and right sleeves inward across the torso;
2. Longitudinal Fold: Reposition both grippers to pinch the bottom hem and fold the lower half of the garment upward along the vertical centerline;
3. Lateral Fold: Grasp the outer folded edge and fold the garment horizontally to form a compact rectangular profile;
4. Smoothing and Alignment: Dynamically smooth out wrinkles and creases across the top layer with downward contact wiping.

A trial is scored as a success if and only if: (1) all four folding and smoothing stages are sequentially completed, (2) the final footprint is reduced cleanly without unfolding or severe edge misalignments, and (3) both grippers safely disengage without dragging or wrinkling the finished garment.

Benchmark 6: Hierarchical Cup Stacking (Cross-Task Shift).

To evaluate the policy’s lifelong adaptation and catastrophic forgetting mitigation under explicit cross-task shift, we introduce the Hierarchical Cup Stacking task. The setup employs three color-coded tapered cups (green, blue, and orange). Identical to the collinear setup in Hierarchical Block Stacking, the three cups are initialized collinearly in front of the robot across all 3!=6 spatial permutations. The green cup serves as the fixed base. The dual-arm system must:

1. Identify the target cup layout, approach, and securely grasp the blue cup, carefully placing it upside down on top of the green base cup with its rim facing downward, without knocking the base cup over;
2. Reposition the manipulators to grasp the orange cup, align its orientation, and stack it neatly atop the blue cup to form a compact three-tier nested cup assembly.

The success criterion strictly mirrors block stacking: a trial is deemed successful if and only if both stacking stages are completed without tipping, and the resulting three-cup tower remains stable and upright for at least 3 seconds after the grippers fully release and clear the scene.

D.3 Training Configurations and Computational Overhead

All models are fine-tuned from base Galaxea G0.5 checkpoints on a GPU cluster with 768GB of total GPU memory and approximately 1.2 PFLOPS of BF16 compute, using the AdamW optimizer with a cosine learning-rate decay schedule. Table 15 lists the hyperparameters shared across the real-world experiments.

During physical deployment, EqC performs K=3 energy gradient ascent steps starting from Gaussian noise \mathcal{N}(0,I) to iteratively refine the action chunk, employing a decaying step-size schedule [\eta_{0},\eta_{1},\eta_{2}]=[0.12,0.10,0.05]. This decaying schedule facilitates rapid initial progress toward the high-compatibility energy basin followed by fine-grained local stabilization, while keeping the overall inference latency within practical limits.

Wall-Clock Fine-Tuning Overhead.

To assess the computational cost of learning an explicit energy function, Table 16 reports wall-clock fine-tuning times for Flower Insertion (6 epochs, 82 demonstrations), Towel Folding (4 epochs, 36 demonstrations), and Debris Sweeping (4 epochs, 142 demonstrations). Computing explicit energy gradients during backpropagation introduces only a modest overhead, increasing training time by 25.9% on Flower Insertion, 22.1% on Towel Folding, and 17.5% on Debris Sweeping.

D.4 Energy-Gated DAgger Protocol

Initial Demonstration Dataset (\mathcal{D}_{0}).

For Hierarchical Block Stacking, the base policy E_{\theta_{0}} is initialized by fine-tuning on a small seed demonstration set \mathcal{D}_{0} comprising N_{0}=18 teleoperated episodes collected exclusively under nominal, collinear block initializations (spanning all 3!=6 color permutations with 3 demonstrations per configuration), yielding a 0\% initial success rate on perturbed setups. For Garment Folding, the base policy is initialized on N_{0}=18 demonstrations on standard unfolded garments, yielding an initial autonomous success rate of 30\%.

Human Intervention Baseline.

To isolate the benefit of automated competence-aware intervention, we additionally evaluate a Human Intervention DAgger baseline under the same data and training budget as Energy-Gated DAgger. The human baseline is initialized from exactly the same seed demonstration dataset \mathcal{D}_{0} containing N_{0}=18 teleoperated episodes. Intervention decisions are made by skilled teleoperators with more than three months of experience collecting VLA training data. At each round, the current checkpoint is deployed on the same nominal collinear configurations used for autonomous evaluation, spanning all 3!=6 possible color permutations for Block Stacking, while the operator continuously monitors whether autonomous execution is likely to fail or has moved beyond the policy’s apparent competence. When the operator identifies such a failure-prone state, autonomous execution is suspended and the operator intervenes to provide a recovery trajectory. For Block Stacking, we collect exactly 6 targeted recovery episodes per round over R=5 rounds, yielding a total of 30 additional recovery demonstrations and expanding the dataset from 18 to 48 episodes. For Garment Folding, we follow the same procedure over R=2 rounds, collecting exactly 6 targeted recovery episodes per round, yielding 12 additional recovery demonstrations and expanding the dataset from 18 to 30 episodes. The human-in-the-loop baseline therefore follows the same task-specific schedule and collection budget as Energy-Gated DAgger: R=5 rounds and 30 recovery demonstrations for Block Stacking, and R=2 rounds and 12 recovery demonstrations for Garment Folding. The two methods share the same initial dataset, collection budget, deployment schedule, and policy-update procedure, and differ in how intervention decisions are triggered: external human judgment for the baseline and the policy-internal energy-consistency signal U_{\Delta E} for our method.

Galaxea G0.5 + Diff-DAgger Baseline.

To provide a rigorous automated baseline for lifelong learning, we adapt Diff-DAgger (Lee et al., 2025) to the physical Galaxea G0.5 policy. Its online failure-detection scoring is identical to the flow-matching formulation detailed in Appendix B.5, evaluating M=10 Monte Carlo samples per replan across active embodiment dimensions. Analogous to EqC, persistent threshold exceedances over m consecutive replanning steps trigger expert recovery intervention. Crucially, G0.5 + Diff-DAgger operates under the exact same experimental conditions: it starts from the identical seed dataset, collects the exact same budget of targeted recovery demonstrations per round, and follows the identical iterative fine-tuning hyperparameters as Energy-Gated DAgger and the human baseline.

Active Deployment and Intervention Mechanism.

During autonomous deployment in round r, the policy executes action prefixes under receding-horizon control while continually monitoring the energy-consistency signal U_{\Delta E}. If U_{\Delta E} exceeds the fixed alarm threshold \gamma for m consecutive replanning steps, autonomous execution is suspended, and an expert intervenes to provide a recovery trajectory \xi_{j}^{\star} that guides the robot to successful task completion. Each round collects |\mathcal{Q}_{r}|=6 targeted recovery episodes.

Iterative Fine-Tuning.

Following Energy-Gated DAgger, each round aggregates newly collected recovery trajectories into the historical dataset (\mathcal{D}_{r+1}\leftarrow\mathcal{D}_{r}\cup\mathcal{Q}_{r}). The policy is re-initialized from the same initial checkpoint and independently trained for 2 epochs on \mathcal{D}_{r+1} using the hyperparameters in Table 15. For Block Stacking, across R=5 rounds, the robot incorporates 30 active recovery demonstrations, expanding the dataset from |\mathcal{D}_{0}|=18 to |\mathcal{D}_{5}|=48 episodes. For Garment Folding, the policy completes lifelong adaptation in R=2 rounds, incorporating 12 targeted recoveries and expanding the dataset from |\mathcal{D}_{0}|=18 to |\mathcal{D}_{2}|=30 episodes. To ensure statistical consistency, all lifelong learning experiments across both tasks and all compared methods are repeated over 3 independent end-to-end runs. After each fine-tuning round, the resulting policy checkpoint is evaluated over 20 independent autonomous trials, and we report the mean success rate and standard deviation across the 3 runs in Table 17 and Figure 4.

Appendix E Robot Agent as Expert

Energy-Gated DAgger connects competence detection, expert recovery, and policy updates through a common intervention interface. We extend this framework by using a robot agent powered by gpt-6-astra as the recovery expert on Garment Folding. Once the expert agent is prepared, both the decision to request assistance and the subsequent recovery are automated: EqC detects when its execution becomes unreliable, and the agent takes over to complete the task without human intervention. Incorporating these recoveries into policy training closes an automated intervention-and-learning loop. This also provides a route to behavioral distillation, through which a compact robot policy acquires capabilities from a larger model through the recovery demonstrations it generates.

Experimental setup.

We use the bimanual platform and garment-folding task described in Appendix D. We initialize from the seed-trained EqC checkpoint of Run 1 at Round 0 in Table 17(b), obtained using the original 18 garment-folding demonstrations. During deployment, EqC generates actions and monitors its energy-consistency signal U_{\Delta E}. The alarm threshold remains fixed at \gamma=0.2, identical to the preceding lifelong learning experiments, and we retain the same persistence criterion for triggering intervention. Upon an alarm, autonomous execution is suspended and the robot agent assumes the expert-recovery role in Algorithm 1.

Expert agent preparation.

We prepare the agent through full-task practice followed by takeover-specific practice (Algorithm 2). We first provide Astra with the original 18 teleoperation demonstrations. The agent inspects these demonstrations offline to understand the folding procedure and writes a robot-control script that approximates the recorded manipulation trajectories. It then performs repeated complete garment-folding attempts on the physical robot. After each attempt, Astra reviews the execution outcome, revises the script, and consolidates its spatial reasoning and manipulation experience into reusable executable skills. This process continues until the agent achieves a success rate of approximately 100% on complete garment-folding attempts.

Because EqC has already performed part of the task when an intervention is triggered, recovery requires adaptation to intermediate garment and robot configurations. We therefore conduct several rounds of takeover-specific practice. Following an EqC alarm, the agent assesses the current scene, estimates spatial relationships and distances, and adapts its control script to complete the task from the intervention state. After execution, it reviews the recovery, refines the reusable control code, and incorporates the new experience into its skill library. Once preparation is complete, the agent enters routine takeover: it uses the prepared code and accumulated skills to adapt an execution script to each intervention state, without further post-execution reflection or skill consolidation. The adapted script is executed directly on the robot, avoiding a separate language-model response for each low-level action.

Takeover and policy learning.

After expert preparation, we deploy the agent for routine takeover, following Stage 3 of Algorithm 2. For each EqC alarm, Astra uses the prepared code and accumulated skills to adapt a recovery script to the current intervention state. The adapted code is then executed to complete the remaining operations.

We collect six agent-generated recovery demonstrations and combine them with the original demonstration set:

|\mathcal{D}_{0}|=18,\quad|\mathcal{Q}_{\mathrm{Astra}}|=6,\quad\mathcal{D}_{\mathrm{1}}=\mathcal{D}_{0}\cup\mathcal{Q}_{\mathrm{Astra}},\quad|\mathcal{D}_{\mathrm{1}}|=24.

After fine-tuning on this combined dataset, the updated EqC policy successfully completes all 20 autonomous garment-folding evaluation trials, achieving 100% success without agent takeover.

Implications.

This proof-of-concept experiment demonstrates how EqC can support a fully automated intervention-and-learning loop after initialization and expert preparation. Its introspective signal determines when assistance is needed, the robot agent supplies recovery behavior, and policy updates incorporate the resulting experience into subsequent autonomous execution. The system also supports behavioral distillation at two levels: the larger model’s reasoning is consolidated into executable robot skills, and the recoveries produced by those skills become supervision for the compact EqC policy. By connecting autonomous competence detection, agent-directed recovery, and policy improvement, this framework provides a promising path toward lifelong robot learning: situations beyond the policy’s current competence become opportunities for targeted learning, allowing the robot to progressively expand its autonomous capabilities through continued deployment.

Appendix F Cross-Task Continual Learning and Catastrophic Forgetting

Protocol.

Real-world lifelong deployment often requires agents to acquire novel manipulation skills sequentially while retaining competence on previously mastered behaviors. To test this capability, we select the fully converged Block Stacking policy checkpoint from Round 5 that achieved 100.0\% autonomous success (corresponding to Run 1 in Table 17, trained on |\mathcal{D}_{5}|=48 demonstrations) and deploy it directly onto the novel Hierarchical Cup Stacking benchmark without any prior cup interactions.

• Round 0: Evaluated directly on the cup task, this base checkpoint achieves 0.0\% success due to geometric discrepancy and altered contact dynamics, while maintaining its initial 100.0\% competence on block stacking. As visualized in Figure 5 (Top row), trajectory drift and grasp slippage immediately trigger persistent U_{\Delta E} violations, prompting expert assistance before hardware collisions occur.
• Round 1: Using EqC’s Energy-Gated DAgger with the frozen threshold \gamma, persistent U_{\Delta E} violations detect out-of-competence execution and prompt human recovery, collecting |\mathcal{Q}_{1}^{\text{cup}}|=6 cup stacking demonstrations. After fine-tuning on \mathcal{D}_{5}\cup\mathcal{Q}_{1}^{\text{cup}}, cup stacking performance jumps to 50.0\%. However, objective conflicts between tasks induced representation interference and parameter drift, causing the original block stacking success to degrade to 60.0\% and manifesting classic catastrophic forgetting in continual learning.
• Round 2: During Round 2 deployment, EqC monitors both tasks simultaneously. It detects residual competence gaps in cup stacking as well as regression in block stacking, triggering targeted interventions that collect an additional 6 cup stacking episodes and 6 block stacking episodes (|\mathcal{Q}_{2}|=12). Fine-tuning on the cumulative dataset successfully resolves task competition: both Hierarchical Block Stacking and Hierarchical Cup Stacking achieve 100.0% autonomous success (with autonomous cup stacking illustrated in Figure 5, Bottom row; Table 18).

This result demonstrates that actively gating data collection for degraded and novel skills provides a practical mechanism for recovering from task-interference-induced forgetting.

Appendix G Ablation Study

G.1 Robustness to Optimization Step Schedules

To further evaluate the sensitivity of EqC to the underlying optimization step-size schedule, we conduct an ablation study across the LIBERO benchmark. Specifically, we compare our default Cosine decay schedule against uniform step schedules (with 15 and 20 optimization steps). Table 19 presents the performance breakdown across the four standard LIBERO task suites (Goal, Spatial, Object, and Long). The results demonstrate that EqC is highly robust to variations in the optimization scheduling strategy. While the Cosine decay schedule yields the best average performance (achieving 94.3), the uniform schedules maintain competitive results, with the 20-step uniform schedule reaching 94.2 and the 15-step schedule maintaining 93.1. This consistent performance indicates that our energy-based failure detection mechanism relies on the fundamental energy landscape rather than highly tuned optimization step sizes, ensuring reliable and stable operation across different scheduling choices.

G.2 Adaptive Inference

To investigate whether our action optimization converges to an equilibrium during inference, we adapt EqC to DiT backbones and analyze the optimization dynamics across the LIBERO benchmark. During inference procedure, we observe that candidate actions rapidly reach a stationary regime, as evidenced by two main aspects in Table 20:

• Rapid Gradient Decay: the mean gradient norm \|\nabla E\| decreases almost monotonically from an initial noise scale (49.3) at step 0 to around 1.0 by step 14, ultimately flattening to a floor of 0.3 at the final step.
• Early Energy Saturation: the normalized energy convergence ratio—defined as (E_{k}-E_{0})/(E_{\mathrm{final}}-E_{0})—saturates remarkably early, reaching 99.0\% by step 7 and exceeding 99.9\% by step 9.

These empirical observations indicate that our method rapidly reaches an equilibrium, as action candidate refinement enters a stable energy state within the first 10–15 steps.

Motivated by this steady-state behavior, we introduce an adaptive early-stopping mechanism that terminates action chunk optimization once the gradient norm falls below a threshold g_{\mathrm{min}}\in\{1.0,3.0\}. As summarized in Table 21, adaptive early stopping significantly enhances inference efficiency without incurring any notable performance degradation. Specifically, setting g_{\mathrm{min}}=1.0 reduces the average optimization steps to 16.8 (a 47.5\% reduction in computation compared to the full budget) while maintaining an equivalent average success rate (93.7 vs. 94.0 baseline). Setting the threshold to g_{\mathrm{min}}=3.0 further truncates the average optimization steps down to 11.1 (saving 65.3\% of inference computation) while achieving a success rate of 94.2. These results confirm that our proposed method—Equilibrium Energy Control—can effectively eliminate redundant computation while preserving policy execution quality through an adaptive gradient-based stopping strategy.

Appendix H Case Study: Detection Timeliness and Failure Modes

A practical introspective robot system must trigger assistance at the competence boundary: alarming neither prematurely on benign execution, nor belatedly after irreversible physical damage has occurred. In this section, we present real-world qualitative case studies on the physical Galaxea bimanual manipulation platform to evaluate detection timeliness across two domains: rigid-body multi-stage contact (Hierarchical Block Stacking) and deformable manipulation (Garment Folding). We first highlight EqC’s proactive detection capability, followed by a detailed diagnostic of the baseline’s characteristic pathology modes.

H.1 Proactive Detection by EqC at the Competence Boundary

As illustrated in Figure 6, EqC’s path-wise energy consistency residual U_{\Delta E} demonstrates sharp sensitivity to the earliest emergence of off-manifold execution drift:

• Rigid-Body Manipulation (Top row): During Hierarchical Block Stacking, when the block is accidentally pushed away (Top-left) or angular misalignment occurs during contact placement (Top-right), U_{\Delta E} persistently exceeds the threshold \gamma, arresting autonomous motion before blocks are knocked over or the tower collapses.
• Deformable Manipulation (Bottom row): In Garment Folding, EqC reliably flags failure to secure the right sleeve during grasping (Bottom-left) and failure to reach the target folding position (Bottom-right), requesting expert takeover while the fabric remains completely recoverable.

H.2 Pathology Diagnostics of Flow-Loss Detection (Diff-DAgger)

In contrast to EqC’s proactive warnings, point-wise velocity error in flow-matching policies displays significant variance and poor temporal sensitivity to compounding geometric error. As illustrated in Figure 7, evaluating Galaxea G0.5 + Diff-DAgger across both rigid stacking (Rows 1–2) and deformable garment folding (Row 3) reveals three distinct, pervasive pathology modes across multiple independent trials:

1. Delayed Detection during Cumulative Drift (Figure 7, Left column): When the manipulators suffer from progressive spatial drift, Diff-DAgger persistently fails to raise an alarm throughout the approach trajectory. Across both block stacking (Rows 1–2) and garment folding (Row 3), it alarms only at a belated stage—after the block cluster is disrupted or the fabric has already suffered a severe angular tilt—severely compressing or completely missing the reaction window required for human teleoperator intervention.
2. Spurious False Alarm on Supported Execution (Figure 7, Middle column): During nominal, error-free execution, Diff-DAgger’s noisy loss expectation frequently induces spurious spikes. This occurs not only during mid-trajectory block transit, but even on completely flat, undisturbed garments (Row 3, Middle), mistakenly triggering false-positive alarms that prematurely halt valid execution.
3. Irreversible Failure Triggering (Figure 7, Right column): When execution diverges critically (e.g., stacking onto the wrong block or severely bunching fabric), Diff-DAgger fails to catch the deviation during transit. It alerts only post-contact or after the garment is balled into an entangled knot (Row 3, Right). At this terminal stage, in-place teleoperation is impossible, requiring a full manual reset of the environment.

Appendix I Code Availability

We have released the code at https://github.com/hrh6666/EqC.