Erst schauen, dann greifen: Fudan-Methode hebt Erfolg bei Verdeckung real von 35/90 auf 83/90

Abstract

Robots manipulating objects in clutter must decide not only how to act, but also where to look. Although active camera motion can expose an occluded target, it may move the robot into an initial observation–state distribution that differs from the manipulation policy’s training distribution. We identify this problem as first-frame distribution sensitivity: under perturbed starts in LIBERO, naively prepending camera repositioning to \pi_{0.5} reduces success from 46.3% to 11.5%. We introduce ActiveAlign, a data-adaptation framework that trains a Vision-Language-Action (VLA) policy from the states at which active perception terminates. A learned viewpoint module first generates a camera trajectory. Offline, ActiveAlign uses closed-loop alignment to connect the trajectory endpoint to an existing demonstration, replays the remaining manipulation trajectory, and retains successful continuations for fine-tuning. At deployment, the same viewpoint module precedes the adapted VLA without modifying its architecture. Across 40 LIBERO tasks, ActiveAlign achieves 70.7% success with \pi_{0} and 72.7% with \pi_{0.5}, outperforming their standard-training baselines by 40.0% and 26.4% respectively. Matched-state and data-matched controls support gains beyond prefix shortening and generic viewpoint augmentation; OpenVLA and GR00T N1 also show the same trend. Success rate under physical occlusion in real world increases from 35/90 to 83/90. These results show that active perception benefits manipulation when policy training is aligned with the observation and robot-state distribution induced by viewpoint selection.

I INTRODUCTION

Effective manipulation begins with an observation that supports the intended action [1, 2]. A robot asked to retrieve fruit from a container or grasp a cup behind clutter may need to move its camera before it can see a useful approach direction. Vision-Language-Action (VLA) models [3, 4, 5, 6, 7] provide a powerful interface from visual observations and language to robot actions, but their input is still determined by where the robot looks. Repositioning an existing wrist camera is an appealing way to expose the target before manipulation. The resulting system must therefore solve two connected problems: selecting an informative view and continuing the task from the state produced by that selection.

The most direct composition is to let an active-perception (AP) module improve the view, then hand control to a VLA trained on conventional demonstrations. Our experiments reveal a counterintuitive outcome: this composition can substantially reduce task success. Under a shared perturbed-start LIBERO protocol, \pi_{0.5} falls from 46.3% to 11.5% average success after camera repositioning, and \pi_{0} falls from 30.7% to 7.0%. The qualitative examples in Fig. 1 illustrate the tension: a policy can act from a familiar but occluded view, yet struggle to exploit a view selected to reveal the target. Improving the observation and making it usable by the action policy are distinct requirements.

We study this interface through first-frame distribution sensitivity. Here, the first frame is the observation at the handoff from active perception to manipulation. Wrist-camera motion changes image location, scale, and occlusion together with arm configuration and the relationship to the remaining task. A policy must consequently infer a suitable continuation from a different combination of visual and proprioceptive cues. Action chunking [8, 9] can propagate an early error through the actions executed before the next observation, even when later decisions use fresh images. This motivates examining the joint observation–action interface; a difference between first-frame image distributions alone does not establish the cause of every failure. Our experiments therefore pair the failure observation with controls that separate viewpoint information from robot-state progress and data volume.

The central design insight is that the deployment viewpoint policy is available during data construction. Rather than requiring a manipulation learner to accommodate arbitrary sensing motions, we can expose it to successful continuations from the endpoints of the very module it will encounter at inference. This turns the composition problem into a question about supervision: how can existing demonstrations be made to begin where active perception stops? Demonstration-generation methods such as MimicGen and DemoGen [10, 11] motivate reusing recorded behavior, while the present setting adds a specific constraint: the starting observation and state are chosen by a task-conditioned camera policy. Visual augmentation alone does not supply the actions needed to continue from that state.

ActiveAlign constructs these continuations offline. Given scene RGBD and a language instruction, a learned module predicts a wrist-camera trajectory. In a restored environment, we execute the trajectory and match its endpoint to a nearby end-effector state in an early demonstration prefix. Closed-loop alignment bridges the residual gap, after which the recorded manipulation suffix is replayed. Successful alignment-and-replay trajectories become training data for the VLA. The active approach replaces part of the original demonstration, while the recorded continuation teaches the policy how to proceed after that approach. At deployment, the same viewpoint module runs once before object contact; the adapted VLA then manipulates using fresh observations. The procedure changes training data and policy weights while preserving the VLA architecture.

Replay feasibility is essential to this construction. End-effector proximity does not ensure that object and contact states agree, so nearest-state matching alone cannot validate a continuation. Successful replay filters the candidate endpoints and limits the training distribution to states for which usable supervision can be reconstructed. ActiveAlign consequently applies to resettable simulators and reconstructable physical workcells. This requirement belongs to offline data collection: deployment requires neither scene reset nor expert replay. The resulting scope is pre-manipulation view selection, with longer-horizon tasks testing the lasting value of that initial view rather than continuous response to moving occluders.

Our evaluation follows this reasoning from composition failure to controlled explanation and physical validation. On LIBERO Spatial, a matched-state camera-extrinsics control holds the demonstration step and manipulation suffix fixed and reaches 54.6%, compared with 78.2% for ActiveAlign. Data-matched random views and broad viewpoint jitter reach 58.0% and 61.4%, respectively. These comparisons support a benefit from the informative, deployment-matched observations beyond shorter demonstrations and generic coverage. Across four suites, ActiveAlign improves \pi_{0} and \pi_{0.5} by 40.0 and 26.4 percentage points; OpenVLA and GR00T N1 extend the evidence to different action formulations. Initial-occlusion simulation tasks, diverse physical tasks, and endpoint perturbations further characterize when the construction is useful and where it breaks down.

Our contributions are threefold:

• We characterize the failure of naive composition of active perception and VLAs, and test competing explanations through matched-state and data-matched controls.
• We introduce observation-aligned demonstration adaptation, connecting active-view endpoints to successful manipulation through prefix matching, closed-loop alignment, and replay filtering.
• We evaluate the construction across four VLA backbones, 40 LIBERO tasks, initial occlusion, and physical manipulation, with sensitivity studies that quantify the feasibility–diversity trade-off and endpoint error tolerance.

II RELATED WORK

Vision-language-action policies. Large-scale language-conditioned robot policies such as RT-1 and RT-2 established a general interface from visual observations and language instructions to robot actions [12, 13]. Recent VLA models instantiate this interface with different action representations: \pi_{0} and \pi_{0.5} use flow matching [3, 5], OpenVLA predicts discrete action tokens [4], and GR00T N1 uses a diffusion-transformer action module [14]. Diffusion policies and alternative action tokenizations provide additional formulations for visuomotor control [9, 15, 16, 17]. Despite these architectural differences, each policy depends on the observation–state distribution represented in its training demonstrations. ActiveAlign is orthogonal to these action parameterizations: it preserves the policy architecture while adapting the fine-tuning distribution to the states induced by a deployment-time viewpoint module which offers a more informative view.

Active and multi-view perception. Active perception [18, 19] and next-best-view methods [20, 21] select sensing actions that improve downstream decisions. Multi-view manipulation methods aggregate or render observations from multiple viewpoints to improve spatial reasoning [22, 23, 24, 25], while SAM2Act, VISO-Grasp, and ViewActive explore related mechanisms for visual representation and viewpoint selection [26, 27, 28]. Our approach, instead, physically repositions a wrist camera before invoking a separately adapted VLA, thus studying physical wrist-camera motion, which changes not only the visual input but also the robot configuration from which manipulation begins. Its focus is therefore the compatibility between a selected better viewpoint and the downstream policy’s manipulation distribution.

Recent robotic systems learn physical sensing motion together with manipulation. Vision in Action couples sensing and bimanual behavior using a 6-DoF neck and VR teleoperation [29]; HoMMI combines egocentric human demonstrations, hand–eye policies, and whole-body control [30]; and ActiveGlasses uses a dedicated perception arm to coordinate camera and manipulation motion [31]. These systems collect or learn paired sensing and manipulation behavior for specialized active-vision embodiments. In contrast, ActiveAlign uses an existing wrist camera for a single pre-manipulation viewpoint trajectory and connects it to an independently trained VLA through offline demonstration adaptation, without altering the VLA architecture or requiring continuous gaze control. GenDoP [32] studies related autoregressive RGBD-and-language camera trajectory generation [33]; our contribution concerns the robot-data adaptation interface.

Demonstration adaptation and distribution shift. DAgger [34] addresses policy-induced state mismatch through iterative collection. Domain randomization and augmentation broaden visual coverage [35, 36, 37], including through generated images [38, 39, 40]. ActiveAlign targets a different source of shift: the handoff states produced by a separate viewpoint policy used at deployment. Demonstration-generation methods reuse recorded behavior to expand the training distribution. MimicGen transforms demonstrations across object configurations [10], while DemoGen adapts trajectories and point-cloud observations for spatial generalization [11]. ActiveAlign instead adapts demonstrations to the physical endpoints induced by a task-conditioned viewpoint policy. Because camera motion changes both arm configuration and visual observations, geometric endpoint matching alone does not guarantee a valid manipulation continuation. ActiveAlign therefore uses closed-loop alignment and successful replay to validate supervision. Our matched-state and data-matched controls further compare this targeted, deployment-matched adaptation with shorter manipulation suffixes and broader random-view or viewpoint-jitter coverage.

III METHOD

III-A Problem Formulation

Let x_{t} denote the physical scene and robot state and let o_{t} denote the complete policy observation, including scene RGB, wrist RGB, and proprioception where supported by the backbone. A policy \pi_{\theta} conditions on o_{t} and instruction l. Demonstrations contain state-observation-action trajectories; denote the observation distribution at original demonstration starts by p_{\mathrm{demo},0}.

An active perception module \mathcal{A} generates a camera trajectory from a scene observation s_{0} and instruction l. Execution induces a new state \tilde{x}_{0} and observation \tilde{o}_{0}. The corresponding initial distribution p_{\mathcal{A},0} may differ from p_{\mathrm{demo},0}. Because the camera is wrist-mounted, this shift generally involves both vision and robot state. A mismatch of initial-image marginals alone neither proves that the input is outside the full training support nor establishes the cause of a policy failure.

Our objective is to improve manipulation from the states induced by \mathcal{A}. We construct adapted demonstrations \widetilde{\mathcal{D}} and fine-tune with the backbone’s supervised learning objective:

\theta^{*}=\arg\min_{\theta}\mathbb{E}_{(o,l,a)\sim\mathcal{D}_{\mathrm{train}}}\bigl[\mathcal{L}_{\mathrm{VLA}}(\theta;o,l,a)\bigr].

Here \mathcal{D}_{\mathrm{train}} denotes the condition’s fine-tuning data, including aligned demonstrations for ActiveAlign; data-matched controls use the same mixing ratio. The target a is an action or chunk, and \mathcal{L}_{\mathrm{VLA}} is its token-prediction or continuous-action loss. Task success is evaluated through rollouts, not optimized directly by this loss.

The construction must preserve more than the appearance of the active view. Each observation needs an action target that is executable from the corresponding robot and scene state. We therefore use the deployed perception module to generate candidate starting states, and use recorded manipulation to supply their continuations. Prefix matching identifies a nearby continuation, alignment connects to it, and replay checks whether the resulting episode remains successful. These three operations jointly determine which candidate views can become useful VLA supervision.

III-B Active Perception Module

View selection is conditioned on both the task and the scene: the instruction identifies what needs to be observed, while the surrounding geometry determines which camera motion may expose it. The module combines RGB features, depth features, and language features to predict a sequence of camera poses. RGB and depth use separate CLIP branches, and a conditional autoregressive decoder generates quantized pose tokens. The decoded extrinsics are passed to the robot controller, which turns the visual prediction into a candidate sensing motion.

Formally, the input is a scene RGBD observation s_{0}=(I_{\mathrm{rgb}},D) and instruction l, and the output is \{T_{1},\ldots,T_{K}\}. The default uses K=30 poses (Fig. 2), represented in SE(3) and transformed into the calibrated robot/world frame. The generated sequence can be interpolated to 120 poses for smooth execution. Training uses scene-instruction-camera-trajectory pairs: 20,000 in simulation and 1,200 for the physical setting, separate from the VLA demonstrations.

Predicted paths are candidate sensing motions, without certified optimality or collision freedom. Simulation uses IK repositioning; the physical robot moves continuously through its controller. Adaptation retains endpoints permitting alignment and successful manipulation replay.

III-C Observation-Aligned Data Adaptation

In Algorithm 1, the environment is restored to each demonstration’s starting state. After executing \mathcal{A}, we identify the nearest end-effector state within a prefix window [\alpha T,\beta T]. Closed-loop Cartesian alignment closes the residual gap, after which the recorded manipulation suffix is replayed. Only the alignment segment and suffix of successful attempts are retained; the active approach is not a VLA training target.

Prefix matching. The default window is [0.05T,0.30T]. Earlier matching retains more of the original approach; later matching shortens manipulation and may require unreproduced scene changes. End-effector proximity does not ensure matching object/contact states, making successful replay a necessary filter. Our window study measures retention, diversity, and success.

Closed-loop alignment. A recorded adjustment connects the active endpoint to the matched suffix using operational space control (OSC) in simulation and Cartesian control on the robot. The default simulation adjustment budget is 20 steps. Directly replaying the suffix from the unmatched endpoint can accumulate control error, so this transition is included in the adapted trajectory. The VLA therefore receives supervision for leaving the active endpoint as well as following the retained manipulation. Unlike IK repositioning, the alignment segment consists of executable VLA training actions.

Filtering and retries. Simulation retries vary the active trajectory budget and alignment steps. The reported adaptation counts are 1,736 retained out of 2,000 attempted in simulation (86.8%) and 86 retained out of 100 attempted on the physical platform (86.0%).

Consequently, \widetilde{\mathcal{D}} represents a feasibility-filtered active-view distribution: filtering and retries can make p_{\widetilde{\mathcal{D}},0} differ from p_{\mathcal{A},0}. Sharing the viewpoint module targets nominal alignment, without guaranteeing distribution equality or optimal task success. The control and perturbation studies test its practical benefit and sensitivity.

III-D Deployment and Applicability

Deployment consists of one active approach followed by manipulation. The viewpoint module is queried once before object contact. Thereafter, the VLA obtains fresh images and state inputs and predicts actions. For the chunked \pi_{0} and \pi_{0.5} implementations,

\{a_{t},\ldots,a_{t+H-1}\}=\pi_{\theta^{*}}(o_{t},l),\quad H=10,\quad R=5,

where the first R actions are executed before replanning. Thus errors can accumulate within the executed portion, but the policy does not commit to all H actions without correction. Other backbones retain their native action-generation interfaces.

The wrist camera moves with the arm during manipulation, but no further viewpoint-selection phase is invoked. Offline adaptation needs simulation reset or a reconstructable physical workcell; deployment needs neither reset nor demonstration replay. Replacing \mathcal{A} may require regenerating data and fine-tuning.

IV EXPERIMENTS

IV-A Experimental Setup

Environments and data. LIBERO [41] provides four suites of 10 tasks, with 50 original demonstrations per task (2,000 total). LIBERO-Occ [42] uses the same suites but occludes the target in the default initial view. The physical evaluation runs on two arm settings (an ARX X5 arm and an ARCHER-Y6 arm), both with scene-level and wrist-mounted Intel RealSense D435 cameras, and 100 original demonstrations. RLBench [43] is used only to visualize zero-shot transfer of the perception module.

Backbones and training. We evaluate \pi_{0}, \pi_{0.5}, OpenVLA, and GR00T N1. We follow the model-specific recommended training recipes, with hyperparameters selected using the validation set. All models share the LIBERO Franka end-effector delta-pose and gripper interface.

Perturbed-start protocol. Each LIBERO result uses 50 rollouts per task, with horizons of 220, 280, 300, and 520 steps for Spatial, Object, Goal, and Long. Initial end-effector pose coordinates receive zero-mean Gaussian perturbations with standard deviation 0.1 (translation in meters and rotation in radians). This perturbed-start protocol differs from the benchmark’s prescribed initial states and is held fixed across all method conditions.

Conditions and statistical reporting. Standard trains on original demonstrations and uses no AP; Naive Active invokes AP only at inference; ActiveAlign trains on observation-aligned adaptations and invokes the same AP module. Data-matched controls fix retained episode count, filtering, suffix length, optimization, and mixing. LIBERO uses the environment completion checker. Physical results report binary counts and Wilson 95% confidence intervals.

IV-B Simulation Results and Analysis

Perturbed-start performance. Table I shows the handoff failure directly: AP without aligned training reduces average success from 30.7% to 7.0% for \pi_{0} and from 46.3% to 11.5% for \pi_{0.5}. ActiveAlign instead reaches 70.7% and 72.7%, gains of 40.0 and 26.4 percentage points (pp) over Standard, with improvements on every suite. These values are comparisons within our perturbed-start protocol.

The gains also extend beyond recovering the loss from naive composition. On Object, for example, ActiveAlign exceeds Standard by 46.0 pp for \pi_{0} and 39.0 pp for \pi_{0.5}, while Naive Active reaches only 0.0% and 2.2%. Thus, the informative starting condition becomes useful when paired with adapted supervision. The two adapted backbones finish within 2.0 pp of each other on average; this motivates studying the data interface alongside model choice, without inferring that model capacity is generally irrelevant.

Table II tests whether the adaptation is tied to the two flow-matching \pi policies. ActiveAlign improves OpenVLA by 13.2 pp and GR00T N1 by 16.0 pp over their Standard conditions. The consistent trend across flow-matching, autoregressive-token, and diffusion-transformer policies supports compatibility with the tested action formulations, without implying universal backbone independence.

Table III shows that under the benchmark-prescribed initial states, Standard and ActiveAlign are nearly matched; as the start-state shift increases, the gap widens.

Viewpoint and data controls. Table IV isolates two competing explanations. The matched-state control fixes k^{*}, suffix length, retained count, training schedule, and mixing, while independently changing camera extrinsics to provide a default view instead of an active informative observation. Its 54.6% success, versus 78.2% for ActiveAlign, leaves a 23.6 pp gap after controlling for prefix shortening. As an auxiliary diagnostic, the target is visible in 43.7% of the matched-state control observations and 74.8% of the corresponding ActiveAlign observations.This visibility difference confirms that the two conditions provide substantially different target information; together with the controlled success comparison, it is consistent with the benefit of training on informative views induced by the deployment viewpoint module. Random-view adaptation and broad viewpoint jitter adaptation use the same data and optimization budgets and reach 58.0% and 61.4%. Generic viewpoint coverage therefore improves over Standard, but remains 20.2 pp and 16.8 pp below ActiveAlign. These results support the value of deployment-matched viewpoint adaptation beyond additional data, shorter continuations, and untargeted viewpoint diversity.

Initial occlusion and perception transfer. Table V evaluates targets hidden in the default first frame. ActiveAlign exceeds Standard by 28.6, 24.6, 22.4, and 34.6 pp on Spatial, Object, Goal, and Long. Naive Active remains below Standard. The Long result shows that a better initial observation can remain useful over a longer manipulation horizon; it does not test occluders that move during execution.

Fig. 3 shows the selected views around clutter. RLBench is a qualitative test of perception transfer only; end-to-end manipulation is evaluated in LIBERO and on the physical robot.

In Fig. 4, adapted demonstrations and AP-conditioned inference occupy overlapping feature regions. This visualization is descriptive; the controlled rollouts provide the causal evidence.

Sensitivity analysis. Table VI Panel (a) exposes the feasibility–diversity trade-off: later windows retain more adaptations but reduce endpoint diversity (measured as mean pairwise terminal-camera translation after scene normalization), and the middle window gives the highest success. Panel (b) perturbs only the predicted terminal camera position at inference. This perturbation is separate from the base protocol’s initial robot-pose noise and tests sensitivity to errors in the AP endpoint. Success falls by 6.0 pp at 0.02 m and 9.0 pp at 0.04 m, then drops below Standard at 0.10 m. The method therefore tolerates moderate translational endpoint error but degrades when the deployment handoff moves substantially beyond the aligned training distribution. Panel (c) shows consistent gains as the trajectory budget grows from 5 to 30. This trend shows that performance benefits from additional viewpoint-trajectory capacity.

IV-C Physical Evaluation

Collection and reset. Marked fixtures restore object and container states within 1 cm and 3^{\circ}. Predicted camera paths are executed continuously; the recorded end-effector prefix is matched, closed-loop Cartesian control reduces the residual, and the manipulation suffix is replayed. An operator provides safety supervision only: near collision or force limits, the operator can abort but cannot correct a motion, teleoperate, or finish the task. The procedure retains 86 of 100 attempts. Reset and replay are used only for offline data construction.

Evaluation protocols. We conduct two separate physical evaluations. The first evaluation measures task diversity using cup grasping, puzzle retrieval, and a longer observe–open–grasp–place task. Each task is evaluated for 20 trials, yielding 60 additional trials per method. The second measures robustness to initial occlusion using three apple-manipulation tasks and three occlusion severities. Each task–severity combination is evaluated for 10 trials, yielding 3\ \text{tasks}\times 3\ \text{severities}\times 10\ \text{trials}=90\ \text{trials per method}.

Physical result interpretation. The 60 trials per method in Table VII are a separate evaluation and are not merged with the occlusion aggregate. ActiveAlign improves over both baselines on every task, with 17/20 successes for cup grasping, 15/20 for container retrieval, and 13/20 for observe–open–grasp–place. Their per-task gains over Standard are 40, 45, and 40 pp, but the 20-trial intervals remain wide. We therefore use these results as evidence of broader task coverage rather than a precise estimate of performance over arbitrary physical tasks. Besides, the lower success on the longer task illustrates the limits of an informative initial view over an extended manipulation horizon. Fig. 5 shows the representative rollouts.

Table VIII aggregates three apple-and-occluder tasks at each severity. ActiveAlign succeeds in 83/90 trials (92.2%), compared with 35/90 (38.9%) for Standard and 17/90 (18.9%) for Naive Active. Their Wilson intervals are [84.8,96.2]%, [29.5,49.2]%, and [12.1,28.2]%, respectively. Relative to Standard, ActiveAlign gains 43.3, 56.7, and 60.0 pp under partial, medium, and severe occlusion.

V LIMITATIONS

ActiveAlign is currently limited by its offline data-construction assumptions. Constructing aligned continuations requires a restorable simulator or a reconstructable physical workcell, as well as demonstration prefixes that remain replayable from the endpoints generated by the viewpoint module. Because training trajectories are retained only after successful alignment and replay, the adapted dataset represents a feasibility-conditioned subset of the deployment endpoint distribution. Difficult endpoints may therefore be underrepresented even though they can still occur at deployment. RLBench evidence is perception-only. The current system performs a single viewpoint-selection phase before manipulation. It does not reselect viewpoints in response to moving occluders. Extending observation-aligned adaptation to non-resettable environments, continuous viewpoint selection, and broader physical deployments remains an important direction.

VI CONCLUSION

ActiveAlign connects the decision of where to observe with the learned behavior needed to act from that observation. By constructing successful continuations from active-view endpoints, it enables existing VLA architectures to benefit from a pre-manipulation sensing phase. LIBERO and physical results show consistent gains, while matched-state and data-matched controls support benefits beyond prefix shortening and generic viewpoint coverage. The resulting lesson is that viewpoint selection and manipulation supervision should be designed together, even when the two policies are trained separately. Extending this construction beyond replayable prefixes and interleaving observation selection with manipulation are promising next steps.

References

1. [1] Z. Liu, Y. Wang, S. Zheng, T. Pan, L. Liang, Y. Fu, and X. Xue (2025) ReasonGrounder: LVLM-guided hierarchical feature splatting for open-vocabulary 3D visual grounding and reasoning. arXiv preprint arXiv:2503.23297.
2. [2] Z. Liu, S. Zheng, S. Chen, C. Zhao, L. Liang, X. Xue, and Y. Fu (2025) A neural representation framework with LLM-driven spatial reasoning for open-vocabulary 3D visual grounding. In Proceedings of the 33rd ACM International Conference on Multimedia, pp. 1042–1051.
3. [3] K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al. (2024) \pi_{0}: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164.
4. [4] M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, et al. (2024) OpenVLA: an open-source vision-language-action model. arXiv preprint arXiv:2406.09246.
5. [5] P. Intelligence, K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, et al. (2025) \pi_{0.5}: A vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054.
6. [6] O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu, et al. (2024) Octo: an open-source generalist robot policy. arXiv preprint arXiv:2405.12213.
7. [7] Z. Liu, Y. Gu, S. Zheng, Y. Fu, X. Xue, and Y. Jiang (2025) TriVLA: a triple-system-based unified vision-language-action model with episodic world modeling for general robot control. arXiv preprint arXiv:2507.01424.
8. [8] T. Z. Zhao, V. Kumar, S. Levine, and C. Finn (2023) Learning fine-grained bimanual manipulation with low-cost hardware. arXiv preprint arXiv:2304.13705.
9. [9] C. Chi, Z. Xu, S. Feng, E. Cousineau, Y. Du, B. Burchfiel, R. Tedrake, and S. Song (2025) Diffusion policy: visuomotor policy learning via action diffusion. The International Journal of Robotics Research 44 (10-11), pp. 1684–1704.
10. [10] A. Mandlekar, S. Nasiriany, B. Wen, I. Akinola, Y. Narang, L. Fan, Y. Zhu, and D. Fox (2023) MimicGen: a data generation system for scalable robot learning using human demonstrations. In Proceedings of the 7th Conference on Robot Learning, Proceedings of Machine Learning Research, Vol. 229, pp. 1820–1864.
11. [11] Z. Xue, S. Deng, Z. Chen, Y. Wang, Z. Yuan, and H. Xu (2025) DemoGen: synthetic demonstration generation for data-efficient visuomotor policy learning. arXiv preprint arXiv:2502.16932.
12. [12] A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, J. Hsu, et al. (2022) RT-1: robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06817.
13. [13] B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, et al. (2023) RT-2: vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pp. 2165–2183.
14. [14] NVIDIA, J. Bjorck, F. Castañeda, N. Cherniadev, X. Da, R. Ding, L. Fan, Y. Fang, D. Fox, et al. (2025) GR00T N1: an open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503.14734.
15. [15] Y. Ze, G. Zhang, K. Zhang, C. Hu, M. Wang, and H. Xu (2024) 3D diffusion policy: generalizable visuomotor policy learning via simple 3D representations. arXiv preprint arXiv:2403.03954.
16. [16] K. Pertsch, K. Stachowicz, B. Ichter, D. Driess, S. Nair, Q. Vuong, O. Mees, C. Finn, and S. Levine (2025) FAST: efficient action tokenization for vision-language-action models. arXiv preprint arXiv:2501.09747.
17. [17] Z. Liu, Y. Wang, K. Wang, L. Liang, X. Xue, and Y. Fu (2025) Spatial-temporal aware visuomotor diffusion policy learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 7122–7131.
18. [18] R. Bajcsy (1988) Active perception. Proceedings of the IEEE 76 (8), pp. 966–1005.
19. [19] S. Chen, Y. Li, and N. M. Kwok (2011) Active vision in robotic systems: a survey of recent developments. The International Journal of Robotics Research 30 (11), pp. 1343–1377.
20. [20] C. Connolly (1985) The determination of next best views. In Proceedings. 1985 IEEE international conference on robotics and automation, Vol. 2, pp. 432–435.
21. [21] A. Doumanoglou, R. Kouskouridas, S. Malassiotis, and T. Kim (2016) Recovering 6d object pose and predicting next-best-view in the crowd. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3583–3592.
22. [22] D. Morrison, P. Corke, and J. Leitner (2019) Multi-view picking: next-best-view reaching for improved grasping in clutter. In 2019 International Conference on Robotics and Automation (ICRA), pp. 8762–8768.
23. [23] A. Goyal, J. Xu, Y. Guo, V. Blukis, Y. Chao, and D. Fox (2023) RVT: robotic view transformer for 3D object manipulation. In Conference on Robot Learning, pp. 694–710.
24. [24] A. Goyal, V. Blukis, J. Xu, Y. Guo, Y. Chao, and D. Fox (2024) RVT-2: learning precise manipulation from few demonstrations. arXiv preprint arXiv:2406.08545.
25. [25] Z. Liu, Y. Gu, Y. Wang, X. Xue, and Y. Fu (2026) ActiveVLA: injecting active perception into vision-language-action models for precise 3D robotic manipulation. arXiv preprint arXiv:2601.08325.
26. [26] H. Fang, M. Grotz, W. Pumacay, Y. R. Wang, D. Fox, R. Krishna, and J. Duan (2025) SAM2Act: integrating visual foundation model with a memory architecture for robotic manipulation. arXiv preprint arXiv:2501.18564.
27. [27] Y. Shi, D. Wen, G. Chen, E. Welte, S. Liu, K. Peng, R. Stiefelhagen, and R. Rayyes (2025) VISO-Grasp: vision-language informed spatial object-centric 6-dof active view planning and grasping in clutter and invisibility. In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 14931–14938.
28. [28] J. Wu, X. Lin, B. He, C. Fermüller, and Y. Aloimonos (2025) ViewActive: active viewpoint optimization from a single image. In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 11812–11818.
29. [29] H. Xiong, X. Xu, J. Wu, Y. Hou, J. Bohg, and S. Song (2025) Vision in action: learning active perception from human demonstrations. arXiv preprint arXiv:2506.15666.
30. [30] X. Xu, J. Park, H. Zhang, E. Cousineau, A. Bhat, J. Barreiros, D. Wang, J. Bohg, and S. Song (2026) HoMMI: learning whole-body mobile manipulation from human demonstrations. arXiv preprint arXiv:2603.03243.
31. [31] Y. Zou, C. Shi, W. Yu, H. Xue, J. Lv, Y. Pan, C. Wen, and C. Lu (2026) ActiveGlasses: learning manipulation with active vision from ego-centric human demonstration. arXiv preprint arXiv:2604.08534.
32. [32] M. Zhang, T. Wu, J. Tan, Z. Liu, G. Wetzstein, and D. Lin (2025) GenDoP: auto-regressive camera trajectory generation as a director of photography. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 18229–18239.
33. [33] Z. Liu, C. Cao, Y. Zhang, X. Zuo, X. Xue, Y. Fu, T. Wang, and C. Guo (2026) OmniCam: omni-camera trajectory generation via geometry-grounded pose token learning. arXiv preprint arXiv:2610.09513.
34. [34] S. Ross, G. Gordon, and D. Bagnell (2011) A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp. 627–635.
35. [35] J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel (2017) Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 23–30.
36. [36] M. Laskin, K. Lee, A. Stooke, L. Pinto, P. Abbeel, and A. Srinivas (2020) Reinforcement learning with augmented data. Advances in neural information processing systems 33, pp. 19884–19895.
37. [37] N. Hansen and X. Wang (2021) Generalization in reinforcement learning by soft data augmentation. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 13611–13617.
38. [38] T. Yu, T. Xiao, A. Stone, J. Tompson, A. Brohan, S. Wang, J. Singh, C. Tan, J. Peralta, B. Ichter, et al. (2023) Scaling robot learning with semantically imagined experience. arXiv preprint arXiv:2302.11550.
39. [39] Z. Chen, S. Kiami, A. Gupta, and V. Kumar (2023) GenAug: retargeting behaviors to unseen situations via generative augmentation. arXiv preprint arXiv:2302.06671.
40. [40] K. Black, M. Nakamoto, P. Atreya, H. Walke, C. Finn, A. Kumar, and S. Levine (2023) Zero-shot robotic manipulation with pretrained image-editing diffusion models. arXiv preprint arXiv:2310.10639.
41. [41] B. Liu, Y. Zhu, C. Gao, Y. Feng, Q. Liu, Y. Zhu, and P. Stone (2023) LIBERO: benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems 36, pp. 44776–44791.
42. [42] T. Li, J. Zhang, S. Wang, Z. Wei, and X. Huang (2026) LIBERO-Occ: evaluating and improving vision-language-action models under scene-induced occlusion via viewpoint imagination. arXiv preprint arXiv:2606.10862.
43. [43] S. James, Z. Ma, D. R. Arrojo, and A. J. Davison (2020) RLBench: the robot learning benchmark & learning environment. IEEE Robotics and Automation Letters 5 (2), pp. 3019–3026.