Benchmarking Interaction-Oriented Aerial Mobility across Heterogeneous Environments

Abstract

Aerial mobile manipulation requires unmanned aerial vehicles (UAVs) to traverse heterogeneous environments such as indoor and outdoor domains, and connect navigation with subsequent physical interaction within the same trajectories. Existing aerial benchmarks are typically confined to single-environment settings, limiting evaluation of agents across continuous environmental transitions. When interaction is evaluated, agents are often initialized near the target, leaving the transition from navigation to interaction largely unassessed. This work introduces CrossFly, a benchmark for long-horizon interaction-oriented aerial mobility across heterogeneous environments. CrossFly contains 15K trajectories across more than 20 reconstructed scenes under a unified simulation and evaluation framework, covering Indoor, Outdoor, and Indoor\leftrightarrowOutdoor trajectories. It further introduces Interaction Readiness Rate (IRR) to assess whether terminal navigation states provide suitable conditions for downstream interaction without assuming a specific manipulation policy. Experiments with five representative aerial vision-language-action (VLA) agents and the proposed CrossFly-Agent show that Indoor\leftrightarrowOutdoor settings are more challenging than single-environment settings, with the highest success rate reaching 9.7% on unseen scenes. A downstream manipulation case study further validates the relevance of interaction-oriented arrival to subsequent physical interaction. We hope CrossFly helps advance aerial embodied intelligence toward unified mobility and physical interaction in heterogeneous environments.

1 Introduction

Aerial mobile manipulation involves UAVs traversing three-dimensional spaces and interacting with objects within a single trajectory (Sun et al., 2026a; Sun et al., 2026b; Wang et al., 2026; Tucker et al., 2026; He et al., 2025). Reaching an interaction site may require flight through rooms, corridors, open spaces, and building boundaries under varying spatial constraints. Such trajectories may span heterogeneous environments (Luo et al., 2026), where flight must not only reach the target region but also terminate in a state suitable for interaction (Zhang et al., 2025a). These requirements motivate interaction-oriented aerial mobility as a core capability for extending goal-directed flight toward interaction.

Existing aerial benchmarks are typically confined to single-environment settings, where evaluation centers on instruction-conditioned or goal-directed flight and generalization to unseen scenes (Wang et al., 2025a; Gao et al., 2026; Liu et al., 2023; Lee et al., 2025; Liu et al., 2026). These protocols largely characterize whether agents reach designated goals, limiting evaluation of their behavior across long-horizon, continuous environmental transitions. When interaction is included, agents are often initialized near the target, providing little assessment of how they progress from extended flight through diverse environments to physical interaction (Sun et al., 2026a; Sun et al., 2026b; Wang et al., 2026).

Supporting this capability requires continuous trajectories that preserve environmental transitions and the approach to interaction sites (Wu et al., 2026a). Such trajectories provide supervision for learning under changing conditions within the same episode, while evaluation should assess whether terminal states support subsequent interaction without assuming a specific manipulation policy.

CrossFly instantiates these requirements as a large-scale benchmark for learning and evaluating interaction-oriented aerial mobility across heterogeneous environments. It contains 15K trajectories across more than 20 reconstructed scenes under a unified simulation framework, spanning diverse Indoor, Outdoor, and Indoor\leftrightarrowOutdoor task suites. Each trajectory records synchronized egocentric RGB observations, 6-DoF flight states, language instructions, waypoints, fine-grained subtasks, and interaction-related annotations throughout the trajectory. For evaluation, CrossFly introduces Interaction Readiness Rate (IRR) to assess whether terminal flight states provide suitable conditions for subsequent physical interaction, making it applicable to different manipulation strategies.

Experiments with five representative aerial agents and the proposed CrossFly-Agent reveal substantial gaps in interaction-oriented aerial mobility. Even the strongest agent reaches only a 9.7% success rate on unseen Indoor\leftrightarrowOutdoor trajectories, indicating difficulty in sustaining goal-directed flight across environmental transitions. IRR shows that goal reaching does not necessarily translate into interaction-oriented arrival, exposing limitations beyond standard navigation metrics. A downstream manipulation case study validates the relevance of interaction readiness to subsequent physical interaction. We hope CrossFly will serve as a standardized platform for evaluating and advancing aerial interaction-oriented mobility.

2 Related Work

Aerial Benchmarks. Language-guided aerial benchmarks cover diverse environments and task settings. AerialVLN (Liu et al., 2023) studies long-horizon UAV instruction following from egocentric observations, while AVDN (Fan et al., 2023) incorporates interactive dialogue. Subsequent benchmarks improve flight realism and expand real-world scene coverage (Wang et al., 2025b; Lee et al., 2025). OpenFly (Gao et al., 2026) integrates data generation, simulation, and agent evaluation across multiple engines, whereas IndoorUAV (Liu et al., 2026) focuses on continuous navigation in indoor environments. Beyond route following, UAV-Flow Colosseo (Wang et al., 2025a) evaluates fine-grained flight control, AIR-VLA (Sun et al., 2026a) couples aerial mobility with manipulation, and HUGE-Bench (Guo et al., 2026) introduces process- and safety-aware evaluation for multi-stage tasks. Ground-based last-mile navigation further targets manipulation-ready positioning (Zhang et al., 2025a; Lee et al., 2026). These benchmarks address complementary aspects of navigation and interaction, but separate indoor and outdoor evaluations do not characterize continuous execution across environmental boundaries. CrossFly brings these dimensions together by combining flight through connected indoor–outdoor environments with terminal interaction-readiness evaluation. Its single-domain and cross-domain suites support systematic comparison of navigation success and arrival quality across different environmental conditions and transitions.

Aerial Agents. Language-conditioned aerial agents span planning-oriented navigation methods and action-oriented VLA policies. Early navigation approaches predict waypoints or hierarchical trajectories from language and visual observations (Fan et al., 2023; Liu et al., 2023; Wang et al., 2025b). Foundation-model-based agents support more robust long-horizon decision making through semantic planning, memory, explicit reasoning, and directional grounding (Zhang et al., 2025b; Cai et al., 2025; Ning et al., 2026; Fan et al., 2026; Qi et al., 2026; Ding et al., 2026), while AerialVLA (Chen et al., 2026a) uses task progress and observation history to support proactive dialogue during flight. Action-oriented policies map multimodal observations more directly to flight commands: OpenFly-Agent (Gao et al., 2026) uses historical observations for action prediction, whereas AeroVLA (Xu et al., 2026) and AutoFly (Sun et al., 2026c) learn end-to-end continuous control from language instructions. Other work explores learning-free navigation, efficient onboard execution, unified indoor–outdoor training, and compressed long-horizon context (Hu et al., 2025; Chen et al., 2026b; Luo et al., 2026; Jiang et al., 2025). Recent aerial manipulation work couples navigation and manipulation over short spatial ranges, leveraging transfer from manipulation-pretrained VLA models (Tucker et al., 2026) and separate decoding of flight and manipulator actions (Sun et al., 2026b). Connecting longer-range navigation to such local interactions requires evaluating whether the terminal flight state supports the subsequent manipulation skill. CrossFly makes this connection explicit through interaction-readiness evaluation. We further introduce CrossFly-Agent as a hierarchical agent, in which a high-level VLM planner specifies the active subtask and a low-level VLA policy generates 6-DoF flight actions and predicts a manipulation-invocation signal.

3 CrossFly Benchmark

Evaluating interaction-oriented aerial mobility across heterogeneous environments requires unified, consistent definitions of the collected data, execution environment, and evaluation criteria. We first summarize the benchmark composition, including trajectory statistics and scene categories. Then, the dataset construction pipeline describes how scenes, trajectories, and subtask annotations are constructed (§3.2). Next, the simulation environment specifies the physics simulator, UAV platform, and action space used for closed-loop execution (§3.3). Finally, the evaluation protocol defines the task suites and metrics used to assess navigation and interaction readiness (§3.4).

3.1 Benchmark Stats

CrossFly contains 15K trajectories with 3M frames, totaling 160+ hours and 120 km of flight. As shown in Fig. 2, the dataset spans diverse indoor, outdoor, and cross-environment Indoor\leftrightarrowOutdoor tasks, where cross-environment trajectories account for the largest portion (51.2%), compared with Indoor (23.7%) and Outdoor (25.1%) trajectories. The instructions are highly compositional, consisting of 4, 6, or 8 subtasks, with 6-subtask instructions being the most common (46.5%), followed by 4-subtask (38.8%) and 8-subtask (14.8%) instructions. The trajectories further exhibit substantial variations in both temporal horizon and spatial extent: most trajectories span 100–400 frames, while trajectory distances are broadly distributed across different ranges, with the majority falling below 20 m and a small number extending to nearly 30 m. Such diversity in trajectory length and spatial scale introduces varying levels of long-horizon navigation and interaction complexity. Moreover, CrossFly covers a wide range of heterogeneous environments with substantially different layouts, scales, and visual appearances, providing diverse settings for evaluating embodied aerial agents.

As shown in Table{}_{\!} 1, CrossFly presents a more challenging setting compared with existing benchmarks. The agent needs to navigate across heterogeneous environments with different spatial constraints, while ensuring that the destinations are suitable for subsequent physical interactions.

3.2 Dataset Construction Pipeline

This pipeline is designed to construct large-scale, executable, and semantically annotated UAV trajectories in a unified framework across diverse heterogeneous environments, including indoor, outdoor, and indoor–outdoor domains. As shown in Fig. 3, it consists of three main stages. First, tasks are defined through objects, key waypoints, and language instructions in self-collected scenes. Next, the resulting task specifications are executed under realistic UAV dynamics to acquire synchronized egocentric observations and flight states, while infeasible rollouts are filtered out automatically. Finally, the waypoint structure and trajectory observations are used to derive fine-grained task annotations, providing structured supervision at both the trajectory and subtask level for effective learning.

Scene Collection. We reconstruct diverse heterogeneous environments from multi-view RGB images and point clouds captured by a handheld 3D scanner, including 3D Gaussian Splatting (3DGS) scenes and mesh representations (Kerbl et al., 2023). The collected scenes cover both individual indoor and outdoor spaces as well as connected regions that allow trajectories to transition between the two domains. These reconstructed environments provide the visual appearance and scene geometry used for subsequent trajectory construction and simulation. For each scene, we annotate target objects, spatial waypoints, and language instructions to define executable trajectories. Each trajectory is specified by four kinds of waypoints: start, intermediate, terminal, and turning points. By combining these annotations with different target objects and language instructions, each reconstructed scene can support diverse configurations with different routes, viewpoints, and interaction goals.

Waypoint Annotation. Within each reconstructed scene, we define a high-level trajectory by annotating target objects, language instructions, and a sparse set of key waypoints. The waypoints serve different roles along the path: the start waypoint specifies the initial UAV pose; intermediate waypoints guide UAV through important spatial regions along the intended route; the terminal waypoint specifies the final destination, typically near the target object. Together, these annotations define a coarse trajectory representation for subsequent trajectory refinement.

Trajectory Acquisition. Given the annotated waypoints, UAV is initialized at the start pose. Then the Ego-Planner (Zhou et al., 2022) generates a collision-free path between consecutive waypoints, transforming the sparse waypoints into an executable reference trajectory. A position-velocity PD tracking controller then converts the trajectory into acceleration commands executed by the low-level flight controller. The resulting motion is simulated under UAV dynamics, introducing realistic flight effects such as small oscillations, tracking deviations, response delays, and viewpoint perturbations.

During each rollout, we record synchronized egocentric RGB observations and 6-DoF UAV states. The complete state sequence provides a temporally aligned representation of how the agent progresses through the task and supports fine-grained evaluation of navigation and interaction behaviors. Rollouts that collide with the reconstructed scene geometry or fail to reach the designated waypoint are automatically rejected. This execution-and-filtering procedure ensures that retained task instances are physically feasible and can be completed under the simulator dynamics.

Task Decomposition. The waypoint structure provides a natural basis for partitioning each trajectory into semantically coherent subtasks. Each trajectory is partitioned into a sequence of subtasks, where each subtask corresponds to the trajectory segment between two adjacent waypoints. These waypoints include start, intermediate, and terminal waypoints. For each segment, we extract the egocentric observations at its start and end frames together with the global instruction \ell, and prompt a pretrained VLM (GPT-5.5) to generate a concise description of the segment-level intent, which serves as the ground-truth subtask for training high-level planning models after human verification.

3.3 Simulation Environment

Physical Simulator. CrossFly is built on NVIDIA Isaac Sim, which provides physics simulation, collision detection, and closed-loop UAV control. UAV dynamics are simulated with PhysX, and the reconstructed 3D Gaussian Splatting scenes serve as the visual environments for navigation. At each simulation step, egocentric observations are rendered according to the current UAV pose, while the predicted actions are executed under the simulator dynamics. The updated UAV state is subsequently used to generate the next observation, forming a closed-loop interaction cycle between the agent and the simulated environment. This setup enables closed-loop evaluation across indoor, outdoor, and heterogeneous environments with realistic visual appearance and physical motion.

UAV Platform. CrossFly supports diverse aerial embodiments through a unified control interface. Our main benchmark experiments are conducted on two mobile manipulation platforms: a quadrotor equipped with SO100-ARM (Cadene et al., 2024) and a hand-like UAV (Wu et al., 2026b).

Action Space. We adopt a 6-DoF action space for UAV navigation, consisting of position (x,y,z) and orientation (roll, pitch, and yaw). Motion commands predicted by the agent are executed through a low-level PD controller. For object-interaction tasks, agents can additionally issue an interaction command to invoke the corresponding manipulation policy after reaching the target.

3.4 Evaluation Protocol

Task Suites. While many existing benchmarks focus on either indoor or outdoor domains, CrossFly provides a cross-domain dataset with aerial interaction-oriented mobility scenarios spanning heterogeneous environments. It provides three fixed suites for evaluation: Indoor, Outdoor, and Indoor\leftrightarrowOutdoor. The first two measure navigation and interaction within a single environmental domain, and the last one evaluates long-horizon execution across environmental transitions.

• Indoor. Indoor tasks are executed inside buildings. Given a natural-language instruction, the agent must navigate through rooms, corridors, and doorways, approach the specified interaction location, and output a special token such as <Manip> when manipulation is required. The suite evaluates instruction following, object grounding, and target approach in constrained indoor spaces.
• Outdoor. Outdoor tasks follow the same formulation in courtyards, building surroundings, and other open areas. They emphasize long-range navigation, instruction grounding, and target localization under larger spatial scales and different visual and geometric conditions.
• Indoor\leftrightarrowOutdoor. These tasks require the agent to navigate across indoor and outdoor environments in either direction, starting in one environment and reaching a target located in the other. The agent must identify and approach an appropriate transition region, such as a doorway or building entrance, cross the environmental boundary, and continue toward the target. This suite evaluates robust navigation under substantial changes in appearance, illumination, and spatial constraints, while maintaining effective navigation performance across heterogeneous environments.

Interaction Readiness Rate. Alongside standard navigation metrics, we report the proposed Interaction Readiness Rate (IRR) to evaluate whether navigation terminates in a state suitable for subsequent physical interaction. Let \mathcal{R} denote the set of episodes in which UAV successfully reaches the target interaction region. We define the interaction-ready terminal state set as:

\mathcal{X}_{\mathrm{int}}=\left\{s\;\middle|\;d(s)\leq d_{\max},\;\psi(s)\in[\psi_{\min},\psi_{\max}]\right\},

where d(s) denotes the distance from the terminal UAV state to the interaction point, and \psi(s) denotes the relative yaw offset between UAV heading and the target-facing direction. IRR is then defined as:

\mathrm{IRR}=\begin{cases}\frac{1}{|\mathcal{R}|}\sum\nolimits_{i\in\mathcal{R}}\mathbb{I}\!\left[s_{i}^{T}\in\mathcal{X}_{\mathrm{int}}\right],&|\mathcal{R}|>0,\\[6.0pt]
0,&|\mathcal{R}|=0.\end{cases}

Here s_{i}^{T} denotes the terminal state of trajectory i and \mathbb{I}[\cdot] is the indicator function. We set d_{\max}=0.626\,\mathrm{m} and [\psi_{\min},\psi_{\max}]=[-50.5^{\circ},47.5^{\circ}]. These thresholds define the required spatial proximity and viewpoint alignment for downstream interaction, and are further analyzed in §4.4.

IRR therefore measures the fraction of target-reaching trajectories whose terminal states satisfy the defined interaction-ready conditions, thereby quantifying terminal-state suitability beyond navigation completion. Conditioning on successful target reaching isolates terminal-state quality from navigation success, making IRR complementary to standard navigation metrics such as SR.

4 Experiment

CrossFly is proposed to support long-horizon aerial interaction-oriented mobility across heterogeneous environments with diverse task settings. To demonstrate its broad applicability and evaluate the capabilities of existing methods, extensive experiments are conducted. First, the evaluation metrics and baseline methods are introduced in §4.1. Second, representative methods are benchmarked on seen scenes and unseen scenes in §4.2. Then, qualitative analyses of representative trajectories are presented in §4.3. Finally, we conduct diagnostic experiments to analyze the effects of environmental transitions and examine the connection between IRR and manipulation in §4.4.

4.1 Experimental Setup

Evaluation Metric. Following standardized protocols (Wang et al., 2025a; Liu et al., 2026), we use three standard metrics for evaluation: i) Success Rate (SR); ii) Navigation Error (NE); and iii) Oracle Success Rate (OSR). In addition, the proposed Interaction Readiness Rate (IRR) is reported to evaluate whether UAV reaches a state suitable for interaction after completing navigation tasks.

Baselines. We evaluate several representative open-source agents spanning different action-generation paradigms and training distributions. \pi_{0} (Black et al., 2024) is a Flow Matching-based foundation model pretrained on large-scale cross-embodiment robot data. \pi_{0}-FAST (Pertsch et al., 2025) uses FAST action tokenization for efficient autoregressive action generation. OpenFly-Agent (Gao et al., 2026) represents agents developed for outdoor aerial scenarios, OpenVLA-OFT (Kim et al., 2025) represents pretrained VLA agents adapted to downstream control tasks through optimized fine-tuning, and GR00T-N1.7 (Bjorck et al., 2025) represents generalist VLA agents developed for humanoid robot control. For a fair comparison, all baselines are retrained on CrossFly training datasets following unified training recipes. More details are provided in §C.

CrossFly-Agent. We construct CrossFly-Agent as a hierarchical agent for long-horizon interaction-oriented aerial mobility (Fig. 4), consisting of a high-level VLM planner and a low-level VLA policy. The planner predicts the subtask from the instruction, gripper state, and visual context comprising the initial frame, three uniformly sampled historical frames, and the current frame. The policy uses the initial frame, a frame from 3 s earlier, and the current frame. Conditioned on this visual context, the predicted subtask, global instruction, and gripper state, the policy adopts QwenPi (StarVLA Community, 2026) to generate action chunks together with a manipulation-invocation signal. The planner and policy are trained independently and coupled through the language subtask interface. The planner is fine-tuned on annotated subtasks, while the policy learns action chunks via flow matching, with supervision for manipulation invocation. The policy uses ground-truth subtasks during training and planner predictions at inference. See §A for more details.

4.2 Performance Benchmarking

Performance on Seen Scenes. Table 2 reports the performance of different methods on seen scenes. These results characterize the capability of existing state-of-the-art methods in environments included in the training distribution. CrossFly-Agent achieves the best performance across all metrics and task suites, indicating that explicit high-level task decomposition is useful for interaction-oriented aerial mobility tasks. Compared with existing methods without high-level planning (Black et al., 2024; Pertsch et al., 2025; Gao et al., 2026; Kim et al., 2025; Bjorck et al., 2025), CrossFly-Agent obtains consistent improvements across different task suites. For example, on the Indoor suite, it achieves SR of 23.5% compared to 17.7% from \pi_{0} (Black et al., 2024), while also improving IRR from 37.2% to 51.0%. These results show that CrossFly evaluates both target reaching and interaction readiness, revealing performance differences among aerial navigation agents.

Generalization on Unseen Scenes. As shown in Table 3, all agents experience performance degradation in unseen environments, while CrossFly-Agent maintains the best performance across all metrics and task suites. In particular, on the Outdoor task suite, CrossFly-Agent improves SR from 16.2% to 18.5%, and reduces NE from 4.22 to 3.83. These results suggest that hierarchical planning and subtask supervision in CrossFly-Agent provide useful guidance in unseen scenes.

4.3 Qualitative Results

Fig. 5 presents the qualitative results of CrossFly-Agent. The example illustrates a challenging Indoor\leftrightarrowOutdoor task, requiring the agent to adapt its navigation behavior when transitioning between indoor and outdoor environments. More visualizations are provided in §A.2. These qualitative results further demonstrate the effectiveness of the hierarchical design and subtask supervision in handling navigation tasks with varying scene constraints and planning horizons.

4.4 Diagnostic Experiments

Controlled Comparison across Task Suites. To investigate whether the performance degradation of Indoor\leftrightarrowOutdoor tasks is associated with environmental transitions rather than differences in other trajectory properties, we conduct a controlled comparison with matched trajectory statistics. Specifically, we construct matched subsets for the Indoor and Outdoor suites and the Indoor\leftrightarrowOutdoor suite by controlling for trajectory length, number of subtasks, and instruction length. We select 5 reconstructed scenes and sample 30 matched trajectories from each suite within each scene. This design minimizes the influence of variations in navigation horizon, scene gaps, and language instruction variations, enabling a more controlled comparison across the three task suites.

As shown in Table 4, all task suites contain comparable trajectories. Despite receiving similar training supervision, agents consistently achieve lower performance on the Indoor\leftrightarrowOutdoor task suite compared with single-domain tasks. These results suggest that the performance gap cannot be fully explained by differences in training exposure or trajectory properties. Instead, cross-domain navigation presents additional challenges that require agents to adapt to changes in visual appearance and spatial constraints when transitioning between different domains.

Manipulation Case Study. To examine whether IRR reflects the suitability of a navigation terminal state for subsequent manipulation, we train a reinforcement learning policy for grasping on a hand-like UAV platform, using the target object pose estimated from the depth map as input. The grasping policy is initialized from the terminal state produced by the navigation agent. As shown in Fig. 6, the top case satisfies the distance and target-view constraints defined by IRR, enabling the policy to successfully grasp the target object. In contrast, the bottom case violates IRR criteria, and the same policy fails to complete the grasp. These examples demonstrate that IRR can distinguish navigation terminal states that lead to different outcomes under the same downstream grasping policy.

Empirical Validation of IRR Criteria. To ground IRR criteria in downstream manipulation capability, we evaluate manipulation success across multiple aerial embodiments, each with its own manipulation policy. For each embodiment, we randomly sample terminal UAV states around fixed target objects and record grasp outcomes while varying the distance to the target and relative yaw. We then estimate the interaction readiness boundaries through maximum-likelihood fitting of the grasp outcomes, using candidate spatial and viewpoint partitions induced by the sampled terminal states. Aggregating the manipulation outcomes across all embodiments and their corresponding policies yields the spatial threshold \hat{d}_{\max}=0.626 m and the valid viewpoint interval [\hat{\psi}_{\min},\hat{\psi}_{\max}]=[-50.5^{\circ},47.5^{\circ}]. The resulting region provides an empirical approximation of the terminal configurations that support downstream grasping. Thus, IRR criteria are grounded in manipulation feasibility across multiple embodiments rather than manually specified thresholds.

5 Limitations and Impact

Limitations. Despite the effectiveness of CrossFly in providing a unified evaluation framework for interaction-oriented aerial mobility across heterogeneous environments, it still has several limitations that could be addressed in future work. First, CrossFly is currently developed in simulation environments. While the benchmark incorporates realistic rendering and physics-based UAV dynamics, the simulation-to-real gap caused by sensor noise, aerodynamic disturbances, and hardware constraints remains unexplored. Future work will investigate real-world deployment and transfer protocols. Second, CrossFly evaluates agents under feasible trajectories generated from successful demonstrations, while real-world long-horizon autonomy requires robust failure recovery and online adaptation. Future benchmarks could incorporate interactive learning, recovery behaviors, and continual adaptation to better evaluate embodied intelligence in open-world environments.

Social Impact. CrossFly advances the evaluation of interaction-oriented aerial mobility by assessing goal reaching and whether terminal states are suitable for interaction across heterogeneous environments. On the positive side, it may facilitate the development of aerial embodied agents. For potential negative impact, CrossFly is simulation-based and does not fully capture hardware constraints or failure recovery. Hence, it requires further safety validation (Feng et al., 2026).

6 Conclusion

In this work, we introduce CrossFly, a benchmark for interaction-oriented aerial mobility across heterogeneous environments. It provides 15K trajectories across 23 reconstructed scenes and evaluates both navigation performance and interaction readiness through a unified protocol. We propose the Interaction Readiness Rate to measure whether the terminal state satisfies the spatial and viewpoint requirements for interaction. Experiments with five aerial agents and CrossFly-Agent show that existing methods face challenges in indoor–outdoor transition tasks, highlighting the difficulty of connecting long-horizon navigation with physical interaction. We hope CrossFly facilitates the development of interaction-oriented aerial mobility by providing a standardized benchmark.

References

1. Anderson et al. (2018) P. Anderson, Q. Wu, D. Teney, J. Bruce, M. Johnson, N. Sünderhauf, I. Reid, S. Gould, and A. Van Den Hengel Vision-and-language navigation: interpreting visually-grounded navigation instructions in real environments. In CVPR,
2. Bjorck et al. (2025) J. Bjorck, F. Castañeda, N. Cherniadev, X. Da, R. Ding, L. Fan, Y. Fang, D. Fox, F. Hu, S. Huang, et al. GR00T N1: an open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503.14734.
3. Black et al. (2024) K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al. \pi_{0}: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164.
4. Cadene et al. (2024) R. Cadene, S. Alibert, A. Soare, Q. Gallouedec, A. Zouitine, S. Palma, P. Kooijmans, M. Aractingi, M. Shukor, D. Aubakirova, M. Russi, F. Capuano, C. Pascal, J. Choghari, K. Meftah, M. Ellerbach, J. Moss, and T. Wolf LeRobot: state-of-the-art machine learning for real-world robotics in PyTorch. Note: https://github.com/huggingface/lerobot
5. Cai et al. (2025) H. Cai, J. Dong, J. Tan, J. Deng, S. Li, Z. Gao, H. Wang, Z. Su, A. Sumalee, and R. Zhong FlightGPT: towards generalizable and interpretable UAV vision-and-language navigation with vision-language models. In EMNLP,
6. Chen et al. (2026a) J. Chen, H. Li, Z. Tang, X. Li, W. Wu, and S. Liu AerialVLA: a vision-language-action model for aerial navigation with online dialogue. In AAAI,
7. Chen et al. (2026b) Q. Chen, N. Gao, S. Huang, J. Low, T. Chen, J. Sun, and M. Schwager GRaD-Nav++: vision-language model enabled visual drone navigation with gaussian radiance fields and differentiable dynamics. IEEE Robotics and Automation Letters 11 (2), pp. 1418–1425.
8. Ding et al. (2026) X. Ding, J. Gao, C. Pan, W. Wang, and J. Qin History-enhanced two-stage transformer for aerial vision-and-language navigation. In AAAI,
9. Fan et al. (2026) C. Fan, C. Pan, Z. Liu, N. Liu, and J. Qin HTNav: a hybrid navigation framework with tiered structure for urban aerial vision-and-language navigation. In CVPR,
10. Fan et al. (2023) Y. Fan, W. Chen, T. Jiang, C. Zhou, Y. Zhang, and X. Wang Aerial vision-and-dialog navigation. In ACL,
11. Feng et al. (2026) T. Feng, Y. Zhang, S. Zhou, Z. Fan, W. Wang, X. Chi, T. Shen, X. Li, X. Bai, Z. Zhu, S. Han, B. Zhou, Z. Wang, S. Chen, Z. Zheng, C. Bai, J. Pang, Y. Yang, H. Zhao, D. Hu, L. Yi, Y. Huang, Q. Wu, H. Zhang, F. Gao, S. Zhang, H. Li, J. Hao, Y. Zhang, Y. Xia, J. Shen, C. Lu, H. Zhang, K. Chen, W. Yu, B. Chen, J. Han, and Y. Guo From bits to atoms: a survey of cross-layer safety in Embodied AI. AI Plus 1 (1).
12. Gao et al. (2026) Y. Gao, C. Li, Z. You, J. Liu, Z. Li, P. Chen, Q. Chen, Z. Tang, L. Wang, P. Yang, Y. Tang, Y. Tang, S. Liang, S. Zhu, Z. Xiong, Y. Su, X. Ye, J. Li, Y. Ding, D. Wang, Z. Wang, B. Zhao, and X. Li OpenFly: a comprehensive platform for aerial vision-language navigation. In ICLR,
13. Guo et al. (2026) J. Guo, Z. Chen, Z. Li, Z. Gao, J. Huang, H. Zhang, F. Huang, Y. Yao, T. Liu, and M. Gong HUGE-Bench: a benchmark for high-level UAV vision-language-action tasks. arXiv preprint arXiv:2603.19822.
14. He et al. (2025) G. He, X. Guo, L. Tang, Y. Zhang, M. Mousaei, J. Xu, J. Geng, S. Scherer, and G. Shi Flying hand: end-effector-centric framework for versatile aerial manipulation teleoperation and policy learning. In RSS,
15. Hu et al. (2025) C. Y. Hu, Y. Lin, Y. Lee, C. Su, J. Lee, S. Tsai, C. Lin, K. Chen, T. Ke, and Y. Liu See, point, fly: a learning-free VLM framework for universal unmanned aerial navigation. In CoRL,
16. Jiang et al. (2025) W. Jiang, L. Wang, K. Huang, W. Fan, J. Liu, S. Liu, H. Duan, B. Xu, and X. Ji LongFly: long-horizon UAV vision-and-language navigation with spatiotemporal context integration. arXiv preprint arXiv:2512.22010.
17. Kerbl et al. (2023) B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis 3D gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics 42 (4).
18. Kim et al. (2025) M. J. Kim, C. Finn, and P. Liang Fine-tuning vision-language-action models: optimizing speed and success. arXiv preprint arXiv:2502.19645.
19. Krantz et al. (2020) J. Krantz, E. Wijmans, A. Majumdar, D. Batra, and S. Lee Beyond the nav-graph: vision-and-language navigation in continuous environments. In ECCV,
20. Lee et al. (2025) J. Lee, T. Miyanishi, S. Kurita, K. Sakamoto, D. Azuma, Y. Matsuo, and N. Inoue CityNav: a large-scale dataset for real-world aerial navigation. In ICCV,
21. Lee et al. (2026) T. Lee, F. Mahmudova, and K. Desingh Learning category-level last-meter navigation from RGB demonstrations of a single-instance. IEEE Robotics and Automation Letters.
22. Liu et al. (2023) S. Liu, H. Zhang, Y. Qi, P. Wang, Y. Zhang, and Q. Wu AerialVLN: vision-and-language navigation for UAVs. In ICCV,
23. Liu et al. (2026) X. Liu, Y. Liu, H. Qiu, Y. Qirong, and Z. Lian IndoorUAV: benchmarking vision-language UAV navigation in continuous indoor environments. In AAAI,
24. Luo et al. (2026) Y. Luo, M. Zhu, X. Liu, H. Su, Y. Hu, Y. Liu, and T. Chen AirUniNav: unified vision-language navigation for UAVs in indoor and outdoor scenes. TechRxiv preprint.
25. Ning et al. (2026) Y. Ning, G. Zhao, Y. Qin, S. Liu, Y. Liu, L. Lin, and G. Li LookasideVLN: direction-aware aerial vision-and-language navigation. In CVPR,
26. Pertsch et al. (2025) K. Pertsch, K. Stachowicz, B. Ichter, D. Driess, S. Nair, Q. Vuong, O. Mees, C. Finn, and S. Levine FAST: efficient action tokenization for vision-language-action models. arXiv preprint arXiv:2501.09747.
27. Qi et al. (2026) Y. Qi, H. Li, S. Huang, T. Hui, Y. Wang, L. Cheng, Z. Zhong, S. Liu, and M. Wang Parse, search, and confirmation: training-free aerial vision-and-dialog navigation with chain-of-thought reasoning and structured spatial memory. In CVPR,
28. Qwen Team (2025) Qwen Team Qwen3 technical report. arXiv preprint arXiv:2505.09388.
29. Schmittle et al. (2018) M. Schmittle, A. Lukina, L. Vacek, J. Das, C. P. Buskirk, S. Rees, J. Sztipanovits, R. Grosu, and V. Kumar OpenUAV: a UAV testbed for the CPS and robotics community. In ICCPS,
30. StarVLA Community (2026) StarVLA Community StarVLA: a lego-like codebase for vision-language-action model developing. arXiv preprint arXiv:2604.05014.
31. Sun et al. (2026a) J. Sun, B. Tian, Q. Zhang, C. Li, Z. Song, Z. Cui, Y. Lv, and Y. Tian AIR-VLA: vision-language-action systems for aerial manipulation. arXiv preprint arXiv:2601.21602.
32. Sun et al. (2026b) J. Sun, B. Tian, Q. Zhang, Z. Liu, Y. Wang, Z. Cui, B. Li, Y. Lv, and Y. Tian AIR-VLA+: decoupling movement and manipulation via cascaded dual-action decoders with asymmetric MoE for aerial robots. arXiv preprint arXiv:2606.12859.
33. Sun et al. (2026c) X. Sun, W. Si, W. Ni, Y. Li, D. Wu, F. Xie, R. Guan, H. Xu, H. Ding, Y. Wu, Y. Yue, Y. Huang, and H. Xiong AutoFly: vision-language-action model for UAV autonomous navigation in the wild. In ICLR,
34. Tucker et al. (2026) J. Tucker, D. Liu, A. Swann, A. Ren, J. Yu, J. Sun, B. Kim, L. McGranahan, Q. Vuong, and M. Schwager \pi, but make it fly: physics-guided transfer of VLA models to aerial manipulation. In CoRL,
35. Wang et al. (2025a) X. Wang, D. Yang, Y. Liao, W. Zheng, W. Wu, B. Dai, H. Li, and S. Liu UAV-Flow Colosseo: a real-world benchmark for flying-on-a-word UAV imitation learning. In NeurIPS,
36. Wang et al. (2025b) X. Wang, D. Yang, Z. Wang, H. Kwan, J. Chen, W. Wu, H. Li, Y. Liao, and S. Liu Towards realistic UAV vision-language navigation: platform, benchmark, and methodology. In ICLR,
37. Wang et al. (2026) Y. Wang, D. Lee, X. Guo, Y. Zhan, Y. Jiang, B. Saravanan, M. Cao, J. Xie, C. Mao, S. Scherer, J. Geng, and G. Shi AM-Bench: a modular simulation suite and benchmark for aerial manipulation policy learning. In CoRL,
38. Wu et al. (2026a) J. Wu, Y. Hu, Z. Han, P. Chang, Y. Dong, B. Rabiei, and M. Ghaffari NavVerse: benchmarking indoor-to-outdoor embodied navigation in continuous robot simulation. arXiv preprint arXiv:2607.19695.
39. Wu et al. (2026b) Y. Wu, F. Yang, R. Jin, Y. Zhong, J. Wang, X. Wu, and F. Gao Hand-like autonomous flying robot for airborne grasping and interaction. Nature Communications 17 (1).
40. Xu et al. (2026) P. Xu, Z. Deng, J. Deng, Z. Gu, and S. Wan AeroVLA: a vision-language-action model for UAV navigation via minimalist end-to-end control. In ECCV,
41. Zhang et al. (2025a) P. Zhang, X. Gao, Y. Wu, K. Liu, D. Wang, Z. Wang, B. Zhao, Y. Ding, and X. Li MoMa-Kitchen: a 100k+ benchmark for affordance-grounded last-mile navigation in mobile manipulation. In ICCV,
42. Zhang et al. (2025b) W. Zhang, C. Gao, S. Yu, R. Peng, B. Zhao, Q. Zhang, J. Cui, X. Chen, and Y. Li CityNavAgent: aerial vision-and-language navigation with hierarchical semantic planning and global memory. In ACL,
43. Zhou et al. (2022) X. Zhou, X. Wen, Z. Wang, Y. Gao, H. Li, Q. Wang, T. Yang, H. Lu, Y. Cao, C. Xu, and F. Gao Swarm of micro flying robots in the wild. Science Robotics 7 (66), pp. eabm5954.

In the appendix, we provide the following items that shed deeper insight into our contributions:

• §A: Additional details on the CrossFly-Agent.
• §B: Supplementary information of benchmark splits.
• §C: Implementation details.
• §D: Discussion.

Appendix A Additional details on the CrossFly-Agent

A.1 Method

Overview. We introduce CrossFly-Agent, a hierarchical agent for aerial mobile manipulation. A high-level VLM planner predicts language subtasks using visual observations sampled across the trajectory. Conditioned on the predicted subtask, a low-level VLA policy uses the initial and recent observations to generate flight action chunks and determine when to invoke manipulation.

High-Level Planning. Let \ell denote the global instruction, o_{t} the current RGB observation, and g_{t} the gripper state. The high-level VLM planner uses Qwen3-VL-4B-Instruct [Qwen Team, 2025] to predict the active language subtask. Its visual context consists of the initial observation o_{0}, three historical frames \mathcal{R}^{\mathrm{plan}}_{t} uniformly sampled between the initial and current observations, and the current observation o_{t}:

z_{t}=P_{\theta}(\ell,g_{t},o_{0},\mathcal{R}^{\mathrm{plan}}_{t},o_{t}).

This five-frame context captures task progress over the observed trajectory.

Low-Level Action Generation. The low-level VLA policy adopts QwenPi [StarVLA Community, 2026], with its vision-language backbone initialized from Qwen3-VL-4B-Instruct. Conditioned on the instruction, predicted subtask, and gripper state, it uses the initial observation, an observation from \Delta=3 s earlier, and the current observation to generate a flight action chunk:

\mathbf{A}_{t}=(a_{t},\ldots,a_{t+H-1})\sim\pi_{\phi}(\,\cdot\mid\ell,z_{t},g_{t},o_{0},o_{t-\Delta},o_{t}),

where H is the chunk length and each flight action specifies a 6-DoF command. The policy also determines when to invoke downstream manipulation, ending the current flight segment and transferring control for grasping or placing.

Training. The planner and policy are trained independently. The planner is fine-tuned on annotated subtasks using a language-modeling loss. The policy learns action chunks through conditional flow matching. Let c_{t}=(\ell,z_{t},g_{t},o_{0},o_{t-\Delta},o_{t}) denote the policy inputs. Given a demonstrated action chunk \mathbf{A}_{t}, noise \epsilon\sim\mathcal{N}(0,I), and flow time \tau, we construct \mathbf{A}_{t}^{\tau}=(1-\tau)\mathbf{A}_{t}+\tau\epsilon and minimize

\mathcal{L}_{\mathrm{FM}}=\mathbb{E}_{\mathbf{A}_{t},c_{t},\epsilon,\tau}\left[\left\|v_{\phi}(\mathbf{A}_{t}^{\tau},\tau\mid c_{t})-(\epsilon-\mathbf{A}_{t})\right\|_{2}^{2}\right],

where v_{\phi} predicts the conditional velocity field. The policy additionally receives supervision for manipulation invocation. It uses ground-truth subtasks during training and planner predictions at inference. Each component uses the same visual sampling strategy during training and inference.

A.2 More Qualitative Results

A.3 Failure Case

Appendix B Supplementary Information of Benchmark Splits

B.1 Scene-Level Split

CrossFly contains 23 independently reconstructed scenes and 15,263 trajectories in total. We construct the benchmark split at the scene level, rather than randomly splitting individual trajectories. Let \mathcal{S} denote the complete set of reconstructed scenes. As shown in Table 5, we divide it into a training-scene set \mathcal{S}_{\mathrm{train}} and an unseen-scene set \mathcal{S}_{\mathrm{unseen}}:

\mathcal{S}=\mathcal{S}_{\mathrm{train}}\cup\mathcal{S}_{\mathrm{unseen}},\qquad\mathcal{S}_{\mathrm{train}}\cap\mathcal{S}_{\mathrm{unseen}}=\varnothing,

where |\mathcal{S}_{\mathrm{train}}|=18 and |\mathcal{S}_{\mathrm{unseen}}|=5, with 18+5=23. The split is fixed and shared by all evaluated methods.

Scene assignment is atomic: all trajectories associated with the same reconstructed scene are assigned to the same scene partition, regardless of whether they belong to the Indoor, Outdoor, or Indoor\leftrightarrowOutdoor task suite. In particular, trajectories from different regions or task suites of the same reconstructed scene are never separated between the training and unseen partitions.

The partition is therefore strictly isolated at the scene level. All trajectories generated from scenes in \mathcal{S}_{\mathrm{unseen}}, together with their RGB observations, 6-DoF UAV states, waypoint annotations, language instructions, and subtask annotations, are excluded from model training. Consequently, no trajectory or observation from an unseen scene is exposed to an agent during training.

B.2 Seen- and Unseen-Scene Evaluation

We distinguish between seen-scene and unseen-scene evaluation according to whether the underlying reconstructed environment is represented in the training set.

Seen-Scene Evaluation. The seen split consists of held-out evaluation trajectories collected from scenes in \mathcal{S}_{\mathrm{train}}. Although the underlying scenes are represented during training, the evaluation trajectories themselves are excluded from optimization. Denoting the training and seen evaluation trajectory sets by \mathcal{D}_{\mathrm{train}} and \mathcal{D}_{\mathrm{seen}}, respectively, we enforce

\mathcal{D}_{\mathrm{train}}\cap\mathcal{D}_{\mathrm{seen}}=\varnothing.

Therefore, seen-scene evaluation measures generalization to new episodes, routes, and task configurations within environments observed during training, rather than performance on trajectories used for optimization.

Unseen-Scene Evaluation. The unseen split consists exclusively of trajectories collected from \mathcal{S}_{\mathrm{unseen}}. Let \mathcal{D}_{\mathrm{unseen}} denote the corresponding evaluation trajectory set. By construction,

\operatorname{scene}(\mathcal{D}_{\mathrm{train}})\cap\operatorname{scene}(\mathcal{D}_{\mathrm{unseen}})=\varnothing.

Thus, unseen-scene evaluation introduces both trajectory-level and scene-level separation from the training data. Agents are evaluated in reconstructed environments whose visual observations, scene geometry, and spatial layouts are not encountered during training. This setting measures generalization to novel environments rather than to new trajectories within familiar scenes.

Appendix C Implementation Details

For reproducibility, we provide the detailed training configuration used for all VLA models. Unless otherwise specified, all VLA models follow the same training protocol, as summarized in Table 6.

We apply this training protocol consistently to all VLA-based models in our experiments.

Appendix D Discussion

Aerial mobile manipulation naturally couples two stages operating at different spatial and control scales: long-range mobility and local physical interaction. While mobility requires UAV to follow instructions, traverse potentially heterogeneous environments, and approach a target, manipulation is typically executed after the platform reaches a suitable pose. The key challenge is the handoff from mobility to manipulation. CrossFly makes this handoff explicit through interaction-oriented mobility and the Interaction Readiness Rate (IRR), which evaluates whether the terminal navigation state satisfies the spatial and viewpoint conditions required for downstream interaction. Once such a state is reached, an interaction-invocation signal can trigger a task-specific manipulation policy.

This formulation provides a modular pathway for extending aerial VLA agents toward mobile manipulation without requiring a single policy to jointly learn long-horizon flight and local interaction. Existing aerial VLA agents can retain their role in instruction following and target approach, while downstream policies handle grasping or other local skills. Our manipulation experiments provide initial evidence for this interface: under the same grasping policy, interaction-ready states can support successful manipulation, whereas unsuitable arrival states may lead to failure. More broadly, this modular view enables mobility and manipulation to be developed independently while remaining connected within a unified embodied execution pipeline.