PathLift: Depth-Conditioned Metric Lifting for Visual-Navigation Priors

Abstract

Generalist visual-navigation policies tell a robot which direction to go but not how far. A separate lifting step is needed at deployment to convert their unitless trajectory into a metric path that a controller can track; existing scalar lifters apply uniform scaling that misses per-waypoint corrections, while in-policy approaches that inject monocular depth into the policy inherit the prediction errors of the depth model near obstacles and at close range. We present PathLift, a per-waypoint depth-conditioned adapter that lifts a pre-trained visual-navigation policy by consuming an explicit, sensor-grade depth source (RGB-D, stereo, or projected LiDAR) and fitting per-waypoint magnitude and lateral residuals to a canonicalized prior. PathLift consistently improves closed-loop navigation across diverse visual-navigation policies in simulation and on a real wheeled robot under both stereo and LiDAR depth. By turning the policy’s unitless intent into a depth-feasible metric path in a single feed-forward pass, the adapter lifts real-world success without test-time optimization.

Keywords: Visual navigation, depth-conditioned prediction

1 Introduction

Visual-navigation policies such as GNM, ViNT, NoMaD, FlowNav, and NaviBridger [21, 22, 27, 4, 15] tell a robot which way to go but not how far. To enable cross-embodiment generalization, these models normalize their training trajectories by a robot-specific constant tied to top speed. As a result, they output a geometrically meaningful but unitless trajectory, which we refer to as the visual-navigation prior. Before a controller can track this prior at deployment time, a separate metric lifting step is required.

Three existing approaches illustrate the gap. Scalar lifting applies one scale uniformly across all waypoints, reducing the prior’s per-waypoint geometry to a single scale factor; it over- or under-shoots when the effective scale varies along the trajectory, and it misses the lateral corrections that depth observation makes possible. In-policy monocular depth [4, 3] injects a depth pathway into the policy itself, which couples the geometric pathway to a specific policy; when monocular depth is the source, the policy also inherits its prediction errors near obstacles and at close range, precisely where the lift must be precise. Test-time geometric trajectory optimization [14, 16, 37] expresses richer trajectories but requires a map and per-frame optimization, and consumes the prior only after a lift has placed it in metric space (Sec. 4.4).

PathLift (Fig. 1) is a per-waypoint depth-conditioned adapter that attaches to any pre-trained visual-navigation policy. It takes shape from a canonicalized prior and scale from an explicit, sensor-grade depth source projected into the policy’s camera frame, supervised at training by a structured admissible family of intent-preserving metric trajectories.

We summarize our contributions:

• Depth-feasible metric path in a single pass. PathLift’s training target is selected from an intent-preserving, depth-feasible subset of candidate lifts. The deployed adapter turns each prior sample into a depth-feasible metric path in a single forward pass.
• Prior-agnostic, sensor-agnostic adapter. A single checkpoint lifts both regression-based (GNM, ViNT) and generative (NoMaD, FlowNav) priors without retraining, and accepts stereo, RGB-D, or projected LiDAR depth through a single camera-frame interface.
• Low-latency adapter. PathLift adds 5 ms adapter overhead per inference, over 10\times faster than the in-policy lifting baseline (Tab. 1).

2 Related Work

Visual-navigation policies and foundation models.

The landscape of generalist navigation models is expanding rapidly, from goal-conditioned image-frame predictors [21, 22, 27, 4, 15, 23] to navigation foundation models trained on web-scale data [11, 36, 6] and world-model-based planners [2]. A common trait remains: their outputs are shape-correct but unitless trajectories in an embodiment-agnostic coordinate frame, requiring a separate lifting step at deployment regardless of how the policy is trained. Without geometric grounding, these policies collide frequently when deployed [5]. A complementary trend scales these models on web video by recovering action labels via deep visual SLAM [28, 29] or structure-from-motion [18, 10, 32], as in CityWalker [11]; the recovered trajectories expose only shape, with deployment scale supplied by a per-robot constant.

Lifting approaches.

The common recipe applies a hand-tuned v_{\max}/f scalar uniformly across waypoints; learned scalar lifters [12] replace it with a regressed single scale, still uniformly applied. A second line couples depth into the policy itself, either through monocular-depth features [4, 34] or through a dedicated depth-input encoder [3]. Both choices tie the depth pathway to a specific policy: monocular features inherit prediction errors near obstacles and at close range, and dedicated depth encoders require retraining when the sensor changes. PathLift instead consumes an explicit, sensor-grade depth source (stereo, RGB-D, or projected LiDAR) at the adapter interface, keeping the upstream policy untouched.

Test-time trajectory optimization.

Recent visual-navigation optimizers [37] solve a smooth-cost problem against a depth-derived traversability field per frame, following a formulation traceable to classical trajectory optimization [14, 16, 19]. These methods solve a single instance optimally given a map.

PathLift offers a distinct approach: an adapter that couples a pre-trained visual-navigation policy with feasible depth via a feed-forward mapping integrating the prior’s intent, depth-observed free space, and expert navigation behavior in a single pass. Test-time collision optimization trades the first of these signals against the second through soft costs; PathLift instead bounds deviation from the prior as a hard constraint, without optimizing an obstacle cost. This adapter design provides compatibility with diverse pre-trained policies and arbitrary depth sources without retraining.

3 Method

3.1 Problem formulation

We formulate metric lifting as structured-target regression. The visual-navigation prior is canonicalized to a unitless sequence of H waypoints \hat{\mathbf{p}}\in\mathbb{R}^{H\times 2} by origin-aligning, arc-length-resampling, and rescaling so that \|\hat{\mathbf{p}}_{H}\|_{2}=1, giving a common input format across policies. The prior’s raw scale follows training convention and is not aligned with the deployment scene, so metric scale must be recovered from depth.

Given \hat{\mathbf{p}}, a depth frame D, and the source-camera projection \pi (intrinsics and the camera-to-body transform T_{BC}), jointly \mathcal{O}=(D,\pi), the network predicts a body-frame trajectory y\in\mathbb{R}^{H\times 2} that tracks an expert path y^{\star} within the depth-observed free space. For each waypoint k\in\{1,\dots,H\}:

y_{k}\;=\;r_{k}\,\hat{\mathbf{p}}_{k}\;+\;\delta_{k}\,\mathbf{n}_{k}(\hat{\mathbf{p}}),

where r_{k}\in\mathbb{R}_{+} scales the prior magnitude at waypoint k and \delta_{k}\in\mathbb{R} applies a lateral residual along the per-waypoint unit normal \mathbf{n}_{k}(\hat{\mathbf{p}}).

The training data provides expert paths y^{\star} but no ground-truth lifts: an expert path is where the robot traveled, not a metric realization of the prior’s shape. Regressing onto y^{\star} directly would let the model discard the prior and leave no way to enforce depth feasibility. At training we therefore select the supervision target from the lifted candidate family \mathcal{C}_{\mathrm{lift}}(\hat{\mathbf{p}}), a finite set of candidates spanning radial scale, progress warp, rotation, and bounded lateral correction (ten sub-families, App. A), keeping the depth-feasible candidate closest to y^{\star}. \mathcal{C}_{\mathrm{lift}} enters only at training, to define the supervision target.

3.2 Supervision target within \mathcal{C}_{\mathrm{lift}}

To ensure the supervision target is physically viable, we restrict \mathcal{C}_{\mathrm{lift}}(\hat{\mathbf{p}}) to a depth-feasible subset \mathcal{C}_{\mathrm{adm}}(\hat{\mathbf{p}},\mathcal{O}) defined by four admissibility predicates:

\mathcal{C}_{\mathrm{adm}}(\hat{\mathbf{p}},\mathcal{O})=\bigl\{\,c\in\mathcal{C}_{\mathrm{lift}}(\hat{\mathbf{p}})\,:\,\mathrm{clr}(c)\!\geq\!\tau_{\mathrm{clr}},\;\mathrm{pen}(c)\!=\!0,\;\mathrm{obs}(c)\!\geq\!\tau_{\mathrm{obs}},\;\mathrm{tube}(c)\!\leq\!\tau_{\mathrm{tube}}\,\bigr\}.

These predicates filter candidates based on safety, visibility, and intent preservation:

• Safety: We enforce a minimum clearance \mathrm{clr}(c)\!=\!\min_{k}\Omega(c_{k}) and a zero penetration count \mathrm{pen}(c)\!=\!|\{k\!:\!\Omega(c_{k})\!<\!\tau_{\mathrm{pen}}\}|, where \Omega is the body-frame clearance field. We use \tau_{\mathrm{clr}}\!=\!0.30 m for a comfort margin and \tau_{\mathrm{pen}}\!=\!0.20 m as a catastrophic safety floor.
• Visibility: The trajectory must stay within the depth-observed region of the BEV. We require \mathrm{obs}(c)=\tfrac{1}{H}|\{k:c_{k}\in\mathcal{V}\}|\geq\tau_{\mathrm{obs}}, where \mathcal{V}\subset\mathbb{R}^{2} is the BEV projection of depth-observed free space. Waypoints outside \mathcal{V} enter regions where occupancy is unknown, so their feasibility cannot be certified. We set \tau_{\mathrm{obs}}=0.90.
• Intent Preservation: To prevent excessive deviation from the prior’s shape, we bound the average residual from its best-fit pure-radial lift, \mathrm{tube}(c)=\tfrac{1}{H}\sum_{k}\|c_{k}-\alpha_{c}\hat{p}_{k}\|\leq\tau_{\mathrm{tube}}, where \alpha_{c}=\langle c,\hat{p}\rangle/\|\hat{p}\|^{2} is the least-squares scalar fit aligning \hat{p} with c. Pure radial candidates (c=R\hat{p}) give zero residual. Rotation or lateral correction increases it. We use \tau_{\mathrm{tube}}=0.30 m, the tightest bound that costs little in supervision quality (App. B).

At training time, the admissible candidate closest to ground truth y^{\star} in average displacement error (ADE) is the supervision target,

c^{\star}\;=\;\argmin_{c\in\mathcal{C}_{\mathrm{adm}}(\hat{\mathbf{p}},\mathcal{O})}\,\mathrm{ADE}(c,y^{\star}).

Ground truth y^{\star} enters only at training, to select c^{\star} from \mathcal{C}_{\mathrm{adm}}. This supervision combines three signals in a single forward pass: the prior’s intent (via \mathcal{C}_{\mathrm{lift}}), the depth-observed free space (via \mathcal{C}_{\mathrm{adm}}), and expert navigation behavior (via the closest admissible candidate to y^{\star}). Fig. 3 illustrates this construction. The candidate family and admissibility predicates are shared across all training data regardless of source policy or camera model.

Scalar lifts are suboptimal.

Empirical analysis justifies our selection-based target construction over simple scalar lifting. In indoor environments (Habitat [17], OpenLORIS [25]), pure-radial candidates, representing the single-scalar regime, account for only 3.1–3.7\% of c^{\star} selections. The remaining >96\% of optimal lifts require progress-dependent scaling, rotation, or lateral corrections. Outdoor driving (CODa [35]) is an exception (27–34\%) due to predominantly straight paths, but the indoor results strongly motivate a per-waypoint output over a single global scalar. Per sub-family admissibility pass rates and the resulting subset composition are reported in App. A.

3.3 Architecture

The PathLift adapter is a feed-forward network f_{\theta}:(\hat{\mathbf{p}},\mathcal{O})\to y that consumes the canonicalized prior and an explicit depth observation, mapping them directly to a metric body-frame trajectory (Fig. 2).

Tokens.

We discretize the depth frame into 256 patch tokens of dimension 16. Each token combines three geometric signals: (i) seven depth statistics within the patch (the median, the p10/p25/p75/p90 quantiles, the variance, and the valid-pixel fraction), (ii) the six coordinates of a Plücker ray [26] for the patch’s central pixel in body frame, and (iii) the three coordinates of a body-frame anchor at the patch’s median-depth point. Prior tokens of dimension 12 encode each waypoint’s canonicalized position, progress fraction, and Plücker line. The unprojection \pi^{-1} supports pinhole, double-sphere fisheye [31], and equirectangular models; once tokenized, the network consumes them uniformly. Any metric depth source (RGB-D or LiDAR) is pre-aligned via re-projection through \pi into the policy’s camera frame, decoupling the adapter from the sensor.

Network.

A 4-layer scene encoder E_{\mathrm{scene}} and a 2-layer prior encoder E_{\mathrm{prior}} process the patch and prior tokens with self-attention. An anchor MLP maps the prior tokens to H anchor queries, which a cross-attention block primes with depth context. Six decoder blocks [24] iteratively refine the queries via alternating self- and cross-attention to E_{\mathrm{scene}} and E_{\mathrm{prior}}. Each refined query emits a magnitude r_{k} and a lateral residual \delta_{k}; the trajectory follows Eq. (1). The full network has 4.3 M parameters.

Training.

Training combines an MSE term to c^{\star} (Eq. (3)) with clearance, prior-coupling, smoothness, and deep-supervision losses [9]. A single adapter is trained on Habitat pinhole and fisheye renders, CODa projected LiDAR and OpenLORIS RGB-D, with priors drawn uniformly from GNM, ViNT and NoMaD (App. C).

4 Experiments and Results

We evaluate PathLift in closed-loop benchmarks in simulation and on a real wheeled robot, paired with diverse visual-navigation priors.

4.1 Simulation Experiments

4.1.1 Experimental Setup

Setup and Baselines.

We benchmark PathLift on the HM3D dataset [13] within the Habitat-sim environment [17]. The benchmark consists of 2,000 trials across 50 unseen scenes (20 trajectories per scene, 2 repetitions, mean length of 27.1 nodes). Navigation is executed in a closed-loop manner via waypoint tracking. We terminate a trial upon the first collision or if the robot fails to reach the goal within 500 steps. Detailed sensor and robot specifications are provided in App. D.1. We evaluate four state-of-the-art image-goal policies with several baselines:

• Visual-navigation policies: We include regression-based feed-forward models, GNM [21] and ViNT [22], alongside generative policies based on diffusion and flow-matching, NoMaD [27] and FlowNav [4]. All four policies output waypoint trajectories in normalized (unitless) form. All policies are fine-tuned on HM3D training scenes disjoint from the evaluation set (App. D.3).
• Adapters: We compare PathLift against MetricNet/MetricNav [12]. Since MetricNav injects guidance during the denoising steps, it is directly compatible only with generative priors. To isolate the impact of the lifting algorithm from the depth source, we report MetricNav∗, which builds the baseline’s TSDF map from the same ground-truth depth used by PathLift, leaving MetricNet unchanged. MetricNav is trained separately for each prior, whereas the same PathLift adapter attaches to all four priors without retraining (App. D.2).

Metrics.

We report Success Rate (SR), Success weighted by Path Length [1] (SPL), Topological-Map Reached (TR), and inference time. TR denotes the fraction of topological nodes traversed before trial termination, providing a finer-grained progress signal than the binary SR. Inference time is measured on an NVIDIA RTX 4090 for the prior and adapter combined.

4.1.2 Results and Analysis

Performance consistency across visual-navigation priors.

Table 1 summarizes the closed-loop benchmark results for the 14 combinations of base priors and adapters. PathLift improves all four base priors without retraining, recovering metric scale from their waypoint outputs, with relative SR gains of +34.8\% (ViNT) to +77.8\% (FlowNav) and an average TR gain of 9.19 pp. The improvement also generalizes to FlowNav, which was not used during PathLift training.

Comparison with MetricNet and MetricNav.

MetricNet provides a direct scalar-lifting baseline on the generative priors. PathLift improves SR over MetricNet by 0.85 pp on NoMaD (29.80\% vs. 28.95\%) and 1.40 pp on FlowNav (28.45\% vs. 27.05\%), while using roughly half the adapter runtime (5.3 vs. 10.7–10.8 ms).

Is geometric lifting alone enough to match an adapter that also selects among generated samples and steers them toward the goal? PathLift matches MetricNav∗ on NoMaD (SR 29.80\% vs. 29.20\%) and trails by 3.15 pp on FlowNav. MetricNav∗ operates inside the denoising loop, drawing multiple trajectory samples per step and scoring them against a goal-informed TSDF cost. PathLift instead lifts a single sample per inference, producing a depth-feasible path without sample selection or goal scoring; even on a sample not best aligned with the goal, the feasible lift yields safe progress that the closed loop can build on. PathLift’s adapter overhead is prior-independent and over an order of magnitude smaller (5 vs 76–104 ms; Tab. 1), and the same adapter attaches to both regression-based (GNM, ViNT) and generative (NoMaD, FlowNav) priors.

4.2 Real-Robot Deployment

4.2.1 Image-Goal Navigation

Setup and Baselines.

We deploy PathLift on an indoor wheeled robot with a forward-facing stereo RGB camera and a LiDAR. The depth pathway accepts either stereo depth (SD) from a real-time stereo model [33] or LiDAR rays re-projected into the camera frame. In each of three different indoor corridor environments (Fig. 4(a); two 20 m, one 10 m), the topological map is pre-built without obstacles, and obstacles are inserted before each trial to create layouts the prior cannot anticipate. A trial succeeds only when the robot terminates within 1 m of the final node; #Coll counts every contact event. Platform and inference details are in Appendix E. We evaluate three policies from Sec. 4.1 (GNM, ViNT, NoMaD) under three adapter conditions: the bare prior scaled by a fixed scale, MetricNet [12], and PathLift. MetricNet is the scalar-scaling front stage of the MetricNav pipeline (Sec. 4.1); for runtime we omit MetricNav’s downstream goal- and collision-cost refinement and apply MetricNet only to NoMaD. PathLift uses the same weights as Sec. 4.1, with no adaptation to the robot, the scene, or the priors at test time.

Real-world consistency across priors.

Across the three evaluated policies (Tab. 2), PathLift with SD achieves a mean 86.7\% SR at 0.82 collisions per trial, above the bare-prior baselines (48.9\% SR / 1.69 collisions). Success rates improve for every policy using the same PathLift adapter without any per-prior adaptation. In contrast, MetricNet on NoMaD improves SR by 20 pp, below PathLift’s 46.7 pp gain on the same prior, consistent with our claim that per-waypoint lifting captures scale variation a uniform scalar cannot.

Consistency across depth sources.

PathLift yields comparable performance regardless of the depth source (Fig. 5), producing comparable collision rates on ViNT (0.60 SD vs. 0.53 LiDAR) and NoMaD (0.87 SD vs. 0.73 LiDAR) while maintaining an aggregated SR within \sim\!7 pp between SD (86.7\%) and LiDAR (80.0\%, n{=}30 each). These results confirm that our depth interface is decoupled from the sensor hardware, unifying the two sources via camera-frame re-projection (App. E.1).

4.2.2 Goal-Free Exploration

Setup.

To probe PathLift’s contribution beyond image-goal navigation, we also evaluate goal-free exploration on the same robot. In the 12\!\times\!10 m indoor area of Fig. 4(b), a robot runs in its no-goal exploration mode from a fixed start pose; each trial ends at the first collision, and the protocol is repeated 10 times per method. We compare the bare NoMaD prior against NoMaD+PathLift (SD, same weights as Sec. 4.2.1).

Longer collision-free exploration.

PathLift extends the mean trajectory before first collision from 7.86 to 11.67 m (+48\%) and the maximum from 13.38 to 22.50 m (+68\%); mean and maximum durations rise from 29.79/47.57 s to 70.66/111.87 s, more than doubling the time before contact (Tab. 3).

4.3 Ablations

We ablate four PathLift design choices on the NoMaD prior over an 800-trial HM3D subset (bare NoMaD: 19.88\% SR; Tab. 4).

Does PathLift use depth meaningfully? Removing the depth input reduces SR to 6.50\%, below the bare-prior baseline. An open-loop sweep over the fraction of invalid depth pixels shows little change until the input becomes entirely invalid (App. F).

Does Plücker tokenization beat raw-depth CNN? Replacing the tokenizer with a CNN drops SR to 14.25\%. The two variants use the same data and 100k-step training schedule, and the gap favors explicit per-patch geometric features for metric lifting. An inaccurate scale estimate can also make the learned lift worse than fixed scaling.

Do lateral corrections help beyond per-waypoint radial scaling? A retrained radial-only variant predicts an independent radial scale for each waypoint but no lateral residual. It reaches 27.62\% SR, compared with 29.62\% for full PathLift, a 2.0 pp difference due to the lateral correction.

Is the admissible-family supervision needed? Training the same network directly on y^{\star}, without \mathcal{C}_{\mathrm{lift}}, reduces SR to 14.88\%, again below the fixed-scale baseline. The expert path y^{\star} is a demonstration trajectory and need not be a feasible metric realization of the current prior.

Combining all components yields the full 29.62\% SR. Additional details are provided in App. F.

4.4 Composition with Local Planners

We compare PathLift with DWA and MPPI under the same protocol as Sec. 4.3. Both planners use a local costmap constructed from the same depth frame provided to PathLift. MPPI reaches 34.12\% SR, compared with 29.62\% for PathLift, while DWA reaches 26.75\% (Tab. 5).

In our setup, DWA and MPPI require a metric local goal that the unitless prior cannot supply, so each runs downstream of a lift rather than replacing it. Using PathLift to provide this goal improves DWA from 26.75\% to 34.62\% SR and MPPI from 34.12\% to 38.00\% SR. PathLift\rightarrowMPPI achieves the highest SR and SPL (38.00\% and 34.57\%, respectively). PathLift\rightarrowDWA also exceeds MPPI alone in both SR and SPL while requiring less runtime.

5 Conclusion and Limitations

We presented PathLift, a depth-conditioned adapter for visual-navigation policies. The adapter decouples three concerns: the upstream policy (shape from a canonicalized prior), the depth interface (sensor-agnostic camera-frame projection of RGB-D, stereo, or LiDAR), and the lift method (per-waypoint magnitude and lateral residual supervised by a structured admissible family of intent-preserving trajectories). A single trained adapter therefore composes with any pre-trained policy.

Against bare priors, PathLift raises closed-loop SR by 35–78\% in simulation and 38 pp on a real wheeled robot under stereo depth, with similar performance using projected LiDAR. Against in-policy lifting trained per prior, our single adapter matches NoMaD, trails FlowNav slightly, and runs over 10\times faster. PathLift can also be used before a local planner, improving both DWA and MPPI (Sec. 4.4).

PathLift’s success is bounded by the base prior’s intent: when the prior persistently points at obstacles or away from the goal, the per-step lift cannot override a persistently misdirected prior. This is the cost of off-the-shelf composability; PathLift’s ceiling tracks the quality of available pre-trained policies. The depth field of view is matched to the prior’s training FOV, trading outside-view obstacle awareness for canonicalization consistency. Real-robot evaluation is restricted to three indoor corridors. The output parameterization (Eq. (1)) expresses rotation only implicitly through per-waypoint magnitude and lateral terms; an explicit rotational degree of freedom, multi-frame depth fusion, higher-speed regimes, outdoor deployment, and VLM coupling are natural next directions.

Acknowledgments

This work was supported by Hyundai Motor Company and Kia, and by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT) [NO.RS-2021-II211343, Artificial Intelligence Graduate School Program (Seoul National University)].

References

1. [1] P. Anderson, A. Chang, D. S. Chaplot, A. Dosovitskiy, S. Gupta, V. Koltun, J. Kosecka, J. Malik, R. Mottaghi, M. Savva, and A. R. Zamir (2018) On evaluation of embodied navigation agents. arXiv preprint arXiv:1807.06757.
2. [2] A. Bar, G. Zhou, D. Tran, T. Darrell, and Y. LeCun (2025) Navigation world models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15791–15801.
3. [3] W. Cai, J. Peng, Y. Yang, Y. Zhang, M. Wei, H. Wang, Y. Chen, T. Wang, and J. Pang (2026) NavDP: learning sim-to-real navigation diffusion policy with privileged information guidance. In 2026 IEEE International Conference on Robotics and Automation (ICRA), pp. 5233–5240.
4. [4] S. Gode, A. Nayak, D. N. Oliveira, M. Krawez, C. Schmid, and W. Burgard (2025) FlowNav: combining flow matching and depth priors for efficient navigation. In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 17762–17768.
5. [5] M. Guerrier, K. Soma, J. Pavlasek, and G. Beltrame (2026) Can vision foundation models navigate? zero-shot real-world evaluation and lessons learned. arXiv preprint arXiv:2603.25937.
6. [6] N. Hirose, C. Glossop, D. Shah, and S. Levine (2026) OmniVLA: an omni-modal vision-language-action model for robot navigation. In 2026 IEEE International Conference on Robotics and Automation (ICRA), pp. 12651–12658.
7. [7] N. Hirose, D. Shah, A. Sridhar, and S. Levine (2024) SACSoN: scalable autonomous control for social navigation. IEEE Robotics and Automation Letters 9 (1), pp. 49–56.
8. [8] H. Karnan, A. Nair, X. Xiao, G. Warnell, S. Pirk, A. Toshev, J. Hart, J. Biswas, and P. Stone (2022) Socially compliant navigation dataset (SCAND): a large-scale dataset of demonstrations for social navigation. IEEE Robotics and Automation Letters 7 (4), pp. 11807–11814.
9. [9] C. Lee, S. Xie, P. Gallagher, Z. Zhang, and Z. Tu (2015) Deeply-supervised nets. In International Conference on Artificial Intelligence and Statistics (AISTATS), pp. 562–570.
10. [10] V. Leroy, Y. Cabon, and J. Revaud (2024) Grounding image matching in 3D with MASt3R. In European Conference on Computer Vision (ECCV), pp. 71–91.
11. [11] X. Liu, J. Li, Y. Jiang, N. Sujay, Z. Yang, J. Zhang, J. Abanes, J. Zhang, and C. Feng (2025) CityWalker: learning embodied urban navigation from web-scale videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6875–6885.
12. [12] A. Nayak, D. O. Makowski, S. Gode, C. Schmid, and W. Burgard (2026) MetricNet: recovering metric scale in generative navigation policies. In 2026 IEEE International Conference on Robotics and Automation (ICRA), pp. 1863–1870.
13. [13] S. K. Ramakrishnan, A. Gokaslan, E. Wijmans, O. Maksymets, A. Clegg, J. M. Turner, E. Undersander, W. Galuba, A. Westbury, A. X. Chang, M. Savva, Y. Zhao, and D. Batra (2021) Habitat-Matterport 3D Dataset (HM3D): 1000 large-scale 3D environments for embodied AI. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), External Links: Link
14. [14] N. Ratliff, M. Zucker, J. A. Bagnell, and S. Srinivasa (2009) CHOMP: gradient optimization techniques for efficient motion planning. In 2009 IEEE International Conference on Robotics and Automation (ICRA), pp. 489–494.
15. [15] H. Ren, Y. Zeng, Z. Bi, Z. Wan, J. Huang, and H. Cheng (2025) Prior does matter: visual navigation via denoising diffusion bridge models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12100–12110.
16. [16] C. Rösmann, W. Feiten, T. Wösch, F. Hoffmann, and T. Bertram (2012) Trajectory modification considering dynamic constraints of autonomous robots. In ROBOTIK 2012; 7th German Conference on Robotics, pp. 1–6.
17. [17] M. Savva, A. Kadian, O. Maksymets, Y. Zhao, E. Wijmans, B. Jain, J. Straub, J. Liu, V. Koltun, J. Malik, D. Parikh, and D. Batra (2019) Habitat: a platform for embodied AI research. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9339–9347.
18. [18] J. L. Schönberger and J. Frahm (2016) Structure-from-motion revisited. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4104–4113.
19. [19] J. Schulman, Y. Duan, J. Ho, A. Lee, I. Awwal, H. Bradlow, J. Pan, S. Patil, K. Goldberg, and P. Abbeel (2014) Motion planning with sequential convex optimization and convex collision checking. The International Journal of Robotics Research 33 (9), pp. 1251–1270.
20. [20] D. Shah, B. Eysenbach, N. Rhinehart, and S. Levine (2022) Rapid exploration for open-world navigation with latent goal models. In Conference on Robot Learning, pp. 674–684.
21. [21] D. Shah, A. Sridhar, A. Bhorkar, N. Hirose, and S. Levine (2023) GNM: a general navigation model to drive any robot. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 7226–7233.
22. [22] D. Shah, A. Sridhar, N. Dashora, K. Stachowicz, K. Black, N. Hirose, and S. Levine (2023) ViNT: a foundation model for visual navigation. In Conference on Robot Learning, pp. 711–733.
23. [23] W. Shen, P. Gu, H. Qin, and Z. Meng (2025) EffoNAV: an effective foundation-model-based visual navigation approach in challenging environment. IEEE Robotics and Automation Letters 10 (7), pp. 6824–6831.
24. [24] S. Shi, L. Jiang, D. Dai, and B. Schiele (2022) Motion transformer with global intention localization and local movement refinement. Advances in Neural Information Processing Systems 35, pp. 6531–6543.
25. [25] X. Shi, D. Li, P. Zhao, Q. Tian, Y. Tian, Q. Long, C. Zhu, J. Song, F. Qiao, L. Song, et al. (2020) Are we ready for service robots? The OpenLORIS-Scene datasets for lifelong SLAM. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 3139–3145.
26. [26] V. Sitzmann, S. Rezchikov, B. Freeman, J. Tenenbaum, and F. Durand (2021) Light field networks: neural scene representations with single-evaluation rendering. Advances in Neural Information Processing Systems 34, pp. 19313–19325.
27. [27] A. Sridhar, D. Shah, C. Glossop, and S. Levine (2024) NoMaD: goal masked diffusion policies for navigation and exploration. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 63–70.
28. [28] Z. Teed and J. Deng (2021) DROID-SLAM: deep visual SLAM for monocular, stereo, and RGB-D cameras. Advances in Neural Information Processing Systems 34, pp. 16558–16569.
29. [29] Z. Teed, L. Lipson, and J. Deng (2023) Deep patch visual odometry. Advances in Neural Information Processing Systems 36, pp. 39033–39051.
30. [30] S. Triest, M. Sivaprakasam, S. J. Wang, W. Wang, A. M. Johnson, and S. Scherer (2022) TartanDrive: a large-scale dataset for learning off-road dynamics models. In 2022 International Conference on Robotics and Automation (ICRA), pp. 2546–2552.
31. [31] V. Usenko, N. Demmel, and D. Cremers (2018) The double sphere camera model. In 2018 International Conference on 3D Vision (3DV), pp. 552–560.
32. [32] J. Wang, M. Chen, N. Karaev, A. Vedaldi, C. Rupprecht, and D. Novotny (2025) VGGT: visual geometry grounded transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5294–5306.
33. [33] B. Wen, S. Dewan, and S. Birchfield (2026) Fast-FoundationStereo: real-time zero-shot stereo matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR),
34. [34] L. Yang, B. Kang, Z. Huang, Z. Zhao, X. Xu, J. Feng, and H. Zhao (2024) Depth Anything V2. Advances in Neural Information Processing Systems 37, pp. 21875–21911.
35. [35] A. Zhang, C. Eranki, C. Zhang, J. Park, R. Hong, P. Kalyani, L. Kalyanaraman, A. Gamare, A. Bagad, M. Esteva, and J. Biswas (2024) Toward robust robot 3-D perception in urban environments: The UT Campus Object Dataset. IEEE Transactions on Robotics 40, pp. 3322–3340.
36. [36] J. Zhang, A. Li, Y. Qi, M. Li, J. Liu, S. Wang, H. Liu, G. Zhou, Y. Wu, X. Li, et al. (2025) Embodied navigation foundation model. arXiv preprint arXiv:2509.12129.
37. [37] L. Zhang, Z. Jiang, and C. Chen (2026) SaferPath: hierarchical visual navigation with learned guidance and safety-constrained control. In 2026 IEEE International Conference on Robotics and Automation (ICRA), pp. 18259–18266.

Appendix A Candidate sub-families

Let \mathrm{arc}\in[0,1]^{H} denote per-waypoint cumulative arc fraction on \hat{\mathbf{p}} and \mathbf{n}(\hat{\mathbf{p}})\in\mathbb{R}^{H\times 2} the per-waypoint unit normals. Let \mathcal{R} be the radius grid \{0.40,0.49,\ldots,3.00\} (19 values mixing linear-dense near 1.0 with log-spaced extension to a per-source R_{\max}: 14.0 m (CODa [35] outdoor), 10.0 m (Habitat [17] indoor), and 6.0 m (OpenLORIS [25] indoor), calibrated to each domain’s path-length distribution). Tiers that compose a shape parameter with radius subsample \mathcal{R} to keep each sub-family’s count bounded.

Table 6 lists the ten sub-families, their shape-parameter spaces, and per-family candidate counts. The total is |\mathcal{C}_{\mathrm{lift}}|=679.

The admissibility filter (Eq. (2)) shapes the supervision target by reducing the 679-candidate family to a non-trivial depth-feasible subset. Standard sub-families pass 35–56\% of the time indoor (Tab. 7), demonstrating the filter’s selectivity, and rise to 82–95\% on outdoor CODa where admissibility constraints are largely unbinding. The same family and thresholds serve all four sources without per-source tuning.

Appendix B Admissibility threshold sensitivity

Recall from §3.2 that \mathrm{tube}(c)=\tfrac{1}{H}\sum_{k}\|c_{k}-\alpha_{c}\hat{p}_{k}\| with \alpha_{c}=\langle c,\hat{p}\rangle/\|\hat{p}\|^{2} measures the average residual after best-fit radial alignment of the prior with the candidate. The admissibility filter (Eq. 2) uses four predicate thresholds, but they play structurally different roles. The clearance (\tau_{\mathrm{clr}}=0.30 m) and penetration (\tau_{\mathrm{pen}}=0.20 m) thresholds derive from the robot’s collision tolerance, and the observation floor (\tau_{\mathrm{obs}}=0.90) is tied to the camera FOV. The intent-preservation tube (\tau_{\mathrm{tube}}=0.30 m) has no such physical anchor. It is a free parameter that trades prior fidelity against candidate breadth. We therefore focus the sensitivity analysis on \tau_{\mathrm{tube}}, sweeping it from 0.15 to 0.50 m around the paper value and reporting the c^{\star} selection change rate against the \tau_{\mathrm{tube}}=0.30 m reference together with \mathrm{ADE}(c^{\star},y^{\star}) (Tab. 8).

Relaxing \tau_{\mathrm{tube}} enlarges the admissible set, so \mathrm{ADE}(c^{\star},y^{\star}) can only decrease, as the table shows. For Habitat pinhole, increasing the threshold from 0.30 to 0.50 m improves ADE by only 3.5\%, while tightening it to 0.15 m increases ADE by 7.5\% and changes the selected target for 48.5\% of samples. We therefore use 0.30 m as a practical balance between preserving the prior and retaining a sufficiently broad candidate set. CODa is more sensitive to larger thresholds because its open outdoor scenes leave more candidates admissible.

Appendix C Architecture, loss weights, and training schedule

Lifting horizon.

The candidate family \mathcal{C}_{\mathrm{lift}} and the supervision target c^{\star} are generated at a fixed maximum horizon H{=}24, a 3\times upsampling of the native 8-waypoint cumsum output common to most visual-navigation policies (NoMaD, ViNT, GNM); this gives finer metric resolution along the path while keeping anchor-decoder computation linear in H. The network is exercised at variable horizons H_{\text{batch}}\!\in\!\{5,8,12,16,24\} per minibatch during training by truncating the prior, supervision target, and waypoint tokens together to H_{\text{batch}}; output length matches the truncated input. At deployment, PathLift can be queried at any H_{\text{deploy}}\!\leq\!24.

Network sizes.

224\!\times\!224 depth input, patch size 14. Hidden width d=128, 4 attention heads; total parameters 4.3 M. Transformer blocks use pre-LayerNorm with GELU and FFN expansion ratio 4; tokens carry no per-position positional encoding, since geometric position is encoded directly in the token features (Plücker ray for patches, canonical position for waypoints).

Loss weights.

Table 9 lists all training loss weights. The principal term is \mathcal{L}_{\mathrm{lift}}, the squared error to the supervision target; auxiliary terms enforce admissibility (clearance hinge), prior coupling (equivariance, tangent alignment), trajectory smoothness, and per-block deep supervision.

Optimization.

AdamW with weight decay 10^{-4}, cosine annealing on a learning rate of 4.24\!\times\!10^{-4} (=3\!\times\!10^{-4}\!\cdot\!\sqrt{2} for batch size 64). Training runs for 100k steps with batch size 64, without mixed precision. Horizons H_{\mathrm{batch}}\!\in\!\{5,8,12,16,24\} are sampled uniformly per minibatch by truncating \hat{\mathbf{p}}, the supervision target c^{\star}, and all auxiliary terms; this matches variable-horizon controller usage and regularizes against endpoint-specific decoder boundary effects. Training runs on a single NVIDIA RTX PRO 6000.

Appendix D Simulation Experiment Details

D.1 Robot, Sensors, and Control

The agent has a radius of 0.3 m, with camera mount height randomized between 0.25 m and 1.0 m per trial. RGB input is captured by a 640\!\times\!480 fisheye camera (\text{HFOV}=170^{\circ}), and depth input by a 224\!\times\!224 pinhole camera (\text{HFOV}=90^{\circ}). Depth values beyond a fixed range are masked as NaN to replicate the noise and range limits of a real RGB-D sensor. Control is performed by teleportation, where a low-level controller tracks the metric waypoint sequence output by the model at each inference step.

D.2 MetricNav with Ground-Truth Depth

MetricNav [12] recovers scale through MetricNet and constructs a TSDF map from DepthAnything [34] to provide collision-cost and goal-cost guidance during the denoising loop of the base prior. Its intended deployment therefore uses DepthAnything as the depth source. In our benchmark, however, PathLift consumes ground-truth depth, so a direct comparison would conflate adapter design with depth-source quality. For a fair comparison on the same input modality, we additionally report MetricNav∗, in which the TSDF is built from the same ground-truth depth used by PathLift; all other components of MetricNav (scalar prediction, sampling, cost computation, denoising-step guidance) are left untouched.

D.3 Finetuning Protocol

The official pretrained checkpoints of each prior are trained on real-world mobile-robot datasets (e.g., SACSoN [7], RECON [20], SCAND [8], TartanDrive [30]) whose embodiments, camera geometry, and scene statistics differ from indoor HM3D [13] scenes, causing domain shift. We collected approximately 20{,}000 trajectories from HM3D scenes disjoint from our benchmark set within the same Habitat-sim setup and converted them into ego-frame waypoints and RGB sequences to finetune all four priors under identical conditions. Preprocessing involves (a) aligning Habitat XZ-plane coordinates to the robot’s local XY-frame, (b) applying a sliding window with an 8-step prediction horizon, and (c) standardizing the normalized cumulative displacement per step. All priors were trained on a single NVIDIA RTX 4090 for 25 epochs with a context size of 3, a batch size of 256, and a learning rate of 5\!\times\!10^{-5}.

Appendix E Real-Robot Deployment Details

This appendix expands the real-robot deployment of Sec. 4.2: the platform specification (Tab. 10) and the depth-pathway pipeline.

E.1 Platform and Sensing

Depth pathway.

Both depth sources feed the same PathLift checkpoint without sensor-specific tuning. Stereo depth (SD) is produced by Fast-FoundationStereo [33] on each forward stereo frame at the controller rate. The LiDAR scan is re-projected through the camera projection \pi (Sec. 3.3) into the forward camera frame, so PathLift consumes a depth image with the same intrinsics and extrinsics regardless of which sensor produced it. This shared interface is what makes the two sources interchangeable in the results of Sec. 4.2.1.

Appendix F Ablation Details

Protocol.

All ablations remove one component at a time from the canonical PathLift configuration, keeping the rest fixed. We use the NoMaD prior on the 800-trial subset (20 HM3D validation scenes \times 20 trajectories \times 2 repetitions); the bare NoMaD baseline reaches 19.88\% SR on this pool (Tab. 11).

Depth input.

We ablate the depth signal by overwriting every pixel of the input depth map with -1.0 m, the value used to mark invalid pixels during training. The shape contract of the adapter is preserved, and the Plücker rays, camera intrinsics, prior trajectory, and scene encoder weights are unchanged. We use this in-distribution value rather than an arbitrary constant (e.g. 5 m), which would instead test an ill-defined failure mode where the model would confidently infer a flat wall everywhere. Under this protocol SR drops to 6.50\%, 13.38 pp below the bare-prior baseline (Tab. 11).

Partial-invalid sweep.

The depth ablation above invalidates the entire input. We additionally vary the fraction of invalid pixels in open loop (Tab. 12). ADE changes by less than 0.001 m up to 95\% invalid pixels and by 0.007 m at 99\%. A clear drop appears only when all pixels are invalid, where the prediction moves toward a pure radial lift. Even with 99\% invalid pixels, the remaining valid depth samples are sufficient to recover metric scale from the patch statistics.

Encoder.

The linear Plücker patch adapter has 2.3 k parameters. The CNN variant has 37 k parameters (\sim\!16\times more) and processes the same 224\!\times\!224 depth map without explicit ray or pose channels. The CNN reaches 14.25\% SR, compared with 29.62\% for the Plücker variant and 19.88\% for the bare prior (Tab. 11).

Output head.

We retrain a radial-only variant that keeps the per-waypoint magnitude head and removes the lateral residual head. It predicts an independent radial scale at each waypoint without lateral correction. This variant reaches 27.62\% SR, compared with 29.62\% for full PathLift (Tab. 11). The remaining 2.0 pp gap corresponds to the addition of lateral corrections in the full model.

Supervision.

Removing \mathcal{C}_{\mathrm{lift}} and training the same architecture directly on the arc-length-resampled expert path y^{\star} reduces SR to 14.88\%, below the fixed-scale baseline. The expert path is a demonstration and need not be a feasible metric realization of the current prior. The admissible family instead restricts the target to depth-feasible realizations of the prior, with y^{\star} used to select among them.