NavHarness: Adaptive Goals for Agentic Vision-Language Navigation

How can a vision-language navigation agent maintain instruction-consistent, long-horizon execution while controlling the growth of accumulated multimodal interaction history?

Existing VLN agents can select locally plausible actions but often fail to ensure those actions remain consistent with the intended route over long horizons, allowing deviations to compound. At the same time, accumulating visual and textual history increases input size and inference cost, while simple truncation can discard landmarks, corrections, or unfinished objectives needed for recovery.

Prior methods include task-specific trained navigation policies and foundation models such as NaVid, NaVILA, StreamVLN, JanusVLN, NavFoM, AwareVLN, OmniNav, and Qwen-RobotNav, as well as general agent workflows like Codex CLI, AgenticNav, HarnessVLN, and SmartWay. These approaches either rely on large-scale trajectory training or on general-purpose reasoning, but they do not tie local action execution to verified progress, and they typically manage context by retaining or truncating history without progress-aware compression.

NavHarness uses four cooperating agents: a Goal Agent that creates adaptive local goals grounded in the instruction, current observation, and history; a Verify Agent that checks after each step whether the goal's observable completion condition has been met using goal-specific verification questions; a Visuomotor Agent that executes movements toward the active goal; and a Memory Agent that compresses multimodal history only at verified goal-completion boundaries, summarizing completed goals into route progress, landmarks, spatial relations, verification outcomes, and keyframes while retaining active-goal evidence and unfinished state. It operates zero-shot with monocular RGB input.

On R2R-CE, NavHarness with GPT-6-Astra achieves 79.0% SR vs 74.0% for Codex CLI (+5.0 pts), with SPL 66.6 vs 62.5 and NE 3.27 m vs 4.49 m. On RxR-CE it achieves 82.5% SR vs 76.3% (+6.2 pts), SPL 64.7 vs 60.0, NE 2.42 m vs 3.14 m. With GPT-5.6-Sol, SR improves by 4.4 pts on R2R-CE and 6.0 pts on RxR-CE, and RxR NE drops from 9.03 m to 7.28 m. NavHarness with GPT-6-Astra exceeds Qwen-RobotNav by 6.9 SR points on R2R-CE and 6.0 on RxR-CE with monocular zero-shot input. Compression reduces total episode tokens compared with NavHarness-off: by 35.94% on R2R-CE and 56.59% on RxR-CE for GPT-5.6-Sol, and by 5.52% and 4.05% for GPT-6-Astra; text per decision falls 55.5-71.6%, and selected decision-input traces show local reductions of 27.7-48.1%. Compressed SR remains above Codex CLI native in all four backbone-benchmark settings, though with SR penalties of 1-2.5 points and an RxR-CE SPL decline of 6.4 points for GPT-6-Astra. In real-world tests on a Unitree Go2 over eight routes and 24 trials, NavHarness attains 83.3% SR and 1.51 m NE vs JanusVLN's 50.0% SR and 2.34 m NE.

The authors state that using remotely served closed-source models causes substantial inference cost and communication latency, slowing the navigation loop, and that a single RGB camera provides limited visual coverage with obstacles outside the field of view potentially going unobserved. The real-world evaluation covers only eight routes and 24 trials on one robot. Compression can reduce route efficiency, especially on RxR-CE with GPT-6-Astra where SPL falls from 64.7 to 58.3. Evaluation on simulation is limited to 100 episodes per benchmark, and no code release is described.