1Zhejiang University
1
Although Large Vision-Language Models (VLMs) have significantly advanced embodied navigation, their direct deployment remains challenging, as existing methods often force VLMs into unnatural action spaces that misalign with their 2D pre-training priors, compounded by rigid reasoning schedules and inefficient memory management. To overcome these limitations, we propose Embodied-Navigator, a unified framework for efficient embodied navigation.
First, we introduce a Pixel-to-3D Action Formulation (Point) that, for the first time, treats pixel pointing as a natural bridge for smoothly transferring visual-grounding capabilities acquired during VLM pre-training to VLN. Specifically, the VLM selects 2D pixels, which are then projected into 3D coordinates for a low-level SLAM controller. Second, we propose an integrated Selective Reasoning and Anchor-Trajectory Memory mechanism (Think and Memorize), which dynamically triggers Chain-of-Thought and retains high-fidelity memory only at critical nodes, compressing redundant trajectories into lightweight Space-Time Indicators, thereby preserving critical historical information and enhancing spatio-temporal perception.
Finally, we design an efficient Two-Level Reinforcement Learning Alignment Paradigm (Align) based on Group Relative Policy Optimization (GRPO). By superimposing global outcome rewards with fine-grained process rewards, this dense supervision tightly aligns the agent's cognitive planning with physical environmental feedback, endowing the model with adaptive reasoning capabilities. Experiments demonstrate that Embodied-Navigator achieves state-of-the-art performance (e.g., 66.2% SR on R2R-CE) with high runtime and sample efficiency (requiring only 90k training trajectories).
One-sentence contribution. Embodied-Navigator aligns where a VLM points, when it thinks, what it remembers, and how reinforcement learning credits its decisions.
Note on system architecture. Embodied-Navigator is a complete navigation system. While the VLM observes RGB only, the full pipeline uses depth for pixel-to-3D projection and odometry for Space-Time Indicator encoding. The policy is trained on MultiNav-CoT, a 90k-trajectory dataset with Gemini 2.5 Flash–generated Chain-of-Thought annotations. The Two-Level GRPO rewards rely on simulator signals (geodesic distance, oracle success, SPL) that may not transfer directly to unstructured real-world data without adaptation.
2
Four aligned components connect high-level visual reasoning to efficient long-horizon execution.
P
Treat pixel pointing as a bridge from VLM pre-training to VLN: select one of four RGB views and a 2D waypoint, then use depth only afterward for 3D projection and SLAM execution.
First to smoothly transfer pretrained grounding into VLN through pixel pointing.
T
Trigger explicit Chain-of-Thought at difficult topological nodes instead of reasoning densely or at fixed intervals.
Allocates computation to consequential decisions.
M
Retain critical states as visual-reasoning anchors and encode routine motion with lightweight Space-Time Indicators.
Preserves topology without storing every frame.
A
Optimize the policy through reinforcement learning with Two-Level GRPO, superimposing local action and global trajectory advantages across multi-branch rollouts.
Assigns credit from immediate decisions to final navigation success.
Local process feedback
Target approach, collision avoidance, stop correctness, reasoning value, and output validity provide dense feedback at each state.
Global outcome feedback
Task success, path efficiency, and reasoning density evaluate complete trajectories and discourage unnecessary computation.
Over 800 training steps, trajectory-level rewards alone converge at a success reward of 0.59. Superimposing local step advantages lifts this to 0.64, and adding annealed guided sampling reaches 0.68 with faster convergence.
3
Embodied-Navigator reports the strongest results among the compared RGB-based navigators on both validation-unseen benchmarks.
Setup. Qwen2.5-VL-7B base model, SFT cold start on MultiNav-CoT (90k trajectories with Gemini 2.5 Flash–generated Chain-of-Thought annotations) followed by Two-Level GRPO, a 4,096-token context window, and NVIDIA A800 GPUs for training, evaluation, and deployment. Metrics are Navigation Error (NE), Oracle Success (OS), Success Rate (SR), Success weighted by Path Length (SPL), and normalized Dynamic Time Warping (nDTW). Sensor modalities: The VLM observes RGB only; the full system uses depth for pixel-to-3D projection and odometry for STI encoding. Compared methods vary in their use of panoramic views, depth input to the VLM, and odometry; Embodied-Navigator uses four 90° RGB views (360° coverage), depth for post-prediction projection, and odometry for memory encoding.
Table 1. R2R-CE val-unseen. NE in meters (lower is better); OS, SR, SPL in percent (higher is better). All listed methods observe RGB only, without odometry or depth input.
| Method | NE ↓ | OS ↑ | SR ↑ | SPL ↑ |
|---|---|---|---|---|
| StreamVLN | 4.98 | 64.2 | 56.9 | 51.9 |
| NavFoM | 4.61 | 72.1 | 61.7 | 55.3 |
| DualVLN | 4.05 | 70.7 | 64.3 | 58.5 |
| Embodied-Navigator (SFT only) | 4.88 | 62.0 | 55.7 | 50.3 |
| Embodied-Navigator | 3.85 | 74.5 | 66.2 | 58.8 |
Table 2. RxR-CE val-unseen. NE in meters (lower is better); SR, SPL, nDTW in percent (higher is better). All listed methods observe RGB only, without odometry or depth input.
| Method | NE ↓ | SR ↑ | SPL ↑ | nDTW ↑ |
|---|---|---|---|---|
| StreamVLN | 6.22 | 52.9 | 46.0 | 61.9 |
| NavFoM | 4.74 | 64.4 | 56.2 | 65.8 |
| DualVLN | 4.58 | 61.4 | 51.8 | 70.0 |
| Embodied-Navigator (SFT only) | 6.10 | 52.4 | 46.2 | 62.1 |
| Embodied-Navigator | 4.32 | 65.7 | 56.9 | 72.4 |
+10.5 pp
R2R-CE success improves from the SFT-only initialization at 55.7% to 66.2% with the full framework, isolating the contribution of Two-Level GRPO alignment. The combined system integrates Pixel-to-3D projection, selective reasoning, Anchor-Trajectory Memory, and the MultiNav-CoT dataset; matched-budget component ablations are reported in Section 4 below.
Values are reported in the manuscript. Depth supports post-prediction projection but is not an input to the VLM.
4
Three complementary views show where cognition is spent, whether memory survives long trajectories, and how hierarchical RL improves optimization.
(a) Think Reasoning allocation
(b) Memorize Long-horizon retention
(c) Align RL learning dynamics
Selective reasoning
Table 3. Reasoning trigger strategies on R2R-CE val-unseen.
| Trigger | CoT ratio | SR ↑ |
|---|---|---|
| Dense CoT | 100.0% | 66.8% |
| Fixed interval (1/3) | 36.2% | 60.1% |
| Adaptive (ours) | 26.3% | 66.2% |
Panel (a) provides the spatial evidence: the share of reasoning assigned to straight corridors falls from 38% after SFT to 11% after RL alignment.
Long-horizon memory
Table 4. Baselines and memory variants on the long-horizon subset: 5,927 R2R-CE and RxR-CE val-unseen trajectories whose expert path exceeds 12.5 meters, over 50 atomic forward actions.
| Model | Long-horizon SR ↑ |
|---|---|
| StreamVLN | 30.9 |
| DualVLN | 41.9 |
| Embodied-Navigator (uniform sampling) | 40.5 |
| Embodied-Navigator (full history) | 42.4 |
| Embodied-Navigator (w/o STI) | 45.6 |
| Embodied-Navigator (AT-Mem) | 49.8 |
Panel (b) shows that the advantage persists across trajectory-length bins. Anchor-Trajectory Memory avoids the information loss of uniform sampling and the attention dilution of full history; removing Space-Time Indicators alone costs 4.2 points in aggregate SR.
Controlled attribution
Every comparison below uses the same Qwen2.5-VL-7B policy, sensing inputs, validation-unseen splits, fixed non-learned SLAM controller, and evaluation protocol. Each block changes only the named component, so differences are conditional marginal contributions rather than additive gains.
Table 5. Matched-budget component ablations. The Action block is SFT-only and uses the same 90k trajectories, approximately 700k interactions, and one training epoch; all other blocks evaluate the corresponding trained navigation policies.
Swipe horizontally to inspect all metrics →
| Block | Variant | R2R-CE val-unseen | RxR-CE val-unseen | ||||||
|---|---|---|---|---|---|---|---|---|---|
| NE ↓ | OS ↑ | SR ↑ | SPL ↑ | NE ↓ | SR ↑ | SPL ↑ | nDTW ↑ | ||
| Action SFT only |
NavFoM-style metric waypoint | 7.18 | 42.0 | 30.9 | 24.7 | 8.47 | 27.2 | 20.6 | 43.1 |
| Pixel-to-3D | 4.88 | 62.0 | 55.7 | 50.3 | 6.10 | 52.4 | 46.2 | 62.1 | |
| GRPO | Global trajectory advantage only | 4.52 | 67.0 | 59.3 | 52.1 | 5.30 | 58.4 | 49.7 | 65.5 |
| Two-Level without annealed guidance | 4.12 | 71.5 | 63.4 | 56.0 | 4.72 | 62.6 | 53.8 | 69.4 | |
| Full Two-Level GRPO | 3.85 | 74.5 | 66.2 | 58.8 | 4.32 | 65.7 | 56.9 | 72.4 | |
| Memory | Uniform sampling | 4.43 | 68.2 | 60.8 | 53.6 | 5.03 | 59.4 | 50.8 | 66.8 |
| Full history | 4.29 | 69.4 | 61.9 | 54.7 | 4.87 | 61.1 | 51.8 | 68.0 | |
| AT-Mem without STI | 4.08 | 71.4 | 63.6 | 56.1 | 4.59 | 62.4 | 52.7 | 69.8 | |
| Full AT-Mem | 3.85 | 74.5 | 66.2 | 58.8 | 4.32 | 65.7 | 56.9 | 72.4 | |
| Reasoning | Dense CoT | 3.79 | 75.1 | 66.8 | 59.3 | 4.25 | 66.1 | 57.4 | 72.8 |
| Fixed interval (1/3) | 4.46 | 67.6 | 60.1 | 52.8 | 5.10 | 58.9 | 50.0 | 65.9 | |
| Adaptive (ours) | 3.85 | 74.5 | 66.2 | 58.8 | 4.32 | 65.7 | 56.9 | 72.4 | |
Pixel-to-3D improves SFT-only SR by 24.8/25.2 points on R2R-CE/RxR-CE. Local advantages add 4.1/4.2 points over global-only GRPO, and annealed guidance adds another 2.8/3.1 points.
Table 6. MultiNav-CoT supervision study. All variants use the same 90k trajectories, action labels, Qwen2.5-VL-7B student, and one-epoch SFT budget without GRPO; only the CoT supervision source changes.
Swipe horizontally to inspect all metrics →
| SFT supervision | R2R-CE val-unseen | RxR-CE val-unseen | ||||||
|---|---|---|---|---|---|---|---|---|
| NE ↓ | OS ↑ | SR ↑ | SPL ↑ | NE ↓ | SR ↑ | SPL ↑ | nDTW ↑ | |
| Action only (no CoT) | 5.31 | 57.0 | 50.6 | 45.0 | 6.71 | 46.1 | 39.8 | 56.2 |
| Qwen2.5-VL-7B CoT | 5.46 | 55.7 | 49.4 | 43.1 | 6.83 | 44.8 | 38.2 | 54.9 |
| GLM-4.5V-108B CoT | 4.99 | 60.8 | 54.4 | 48.9 | 6.24 | 51.2 | 44.9 | 60.8 |
| Gemini 2.5 Flash CoT | 4.88 | 62.0 | 55.7 | 50.3 | 6.10 | 52.4 | 46.2 | 62.1 |
Weak CoT supervision falls below action-only SFT, while the open-weight GLM-4.5V-108B teacher approaches Gemini 2.5 Flash on both benchmarks.
763k trajectories for DualVLN; 3.37M interactions for NavFoM
approximately 30 for StreamVLN and DualVLN
37.47 s StreamVLN; 41.46 s DualVLN
closed-loop re-evaluation corrects single-step geometric error
5
Pixel waypoints and sparse reasoning transfer from continuous simulation to an unmapped real-world environment.
Representative trials from the 100-episode real-world evaluation on the Unitree Go2 quadruped, executed zero-shot in unmapped environments. Success examples are shown first; a representative failure case is shown last.
6
arXiv
Code
Hugging Face
BibTeX
@inproceedings{feng2026embodiednavigator,
title = {Embodied-Navigator: Point, Think, Memorize, and Align
for Efficient Embodied Navigation},
author = {Feng, Hongyan and Chen, Sunlai and Liu, Xuanyu and Pan, Miao and
Xie, Yangfan and Cui, Yuxiang and Zhou, Zhongxiang and
Xiong, Rong and Zhang, Wenqi and Yin, Jianwei and
Zhuang, Yueting and Zhang, Xuhong},
year = {2026}
}Open research
This repository contains the navigation policy, the navigation-adapted VLM, SFT and Two-Level GRPO training code, the continuous-navigation environment and evaluation harness, the MultiNav-CoT processing pipeline with a data subset, and the real-robot serving interfaces.
Key dependencies: The method is trained on MultiNav-CoT, a 90k-trajectory dataset with Gemini 2.5 Flash–generated Chain-of-Thought annotations. The Two-Level GRPO rewards use simulator signals (geodesic distance, oracle success, SPL) that require adaptation for real-world training. The full system uses depth for pixel-to-3D projection and odometry for Space-Time Indicator encoding, though the VLM observes RGB only.
The released Embodied-Navigator-7B-GRPO checkpoint is available on Hugging Face. The full licensed simulation assets, complete training corpus, additional deployment videos, manuscript, and paper source are distributed separately and are not part of this repository.