GroundAnything: Reconciling Parallel Decoding with Precise Visual Grounding at Flash Speed

1]XPeng Inc. 2]Peking University 3]The University of Hong Kong 4]University of California, Berkeley 5]Princeton University 6]National University of Singapore 7]HKUST (GZ) \contribution[*]Equal contribution \contribution[‡]Project lead \contribution[†]Corresponding authors \projecthttps://groundingpi.github.io/groundanything/ \codehttps://github.com/groundingpi/GroundAnything \modelhttps://huggingface.co/GroundingPI/GroundAnything https://huggingface.co/GroundingPI/GroundAnything-VLM \titleteaserfigures/teaser.pdf

Abstract

Autoregressive (AR) grounding models serialize spatial predictions, introducing sequential latency and imposing a causal order on output tokens. We view grounding as visual evidence extraction: objects, locations, and spatial relations are jointly constrained by the image and query, yet their dependencies do not imply an intrinsic left-to-right generation order. This distinction makes bidirectional diffusion a natural fit, allowing spatial hypotheses to emerge in parallel and be jointly refined through iterative denoising. We introduce GroundAnything, a 4B-parameter grounding foundation model that reconciles fast parallel decoding with precise localization through blockwise denoising. Training combines grounding pretraining from public datasets and dedicated data engines, direct AR-to-diffusion conversion with joint AR and diffusion objectives, supervised fine-tuning, and GRPO-based reinforcement post-training. Across 30 grounding benchmarks, our autoregressive variant, GroundAnything-VLM, establishes a new overall state of the art among similarly sized models at 72.42%, remaining competitive with GPT-6 Astra (71.35%). With entropy-guided decoding, GroundAnything also surpasses the prior state of the art at this scale, averaging 61.75% versus 53.32% for the fast MTP-based LocateAnything model. We further explore decoding strategies, showing that an optional self-speculative mode achieves a 4.51\times speedup over the AR counterpart with a 0.74 percentage-point drop in COCO F1mIoU. Infrastructure experiments show that progressive inference optimizations translate parallel decoding into practical speedups. These support efficient visual grounding in latency-sensitive real-world systems.

Abstract

\abstractlist

Contents

Appendix Contents

1 Introduction

Visual grounding turns visual inputs into structured spatial predictions across object detection, referring expression comprehension, OCR, GUI interaction, and embodied perception. Next-token prediction has become a mainstream paradigm for generative grounding, with models such as Pix2Seq, Florence-2, and Rex-Omni expressing visual predictions as structured token sequences (Jiang et al., 2026; Chen et al., 2022; Xiao et al., 2024). However, autoregressive (AR) decoding serializes labels and coordinates, introducing sequential latency and restricting output interactions to the generated prefix(Chen et al., 2022; Cheng et al., 2024). These limitations are particularly costly in dense scenes and complex layouts.

To reduce AR decoding latency, LocateAnything uses multi-token prediction (MTP) for parallel box decoding (Wang et al., 2026). Diffusion VLMs support parallel generation and bidirectional refinement, with advances in multimodal understanding (You et al., 2026b; Wu et al., 2026a), GUI grounding (Kumbhar et al., 2026), and document OCR (Dong et al., 2026). Yet these grounding applications remain domain-specific, leaving diffusion’s broader potential for unified structured prediction across diverse visual tasks largely untapped.

To unlock this potential and reconcile fast parallel decoding with broad and precise visual grounding, we start from a familiar observation: we may notice objects or read separate signs in different orders, yet reach the same judgment about what is present and where. We therefore view grounding as structured visual evidence extraction: spatial variables are jointly constrained by the image and query, without an intrinsic left-to-right generation order. Bidirectional masked diffusion naturally supports this process, allowing hypotheses to emerge in parallel and be refined using shared visual evidence and partially recovered structure ().

We introduce GroundAnything, a 4B-parameter grounding foundation model combining broad perceptual capabilities with blockwise parallel decoding. A shared vocabulary with quantized coordinates unifies diverse grounding tasks. Its decoder combines bidirectional attention within blocks with causal dependencies across blocks, enabling parallel prediction and KV-cache reuse (Arriola et al., 2025).

Grounding pretraining draws on public datasets and dedicated data engines. We build GroundAnything-VLM and GroundAnything from a shared pretrained checkpoint, applying AR-to-diffusion conversion with joint AR and diffusion objectives to the latter. Both variants then undergo supervised fine-tuning and GRPO-based reinforcement post-training.

Across 30 grounding benchmarks, GroundAnything-VLM establishes a new overall state of the art among similarly sized models at 72.42%, remaining competitive with GPT-6 Astra (71.35%). With entropy-guided decoding, GroundAnything is, to our knowledge, the first diffusion language model to surpass the prior overall AR state of the art at this scale, averaging 61.75% at over 2\times the speed of GroundAnything-VLM. We use this mode throughout the main benchmarks to evaluate pure diffusion decoding without AR verification. Additional ablations examine key model and training design choices.

We further conduct a systematic speed analysis of decoding strategies and their trade-offs, including comparisons with MTP-based generation (Wang et al., 2026). An optional self-speculative mode combines diffusion drafting with AR verification and improves both speed and accuracy over entropy-guided decoding on COCO, achieving a 4.51\times speedup over GroundAnything-VLM with only a 0.74 pp F1mIoU drop. Systematic infrastructure experiments with SGLang, CUDA Graph, and FP8 demonstrate further practical speedups through progressive inference optimizations, supporting efficient visual grounding in real-world applications.

In summary, our major contributions are as follows:

• Unlocking diffusion for broad visual grounding. We introduce GroundAnything, a 4B foundation model that unifies diverse grounding tasks with precise localization and parallel decoding, trained through grounding pretraining, AR-to-diffusion conversion with joint objectives, supervised fine-tuning, and GRPO-based reinforcement learning.
• State-of-the-art grounding at 4B scale. Across 30 benchmarks, GroundAnything surpasses the prior overall state of the art among similarly sized AR models and outperforms the larger Qwen3.7-Max. GroundAnything-VLM establishes a new overall state of the art at this scale and remains competitive with GPT-6 Astra.
• Systematic acceleration studies. We analyze decoding strategies in detail, compare with MTP-based generation, and evaluate progressive infrastructure optimizations for practical deployment.

2 Related Work

2.1 Visual Grounding

Object detectors such as YOLO establish efficient localization within predefined category vocabularies (Redmon et al., 2016). Grounded vision–language pretraining extends detection to language-conditioned, open-set localization (Li et al., 2022; Liu et al., 2024). Generative approaches formulate detection as autoregressive next-token prediction (Chen et al., 2022) and integrate spatial grounding into multimodal language modeling (Peng et al., 2023). Structured generation unifies diverse perception tasks (Xiao et al., 2024), while quantized coordinates, large-scale grounding data, and reinforcement post-training strengthen localization (Jiang et al., 2026). However, AR generation serializes spatial predictions, increasing decoding costs for dense outputs. MTP-based parallel box decoding reduces this overhead by treating boxes and points as atomic generation units (Wang et al., 2026). This progression motivates exploring alternative parallel generation mechanisms for broad and precise grounding.

2.2 Diffusion Vision-Language Models

Masked diffusion offers a complementary route to parallel generation through bidirectional conditioning and iterative token recovery. Diffusion VLMs demonstrate its potential for visual instruction following and multimodal understanding (You et al., 2026b; Li et al., 2026b). Direct AR-to-diffusion conversion adapts pretrained VLMs for efficient blockwise generation (Wu et al., 2026a). Beyond general understanding, specialized applications include GUI grounding with geometry-aware masking (Kumbhar et al., 2026) and structured document OCR with blockwise denoising (Dong et al., 2026). These studies demonstrate the promise of diffusion in both general multimodal understanding and specialized visual tasks. Nevertheless, a unified diffusion grounding foundation that preserves precise localization across heterogeneous tasks remains underexplored. GroundAnything addresses this gap through grounding-focused training and a shared structured output interface, combining broad perceptual capabilities with efficient parallel decoding.

2.3 Parallel Decoding

Prior work explores blockwise generation (Arriola et al., 2025), adaptive parallel sampling (Wu et al., 2026b; Ben-Hamu et al., 2026), and diffusion-based speculative decoding (Liu et al., 2026; Wu et al., 2026a). Further work emphasizes practical acceleration through optimized inference infrastructure (Liu et al., 2025). Motivated by these advances, we examine decoding choices for GroundAnything in detail, studying their speed–accuracy trade-offs and extending the analysis to comparisons with MTP-based generation and progressive infrastructure optimizations.

3 Method

3.1 Problem Formulation: Grounding as Structured Visual Evidence Extraction

Given an image I and a task query Q, grounding extracts semantic identifiers and their spatial evidence, represented by boxes, points, or ordered point sequences. We serialize this evidence as Y=(y_{1},\ldots,y_{L}) using a shared vocabulary of text, structural markers, and quantized coordinates (Jiang et al., 2026); input/output conventions are detailed in Section 6.1.

These variables are jointly constrained by the image and query, but need not be recovered in a fixed left-to-right order (). We therefore use blockwise masked diffusion (Arriola et al., 2025; Wu et al., 2026a; You et al., 2026a; Yu et al., 2026), modeling

p_{\theta}(Y\mid I,Q)=\prod_{k=1}^{K}p_{\theta}\!\left(Y^{(k)}\mid I,Q,Y^{(<k)}\right)

over K consecutive response blocks. Bidirectional attention supports parallel token recovery within each block, while completed blocks form a causal prefix. Semantic ordering, including trajectory time order, is preserved.

3.2 Model Architecture

GroundAnything is a diffusion grounding model obtained by directly converting GroundAnything-VLM, an autoregressive grounding model trained through multimodal and spatial pretraining, supervised fine-tuning, and reinforcement post-training. It combines a MoonViT-V2 (Kimi K3) visual encoder (Team et al., 2026), a projector with 2\times 2 spatial aggregation and a two-layer MLP, and a Qwen3-4B language decoder (Yang et al., 2025) (Figure 1). Its variable-length output interface uses a shared vocabulary for semantic labels, protocol markers, and 1,000 quantized coordinate tokens (<0>–<999>); absent queried categories receive a None payload. The conversion introduces a mask token and jointly trains causal prediction and blockwise denoising with B=32; both modes share the decoder and vocabulary head, enabling parallel grounding and optional self-speculative decoding (Section 3.4.2).

3.3 GroundAnything Data

Training data combines public datasets with annotations produced by our data engine.

Our data engine (Figure 2) fuses multi-teacher annotations at the field level and applies task-specific validation. Accepted labels train a unified grounding expert for iterative annotation; complementary teachers and local observations resolve uncertain cases. This process expands supervision while refining existing labels. Field validity and query-level coverage are checked separately, so an empty result alone does not establish absence. Fusion, validation, and expert iteration are detailed in Section 6.3.

3.4 Training Design

3.4.1 Base VLM Training

Base pretraining has three stages. Vision–language connector alignment trains the projector on image–text pairs with both backbones frozen. Joint multimodal pretraining updates all modules using text and multimodal supervision. General visual and video understanding develops instruction following and image/video understanding. This autoregressive foundation supports subsequent spatial specialization through supervised fine-tuning (SFT) and reinforcement post-training. GroundAnything additionally undergoes the conversion below; training settings are provided in Sections 6.5 and 6.4.

3.4.2 AR-to-Diffusion Conversion

Following multimodal AR-to-diffusion conversion (Wu et al., 2026a; Dong et al., 2026), we retain the pretrained backbone and grounding vocabulary and add a mask token. For a clean sequence x=[V,P,Y] of visual embeddings, text prompt, and response, we partition Y into B=32 token blocks, independently of geometric-tuple boundaries. Sampling t_{b}\sim\mathcal{U}(0,1) independently per block, we corrupt only response positions to form complementary noisy views w^{(a)}=[P,\widetilde{Y}^{(a)}], a\in\{1,2\}; EOS is masked in both. The joint input [w^{(1)};w^{(2)};x] contains visual embeddings only in x.

Under the mask in Figure 3, each noisy response block attends bidirectionally within itself and reads the clean image/prompt context and strictly preceding clean response blocks. Clean tokens use causal attention and cannot read noisy views. The shared decoder minimizes

\mathcal{L}=0.5\mathcal{L}_{\mathrm{MDM}}+0.5\mathcal{L}_{\mathrm{AR}},

where \mathcal{L}_{\mathrm{MDM}} supervises masked response targets and \mathcal{L}_{\mathrm{AR}} supervises clean next-token prediction. Both retain the pretrained token shift. Full loss definitions and training settings appear in Sections 6.7 and 6.4.

3.4.3 Reinforcement Post-Training

We apply GRPO (Shao et al., 2024; Ping et al., 2026) using causal rollouts and exact causal token likelihoods (Figure 4). Box grounding combines set-completeness R_{\mathrm{set}} and strict localization R_{\mathrm{strict}}; OCR scores annotation-aware text–geometry agreement; pointing scores location, count, label, and format. For each prompt group, advantages are A_{i}=Z_{G}(\sum_{c}w_{c}Z_{G}(R_{c,i})), where Z_{G} standardizes within the group. Box-reward weights are (0.7,0.3); OCR and pointing each provide one scalar reward. We optimize the clipped objective

\mathcal{L}_{\mathrm{GRPO}}=-\mathbb{E}_{i}\!\left[|\mathcal{T}_{i}|^{-1}\!\sum\nolimits_{t\in\mathcal{T}_{i}}\min\!\bigl(r_{i,t}A_{i},\operatorname{clip}(r_{i,t},1-\epsilon,1+\epsilon)A_{i}\bigr)\right],

where r_{i,t} is the current-to-old causal token-probability ratio at the rollout temperature, \mathcal{T}_{i} contains valid response tokens including EOS, and \epsilon=0.2. One update per rollout trains the projector and language parameters with the vision encoder frozen and no reference-policy KL term. The updated weights are reused for diffusion inference. Reward definitions and variant-specific reinforcement settings appear in Sections 6.10, 6.8 and 6.9.

3.5 Efficient Parallel Inference

Parallel diffusion decoding.

After image/query prefill, GroundAnything denoises response blocks with bidirectional attention and caches the completed prefix. Sub-blocks are processed from left to right. Our default Entropy-Guided Decoding (Liu et al., 2025) commits masked positions with H_{j}\leq\tau, where H_{j}=-\sum_{v}p_{j}(v)\log p_{j}(v) is the entropy of the unmodified token distribution. If none qualifies, the lowest-entropy position is committed to ensure progress. Committed tokens remain fixed, and a causal pass constructs the completed block’s cache without AR verification. Dynamic and Static Decoding instead use confidence thresholds and fixed quotas; commitment and cache construction are detailed in Section 7.1.

Optional self-speculative decoding.

The shared weights also support diffusion drafting with causal verification (Wu et al., 2026a), accepting the longest matching prefix (Figure 5). Linear decoding uses two network function evaluations (NFEs) and 2B query tokens per round; quadratic decoding fuses verification and proposal generation in one NFE after initialization, using B(B+1) query tokens. These counts describe queries, not Transformer FLOPs. Acceptance and proposal reuse are detailed in Section 7.2; CUDA Graph and selective FP8 optimizations appear in Section 7.3.

4 Experiments

In this section, we ask:

4.1 Implementation Details and Evaluation Setup.

We evaluate GroundAnything-VLM and GroundAnything against 44 baselines on 30 benchmarks spanning 11 perceptual capabilities, covering specialized detectors, general-purpose VLMs, grounding specialists, and embodied foundations. All numerical results and conclusions involving our models are based on the mean of ten runs, using five random seeds with two runs per seed. Owing to the high computational cost, other baselines that we evaluate locally for the main leaderboard are averaged over three runs, using three random seeds with one run per seed. GPT-6 Astra uses thinking effort High. We report F1mIoU for box grounding and OCR, F1@Point for object pointing, and task-specific accuracy for spatial and GUI grounding.

For a fair comparison of diffusion language model (DLM) decoding, all GroundAnything results in the main benchmark tables (Tables 1–4) use entropy-guided diffusion decoding, without self-speculative decoding. The self-speculative setting provides higher throughput and quality at the reported operating point, but combines diffusion drafting with causal autoregressive (AR) verification and correction; it is therefore not pure DLM decoding. We analyze this setting separately in Section 4.3 and highlight it in the teaser (Figure ).

4.2 Main Results

Benchmark suite.

Detection covers common and long-tailed objects on COCO (Lin et al., 2014) and LVIS (Gupta et al., 2019), and dense and tiny objects on Dense200 (Jiang et al., 2026) and VisDrone (Zhu et al., 2018). Referring grounding uses RefCOCO, RefCOCO+, and RefCOCOg (Yu et al., 2016; Mao et al., 2016; Nagaraja et al., 2016); object pointing covers the four detection datasets and RefCOCOg. Spatial and GUI grounding use RoboSpatial (Song et al., 2025), RefSpatial (Zhou et al., 2026), ScreenSpot-V2 (Wu et al., 2025), ScreenSpot-Pro (Li et al., 2025), and OSWorld-G (Xie et al., 2026). OCR uses HierText (Long et al., 2022), ICDAR2015 (Karatzas et al., 2015), TotalText (Ch’Ng & Chan, 2017), and SROIE (Huang et al., 2019); layout grounding uses DocLayNet (Pfitzmann et al., 2022) and M6Doc (Cheng et al., 2023). Exemplar-based visual prompting on FSC147 (Ranjan et al., 2021) and Dense200 is also included in the main results.

GroundAnything-VLM establishes a new overall best among similarly sized models at 72.42%, remaining competitive with GPT-6 Astra (71.35%). GroundAnything reaches 61.75% with pure diffusion decoding, surpassing the prior overall AR state of the art at comparable scale. Figure 6 summarizes this breadth; the comparisons below examine localization precision and task-dependent diffusion–AR gaps.

Reporting conventions.

Scores are percentages; bold marks column bests, including ties (lower is better only for parse error). Model-name stars denote externally reported rows; entry-level stars denote source or task-interface exceptions. --, N/A, and UNK indicate unreported values, unsupported or unresolved evaluations, and unspecified zero-shot status, respectively. Daggers flag uncertain prompt/protocol alignment, so affected scores are descriptive. Under our protocols, GroundingDINO lacks GUI/OCR/layout interfaces, and Kimi-K3 lacks compatible pointing/OCR/layout outputs; SenseNova-Vision’s GUI evaluation remains unresolved. LocateAnything lacks a supported visual-prompt interface. Starred SenseNova-Vision HierText/ICDAR2015 scores follow Han et al. (2026); its other OCR scores are local. Complete sources and exceptions appear in Section 10.

Detection and referring grounding.

GroundAnything reaches 70.04 F1mIoU on Dense200 and 91.61/91.10 on RefCOCOg val/test (Table 1), extending precise diffusion grounding to crowded scenes and language-conditioned targets, although tiny-object localization remains challenging. RefCOCO avg is the unweighted mean of RefCOCO, RefCOCOg, and RefCOCO+.

Robot, spatial, and GUI grounding.

GroundAnything matches its AR counterpart on RefSpatial Unseen (62.34%) and improves ScreenSpot-Pro from 65.34% to 75.96% (Table 2), while Astra remains stronger on RefSpatial and GUI tasks. RefSpatial avg averages the Location and Placement splits.

OCR, layout, and visual prompting.

The shared interface extends to text, document regions, and exemplar-matched instances: GroundAnything-VLM reaches 85.78 on DocLayNet and 74.76 on M6Doc, while GroundAnything scores 63.52 on Dense200 visual prompting (Table 3). OCR and layout exhibit larger diffusion–AR gaps than referring and GUI grounding, identifying where parallel generation still sacrifices precision.

Object pointing.

Following Jiang et al. (2026), SAM-derived masks (Kirillov et al., 2023) determine point correctness, with F1@Point balancing misses and false positives. GroundAnything-VLM leads five of the six selected comparisons.

4.3 Speed Evaluation

4.3.1 Decoding Strategy Analysis

We report tokens per forward (TPF), output tokens per second (TPS), and COCO F1mIoU. Decoding protocols and additional analysis appear in Section 9.1.

Forward efficiency versus latency.

At B=32, quadratic self-speculation reaches higher TPF than the linear schedule (3.17 versus 2.18), yet substantially lower throughput (49.9 versus 223.8 TPS; Figure 7). Its O(B^{2}) query-token cost offsets the reduction in forward calls, favoring the linear schedule at this operating point.

Entropy threshold and block size.

Relaxing the entropy threshold increases parallel commitment, but COCO quality falls sharply beyond \tau=0.8 (Figure 8). At B=32, \tau=0.8 achieves the highest F1mIoU in this threshold sweep (60.72) at 127.0 TPS, or 2.56\times GroundAnything-VLM throughput. Block size also changes this balance: entropy-guided quality peaks at B=16 in this COCO sweep, whereas linear self-speculation peaks at B=32 (Figure 9); larger blocks are therefore not uniformly better.

4.3.2 DLM vs. MTP

We match visual inputs, adaptation data, tokenization, and prefixes, and freeze the same converted model’s causal branch as verifier. The primary MTP baseline is an in-house causal drafter; bidirectional block-MTP and retrained causal-DLM controls test the role of attention (Section 8).

We address four questions:

Draft acceptance.

At K=16, one-step DLM accepts 3.48 draft tokens per round versus 2.69 for causal MTP (Figure 11), yielding more useful proposals at the same candidate length.

Cost per accepted draft token.

Including drafting, verification, and rejected work, one-step DLM reduces full-round cost per accepted draft token from 7.71 to 5.38 ms (Figure 11). Higher acceptance therefore translates into lower wall-clock cost at this operating point.

Refinement versus latency.

At K=16, a second DLM refinement step lowers this cost to 4.70 ms; further refinement increases it. With validation-selected policies on the long-output subset, DLM commits 97.1 versus 84.6 tokens within 400 ms, although MTP leads at 50 ms (Figure 12). Refinement is thus useful when its acceptance gain repays its additional latency.

Bidirectional attention and grounding failures.

Bidirectional DLM reduces Dense200’s unmatched valid-proposal rate from 24.89% for causal MTP to 14.79%; its retrained causal counterpart scores 21.74% (Figure 13). This supports bidirectional attention for drafting jointly constrained spatial predictions. Bidirectional block-MTP also improves draft quality, so the benefit is not exclusive to diffusion.

These gains concern draft efficiency and reliability; exact shared verification preserves the AR verifier’s completed output.

4.3.3 Infrastructure Analysis

Progressive integration with SGLang, CUDA Graph, and FP8 reduces time per output token across all four decoding modes (Figure 14). The complete stack yields 3.45–4.06\times speedups over each mode’s native PyTorch implementation, with self-speculation reaching 3.73 ms/token. Implementation gains are measured within each decoding mode, separately from algorithmic speedups (Section 9.2).

4.4 Ablation Studies

Coordinate representation.

Quantized coordinates improve the reported ablation average by 1.43 pp for GroundAnything-VLM and 5.68 pp for GroundAnything over textual coordinates (Figure 15).

ViT choice.

MoonViT-V2 gives the strongest AR and DLM scores among the tested encoders. Section 9.3.1 analyzes when multi-level visual evidence can help coordinate prediction; the encoder ablation supports our choice under this recipe.

Training.

Removing Stage III causes the largest DLM drop (16.62 pp).

Compact outputs and dense scenes.

The shared output format uses 7.6 tokens/box on COCO and 5.1 on Dense200, compared with 148.8 and 74.5 for the SEED1.5-VL reference reported by Jiang et al. (2026) (Table 5). Figure 16 shows how generation time and output length vary with predicted object count: GroundAnything reduces generation time across the displayed ranges, with larger absolute savings for longer outputs (Section 9.4).

4.5 Discussion

5 Conclusion

We introduced GroundAnything, reconciling fast parallel decoding with broad and precise visual grounding. Structured grounding naturally suits DLMs: its predictions are jointly constrained by shared visual evidence. Diffusion complements the Transformer backbone through a generation process with distinctive strengths in parallel filling and structured outputs. Causal conditioning supports long outputs across blocks, while diffusion accelerates prediction within them. GroundAnything thus highlights a broader principle: generation should follow task structure, and spatial evidence need not be inferred in the order it is serialized.

AI Use Statement

In this work, we used generative AI tools to assist with code development, to polish the writing, and to produce some of the figures. We also used generative AI models as part of the data annotation pipeline to generate pseudo-labels for model training. We did not use generative AI tools to develop the research ideas or methodology, to design or interpret the experiments, or to outline the paper. We have reviewed all AI-assisted work. We take responsibility for the final content of this work, including text, claims, data annotations, or artifacts produced with the aid of generative AI.

Ethics Statement

This work does not involve human-subject studies. All datasets and evaluation benchmarks used in this work were obtained from publicly available or appropriately licensed sources in accordance with their respective terms and licenses. The experiments focus on visual grounding and structured visual understanding and do not involve real-world robotic control, autonomous driving, or interaction with people. GroundAnything is developed as a general-purpose visual grounding model for research rather than for safety-critical real-world deployment. Any future integration into embodied or autonomous systems should be subject to appropriate safety evaluation and human oversight. The authors declare no conflicts of interest.

Reproducibility Statement

We will publicly release the code, model weights of GroundAnything and GroundAnything-VLM, and evaluation data to support systematic and reproducible evaluation. The release will include training and decoding configurations, inference implementations, and evaluation scripts with standardized task prompts, output parsers, and metric implementations. The shared architecture, tokenizer, visual processing, and input/output protocol are specified in Sections 6.2 and 6.1; the data engine and validation pipeline in Section 6.3; and base VLM training, AR-to-diffusion conversion, and continued supervised fine-tuning in Sections 6.5, 6.7 and 6.4. Variant-specific GRPO configurations and task rewards are detailed in Sections 6.8, 6.9 and 6.10. Direct diffusion decoding, optional self-speculative decoding, and execution optimizations are described in Sections 7.1, 7.2 and 7.3. Controlled DLM–MTP comparisons are documented in Section 8, while speed metrics, decoding sensitivity, and infrastructure comparisons appear in Sections 9.1 and 9.2. Architecture and training ablations and output-token efficiency analyses are provided in Sections 9.3 and 9.4. The benchmark suite, evaluation metrics, reporting conventions, and complete task-level results are given in Section 10.

Research Scope, Data Use, and Institutional Disclaimer

This work originated from exploratory academic research undertaken by the project leader Qize Yu during their internship at Xpeng Inc. The project was conducted exclusively for scientific investigation and academic publication and does not involve commercial applications, product development, or commercial deployment. All data used in this project were used solely for academic research and maintained under strict segregation from the company’s commercial model development and deployment activities. No project data were used to train, fine-tune, evaluate, or otherwise support commercial models, products, or services. Internal legal review of the dataset materials was completed on September 21, 2026. This work neither uses nor discloses business data containing users’ private or personally identifiable information. The research-only scope described here does not modify or supersede the applicable terms and licenses of the source datasets.

The views, methods, findings, and conclusions presented in this paper are those of the authors and do not represent the official positions, technical direction, product roadmap, or commercial commitments of Xpeng Inc. The company’s support for this research should not be construed as endorsement of any commercial application. Neither the research findings nor their publication constitute a claim of readiness, safety, or suitability for commercial deployment.

Acknowledgments

We thank Xpeng Inc. for providing computational and data resources in support of this academic research, and the data team for their assistance with data preparation and research support. We are particularly grateful to Professor Ping Luo for his guidance on the research ideas and manuscript writing. We also thank Xinghang Li, Qing Li, Baiqiao Yin, Xinyu Wei, Jiadi You, Linhao Zhou, Qiman Wu, Ziteng Cui, Yi Zou, Wei Wei, Hanzhen Zhang and Zhuo Li for their valuable suggestions and constructive feedback.

References

1. Arriola et al. (2025) [1] Arriola, M., Gokaslan, A., Chiu, J., Yang, Z., Qi, Z., Han, J., Sahoo, S., and Kuleshov, V. Block diffusion: Interpolating between autoregressive and diffusion language models. In International Conference on Learning Representations, volume 2025, pp. 50726–50753, 2025.
2. Bai et al. (2025a) [2] Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., Ge, W., Guo, Z., Huang, Q., Huang, J., Huang, F., Hui, B., Jiang, S., Li, Z., Li, M., Li, M., Li, K., Lin, Z., Lin, J., Liu, X., Liu, J., Liu, C., Liu, Y., Liu, D., Liu, S., Lu, D., Luo, R., Lv, C., Men, R., Meng, L., Ren, X., Ren, X., Song, S., Sun, Y., Tang, J., Tu, J., Wan, J., Wang, P., Wang, P., Wang, Q., Wang, Y., Xie, T., Xu, Y., Xu, H., Xu, J., Yang, Z., Yang, M., Yang, J., Yang, A., Yu, B., Zhang, F., Zhang, H., Zhang, X., Zheng, B., Zhong, H., Zhou, J., Zhou, F., Zhou, J., Zhu, Y., and Zhu, K. Qwen3-vl technical report, 2025a. URL https://arxiv.org/abs/2511.21631.
3. Bai et al. (2025b) [3] Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., Dang, K., Wang, P., Wang, S., Tang, J., Zhong, H., Zhu, Y., Yang, M., Li, Z., Wan, J., Wang, P., Ding, W., Fu, Z., Xu, Y., Ye, J., Zhang, X., Xie, T., Cheng, Z., Zhang, H., Yang, Z., Xu, H., and Lin, J. Qwen2.5-vl technical report, 2025b. URL https://arxiv.org/abs/2502.13923.
4. Ben-Hamu et al. (2026) [4] Ben-Hamu, H., Gat, I., Severo, D., Nolte, N. S., and Karrer, B. Accelerated sampling from masked diffusion models via entropy bounded unmasking. Advances in Neural Information Processing Systems, 38:55981–56007, 2026.
5. Carion et al. (2020) [5] Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. End-to-end object detection with transformers. In European conference on computer vision, pp. 213–229. Springer, 2020.
6. Chen et al. (2022) [6] Chen, T., Saxena, S., Li, L., Fleet, D. J., and Hinton, G. Pix2seq: A language modeling framework for object detection, 2022. URL https://arxiv.org/abs/2109.10852.
7. Cheng et al. (2023) [7] Cheng, H., Zhang, P., Wu, S., Zhang, J., Zhu, Q., Xie, Z., Li, J., Ding, K., and Jin, L. M6doc: A large-scale multi-format, multi-type, multi-layout, multi-language, multi-annotation category dataset for modern document layout analysis, 2023. URL https://arxiv.org/abs/2305.08719.
8. Cheng et al. (2024) [8] Cheng, Z., Li, K., Jin, P., Li, S., Ji, X., Yuan, L., Liu, C., and Chen, J. Parallel vertex diffusion for unified visual grounding. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pp. 1326–1334, 2024.
9. Ch’Ng & Chan (2017) [9] Ch’Ng, C. K. and Chan, C. S. Total-text: A comprehensive dataset for scene text detection and recognition. In 2017 14th IAPR international conference on document analysis and recognition (ICDAR), volume 1, pp. 935–942. IEEE, 2017.
10. Comanici et al. (2025) [10] Comanici, G., Bieber, E., Schaekermann, M., Pasupat, I., Sachdeva, N., Dhillon, I., Blistein, M., Ram, O., Zhang, D., Rosen, E., et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261, 2025.
11. Cui et al. (2025) [11] Cui, C., Sun, T., Lin, M., Gao, T., Zhang, Y., Liu, J., Wang, X., Zhang, Z., Zhou, C., Liu, H., Zhang, Y., Lv, W., Huang, K., Zhang, Y., Zhang, J., Zhang, J., Liu, Y., Yu, D., and Ma, Y. Paddleocr 3.0 technical report, 2025. URL https://arxiv.org/abs/2507.05595.
12. Dai et al. (2021) [12] Dai, X., Chen, Y., Xiao, B., Chen, D., Liu, M., Yuan, L., and Zhang, L. Dynamic head: Unifying object detection heads with attentions. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7369–7378. ieee, 2021.
13. Dang et al. (2026) [13] Dang, R., Guo, J., Hou, B., Leng, S., Li, K., Li, X., Liu, J., Mao, Y., Wang, Z., Yuan, Y., Zhu, M., Lin, X., Bai, Y., Jiang, Q., Zhao, Y., Zeng, M., Gao, J., Jiang, Y., Cen, J., Huang, S., Wang, L., Zhang, W., Liu, C., Yang, J., Lu, S., and Zhao, D. Rynnbrain: Open embodied foundation models, 2026. URL https://arxiv.org/abs/2602.14979.
14. Deitke et al. (2025) [14] Deitke, M., Clark, C., Lee, S., Tripathi, R., Yang, Y., Park, J. S., Salehi, M., Muennighoff, N., Lo, K., Soldaini, L., et al. Molmo and pixmo: Open weights and open data for state-of-the-art vision-language models. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 91–104. IEEE, 2025.
15. Deng et al. (2025) [15] Deng, C., Zhu, D., Li, K., Gou, C., Li, F., Wang, Z., Zhong, S., Yu, W., Nie, X., Song, Z., Shi, G., and Fan, H. Emerging properties in unified multimodal pretraining, 2025. URL https://arxiv.org/abs/2505.14683.
16. Dong et al. (2026) [16] Dong, H., Niu, J., Wang, B., Zeng, W., Zhang, W., and He, C. Mineru-diffusion: Rethinking document ocr as inverse rendering via diffusion decoding. In European Conference on Computer Vision, pp. 439–456. Springer, 2026.
17. Guo et al. (2025) [17] Guo, D., Wu, F., Zhu, F., Leng, F., Shi, G., Chen, H., Fan, H., Wang, J., Jiang, J., Wang, J., et al. Seed1. 5-vl technical report. arXiv preprint arXiv:2505.07062, 2025.
18. Gupta et al. (2019) [18] Gupta, A., Dollar, P., and Girshick, R. Lvis: A dataset for large vocabulary instance segmentation. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5351–5359. IEEE, 2019.
19. Han et al. (2026) [19] Han, X., Li, J., Deng, K., Chen, Z., Shi, X., Wang, S., Li, B., Wang, L., Xie, S., You, X., Quan, J., Cai, Z., Diao, H., Liu, Z., Yang, L., Lin, D., and Wang, Q. Vision as unified multimodal generation, 2026. URL https://arxiv.org/abs/2607.06560.
20. Huang et al. (2019) [20] Huang, Z., Chen, K., He, J., Bai, X., Karatzas, D., Lu, S., and Jawahar, C. Icdar2019 competition on scanned receipt ocr and information extraction. In 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 1516–1520. IEEE, 2019.
21. Jiang et al. (2026) [21] Jiang, Q., Huo, J., Chen, X., Xiong, Y., Zeng, Z., Chen, Y., Ren, T., Yu, J., and Zhang, L. Detect anything via next point prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 25472–25483, 2026.
22. Karatzas et al. (2015) [22] Karatzas, D., Gomez-Bigorda, L., Nicolaou, A., Ghosh, S., Bagdanov, A., Iwamura, M., Matas, J., Neumann, L., Chandrasekhar, V. R., Lu, S., Shafait, F., Uchida, S., and Valveny, E. Icdar 2015 competition on robust reading. In 2015 13th International Conference on Document Analysis and Recognition (ICDAR), pp. 1156–1160, 2015. 10.1109/ICDAR.2015.7333942.
23. Kirillov et al. (2023) [23] Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., et al. Segment anything. In 2023 IEEE/CVF international conference on computer vision (ICCV), pp. 3992–4003. IEEE, 2023.
24. Kumbhar et al. (2026) [24] Kumbhar, S., Liao, H., Appalaraju, S., and Singh, K. Y. Towards gui agents: Vision-language diffusion models for gui grounding, 2026. URL https://arxiv.org/abs/2603.26211.
25. Li et al. (2025) [25] Li, K., Meng, Z., Lin, H., Luo, Z., Tian, Y., Ma, J., Huang, Z., and Chua, T.-S. Screenspot-pro: Gui grounding for professional high-resolution computer use. In Proceedings of the 33rd ACM International Conference on Multimedia, pp. 8778–8786, 2025.
26. Li et al. (2026a) [26] Li, K., Hou, B., Zhu, M., Zhang, T., Cheng, Z., Wang, Z., Leng, S., Li, X., Lin, X., Yao, B., Zeng, M., Liu, J., Dang, R., Guo, J., Huang, S., Zhao, H., Ping, H., Zhao, Y., Zhao, T., Wang, K., Lu, T., Xue, S., Tang, J., Wang, Y., Wang, Z., Gao, J., Lu, S., Liu, C., Yang, J., Chen, M., and Zhao, D. Rynnbrain 1.1: Towards more capable and generalizable embodied foundation model, 2026a. URL https://arxiv.org/abs/2607.17977.
27. Li et al. (2022) [27] Li, L. H., Zhang, P., Zhang, H., Yang, J., Li, C., Zhong, Y., Wang, L., Yuan, L., Zhang, L., Hwang, J.-N., et al. Grounded language-image pre-training. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10955–10965. IEEE, 2022.
28. Li et al. (2026b) [28] Li, S., Kallidromitis, K., Bansal, H., Gokul, A., Kato, Y., Kozuka, K., Kuen, J., Lin, Z., Chang, K.-W., and Grover, A. Lavida: A large diffusion language model for multimodal understanding. Advances in Neural Information Processing Systems, 38:105101–105134, 2026b.
29. Lin et al. (2014) [29] Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C. L. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740–755. Springer, 2014.
30. Liu et al. (2025) [30] Liu, A., He, M., Zeng, S., Zhang, S., Zhang, L., Wu, C., Jia, W., Liu, Y., Zhou, X., and Zhou, J. Wedlm: Reconciling diffusion language models with standard causal attention for fast inference, 2025. URL https://arxiv.org/abs/2512.22737.
31. Liu et al. (2026) [31] Liu, J., Dong, X., Ye, Z., Mehta, R., Fu, Y., Singh, V., Zhang, C., and Molchanov, P. Tidar: Think in diffusion, talk in autoregression. Proceedings of Machine Learning and Systems, 8:748–762, 2026.
32. Liu et al. (2024) [32] Liu, S., Zeng, Z., Ren, T., Li, F., Zhang, H., Yang, J., Jiang, Q., Li, C., Yang, J., Su, H., et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European conference on computer vision, pp. 38–55. Springer, 2024.
33. Long et al. (2022) [33] Long, S., Qin, S., Panteleev, D., Bissacco, A., Fujii, Y., and Raptis, M. Towards end-to-end unified scene text detection and layout analysis. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1039–1049. IEEE, 2022.
34. Lu et al. (2026) [34] Lu, Z., Chai, Y., Guo, Y., Yin, X., Liu, L., Wang, H., Xiao, H., Ren, S., Zhao, P., Liu, G., et al. Ui-r1: Enhancing efficient action prediction of gui agents by reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pp. 17608–17616, 2026.
35. Mao et al. (2016) [35] Mao, J., Huang, J., Toshev, A., Camburu, O., Yuille, A. L., and Murphy, K. Generation and comprehension of unambiguous object descriptions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 11–20, 2016.
36. Meng et al. (2024) [36] Meng, L., Yang, J., Tian, R., Dai, X., Wu, Z., Gao, J., and Jiang, Y.-G. Deepstack: Deeply stacking visual tokens is surprisingly simple and effective for lmms. Advances in Neural Information Processing Systems, 37:23464–23487, 2024.
37. Nagaraja et al. (2016) [37] Nagaraja, V. K., Morariu, V. I., and Davis, L. S. Modeling context between objects for referring expression understanding. In European conference on computer vision, pp. 792–807. Springer, 2016.
38. Peng et al. (2023) [38] Peng, Z., Wang, W., Dong, L., Hao, Y., Huang, S., Ma, S., and Wei, F. Kosmos-2: Grounding multimodal large language models to the world, 2023. URL https://arxiv.org/abs/2306.14824.
39. Pfitzmann et al. (2022) [39] Pfitzmann, B., Auer, C., Dolfi, M., Nassar, A. S., and Staar, P. Doclaynet: A large human-annotated dataset for document-layout segmentation. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, pp. 3743–3751, 2022.
40. Ping et al. (2026) [40] Ping, B., Chen, Z., Hui, T., Yu, Q., Li, C., Yan, J., and Chang, B. LongAct: Harnessing intrinsic activation patterns for long-context reinforcement learning, 2026. URL https://arxiv.org/abs/2604.14922.
41. Qin et al. (2025) [41] Qin, Y., Ye, Y., Fang, J., Wang, H., Liang, S., Tian, S., Zhang, J., Li, J., Li, Y., Huang, S., Zhong, W., Li, K., Yang, J., Miao, Y., Lin, W., Liu, L., Jiang, X., Ma, Q., Li, J., Xiao, X., Cai, K., Li, C., Zheng, Y., Jin, C., Li, C., Zhou, X., Wang, M., Chen, H., Li, Z., Yang, H., Liu, H., Lin, F., Peng, T., Liu, X., and Shi, G. Ui-tars: Pioneering automated gui interaction with native agents, 2025. URL https://arxiv.org/abs/2501.12326.
42. Qwen Team (2026a) [42] Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026a. URL https://qwen.ai/blog?id=qwen3.5.
43. Qwen Team (2026b) [43] Qwen Team. Qwen3.6-27B: Flagship-level coding in a 27B dense model, April 2026b. URL https://qwen.ai/blog?id=qwen3.6-27b.
44. Qwen Team (2026c) [44] Qwen Team. Qwen3.7: The agent frontier, May 2026c. URL https://qwen.ai/blog?id=qwen3.7.
45. Qwen Team (2026d) [45] Qwen Team. Qwen3.8-Max: A new bar for coding and cowork, August 2026d. URL https://qwen.ai/blog?id=qwen3.8.
46. Ranjan et al. (2021) [46] Ranjan, V., Sharma, U., Nguyen, T., and Hoai, M. Learning to count everything. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3393–3402. IEEE, 2021.
47. Redmon et al. (2016) [47] Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 779–788, 2016.
48. Shao et al. (2024) [48] Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y. K., Wu, Y., and Guo, D. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URL https://arxiv.org/abs/2402.03300.
49. Song et al. (2025) [49] Song, C. H., Blukis, V., Tremblay, J., Tyree, S., Su, Y., and Birchfield, S. Robospatial: Teaching spatial understanding to 2d and 3d vision-language models for robotics. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15768–15780. IEEE, 2025.
50. Team et al. (2026) [50] Team, K., Bai, T., Bai, Y., Bao, Y., Cai, J., Cai, X., Cao, P., Cao, Y., Chai, Z., Charles, Y., et al. Kimi k3: Open frontier intelligence. arXiv preprint arXiv:2607.24653, 2026.
51. Wang et al. (2026) [51] Wang, S., Liu, S., Kuang, Y., Wei, X., Liu, Y., Li, Z., Man, Y., Chen, G., Tao, A., Liu, G., et al. Locateanything: Fast and high-quality vision-language grounding with parallel box decoding. In European Conference on Computer Vision, pp. 336–357. Springer, 2026.
52. Wu et al. (2026a) [52] Wu, C., Lan, S., Fu, Y., Gao, S., Wang, J., Yu, J., Alvarez, J. M., Molchanov, P., Luo, P., Han, S., Zhu, L., and Xie, E. Fast-dvlm: Efficient block-diffusion vlm via direct conversion from autoregressive vlm, 2026a. URL https://arxiv.org/abs/2604.06832.
53. Wu et al. (2026b) [53] Wu, C., Zhang, H., Xue, S., Liu, Z., Diao, S., Zhu, L., Luo, P., Han, S., and Xie, E. Fast-dllm: Training-free acceleration of diffusion llm by enabling kv cache and parallel decoding. In International Conference on Learning Representations, volume 2026, pp. 57027–57051, 2026b.
54. Wu et al. (2024) [54] Wu, Z., Chen, X., Pan, Z., Liu, X., Liu, W., Dai, D., Gao, H., Ma, Y., Wu, C., Wang, B., et al. Deepseek-vl2: Mixture-of-experts vision-language models for advanced multimodal understanding. arXiv preprint arXiv:2412.10302, 2024.
55. Wu et al. (2025) [55] Wu, Z., Wu, Z., Xu, F., Wang, Y., Sun, Q., Jia, C., Cheng, K., Ding, Z., Chen, L., Liang, P. P., et al. Os-atlas: Foundation action model for generalist gui agents. In International Conference on Learning Representations, volume 2025, pp. 5090–5108, 2025.
56. Xiao et al. (2024) [56] Xiao, B., Wu, H., Xu, W., Dai, X., Hu, H., Lu, Y., Zeng, M., Liu, C., and Yuan, L. Florence-2: Advancing a unified representation for a variety of vision tasks. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4818–4829. IEEE, 2024.
57. Xie et al. (2026) [57] Xie, T., Deng, J., Li, X., Yang, J., Wu, H., Chen, J., Hu, W., Wang, X., Xu, Y., Wang, Z., et al. Scaling computer-use grounding via user interface decomposition and synthesis. Advances in Neural Information Processing Systems, 38, 2026.
58. Yang et al. (2025) [58] Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025.
59. Ye et al. (2025) [59] Ye, J., Zhang, X., Xu, H., Liu, H., Wang, J., Zhu, Z., Zheng, Z., Gao, F., Cao, J., Lu, Z., Liao, J., Zheng, Q., Huang, F., Zhou, J., and Yan, M. Mobile-agent-v3: Fundamental agents for gui automation, 2025. URL https://arxiv.org/abs/2508.15144.
60. You et al. (2026a) [60] You, J., Yu, Q., Chen, Y., Cai, M., Zhong, Z., Wang, Y., Ping, B., Liang, J., Shen, Z., Yan, H., Li, Y., Wu, R., Qi, X., and Chen, Y. AffordanceWAM: Affordance-aware joint world-action modeling for robot manipulation, 2026a. URL https://arxiv.org/abs/2609.22332.
61. You et al. (2026b) [61] You, Z., Nie, S., Zhang, X., ZHOU, J., Lu, Z., Wen, J.-R., and Li, C. Llada-v: Large language diffusion models with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10093–10105, 2026b.
62. Yu et al. (2016) [62] Yu, L., Poirson, P., Yang, S., Berg, A. C., and Berg, T. L. Modeling context in referring expressions. In European conference on computer vision, pp. 69–85. Springer, 2016.
63. Yu et al. (2026) [63] Yu, Q., You, J., Wang, Y., Liang, J., Ping, B., Tian, Y., Chen, Y., Cai, M., Gong, Z., Wu, R., Li, Y., Liang, J., and Chen, Y. AffordanceVLA: A vision-language-action model empowering action generation through affordance-aware understanding, 2026. URL https://arxiv.org/abs/2606.06155.
64. Yuan et al. (2024) [64] Yuan, W., Duan, J., Blukis, V., Pumacay, W., Krishna, R., Murali, A., Mousavian, A., and Fox, D. Robopoint: A vision-language model for spatial affordance prediction for robotics, 2024. URL https://arxiv.org/abs/2406.10721.
65. Yue et al. (2025) [65] Yue, Z., Lin, Z., Song, Y., Wang, W., Ren, S., Gu, S., Li, S., Li, P., Zhao, L., Li, L., et al. Mimo-vl technical report. arXiv preprint arXiv:2506.03569, 2025.
66. Zhang et al. (2022) [66] Zhang, H., Li, F., Liu, S., Zhang, L., Su, H., Zhu, J., Ni, L. M., and Shum, H.-Y. Dino: Detr with improved denoising anchor boxes for end-to-end object detection, 2022. URL https://arxiv.org/abs/2203.03605.
67. Zhao et al. (2024) [67] Zhao, Z., Kang, H., Wang, B., and He, C. Doclayout-yolo: Enhancing document layout analysis through diverse synthetic data and global-to-local adaptive perception, 2024. URL https://arxiv.org/abs/2410.12628.
68. Zhou et al. (2026) [68] Zhou, E., An, J., Chi, C., Han, Y., Rong, S., Zhang, C., Wang, P., Wang, Z., Huang, T., Sheng, L., et al. Roborefer: Towards spatial referring with reasoning in vision-language models for robotics. Advances in Neural Information Processing Systems, 38:28404–28481, 2026.
69. Zhu et al. (2018) [69] Zhu, P., Wen, L., Bian, X., Ling, H., and Hu, Q. Vision meets drones: A challenge, 2018. URL https://arxiv.org/abs/1804.07437.

\beginappendix

\WF@box

6 Model and Training Details

\WF@box

6.1 Input/Output Protocol

GroundAnything and GroundAnything-VLM share a structured interface: the query specifies what evidence to extract, and each response entry associates a semantic identifier with spatial coordinates. The geometry is task-dependent, while the enclosing grammar remains unchanged, following the unified grounding formulation of Jiang et al. (2026).

Query templates.

Table 6 lists the task instructions. Bracketed terms are placeholders. A category list joins the requested names with </c>, without intervening spaces; for example, person</c>car. Visual reference boxes use the same coordinate tokens as outputs. The image and instruction form the user turn; the structured response forms the assistant turn.

Response grammar.

The entry template and illustrative payloads are shown below. Displayed line breaks are for readability and are omitted in the serialized response.

Boxes use four atomic coordinate tokens in (x_{1},y_{1},x_{2},y_{2}) order; points use two in (x,y) order. Instances sharing an identifier occupy one wrapper, with tuples separated by commas without spaces. Distinct entries are separated by a comma and a space. Ordered point sequences retain their temporal order. The ordinary text None denotes an absent queried target, not a padding element. OCR transcriptions occupy the identifier field rather than a separate text channel.

Coordinates and canonicalization.

Coordinates are normalized by image width and height and quantized to <0>–<999>. For annotations on a [0,1000] grid, the conversion is

q(v)=\left\lfloor\frac{999v+500}{1000}\right\rfloor.

Input xywh boxes are converted to xyxy; polygons used for box supervision become axis-aligned enclosing boxes. Non-finite or out-of-range coordinates and inverted or degenerate boxes fail validation. Box instances are stably sorted by x_{1}, and unordered points by x; semantic sequence order is never replaced by spatial sorting.

Structural tokens and termination.

Table 7 distinguishes entry boundaries from response termination. Structural and coordinate tokens are retained when decoding text for the parser. The diffusion mask is an internal input token and is excluded from the generatable vocabulary.

\WF@box

6.2 Architecture, Tokenizer, and Visual Processing

The variants share a MoonViT-V2 (Kimi K3) visual encoder, a multimodal projector, and a Qwen3-4B language architecture. All language layers use full attention and one-dimensional rotary position indices. Dynamic image processing supplies at most 1024 projected visual tokens, corresponding to 4096 patches before 2\times 2 aggregation; channel means and standard deviations are both (0.5,0.5,0.5). Table 8 gives the module specifications.

Vocabulary and parameter sharing.

GroundAnything-VLM extends the 151,669-entry base vocabulary with 1000 coordinate tokens and </c>, giving 152,670 entries. Coordinate IDs are 151669–152668; the category delimiter has ID 152669. GroundAnything adds |<MASK>| at ID 152670, giving 152,671 entries. EOS and padding have distinct IDs, 151645 and 151643; the image, vision-start, and vision-end IDs are 151655, 151652, and 151653.

Input embeddings and output heads are untied. Within GroundAnything, causal and diffusion modes share all model weights. The new mask row is initialized separately in each vocabulary matrix using the FP32 mean of the rows for <|im_start|>, <|im_end|>, <|vision_start|>, and <|vision_end|>, then cast to the model dtype. This adds 5120 parameters. The complete multimodal model has approximately 4.844B parameters (Table 9).

\WF@box

6.3 Expert-Driven Data Engine

Field-level fusion.

Complementary teachers provide semantic, localization, segmentation, text, interface, and layout evidence. Category discovery precedes category-conditioned localization; category names are normalized while attributes needed for referring expressions are retained. Predictions are mapped back to the original image and aligned only when their query scope and annotation granularity agree. Fusion operates on individual fields, allowing reliable text or geometry to be retained without accepting an entire inconsistent annotation.

Validation and coverage.

Task-specific checks assess geometric validity, semantic consistency, text–region agreement, and cross-view support. Each dependency has a validation state a(f)\in\{\mathrm{pass},\mathrm{fail},\mathrm{pending}\}; query-level coverage c(I,Q) uses the same state space. Coverage checks whether the required query scope has been examined and relevant unresolved proposals and matching conflicts have been addressed. Unresolved dependencies trigger complementary teachers or targeted local observations.

Required dependencies and acceptance.

Let \mathcal{F}(I,Q,Y) contain the fields and semantic facts required to validate candidate Y, including query–target correspondence. The task protocol and query semantics determine these requirements, which are then instantiated for the candidate. Missing required fields remain in \mathcal{F} with status \mathrm{pending}: omitting a field cannot remove its validation requirement. Let h(Q)\in\{0,1\} indicate whether the query requires exhaustive instance coverage. Acceptance requires

\mathcal{A}(I,Q,Y)=\mathbf{1}\!\left[\forall f\in\mathcal{F}(I,Q,Y),\ a(f)=\mathrm{pass}\right]\mathbf{1}\!\left[h(Q)=0\ \lor\ c(I,Q)=\mathrm{pass}\right],

so every required dependency must pass, and exhaustive queries additionally require validated coverage. For example, validating the returned boxes does not establish that every queried instance has been found; an all-instance query remains unaccepted while coverage is pending.

Empty-target claims.

An answer with no instances still requires verification of its query–answer relation. Its dependency set explicitly includes a query-scoped absence fact f_{\mathrm{abs}}\in\mathcal{F}(I,Q,Y), so an empty prediction never yields an empty dependency set. An unsupported absence claim remains pending; contrary evidence makes it fail. It passes only when absence is verified and c(I,Q)=\mathrm{pass}, even when h(Q)=0. Missing outputs, failed annotation runs, and unresolved empty predictions do not constitute evidence of absence. These requirements prevent empty predictions from being accepted through a vacuously true check over an empty set.

Derived box, point, referring, and visual-prompt supervision preserves its dependencies on the validated source evidence; crops and coordinate transforms remain consistent with visible content.

Expert iteration.

Accepted annotations train a unified grounding expert, which then proposes annotations for further validation. Difficult cases receive targeted crops or complementary teacher evidence. New evidence can correct or retire previous labels, with versioned annotation snapshots keeping each training input fixed. This loop improves both coverage and field consistency rather than merely accumulating predictions.

\WF@box

6.4 Training Pipeline

GroundAnything follows four training phases: Base VLM Training (Pretrain 1), Coordinate Alignment (Pretrain 2, also termed SFT), AR-to-Diffusion conversion, and reinforcement learning (RL). GroundAnything-VLM follows three phases: Base VLM Training, Coordinate Alignment, and RL. The Stage 1–4 numbering below refers specifically to pretraining and alignment: Stages 1–3 belong to Pretrain 1, while Stage 4 is Pretrain 2. Table 10 summarizes the two pipelines and their relation to Figure 4.

\WF@box

6.5 Base VLM Training (Pretrain 1)

Stage 1: vision–language connector alignment.

The vision encoder and language backbone are frozen, and only the projector is trained on image-caption data. Assistant-only causal cross-entropy aligns the visual features with the language input space.

Stage 2: joint multimodal pretraining.

All modules are unfrozen and jointly trained on pure text and general image–text data. Pure-text examples supervise all valid causal text targets; multimodal examples supervise assistant responses, excluding visual inputs, prompts, and masked template positions. The text and multimodal losses are normalized separately over their valid target tokens and then summed, \mathcal{L}_{\mathrm{mix}}=\mathcal{L}_{\mathrm{text}}+\mathcal{L}_{\mathrm{vlm}}. Counts are aggregated across data-parallel workers so that each domain contributes its global token mean. Packing preserves independent subsequence boundaries and supervision masks.

Stage 3: general visual and video understanding.

All modules continue training on general visual question answering and instruction-following data, image captions, and video data with assistant-only causal cross-entropy. The configured context and packing lengths are both 32K. This stage has no separate pure-text quota or dedicated spatial-specialization mixture; spatial supervision is concentrated in Pretrain 2. Aggregate base-pretraining exposure exceeds 700B tokens.

\WF@box

6.6 Coordinate Alignment (Pretrain 2 / SFT)

Stage 4: spatial specialization.

Initialized from the Stage 3 checkpoint, coordinate alignment updates the vision encoder, projector, and language parameters on eight spatial task groups: detection, GUI grounding, referring grounding, referring pointing, OCR, document layout, dense pointing and counting, and visual prompting. Boxes, points, semantic identifiers, and protocol markers are supervised jointly as assistant-response tokens. The coordinate tokens <0> through <999> use the same causal cross-entropy objective as the other valid response tokens. This single alignment phase is the supervised fine-tuning (SFT) phase referred to in the main text. Its context and packing lengths are 8K.

Table 11 gives the configured recipe for both pretraining phases. After coordinate alignment, GroundAnything-VLM proceeds to RL, while GroundAnything first undergoes the AR-to-Diffusion conversion in Section 6.7.

All four pretraining/alignment stages use BF16, ZeRO-1, AdamW with (\beta_{1},\beta_{2})=(0.9,0.95), FlashAttention, and activation checkpointing. Global batches count packed sequences and equal the number of GPUs times the per-device batch times gradient accumulation. Here 8K and 32K denote 8192 and 32768 tokens, respectively, and one epoch denotes the configured training budget. Language parameters include the untied input embedding and output head.

\WF@box

6.7 AR-to-Diffusion Conversion (GroundAnything Only)

GroundAnything is initialized from the autoregressive checkpoint after coordinate alignment. Direct multimodal conversion (Wu et al., 2026a) retains its grounding vocabulary and output grammar. Let x=[V,P,Y] contain visual embeddings V, the text prompt P encoding query Q, and response Y. Supervised response spans are partitioned into consecutive blocks of B=32 tokens. Blocks may cross geometric-tuple boundaries, stop at response-span boundaries, and have a shorter final block; they require no semantic padding.

Complementary corruption.

For each response block b, sample t_{b}\sim\mathcal{U}(0,1). For every supervised non-EOS position i in that block, draw m_{i}^{(1)}\sim\operatorname{Bernoulli}(t_{b}) and set m_{i}^{(2)}=1-m_{i}^{(1)}. View a replaces the target by |<MASK>| if m_{i}^{(a)}=1, giving w^{(a)}=[P,\widetilde{Y}^{(a)}]. EOS is masked in both views. Images, prompts, headers, unsupervised turns, and empty thinking prefixes are not corrupted.

Attention visibility.

The joint input [w^{(1)};w^{(2)};x] shares one clean multimodal stream and one visual encoding; each noisy view retains an uncorrupted prompt copy. Within a view, a noisy response block attends bidirectionally to itself and reads the clean visual/prompt context and strictly preceding clean response blocks. It cannot read current or future clean response targets or the other noisy view. Clean tokens use token-level causal attention and cannot attend to either noisy view. Figure 3 illustrates one view with two-token blocks for clarity.

Joint objective.

For masked target sets \mathcal{M}^{(1)} and \mathcal{M}^{(2)}, and valid clean response targets \mathcal{T}, the two losses are

\displaystyle\mathcal{L}_{\mathrm{MDM}} \\ \displaystyle=-\frac{\sum_{a=1}^{2}\sum_{i\in\mathcal{M}^{(a)}}\log p_{\theta}(y_{i}\mid x,w^{(a)};\mathcal{A}_{a})}{|\mathcal{M}^{(1)}|+|\mathcal{M}^{(2)}|}, \\ \displaystyle\mathcal{L}_{\mathrm{AR}} \\ \displaystyle=-\frac{1}{|\mathcal{T}|}\sum_{i\in\mathcal{T}}\log p_{\theta}(y_{i}\mid V,P,y_{<i}).

Here \mathcal{A}_{a} enforces the visibility rules above, so writing x as an input does not expose the clean target. The MDM mean includes both masked EOS targets and has no inverse-noise weighting. Both losses retain the next-token shift: hidden position i-1 predicts target i. Prompts, visual positions, headers, padding, unsupervised turns, and empty thinking prefixes are excluded. Valid target losses are averaged across the training batch and combined as in Equation 2.

Conversion optimization.

Within this conversion phase, optimization has two substages: Phase I freezes the vision encoder and trains the projector and language parameters; Phase II updates all modules. Both retain the fixed block size and joint objective. The optimizer and scheduler restart between phases. Table 12 reports the settings.

Conversion uses BF16, ZeRO-1, one gradient-accumulation step, and AdamW with betas (0.9,0.95), epsilon 10^{-8}, weight decay 0.1, and gradient clipping at 1.0. Both phases use cosine decay with 3% warmup.

\WF@box

6.8 Reinforcement Learning (RL): Causal GRPO

RL is the final training phase for both variants, following coordinate alignment for GroundAnything-VLM and AR-to-Diffusion conversion for GroundAnything. GroundAnything uses causal attention for both rollout generation and likelihood evaluation. For each prompt, G=8 responses are sampled at temperature T=0.7. Before updating, a no-gradient teacher-forcing pass records detached old-policy log-probabilities; a second pass computes current-policy gradients. Both use FP32 log-softmax at the rollout temperature. These are exact likelihoods of the causal policy, not likelihoods of a diffusion denoising trajectory.

Advantage normalization.

For a scalar component u within one prompt group, define

Z_{G}(u_{i})=\frac{u_{i}-\bar{u}}{\sqrt{G^{-1}\sum_{j=1}^{G}(u_{j}-\bar{u})^{2}}+10^{-6}},\qquad\bar{u}=G^{-1}\sum_{j=1}^{G}u_{j}.

Each active reward component is normalized before weighting, and their sum is normalized again within the same group:

A_{i}=Z_{G}\!\left(\sum_{c\in\mathcal{C}_{\mathrm{task}}}w_{c}Z_{G}(R_{c,i})\right).

Box rewards have weights (0.7,0.3); OCR and pointing each supply one complete scalar reward with unit weight. Internal OCR subterms are not normalized separately. Constant groups produce zero advantage.

Policy objective.

Let \pi_{\theta,T} denote the causal policy at temperature T, and let \mathcal{T}_{i} contain response tokens including EOS, excluding prompts and post-EOS padding. Expanding the response-wise averaging in Equation 3, the update minimizes

\displaystyle r_{i,t}(\theta) \\ \displaystyle=\frac{\pi_{\theta,T}(y_{i,t}\mid I,Q,y_{i,<t})}{\pi_{\mathrm{old},T}(y_{i,t}\mid I,Q,y_{i,<t})}, \\ \displaystyle\mathcal{L}_{\mathrm{GRPO}} \\ \displaystyle=-\frac{1}{G}\sum_{i=1}^{G}\frac{1}{|\mathcal{T}_{i}|}\sum_{t\in\mathcal{T}_{i}}\min\!\left(r_{i,t}A_{i},\operatorname{clip}(r_{i,t},1-\epsilon,1+\epsilon)A_{i}\right),

with \epsilon=0.2. There is one update per rollout and no reference-policy KL term. The vision encoder is frozen; the projector and language parameters are updated. Causal and diffusion attention modes subsequently reuse these parameters without changing the output protocol.

\WF@box

6.9 GroundAnything-VLM Reinforcement Settings

GroundAnything-VLM uses the same task-reward definitions but a distinct optimization configuration. It freezes both the vision encoder and projector and retains a frozen supervised reference with KL coefficient 0.02. Active reward components are standardized within each prompt group using the sample standard deviation and epsilon 10^{-8}; after weighted summation, advantages are standardized across the complete response batch. GroundAnything instead uses population standard deviations and a second within-group normalization. Table 13 keeps these recipes separate.

GroundAnything maintains FP32 optimizer states. Its rollout and gradient passes use the same causal policy; the inference implementations described below do not alter this training factorization.

\WF@box

6.10 Task Rewards

The rewards below evaluate complete structured responses. We write n and m for the numbers of predictions and reference instances, respectively, and \operatorname{clip}_{[0,1]} for clipping to the unit interval. Content scores below assume nonempty references; missing predictions receive zero content agreement.

\WF@box

6.10.1 Box Grounding

Set completeness.

For each reference box g_{j}, select the predicted box with maximum IoU before checking its label:

i^{*}(j)=\arg\max_{i}\operatorname{IoU}(b_{i},g_{j}),\qquad s_{j}=\operatorname{IoU}(b_{i^{*}(j)},g_{j})\,\mathbf{1}[\ell_{i^{*}(j)}=\ell_{j}].

With S=\sum_{j}s_{j}, soft precision and recall are P=S/n and R=S/m, and

R_{\mathrm{set}}=\frac{2PR}{P+R+10^{-8}}.

This GT-wise matching permits a prediction to support multiple references. Consequently, R_{\mathrm{set}} is a completeness proxy rather than a bounded one-to-one F1 metric. It is combined with a stricter reward that penalizes duplicate and oversized predictions.

Strict localization.

Let \bar{F} average localization F1 at IoU thresholds 0.50, 0.75, and 0.95. The strict reward is

\begin{split}R_{\mathrm{strict}}=\operatorname{clip}_{[0,1]}\bigl(&0.10R_{\mathrm{fmt}}+0.10R_{\mathrm{count}}+0.50\bar{F}+0.25R_{\mathrm{IoU}}+0.05R_{\mathrm{sort}}\\
&-0.07P_{\mathrm{big}}-0.03P_{\mathrm{dup}}\bigr).\end{split}

The auxiliary terms score protocol validity, instance-count agreement, matched IoU, and canonical ordering. Penalties discourage unsupported large boxes and duplicate predictions. The set and strict rewards are normalized separately before combination, rather than treating their raw weighted sum as a single reward.

\WF@box

6.10.2 OCR

Text–geometry matching.

OCR uses one-to-one Hungarian matching. The main text key applies Unicode NFKC normalization and case folding and retains alphanumeric characters; symbol-only strings retain codepoint-based keys. Let E_{ij} be one minus normalized Levenshtein distance between the resulting keys, and I_{ij} the box IoU. For an affinity matrix A, define

F(A)=\frac{2}{n+m}\max_{\mathcal{M}}\sum_{(i,j)\in\mathcal{M}}A_{ij},

where \mathcal{M} is a one-to-one assignment. Hard affinity requires equal text keys and I_{ij}\geq\tau. Write its score as H_{\tau} and its average over \tau\in\{0.50,0.55,\ldots,0.95\} as \bar{H}. The soft affinity and single-reference score are

\displaystyle A^{\mathrm{soft}}_{ij} \\ \displaystyle=\sqrt{I_{ij}}\,E_{ij}\,\mathbf{1}[I_{ij}\geq 0.10,\ E_{ij}\geq 0.20], \\ \displaystyle V(P,T) \\ \displaystyle=\operatorname{clip}_{[0,1]}\left(0.45\bar{H}+0.35F(A^{\mathrm{soft}})+0.10H_{0.50}+0.10C\right),

with count agreement C=\min(n,m)/\max(n,m).

Word/line consistency.

Let V_{\mathrm{hi}} and V_{\mathrm{lo}} be the higher and lower scores against complete word- and line-level references. For a region with alternative granularities, local agreement selects the better complete representation, not independently chosen fragments. Singleton consensus uses the maximum text and box similarities among the reference alternatives, followed by one-to-one assignment, with score 0.60\bar{H}^{*}+0.40S^{*}. Aggregating local agreements gives \Gamma, with lower weights for text-conflict regions. The global and local terms are combined as

D=\alpha V_{\mathrm{hi}}+(1-\alpha)V_{\mathrm{lo}},\qquad Q_{g}=\beta D+(0.90-\beta)\Gamma,

with (\alpha,\beta)=(0.90,0.35) for conflicting granularities and (0.75,0.50) otherwise. Define

m_{g}=\max(|T_{\mathrm{word}}|,|T_{\mathrm{line}}|,1),\qquad C_{g}=\frac{\min(n,m_{g})}{m_{g}}.

The reward is

R_{g}=\operatorname{clip}_{[0,1]}\left((Q_{g}+0.10f_{g}C_{g})f_{g}^{2}-0.05P_{\mathrm{dup}}-0.10P_{\mathrm{over}}-0.05P_{\mathrm{invalid}}\right).

Complementary-reference consistency.

For complex text, each reference score additionally preserves surface-form accuracy: W=0.85V+0.15U, where U is exact-string F1 after NFKC and case folding, retaining punctuation and requiring IoU at least 0.50. Two primary references and a complementary reference yield

W_{\mathrm{text}}=0.82\!\left(0.70\max(W_{a},W_{b})+0.30\min(W_{a},W_{b})\right)+0.18W_{c}.

When auxiliary geometry is available, Q_{c}=0.95W_{\mathrm{text}}+0.05G_{\mathrm{aux}}; otherwise Q_{c}=W_{\mathrm{text}}. The geometry-only score is G_{\mathrm{aux}}=0.60\bar{H}_{\mathrm{geo}}+0.30S_{\mathrm{geo}}+0.10C_{\mathrm{geo}}, excluding transcription agreement. Let m_{c} be the rounded median reference count and C_{c}=\min(n,m_{c})/m_{c}. Then

\begin{split}R_{c}=\operatorname{clip}_{[0,1]}\bigl(&(0.90Q_{c}+0.10f_{c}C_{c})f_{c}^{2}-0.05P_{\mathrm{dup}}-0.10P_{\mathrm{over}}\\
&-0.05P_{\mathrm{invalid}}-0.08P_{\mathrm{large}}\bigr).\end{split}

Format gating and penalties.

The parse grades in Table 14 gate the full content score quadratically. Penalties measure duplicated, excessive, or invalid predictions; P_{\mathrm{large}} additionally penalizes boxes covering over 80% of the image without geometric reference support. The annotation structure selects R_{g} or R_{c}, which enters GRPO as one scalar reward.

\WF@box

6.10.3 Pointing

Point predictions are matched one-to-one within each label. For distance d on the normalized 0–999 coordinate grid, similarity is

s(d)=\begin{cases}\exp[-\tfrac{1}{2}(d/75)^{2}],&d\leq 200,\\
0,&d>200.\end{cases}

The matched similarities define soft precision P and recall R. With F_{2}=5PR/(4P+R), defined as zero when the denominator vanishes, the reward is

R_{\mathrm{point}}=0.55F_{2}+0.25R_{\mathrm{count}}+0.10F_{1,\mathrm{label}}+0.10R_{\mathrm{fmt}}.

Count agreement is the smaller-to-larger count ratio when the larger count is positive. Format validity is 1 for a valid response, 0.5 when valid points remain recoverable from a malformed response, and 0 otherwise.

\WF@box

7 Parallel Inference

\WF@box

7.1 Direct Diffusion Decoding

Block state and token alignment.

Image/query prefill establishes a causal prefix cache and the first anchor token. A physical block contains this known anchor followed by B-1 mask positions. Predictions retain the training token shift. Sub-blocks are completed from left to right; every denoising forward still evaluates the full physical block. Completed tokens remain visible to subsequent masked positions, and the prefix cache is reused throughout.

Entropy-guided commitment.

Following the use of entropy as a reliability signal in diffusion decoding (Liu et al., 2025), we compute

p_{j}=\operatorname{softmax}(z_{j}),\qquad H_{j}=-\sum_{v\in\mathcal{V}_{\mathrm{gen}}}p_{j}(v)\log p_{j}(v),

over the generatable vocabulary, excluding the mask token. For the still-masked positions \mathcal{U} of the active sub-block, commit all positions in \mathcal{S}=\{j\in\mathcal{U}:H_{j}\leq\tau\}. If \mathcal{S} is empty, commit only \arg\min_{j\in\mathcal{U}}H_{j}. Candidate sampling and reliability assessment are separate operations. The procedure repeats until the sub-block is complete; committed tokens are neither remasked nor revised.

Causal cache construction.

Bidirectional hidden states depend on positions to their right and cannot serve directly as causal history. After completing a block, one causal forward recomputes its authoritative KV entries and predicts the next anchor. This pass does not verify, reject, or replace the completed block. Thus, a block requiring D denoising passes uses D+1 NFEs, excluding initial prefill. The number of passes depends on commitment decisions rather than a fixed per-response denoising count.

Other commitment policies.

Dynamic Decoding uses a confidence threshold with a forced best-position fallback. Static Decoding commits the highest-confidence remaining positions under a prescribed per-pass quota, distributing the remaining positions across the remaining passes. These policies change token commitment, while preserving blockwise generation. They are distinct from AR verification; their confidence scores should not be identified with the unmodified-logit entropy used above.

\WF@box

7.2 Optional Self-Speculative Decoding

Self-speculative decoding (Wu et al., 2026a) uses GroundAnything’s shared weights for bidirectional drafting and causal verification, as illustrated in Figure 5. Verification accepts only the longest consecutive draft prefix agreeing with the causal predictions. The first disagreement ends acceptance; later coincidental matches are discarded. Rejected suffix states are removed from the cache.

Linear schedule.

Given a committed seed, a bidirectional pass predicts a candidate block, and a causal pass verifies it in parallel. Accepted draft tokens are committed with the verifier correction at the first mismatch. Each round uses two NFEs and 2B query tokens. The seed is counted once and is excluded from draft-acceptance statistics.

Quadratic schedule.

The fused query contains B groups of B+1 tokens, with group i arranged as [d_{i},M,\ldots,M] at overlapping positions [t+i,\ldots,t+i+B]. Row leaders support causal verification; each row’s mask positions form a bidirectional proposal group. All queries read the cached prefix. Compare row prediction a_{i} with d_{i+1}, starting the accepted count at one for the known leading token d_{0}. At the first mismatch, select that row’s proposal as the next draft. This selection reuses computation but does not verify the selected proposal. After initialization, each round uses one NFE and B(B+1) query tokens. The linear/quadratic terminology concerns query-token counts, not total Transformer FLOPs or a fixed number of accepted tokens.

These schedules use greedy causal comparisons. Their verification semantics concern the converted model’s causal branch and do not establish equivalence to sampling an arbitrary autoregressive distribution or to an earlier model’s outputs.

\WF@box

7.3 Execution Optimizations

The implementation progresses from Native PyTorch (Eager) to SGLang (Eager), CUDA Graph replay, and selective FP8 execution. Denoising and causal passes retain their respective attention and cache semantics. FP8 applies to eligible language-model linear operations, with the remaining components kept in BF16. These changes reduce execution overhead or arithmetic cost; quantization can still alter logits and decoding decisions. The corresponding latency results are analyzed in Section 9.2.

\WF@box

8 DLM–MTP Comparison: Experimental Details

\WF@box

8.1 Controlled Model Comparisons

The drafters share visual preprocessing, the projector, coordinate vocabulary, serialization, and autoregressive backbone initialization. A single frozen causal branch of GroundAnything verifies all proposals. Adaptation matches ordered examples, supervised-token exposure, optimizer updates, effective batch size, and candidate-length sampling. These controls do not imply equal trainable parameter counts or training FLOPs.

The primary in-house MTP baseline uses a causal recurrent future-token module with weights shared across prediction horizons. It is distinct from LocateAnything, which uses bidirectional attention within box-aligned MTP blocks and autoregressive fallback in Hybrid Mode (Wang et al., 2026). An additional in-house block-MTP control predicts a fully masked candidate block in one bidirectional pass; it shares our data and token format but is not a reproduction of LocateAnything.

The attention ablation compares separately trained causal and bidirectional DLMs with the same architecture, initialization, denoising objective, corruption distribution, training exposure, and refinement schedule. Only candidate-block attention differs during training and inference. All candidate positions can read the committed prefix; the prefix cannot read candidates. This pair isolates attention, whereas comparing block MTP with DLM also changes the objective and refinement procedure.

\WF@box

8.2 Draft Acceptance and Token Cost

All configurations receive the same bank of verifier-generated prefixes from held-out COCO prompts, with validation and test prefixes separated by image. The deadline comparison uses long-output prefixes and therefore characterizes decode progress on that subset rather than full-dataset request throughput. Candidate length K counts newly proposed tokens, excluding the committed anchor, padding, and verifier correction or bonus. Greedy verification accepts the longest matching prefix, then emits a correction at the first mismatch or a bonus after complete acceptance, subject to response termination.

For round r, let K_{r}, A_{r}, and C_{r} denote proposed, accepted-draft, and committed-output token counts, and let T_{r} be full-round latency in milliseconds. We report

\displaystyle R_{\mathrm{draft}} \\ \displaystyle=\frac{\sum_{r}A_{r}}{\sum_{r}K_{r}}, \\ \displaystyle\bar{A} \\ \displaystyle=\frac{1}{N_{r}}\sum_{r}A_{r}, \\ \displaystyle\bar{C} \\ \displaystyle=\frac{1}{N_{r}}\sum_{r}C_{r}, \\ \displaystyle c_{A} \\ \displaystyle=\frac{\sum_{r}T_{r}}{\sum_{r}A_{r}}, \\ \displaystyle c_{C} \\ \displaystyle=\frac{\sum_{r}T_{r}}{\sum_{r}C_{r}}, \\ \displaystyle\bar{T} \\ \displaystyle=\frac{1}{N_{r}}\sum_{r}T_{r},

where N_{r}>0 is the number of rounds; each ratio requires a positive denominator. Ratios of totals retain the cost of zero-acceptance and malformed-proposal rounds. For a full nonterminal round, C_{r}=A_{r}+1; terminal rounds use actual emitted counts. Output throughput is 1000/c_{C}, while 1000/c_{A} measures accepted draft tokens per second.

Round timing begins after the shared prefix cache is ready and ends after output and cache commitment. It includes drafting, verification, rejected work, candidate selection, scheduling, and cache maintenance; shared visual encoding and prefix prefill are outside this decode-only cost. All methods use the same execution conditions.

Acceptance and latency results.

At K=16, one-step DLM accepts 21.76% of candidates (3.48 tokens per round), compared with 16.81% (2.69 tokens) for causal MTP. This reduces c_{A} from 7.71 to 5.38 ms, a 30.2% reduction at matched candidate length. Across the tested lengths, DLM’s lowest cost is 5.38 ms at K=16, while MTP’s is 6.58 ms at K=8. However, MTP is preferable at K=4: its cost is 7.15 ms versus 9.31 ms for DLM. At K=32, DLM’s accepted prefix decreases to 3.30 tokens despite the longer proposal. These results favor selecting candidate length by accepted work and elapsed time rather than nominal parallelism.

\WF@box

8.3 Refinement and Fixed-Deadline Decoding

The refinement comparison fills all K candidate positions on the first pass. Before pass j\in\{2,\ldots,S\}, it remasks the lowest-confidence

m_{j}=\left\lceil\frac{K(S-j+1)}{S}\right\rceil

positions and predicts them jointly. Both DLM attention variants use this schedule, followed by causal verification. This controlled draft-refinement experiment is distinct from monotone entropy-guided commitment in direct generation.

For two refinement depths S and S^{\prime} with positive mean latency and accepted-token counts, extra refinement lowers accepted-token cost precisely when

\frac{\bar{T}_{S^{\prime}}}{\bar{T}_{S}}<\frac{\bar{A}_{S^{\prime}}}{\bar{A}_{S}}.

The condition for committed-token cost replaces \bar{A} by \bar{C}. Thus, improved draft acceptance alone does not establish a wall-clock benefit.

For a deadline D, candidate length and denoising depth are selected on validation prefixes and fixed for testing. Methods may execute repeated rounds, and only rounds completed by the deadline contribute:

\bar{N}(D)=\frac{1}{N}\sum_{i=1}^{N}\sum_{r}C_{i,r}\mathbf{1}[t^{\mathrm{finish}}_{i,r}\leq D].

This measures progress at equal elapsed time, counting verifier corrections and bonuses.

Refinement and deadline results.

At K=16, two DLM passes reduce c_{A} from 5.38 to 4.70 ms, whereas four and eight passes increase it to 6.84 and 11.83 ms. Cost per committed output token decreases by only 3.8%, because the verifier contributes a correction or bonus token in each full round. At 400 ms, DLM commits 97.1 tokens versus 84.6 for MTP; at 50 ms, MTP leads with 10.6 versus 8.6. Thus, refinement is beneficial over part of the latency range, but additional passes and very short budgets can reverse the advantage.

\WF@box

8.4 Structural and Geometric Diagnostics

Each drafter receives the same diagnostic windows: category transitions from COCO, crowded objects from Dense200, document regions from DocLayNet, and GUI targets from ScreenSpot-Pro. Candidate length is matched, both DLM variants use two refinement steps, and the bidirectional block-MTP control uses one pass. Diagnostic windows are selected from the common reference serialization before inspecting proposals. Raw format error detects illegal structural transitions, token types, or coordinate arity, using the parser state inherited from the prefix. Ending partway through an otherwise valid tuple at the candidate boundary is not itself a format error. Diagnostics precede grammar filtering, fallback, and verification.

The valid-box miss rate is

M_{\mathrm{box}}=\frac{\#\text{ unmatched complete, syntactically valid proposed boxes}}{\#\text{ complete, syntactically valid proposed boxes}}.

Matching is one-to-one against relevant remaining references, constrained by category or referring target and IoU at least 0.5. Unmatched duplicates, reversed corners, and zero-area boxes count as geometric misses. The numerator counts unmatched complete, syntactically valid proposed boxes; the denominator counts all complete, syntactically valid proposed boxes. The rate is undefined when this denominator is zero. It measures the fraction of eligible proposals that remain unmatched and is interpreted alongside format errors; it is not a ground-truth miss rate.

Verifier agreement, structural validity, and geometric accuracy measure different properties. Exact greedy verification produces the same sequence as the shared causal verifier; a finite deadline can yield different-length prefixes of that sequence. These diagnostics therefore assess draft reliability and rejected computation, rather than improved final localization under a fixed exact verifier.

Where bidirectional refinement helps.

Relative to causal MTP, bidirectional DLM reduces COCO format errors from 5.01% to 1.80%. Its valid-box miss rate decreases by 10.10 pp on Dense200 and 7.52 pp on DocLayNet, compared with only 0.46 pp on GUI. These differences are consistent with greater benefits for proposals containing several spatially related elements. The retrained DLM pair provides the attention control: bidirectional attention lowers the Dense200 rate from 21.74% to 14.79% at the same refinement depth. Bidirectional block-MTP also improves over causal MTP, so bidirectionality itself is not a uniquely diffusion-based advantage. Its one-pass comparison with two-step DLM changes objective and refinement cost; it does not isolate diffusion training at equal elapsed time. The separate deadline analysis provides the matched-time comparison.

Relation to the grounding formulation.

Together, the acceptance, cost, and deadline results support Finding 2.2: bidirectional diffusion with limited refinement makes draft generation more efficient than the tested causal MTP baseline over the reported operating range. The attention control supports joint access to visual evidence, while the refinement sweep identifies when an additional denoising pass repays its cost; neither establishes superiority over every MTP design. The geometric diagnostic measures proposal reliability before verification, separately from ScreenSpot accuracy, F1mIoU, and ground-truth recall. Exact verification preserves the shared causal model’s completed output.

\WF@box

9 Additional Experimental Analysis

\WF@box

9.1 Speed Reporting and Decoding Sensitivity

Metrics and scope.

TPF denotes committed output tokens per model forward; TPS denotes output tokens per second. Forward counts must include the causal cache-construction or verification passes required by the relevant schedule. The physical block size B in the decoding sweeps includes the known anchor, whereas candidate length K in the DLM–MTP comparison counts newly proposed tokens. The two quantities should not be identified. All quality values in the decoding sweeps are COCO F1mIoU, and their speed–accuracy trade-offs should not be extrapolated to a 30-benchmark quality average.

Reference operating points.

The AR reference is 49.6 TPS with 63.70 F1mIoU. At B=32, throughput is 223.8 TPS for linear self-speculation, 49.9 TPS for quadratic self-speculation, and 127.0 TPS for entropy guidance at \tau=0.8. The corresponding COCO quality is 62.96 for linear self-speculation and 60.72 for entropy guidance. Thus, the two main speedups are 223.8/49.6=4.51\times and 127.0/49.6=2.56\times, with quality gaps of 0.74 and 2.98 pp.

\WF@box

Interpreting the Decoding Sweeps

Linear versus quadratic self-speculation.

Linear TPF rises from 1.41 at B=4 to 2.21 at B=16, then changes little at B=32 (2.18). Quadratic TPF reaches 3.17 at both B=16 and B=32. At B=32, its 45.4% higher TPF nevertheless accompanies much lower TPS than the linear schedule. Fusing verification and proposal reduces forward-call count but increases the number of query tokens, illustrating why TPF alone is not a latency metric. From B=16 to B=32, linear throughput changes from 226.6 to 223.8 TPS, whereas quadratic throughput falls from 88.0 to 49.9 TPS.

Entropy threshold.

As \tau increases from 0.2 to 1.2, TPF increases from 1.02 to 1.60 and TPS from 107.3 to 148.7. COCO F1mIoU remains near 60 for \tau\leq 0.8, peaks at 60.72 at \tau=0.8, falls to 54.06 at \tau=1.0, and partly recovers to 56.14 at \tau=1.2. The non-monotonic quality curve supports a speed–accuracy trade-off rather than a universal monotonic accuracy law. Increasing \tau permits more uncertain tokens to be committed together; the observed quality drop is consistent with the risk of premature commitment.

Block size and decoding mode.

Entropy-guided TPF changes only from 1.18 to 1.24 over the tested block sizes, while its best COCO score is 61.58 at B=16. Linear self-speculation attains its best tested COCO score at B=32. Consequently, the B=32 operating point used for the headline comparison is not the maximum-quality choice for every decoding strategy. Block-size selection therefore depends on task quality and output length; the COCO sweep alone does not establish a universal optimum.

What exact verification preserves.

Self-speculation verifies against the converted model’s causal branch. Exact greedy verification preserves that branch’s outputs, not necessarily the outputs of the separately trained GroundAnything-VLM. The 0.74 pp gap to GroundAnything-VLM therefore does not imply that the verifier accepts mismatching tokens. Direct entropy-guided decoding has no AR acceptance test and remains the appropriate main-table setting for assessing diffusion generation itself.

\WF@box

9.2 Infrastructure Analysis

Figure 14 adds SGLang (Eager), CUDA Graph, and FP8 sequentially to each decoding mode. The final time per output token is 6.59 ms for Dynamic Decoding, 6.76 ms for Static Decoding, 7.15 ms for Entropy-Guided Decoding, and 3.73 ms for Self-Speculative Decoding. Relative to each mode’s Native PyTorch (Eager) implementation, the cumulative speedups are 3.48\times, 3.45\times, 3.51\times, and 4.06\times, respectively. The consistent reductions support the claim that parallel decoding requires corresponding execution support to realize practical acceleration.

These ratios compare implementations within a decoding mode. They are not multiplied by the separate 4.51\times algorithmic comparison, whose denominator is GroundAnything-VLM at a different reported operating point. The figure reports latency rather than a matched quality ablation for every infrastructure stage. In particular, it does not establish that FP8 preserves accuracy; selective quantization can change logits and thus commitment or verification decisions. The implementation-level distinction is described in Section 7.3.

\WF@box

9.3 Architecture and Training Ablations

Coordinate representation.

In Figure 15, replacing quantized coordinates with textual coordinates reduces the AR score from 73.03 to 71.60 and the DLM score from 61.66 to 55.98. The respective 1.43 and 5.68 pp drops support the compact coordinate vocabulary as a useful part of both variants, with a larger effect in the diffusion setting. The figure’s 0.25\times annotation concerns the textual-representation comparison; it should not be identified with the external SEED1.5-VL token-count ratios.

Visual encoder.

With MoonViT (Kimi-VL), the AR and DLM scores are 72.81 and 61.03; with Qwen3-ViT, they are 71.12 and 60.12. MoonViT-V2 is consistently strongest among these tested choices. These comparisons support the selected encoder within this training recipe; they do not establish a universal ranking across vision architectures. Section 9.3.1 analyzes the task-dependent value of multi-level visual injection at the coordinate-token readout; this is a separate design question from the encoder comparison.

Training-phase ablation.

The ablation labeled “Stage III” in Figure 15 gives 71.74 for GroundAnything-VLM and 45.04 for GroundAnything, drops of 1.29 and 16.62 pp. We retain the figure’s ablation label here; it is distinct from the Stage 1–4 pretraining/alignment numbering above. The larger DLM sensitivity indicates that the ablated training phase contributes substantially to useful grounding behavior after conversion. The experiment changes a complete phase and does not isolate individual reward components or optimization choices within it.

Comparison scope.

The full ablation scores (73.03/61.66) differ from the final 30-benchmark summary (72.42/61.75). We retain the ablation values and interpret only within-comparison changes, without replacing either set of scores or assuming identical aggregation.

\WF@box

9.3.1 Multi-level Visual Injection and Coordinate Readout

Connection to the encoder choice.

GroundAnything uses MoonViT-V2 with an input-level projector and no intermediate visual injection. DeepStack supplies additional visual evidence at intermediate language layers (Meng et al., 2024); Qwen3-VL adds three projected ViT feature levels to its first three language layers and reports fine-grained understanding gains (Bai et al., 2025a). The relevant question here is whether grounding adaptation makes this evidence useful to the coordinate-token readout under both causal and masked contexts. Our encoder ablation supports MoonViT-V2 under the current recipe but does not isolate the injection scheme.

Readout sensitivity.

Fix model parameters, image, query, and one decoding context: an AR prefix or a DLM block with fixed masks and visible tokens. For vectorized hidden states, write h_{\ell+1}=F_{\ell}(h_{\ell})+S_{\ell}u_{\ell} and q=g(h_{L}), where u_{\ell} contains projected visual features, S_{\ell} inserts them at visual-token positions, and q\in\mathbb{R}^{V} contains logits over a fixed vocabulary at one coordinate position. The readout g includes final normalization and the output head; \mathcal{I} indexes injection layers. With h_{0} fixed, changing injected features by \delta u_{\ell} gives

\displaystyle d \\ \displaystyle=\sum_{\ell\in\mathcal{I}}B_{\ell}\delta u_{\ell}+O(\lVert\delta u\rVert_{2}^{2}), \\ \displaystyle B_{\ell} \\ \displaystyle=Dg(h_{L})J_{L-1}\cdots J_{\ell+1}S_{\ell},\qquad J_{j}=DF_{j}(h_{j}).

Here d is the exact logit change, \delta u concatenates the feature changes, and empty products are identities; all derivatives are evaluated at the unchanged interface. The chain rule gives each path contribution; locally bounded second derivatives give the remainder. This is sensitivity relative to the specified interface, not prediction error relative to an ideal alignment.

Task alignment.

Let y denote the reference coordinate token and \mathcal{L}_{y}(q)=-\log p_{y} its cross-entropy loss. Write p=\operatorname{softmax}(q) and p^{\prime}=\operatorname{softmax}(q+d) for the original and updated token distributions. For the exact change d,

\displaystyle\mathcal{L}_{y}(q+d)-\mathcal{L}_{y}(q) \\ \displaystyle=\log\!\left(\sum_{v=1}^{V}p_{v}e^{d_{v}}\right)-d_{y} \\ \displaystyle=(p-e_{y})^{\top}d+D_{\mathrm{KL}}(p\|p^{\prime}),

where e_{y} is the one-hot target and D_{\mathrm{KL}}(p\|p^{\prime})=\sum_{v}p_{v}\log(p_{v}/p^{\prime}_{v}). The first equality follows from log-softmax; expanding the KL divergence gives the second. Additional evidence lowers coordinate loss precisely when its contribution opposing the current loss gradient exceeds the nonnegative KL remainder. Averaging over matched reference coordinate positions and decoding contexts gives the corresponding token-loss comparison. The loss identity is exact; substituting the linearized change from Equation 31 requires controlling its remainder.

If additional branches can be zeroed with all other components unchanged, the multi-injection model retains the single-interface model as a special case; path count alone cannot raise optimal task loss. The testable issue is whether finite grounding data and optimization learn useful contributions across causal and masked contexts. A controlled comparison should fix encoder and language-backbone initialization, coordinate vocabulary, data, and training budget, then compare input-only and multi-level interfaces with comparable capacity. Coordinate loss across adaptation budgets and dense/tiny-object localization would test this hypothesis; entropy-based commitment and final box metrics require separate evaluation beyond the fixed-context analysis. Disabling branches after training measures sensitivity only. Thus, the current ablation supports our encoder choice, while attributing its advantage to the absence of DeepStack requires an injection-specific experiment.

\WF@box

9.4 Output Token Efficiency and Generation Cost

Compact serialization.

Table 5 follows the output-length analysis of Jiang et al. (2026). Its SEED1.5-VL row is taken from that work. The GroundAnything variants use the same structured vocabulary and have identical reported output statistics: 7.6 tokens/box on COCO and 5.1 on Dense200. Compared with 148.8 and 74.5 tokens/box for SEED1.5-VL, these are approximately 19.6\times and 14.6\times lower token counts per box. The ratios reflect serialization length, not matched-hardware latency; differences in detected instance counts and model execution preclude reading them as speedups.

Generation cost across object counts.

Figure 16 groups outputs by predicted box count and reports average generation time alongside output-token count. Longer coordinate lists increase decoding work for both variants, while GroundAnything reduces generation time across the displayed ranges, with larger absolute savings for longer outputs. The token-efficiency table characterizes serialization length; this figure characterizes generation cost as the number of predicted instances grows. The speed–quality comparisons in Section 9.1 complement this output-length analysis.

\WF@box

10 Comprehensive Grounding Benchmark Results

We expand the selected main-text comparisons with complete baseline tables, threshold-specific localization metrics, and task-level analysis. GroundAnything uses Entropy-Guided Decoding throughout these tables; GroundAnything-VLM uses AR decoding. External results and unresolved evaluations retain their original qualifications.

\WF@box

Benchmark Suite and Metrics

Tasks and datasets.

The suite covers common and long-tailed detection on COCO (Lin et al., 2014) and LVIS (Gupta et al., 2019); dense and tiny-object detection on Dense200 (Jiang et al., 2026) and VisDrone (Zhu et al., 2018); referring grounding on RefCOCO, RefCOCO+, and RefCOCOg (Yu et al., 2016; Mao et al., 2016; Nagaraja et al., 2016); and object pointing on COCO, LVIS, Dense200, VisDrone, and RefCOCOg val/test. Spatial pointing uses RefSpatial (Zhou et al., 2026) and RoboSpatial (Song et al., 2025), while GUI grounding uses ScreenSpot-V2 (Wu et al., 2025), ScreenSpot-Pro (Li et al., 2025), and OSWorld-G (Xie et al., 2026). OCR covers HierText (Long et al., 2022), ICDAR2015 (Karatzas et al., 2015), TotalText (Ch’Ng & Chan, 2017), and SROIE (Huang et al., 2019); layout grounding uses DocLayNet (Pfitzmann et al., 2022) and M6Doc (Cheng et al., 2023); visual prompting uses FSC147 (Ranjan et al., 2021) and Dense200.

Metrics and aggregation.

Following the grounding evaluation convention of Jiang et al. (2026), box tasks report recall, precision, and F1 at IoU 0.50 and 0.95, together with their reported aggregates over IoU thresholds from 0.50 to 0.95. F1mIoU denotes the threshold-aggregated F1 score, not mean matched-box IoU. OCR additionally requires transcription agreement under the loose-match evaluation protocol and reports parse-error rates separately. Object pointing uses point-in-mask F1; spatial pointing uses point-in-mask accuracy, ScreenSpot uses action accuracy, and OSWorld-G uses exact accuracy. RefCOCO avg is the unweighted mean of the three family entries; RefSpatial avg averages Location and Placement. Reported benchmark entries are preserved, and missing metrics are not reconstructed from other scores.

Reporting conventions and external scores.

The notation in Section 4.2 applies throughout. R and P denote recall and precision; lower is better only for parse error. Model-name stars identify external-only rows, while entry-level stars identify individual substitutions or task-interface exceptions. The relevant captions specify their sources. In particular, selected detector, SEED1.5-VL, Molmo, spatial, and GUI references are taken from Jiang et al. (2026); GUI-Owl references follow Wang et al. (2026). Size groups refer to language-backbone configurations; the 7B understanding branch is used for MoT models and total language parameters for MoE models. Qwen3.7-Max, Kimi-K2.6, Kimi-K3, and GPT-6 Astra are grouped separately in the >1T category.

Interpreting unsupported or uncertain evaluations.

N/A means that the required task interface or a reliable prompt/parser combination was unavailable; it does not establish that a model intrinsically lacks the capability. The † marker identifies uncertain prompt/protocol alignment, including the affected Kimi, DeepSeek, and SenseNova-Vision evaluations. Repeated DeepSeek GUI attempts did not yield a suitable prompt/parser combination. Starred BAGEL and MiMo observations under suspected protocol incompatibility are retained descriptively. Where external scores replace unreproduced local results, only the matching model and available metrics are used: DeepSeek-VL2-Small scores are never substituted for the full model, and missing parse-error rates remain unreported. The following comparisons use the reported compatible entries.

\WF@box

10.1 Common and Long-tailed Object Detection

COCO.

GroundAnything-VLM achieves 63.70 F1mIoU, while direct diffusion retains 60.72, exceeding Rex-Omni by 4.44 pp. The threshold breakdown qualifies this result: GroundAnything’s F1 at IoU 0.95 is 22.39, below LocateAnything Hybrid’s 27.61, despite its higher aggregate. Thus, the overall gain does not imply uniformly tighter boxes at the strictest threshold.

LVIS.

GroundAnything reaches 52.59 F1mIoU, above Rex-Omni (46.74) and LocateAnything Fast (42.90), while GroundAnything-VLM reaches 56.63. This extends competitive grounding beyond common categories, although SenseNova-Vision remains stronger than the DLM variant at 56.12. The AR variant’s 75.46 F1 at IoU 0.50 versus 23.23 at IoU 0.95 also shows that rare-category coverage and very tight localization remain distinct challenges.