CrowdOcc: Monocular Semantic Scene Completion for Quadruped Robots in Crowded Indoor Environments
The authors present CrowdOcc, a preprint introducing an RGB-D dataset of 25.1K quadruped-captured frames in crowded indoor scenes and a monocular semantic scene completion model with surface-normal fusion and sparse human-centric 3D interaction. On held-out scenes it reaches 15.80 IoU, 11.40 mIoU, and 46.23 Human IoU, outperforming prior baselines and improving human, wall, and floor reconstruction. This matters because onboard single-camera dense 3D occupancy for low-viewpoint robots in cluttered human environments remains underexplored.
Can monocular RGB input predict dense 3D semantic occupancy for a quadruped robot in crowded indoor environments with strong human-scene occlusion and cross-scene generalization?
Existing semantic scene completion datasets and methods target mostly static indoor or autonomous-driving scenes, so they do not capture low-viewpoint quadruped observations with dense, moving humans. Under human-scene occlusion, depth-based lifting produces discontinuous wall and floor geometry, and human occupancy is often incomplete or spatially misplaced.
Earlier monocular SSC pipelines such as MonoScene, VoxFormer, NDC-Scene, and ISO lift 2D image features into a 3D volume and rely heavily on monocular depth priors, while datasets such as NYUv2, Occ-ScanNet, SemanticKITTI, and Occ3D are built for static indoor or vehicle-centric settings. These approaches lack targeted human-centric spatial reasoning and surface-orientation constraints, so they degrade in crowded unseen indoor scenes.
CrowdOcc collects synchronized RGB-D, LiDAR, and IMU data with a Unitree Go1 quadruped and constructs static occupancy from human-free sequences while deriving current-frame human occupancy using depth-guided refinement, uncertainty masks, and validity masks. The model uses frozen MoGe-2 depth and surface-normal estimates; NGSGF fuses masked surface-normal features with image features in local windows before depth-based lifting, and HCSI predicts human voxel responses, applies 3D non-maximum suppression and Top-K candidate selection, then performs self-attention among human candidates and cross-attention with local 3D scene features, writing residuals back only to selected voxels.
On the CrowdOcc test set of 3 unseen scenes and 3,914 frames, the full model achieves 15.80 IoU, 11.40 mIoU, and 46.23 Human IoU, exceeding ISO by +1.31 IoU, +0.68 mIoU, and +2.44 Human IoU; human recall is 37.02 versus 34.00 for NDC-Scene and 31.32 for ISO. Class IoUs include floor 21.12, wall 11.80, human 46.23, and door 4.64. Ablations from the ISO baseline at 14.49 IoU, 10.72 mIoU, and 43.79 Human IoU show MoGe-2 depth alone reaches 14.89 IoU, 11.25 mIoU, and 42.88 Human IoU; adding HCSI reaches 15.34 IoU, 11.26 mIoU, and 45.77 Human IoU; adding NGSGF reaches 15.80 IoU, 11.40 mIoU, and 46.23 Human IoU.
The paper does not state an explicit limitations section. Evident limits include evaluation on only three held-out indoor scenes from the same dataset; training used four RTX 4090-D GPUs with a total batch size of 4 and a frozen MoGe-2 ViT-L model, indicating substantial compute requirements. Annotations rely on a vision foundation model plus manual refinement and current-frame human depth, so label noise and incomplete human occupancy remain possible. No code release, real-time deployment results, or multi-robot validation are reported, and the dataset covers a single Unitree Go1 platform with 8 semantic classes.
This could benefit quadruped and indoor mobile robot perception stacks for logistics, inspection, hospital, retail, or service robots, especially quadruped robot makers such as Unitree. Real-time product use is likely 1–3 years away; first the method must be optimized for onboard inference, expanded to more sensors, robots, and buildings, and validated for failure modes before safety-critical deployment.
The full text is not republished here because the paper's license does not allow it. Read the original on arXiv.