Towards Spatial Perception for Heterogeneous Robot Collaboration in Subterranean Mining Environments

Can a lightweight underground explorer recognize operator-specified minerals with zero-shot vision-language segmentation and convert noisy detections into compact, geometrically actionable inspection targets for a second robot?

Abandoned and partially exploited underground mines are dangerous and expensive to re-enter, and no single robot can both cover long distances and carry the advanced payloads needed for close-range mineral characterization. The Explorer must send the Inspector more than a point cloud: it needs where minerals are, what they are, and how to observe them, but this must be produced onboard, without mine-specific training data, and compact enough for intermittent links. Existing dense open-vocabulary 3D maps and raw probability heatmaps do not provide ready-to-plan surface descriptions.

DARPA SubT systems such as CERBERUS, NeBula, CSIRO Data61, and CTU-CRAS-NORLAB demonstrated heterogeneous fleets for mapping and traversal, but treat the environment as geometry rather than semantically understood targets. Open-set 3D mapping systems including ConceptFusion, OpenScene, and ConceptGraphs create rich queryable maps but are heavy to compute, store, and transmit. Inspection planners often assume a hand-specified surface with orientation, lacking a compact semantic target interface from another robot.

The Explorer runs STAGE exploration and LiDAR-inertial odometry. CLIPSeg produces per-pixel confidence from natural-language prompts; a background threshold converts them to masks, and contours filtered by area yield instances. LiDAR points are projected into the camera frame and inherit mineral label, confidence, instance, and color. Per-instance DBSCAN filtering removes outliers, then detections are transformed to world frame and associated across views using intersection-over-minimum-volume on bounding boxes. Each persistent region is fitted with a plane by RANSAC; points are projected onto the plane, and a minimum-area bounding rectangle becomes an oriented polygon with surface normal. Polygons are organized into a three-level 3D scene graph and transmitted with the PCD map to the Inspector.

At the LTU SubT facility, per-point CLIPSeg confidence separated mineral detections (mean approximately 0.60) from host rock (mean approximately 0.02). During the run, approximately 38,000 raw mineral points were condensed into a few dozen persistent deposits and inspection polygons; the handover was roughly two orders of magnitude smaller than the raw semantic cloud. The same pipeline, with only the text prompt changed, detected magnesite in an active Grecian Magnesite mine. RANSAC plane fits were good for compact deposits, with inlier ratio decreasing for the largest, less-planar deposits, indicating those warrant multiple polygons. No baseline detection success rate or latency numbers are reported.

The largest mineral deposits are less planar and the authors note they are better represented by multiple polygons, so the single-plane abstraction has limits. Experiments used one Inspector, so multi-robot prioritization remains untested. No latency, power, bandwidth, or onboard compute measurements are reported, and no quantitative comparison against alternative open-set 3D representations or detection baselines is given. The evaluation is limited to two sites and two mineral types, and no code release or peer-reviewed venue is noted.