I2CD: Direct Image-to-Convex Decomposition for Simulation-Ready Collision Geometry

Can a frozen pretrained image-to-3D foundation model be adapted to directly emit convex-decomposition collision geometry from one RGB image, avoiding the slow and brittle mesh repair and approximate convex decomposition pipeline?

Robotics simulators and motion planners require convex collision geometry, but modern image-to-3D generative models output dense visual meshes with non-manifold edges, open boundaries, and stray fragments. Existing conversion workflows involving mesh repair, decimation, watertightening, and then approximate convex decomposition are slow, error-prone, and can be silently mishandled by engines that substitute a single convex hull.

Previous systems either used reconstruct-then-decompose pipelines, pairing image-to-3D generators such as Hunyuan3D-2 or SAM3D with V-HACD/CoACD or learned CvxNet/BSP-Net decomposers, or loaded raw generated meshes into engines. Classical decomposers degrade on non-manifold generated meshes and take seconds to tens of seconds per object; single-view CvxNet/BSP-Net are category-specific with no open-world weights; and raw meshes often get silently replaced by convex hulls.

I2CD keeps Hunyuan3D-2-mini's image-conditioned diffusion transformer and shape decoder frozen and trains only a lightweight cross-attention head. K learned convex-slot tokens attend to the frozen shape features and output halfplane normals, offsets, translation, and blending sharpness for K convex polytopes in the CvxNet representation. Training uses cached MolmoSpaces latents with class-balanced binary cross-entropy plus a convex-pair contrastive loss, under ten GPU-hours; K=H=12. The output is meshed per convex by marching cubes and loads directly into physics engines.

On 227 held-out OmniObject3D and Google Scanned Objects instances, I2CD reaches pooled IoU 65.7±13.2% (Omni 63.1, GSO 68.9), versus 62.8±19.0 for the best baseline HY3D+CoACD; end-to-end time is 0.51±0.04s vs 3.05–19.04s for pipelines. Surface metrics: F 15.6 vs best SAM3D+CoACD 19.2, CD 6.28 vs 5.87 mm. Cross-engine drop tests on 127 objects preserve I2CD geometry as delivered at 99.2% in MuJoCo/PyBullet/Genesis and 98.4% in Isaac Sim, while raw default meshes are preserved at only 40.2–40.5%; PhysX SDF preserves 100% but takes 3.75s and crashes on 6 objects, and PyBullet concave preserves 100% but registers zero object-object contacts. On a physical xArm7 in a 20-object scene, I2CD takes 11s to planner-ready geometry vs 328s for SAM3D+CoACD, MuJoCo load 0.9s vs 1.7, and achieves 85/100 pick-and-place success vs 90/100; auto-hull baseline takes 198s and succeeds 5/100. Ablations: balanced BCE only reaches 62.2 IoU, unbalanced BCE 54.7, MSE fails 127/127 empty; adding contrastive loss gives +1.6 IoU; K=6/H=12 reaches 65.5 but default K=H=12 is kept for equal hull budget.

Stated limitations: I2CD inherits frozen backbone errors such as missed thin structures and hallucinations, plus a 3.2 IoU gap between mesh-encoded and image-sampled latents; surface fidelity trails the best pipeline by 3.6 F-score points because of the K=12 predicted geometry, not export; convex slots group by proximity rather than semantics; and the real-robot study covers one tabletop quasi-static scene. Evident limitations: benchmark is 227 objects, robot experiments use one arm and one scene, code/models are not yet available at time of writing, and contact-rich tasks remain untested.