CrossBFM: Distilling a Shared Latent Behavior Space Across Humanoid Embodiments

Whether the promptable behavior space learned for one humanoid can be handed to humanoids with different joint counts, body layouts and joint order by supervised distillation, instead of rerunning the expensive training on every new robot.

Behavior foundation models give a humanoid one policy steered by a single latent vector, which can stand for a motion to imitate, a pose to reach or a reward to maximise. Forward-Backward representations, as used in BFM-Zero, learn such a space well, but the authors put one training run at more than 100 GPU-hours per robot. Each run also produces its own latent space, unrelated to that of any other robot, so prompts, generators and planners built for one humanoid do not carry over to the next. As new humanoid bodies keep arriving, that cost and that fragmentation both compound.

The usual path is to rerun the full unsupervised Forward-Backward reinforcement learning pipeline for every embodiment. The cheaper alternative is a motion-tracking policy fed retargeted joint references, such as those trained with TWIST2; it tracks well but offers one mode of use and no shared space for goal or reward prompts. Cross-embodiment methods that condition one policy on a body description generalise to new robots, but the authors note they usually cover a single task such as velocity tracking.

CrossBFM freezes the BFM-Zero model pretrained on the Unitree G1 and keeps only its backward map, the part that turns a motion, pose or reward into a latent. Because each motion clip is retargeted to every robot on the same timeline, the G1 latent for each frame can serve as a regression label for the matching frame on any other robot, so the target side needs no simulator, reinforcement learning or adversarial training to build the space. Stage 1 trains one encoder for all target robots at once. It has no robot-specific weights: each robot's state is written into a fixed 163-dimensional input with 33 canonical joint slots, 8 key-body slots and a root block, missing slots are flagged, and a small pre-LayerNorm transformer (4 layers, width 256, 3.27 million parameters) is trained with a frame-wise cosine loss against the frozen 256-dimensional G1 latent. That takes under an hour on one consumer GPU for all robots together. Stage 2 trains a latent-conditioned tracking policy per robot with PPO in under ten hours on one RTX 4090; the policy sees only the shared latent and its own proprioception. For reward prompts, the method estimates the state distribution from each robot's retargeted data and rescales reward thresholds by percentile so that, for example, a hand-height target fits a smaller or larger body. A rectified-flow model trained over the shared latent adds a fourth way to prompt.

The three targets, the authors' in-house M3 (27 joints), Booster T1 (23) and Fourier N1 (23), differ from the 29-joint G1 and from one another in joints, bodies and topology. On LAFAN motion tracking, latent-conditioned policies reached mean joint errors of 0.2024 rad on M3, 0.1901 on T1 and 0.1391 on N1, against 0.1777, 0.1844 and 0.1362 rad for TWIST2-style policies fed joint references, a gap of at most 0.025 rad. Goal reaching moved between poses with no falls at errors of 0.2345, 0.1967 and 0.1512 rad. All 41 reward prompts designed for the G1 ran on the new robots, and the target-side method lifted the mean normalised return from 0.183 to 0.431 compared with reusing the G1's own state buffer, a 2.35-fold gain. Training the encoder on a quarter of the motion data raised M3 tracking error only from 0.2132 to 0.2241 rad, about 5%; at 10% it rose to 0.2706, against 0.4411 for random latents. In a leave-one-out test, an encoder trained on two robots and applied to the third closed 89.3% of the gap between random and fully trained latents on M3, 55.5% on N1 and 12.8% on T1, the smallest and most different body. On hardware, M3 tracked five clips at 0.2206 rad (0.2024 in simulation), reached nine poses at 0.2675 rad and scored 0.562 on three reward tasks, and both M3 and T1 ran latents produced by the flow model.

This is a preprint that the authors list as under review. Transfer depends on body similarity: the 12.8% result for T1 shows that a robot with no close relative in the training set still needs its own data. Every new robot needs good retargeted motion, and the experiments use only the LAFAN dataset, whose 40 motions cover eight behaviour types. In MEKAWIRE's reading, the latent policies trail joint-reference trackers on every robot in simulation and on M3 hardware, real-world numbers are given only for M3, and the work covers whole-body motion, not object manipulation. The latent space can only be as rich as the G1 model it inherits.