Multi-Teacher On-Policy Distillation for Flow-Based Vision-Language-Action Models
Huawei researchers show how to merge several specialist robot-control models into one generalist by letting the generalist practise and having the specialists correct it at every step of its action generation. In three simulation benchmarks the merged model gained 22 to 33 points of success over a standard fine-tune, though it still trails the specialists.
How can skills held by separate domain-specialised flow-based VLA policies be consolidated into one shared student policy without the student collapsing when it meets situations the teachers never showed it?
Fine-tuning one model on several domains at once (joint supervised fine-tuning) loses much of what each specialist knows. Copying teacher demonstrations does not help when the student makes its own mistakes, because a missed grasp creates observations and recovery needs the teachers' data never covered. For flow-based VLAs the mismatch exists twice: in the environment states the student visits and in the intermediate noise-to-action states its generator passes through.
Teacher-generated data (teacher-rollout SFT) and on-policy distillation that matches only the final action chunk (Action Loss OPD) address one level at most.
The paper's two-level on-policy distillation lets a shared pi-0.5 student act in each domain, then asks the frozen domain specialist for the velocity it would predict at every intermediate state along the student's own flow trajectory. Weighted losses from all domains update the student once per round, and the next round collects fresh environment experience and flow paths from the updated student. Teachers are used only during training.
After 80 rounds, Velocity Loss OPD reached 75.7% on LIBERO-40 (joint SFT 53.15%, teacher-rollout SFT 69.3%, action-loss OPD 66.2%), 60.6% on Meta-World MT50 (SFT 38.0%) and 72.9% on completing all five CALVIN instructions (SFT 39.5%), with average CALVIN sequence length 4.362 versus 3.235. Gains over teacher-rollout SFT grew with task length: 13.8 points on LIBERO-10 versus 1.6 to 5.8 on the shorter suites, and from 0.9 points on the first CALVIN instruction to 9.7 on all five.
All results are in simulation; no real robot was tested. The merged student still falls short of the specialists (LIBERO-40 75.7% versus 98.0%; MT50 60.6% versus 76.2%), and the authors note that the domains use inconsistent action conventions (the gripper sits in different output slots). The method needs frozen teachers per domain and many rollout rounds, so training cost scales with the number of domains.
Companies building one robot model for many customers, from Physical Intelligence to Chinese platform players such as Huawei itself, face exactly this consolidation problem as they fine-tune per site and want to fold improvements back. The result suggests distillation targets inside the generator matter more on long tasks. Relevance in 6 to 18 months for model training pipelines; real-robot evidence is still needed.