CoDA: Co-Evolution of Data Engines and Action Policies

When a simulator can generate endless robot training environments, how do you choose the ones that fix what the current policy gets wrong, and what training signal helps it learn from them?

Generative simulation has made scenes cheap, but a physically valid scene is not necessarily a useful one. Existing data engines vary scenes within hand-set ranges, stop once a scene passes physics checks, or spend budget by per-task success rate, none of which says why a policy fails or how to supervise it past that failure. Sparse success rewards also make each update slow to pay off.

Domain randomisation and uniform variants sample changes blindly; success-rate-guided generation allocates scenes by a single score; skill libraries store reusable code but mix what was learned about building scenes with what was learned about training. None keeps a structured record of why the policy failed.

CoDA runs a loop between a data engine and the policy around a shared memory with separate stores for scene construction, diagnosed failures and learning progress. A vision-language critic (GPT-5.5 by default) watches failed rollouts and proposes diagnoses; only those confirmed by control experiments enter memory. The memory then compiles generation programs that recreate the difficulty and training programs that help overcome it. Text2potential turns each diagnosis into staged subgoal potentials for learning the value of action chunks, a dense signal that, under standard reward-shaping assumptions, does not change the best policy. Measured progress decides the next round.

With pi0.5 as the policy, average out-of-distribution success across pick-place, articulated, put-inside and stack-or-insert families reached 27.6%, against 5.3% after fine-tuning on seed demonstrations, 7.9% for domain randomisation, 8.5% for uniform variants and 18.1% for success-rate-guided generation (52% relative gain); fifteen rounds took it to 42.1%. On LIBERO-Pro, success rose from 25% to 84% under position changes and from 1% to 76% under instruction changes, 7 and 9 points above ASPIRE, while standard LIBERO stayed at 97.4% (97.1% before). Text2potential reached 25% OOD success after 21,000 policy updates against 31,000 with sparse rewards, about 32% fewer. Swapping the critic for Claude Opus 5, InternVL3-78B or Qwen3-VL-235B cost 1.5 to 2.7 points. Memory gathered with pi0.5 raised OpenVLA-OFT from 3.8% to 21.1% with no new scenes. Cost: 7.4 million language-model tokens and 66 GPU-hours per 1,000 accepted environments.

This preprint evaluates simulated manipulation only; the authors state that transfer to physical robots, especially for contact-sensitive tasks, remains to be shown. The method depends on several foundation models for scene generation, diagnosis and potential compilation, so errors in their physical understanding flow into the data, and results may shift as model versions change. Absolute OOD success after five rounds is still only 27.6%, and each round adds simulation and inference cost that limits how many rounds and task families a team can afford.