InterEvolve: Test-Time Evolution of Reward Programs for Humanoid Loco-Manipulation
InterEvolve is a preprint framework that evolves staged reward programs at test time so a frozen humanoid controller can solve new loco-manipulation tasks without retraining. It raises average success from 34.6% for the best calibrated fixed reward to 86.5% across eight goal-conditioned task families and transfers evolved skills to a real Unitree G1 using onboard perception. This turns task adaptation into a search over objectives rather than policy retraining, with more test-time compute improving results.
Can a pre-trained whole-body humanoid controller solve unseen, contact-rich loco-manipulation tasks by evolving reward programs at test time, without changing the controller weights?
Humanoids in open-ended environments encounter tasks and scenes not present in training data. Common task interfaces such as motion references, goal states, and skill labels are hard to design for nuanced multi-stage contact-rich interactions, and conventional reward revision requires training a new policy for each candidate reward. As a result, adaptation is slow and much of the controller's existing motor competence remains unused.
Prior methods specify tasks through motion references, goal states, skill labels, or language-conditioned planners. Reward-design agents such as those in Eureka and related work generate rewards from language, but each candidate reward is realized by training a new policy. Body-only forward-backward behavioral foundation models like BFM-Zero can execute new rewards without retraining but observe only the body, so rewards that differ only in object goals can collapse into one behavior.
InterEvolve uses an object-aware forward-backward behavioral foundation model: frozen body networks from humanoid control are extended with trainable object residuals on human-object interaction data. Tasks are expressed as staged reward programs with completion conditions and tunable constants. A DeepSeek LLM agent revises program structure from parallel simulation feedback, CMA-ES tunes constants, and verified programs are stored in a skill library for reuse across tasks.
On held-out reference tracking, the evolved-reward controller reaches 72% success versus 66% for the ULTRA tracker, 60% for the same controller without evolution, and 8% for BFM-Zero; body/object errors are 19.80/24.92 cm. Across eight goal-conditioned task families, InterEvolve averages 86.5% success versus 34.6% for the best calibrated fixed agent-written program, 18.0% for a calibrated human reward, and 8.0% for an uncalibrated human reward, with 95.6% earned criteria. Ablations show single-stage programs drop success to 44.7%, removing CMA-ES drops to 51.6%, and removing targeted edits or multi-scenario evaluation drops to 68.9% and 68.4%. Evolution reaches 75.0% after one round and 86.5% after four. On composite tasks, the full skill library solves 8/10, 4/10, and 5/10 episodes for relocate, stack, and carry-place-kick respectively, versus 0/10, 0/10, and 1/10 with no library. Real-robot runs on a Unitree G1 use an onboard camera and FoundationPose for two evolved tasks.
Authors note that simulation rollouts dominate compute cost and that kicking remains the hardest task. The paper is a preprint with simulation-centric evaluation; real-robot transfer is demonstrated on only two tasks with one robot platform. Composite task success is around half, and the object set is limited to box-like objects and suitcases plus qualitative dexterous grasping. No peer-reviewed venue or code release is reported in the paper.
Humanoid robot makers such as Unitree and other whole-body manipulation vendors, warehouse and logistics automation providers, and manufacturers could use this to reprogram robots for new contact-rich tasks without retraining. In 1-3 years, controlled industrial pilots could adopt test-time reward search if perception and safety validation mature. Broad unstructured or home use is 3+ years out because sim-to-real transfer, compute cost, and certification remain open.
The full text is not republished here because the paper's license does not allow it. Read the original on arXiv.