DiPLO: Self-distilled Diffusion Planning for Loco-manipulation

Abstract

Humanoid loco-manipulation is particularly challenging because it requires coordinating locomotion, balance, and object interaction, making it difficult to learn from limited demonstrations or from scratch through Reinforcement Learning (RL). We introduce DiPLO (self-distilled Diffusion Planning for Loco-manipulation), a framework that combines a self-distilled diffusion planner with a low-level whole-body tracking policy. Our planner is pretrained via supervised learning on motions derived from human-to-humanoid retargeting, while the tracker is trained via RL to execute them. DiPLO centers around two complementary post-training stages. First, we introduce Self-Distillation Diffusion Policy Optimisation (SD:DPO) to remove the planner’s dependence on strong task conditioning, which we identify as a crucial ingredient for effective pretraining, as downstream fine-tuning does not recover comparable performance when it is omitted.. We then fine-tune this distilled model with Diffusion-Augmented Proximal Policy Optimization (DA:PPO), a maximum-entropy RL method for diffusion models, to improve task success and align generated plans with the frozen tracker’s execution capabilities. Together, these stages enable motion generation through state-history and goal conditioning alone, enabling \sim 99\% task success in pick-carry-place tasks, compared to just 45\% for the pretrained model. The resulting planner also generates motions for unseen object goals, with further evaluations on kicking and pushing/pulling tasks. Finally, we deploy our pipeline on the hardware, using our DA:PPO planner-tracker on a Unitree G1, achieving repeated successful executions without any real-world fine-tuning. Video: https://youtu.be/rDrdnYQUYoM

Index Terms:

Humanoid Robots, Diffusion Models, Reinforcement Learning, Post-Training, Generalisation

I Introduction

Humanoid loco-manipulation requires coordinating locomotion, balance, and object interaction. Human motion data provides useful supervision for learning these behaviors, but collecting large datasets is costly [1, 2]. Our goal is to learn from a small dataset of retargeted motions while keeping task specification simple: the user provides the desired object position and orientation, and the robot determines how to reach it. This requires a planner that can generate trackable motions for different object goals, including goals beyond those represented in the demonstrated behaviours.

We introduce DiPLO (self-distilled Diffusion Planning for Loco-manipulation), which combines a diffusion motion planner with an RL-trained tracker. Starting from 29 retargeted robot–object motion sequences from an existing dataset [3] (which amounts to just 2.7 mins of data), we use trajectory optimization [4] and augmentation to generate dynamically feasible training motions for both components. During execution, the diffusion planner generates motion chunks and replans from the measured state as the tracker executes them. Such a pretrained planner still relies strongly on kinematic reference conditioning, which limits its ability to generate motions to new goals. To remove this dependence after pretraining, we propose Self-Distillation Diffusion Policy Optimization (SD:DPO). SD:DPO transfers guidance from a reference-conditioned teacher to a student conditioned only on state history and the object goal. The student learns to match the teacher’s denoising predictions at states encountered during environment rollouts with the tracker. These rollouts expose the student to the consequences of planning and tracking errors, extending its supervision beyond the contexts represented in the training data. SD:DPO thereby enables high survival without requiring a kinematic reference at deployment.

Trackable motions alone, however, do not ensure that the object reaches its goal. We therefore follow self-distillation with planner fine-tuning using DA:PPO [5], a maximum-entropy RL method for diffusion models. With the tracker fixed, DA:PPO improves task completion through closed-loop interaction while encouraging diversity in generated plans. SD:DPO and DA:PPO thus address complementary requirements: removing dependence on reference conditioning and improving task success. Our main contributions are:

• We introduce DiPLO, a framework that combines a self-distilled diffusion-based planner for humanoid loco-manipulation that jointly generates whole-body and object motion for execution by a single generalist RL tracking policy.
• We propose SD:DPO, which distills a reference-conditioned diffusion planner into a goal-conditioned planner, removing the need for kinematic reference conditioning at deployment. We further apply DA:PPO fine-tuning through closed-loop interaction with a fixed genralist tracker, improving object-goal completion while maintaining diverse, trackable plans.
• We provide extensive ablations across a diverse set of loco-manipulation tasks, demonstrating the effectiveness of DiPLO and its components compared with state-of-the-art baselines, together with zero-shot transfer to a real humanoid robot.

II Related Work

II-1 Learning from retargeted motion data

Retargeted motions provide supervision for learning whole-body tracking policies [6, 7, 8], but kinematic retargeting [9, 3] alone does not ensure physically feasible robot–object interaction. DynaRetarget [4] addresses this limitation through sampling-based trajectory optimization (SBTO), which refines a kinematic reference into a dynamically feasible motion. To scale demonstrations beyond those recorded, HumanoidMimicGen [10] augments loco-manipulation data by adapting object-centric skill segments of a source demonstration to new object poses. However, it plans kinematically and decouples locomotion from stationary manipulation, so dynamic feasibility is not enforced during generation. In contrast, we repurpose SBTO, originally proposed to refine individual demonstrations, for data augmentation: starting from perturbed initial conditions, we exploit the stochasticity of the sampling-based optimizer to synthesize locally diverse, dynamically feasible robot–object motions in which locomotion and manipulation remain coupled.

II-2 Diffusion planning for loco-manipulation

Diffusion Policy [11] models multimodal action sequences for robotic manipulation. In humanoid locomotion, however, maintaining balance while accomplishing the task motivates delegating execution to a low-level controller [12]. PDP [13] learns action generation from expert policies, while BeyondMimic [14] uses guided state–action diffusion to generate goal-directed behaviors. An alternative is to separate motion generation from execution: [15] combine a diffusion motion generator with a tracking policy for terrain locomotion. Recent works have shown the success of this separation in loco-manipulation settings [16, 17]. REFINE-DP [16] is closest to our approach in combining diffusion planning with RL fine-tuning for humanoid loco-manipulation. Its planner generates motion commands for separate upper- and lower-body controllers. DiPLO instead jointly plans whole-body and object motion for a single generalist tracking policy. Our post-training procedure also differs. We first use SD:DPO to remove the dependence on kinematic reference conditioning introduced during pretraining on retargeted motions. We then apply maximum-entropy fine-tuning with DA:PPO while keeping the tracker fixed. Finally, our generalization evaluation examines whether the same trained planner can reach different requested object positions and orientations that are absent from the retargeted dataset.

Our planner also treats rotational features through diffusion on \mathrm{SO}(3). Such processes have been developed for rotational alignment [18, 19], while DiffDock [20] combines diffusion over translational, rotational, and torsional degrees of freedom. We apply this geometric formulation to humanoid–object motion, evaluate its effect on closed-loop survival, and retain it during planner post-training.

II-3 Distillation in Robot Learning

Teacher–student distillation allows policies to benefit from information unavailable at deployment [21]. Following such a recipe, OmniH2O [8] distills a controller with privileged observations into a policy using deployable observations and sparse motion commands. Our objective concerns the high-level task specification: removing the diffusion planner’s dependence on strong task conditioning (kinematic reference trajectory).

The closest methodological precedent is Self-Distillation Policy Optimization (SDPO) [22], which uses a language model conditioned on additional feedback as a self-teacher and distills its next-token predictions into a student. Inspired by SDPO, we propose SD:DPO for diffusion motion planning. A teacher with access to privileged task-prior conditioning supervises the student’s denoising predictions on states encountered during closed-loop environment rollouts with the tracker, enabling reward-free credit assignment through denoising-step matching. This distills the guidance provided by the privileged task prior into a planner conditioned only on the robot’s state history and object goal, allowing the task prior to be removed at deployment. We evaluate both the resulting improvement in survival and the ability of the distilled planner to generate motions for object goals absent from the retargeted dataset.

II-4 Fine-tuning of diffusion planners

Diffusion-Policy Policy Optimization (DPPO) [23] enables reinforcement learning for diffusion policies by treating denoising transitions as decisions with tractable likelihoods and applying PPO to the resulting augmented process. REFINE-DP [16] applies diffusion-policy fine-tuning to humanoid loco-manipulation, considering both a fixed controller and alternating updates of the planner and controllers. However, DPPO promotes exploration by truncating the denoising process during training, which can lead to jittery motions. Such behavior is particularly undesirable for real-world humanoid control, where smooth and dynamically consistent motions are essential.

Diffusion-Augmented MDPs [5] instead provide a maximum-entropy formulation in which ratios of reverse and forward transition densities yield tractable entropy regularization at each denoising step. We use their PPO instantiation, DA:PPO, to fine-tune the planner after SD:DPO. Our setting combines Euclidean and \mathrm{SO}(3) diffusion and evaluates generated motions through a frozen generalist tracker. DA:PPO optimizes task completion after kinematic reference conditioning has been removed, while entropy regularization encourages diversity in the generated plans. We separately evaluate tracker adaptation to distinguish improvements due to planner fine-tuning from those obtained by adapting the tracker.

III Preliminaries

III-A Motion Planning and Tracking

We consider humanoid loco-manipulation with a high-level motion planner and a low-level tracking policy. The planner generates desired whole-body and object motion over a finite horizon, while the tracker executes this motion by adjusting joint actions based on the measured robot and object states. This hierarchical decomposition is motivated by the underactuated nature of humanoid robots and the need to maintain balance while tracking task-level motions.

Let s_{t} denote the measured robot–object motion state and g_{t} the desired object pose expressed relative to its current pose. At planning step t, the planner G_{\phi} receives the state history s_{t-h:t} over h steps and the goal g_{t}. It may additionally receive a kinematic reference s^{IK}_{\tau} obtained by inverse-kinematics (IK) retargeting. These inputs condition the generation of an n-frame motion chunk over \tau=t+1:t+n:

\displaystyle c_{t} \\ \displaystyle=(s_{t-h:t},g_{t}), \\ \displaystyle\hat{s}^{D}_{\tau} \\ \displaystyle\sim G_{\phi}(\cdot\mid c_{t},s^{IK}_{\tau}).

Here, \hat{s}^{D}_{\tau} denotes a generated motion chunk whereas s^{D}_{\tau} denotes a motion chunk from the dataset. When the kinematic reference is unavailable, its input is replaced by a null token \varnothing. We use z_{\tau} to collect the planner’s conditioning: z_{\tau}=(c_{t},s^{IK}_{\tau}) with the reference and z_{\tau}=c_{t} without it. Section IV-A specifies the motion state s_{t}.

The generated chunk provides the reference motion for the tracker. At control step j, the policy \pi_{\theta} receives robot and object observations o_{j}, its previous action a_{j-1}, and planned motion features \mathcal{R}_{j}(\hat{s}^{D}_{\tau}). Here \mathcal{R}_{j} extracts reference robot and object states at specified future offsets from j. The policy predicts a residual correction to the reference joint positions q_{j}^{\mathrm{ref}}. Denoting the scaled residual by a_{j}, the joint controller’s target is

\displaystyle a_{j} \\ \displaystyle\sim\pi_{\theta}\!\left(\cdot\mid o_{j},a_{j-1},\mathcal{R}_{j}(\hat{s}^{D}_{\tau})\right), \\ \displaystyle q_{j}^{\mathrm{PD}} \\ \displaystyle=q_{j}^{\mathrm{ref}}+a_{j}.

A proportional–derivative (PD) controller uses q_{j}^{\mathrm{PD}} as its joint-position target. The tracker executes the first m\leq n frames of the chunk, at control steps j=t,\ldots,t+m-1. The planner then generates a new chunk from the updated measured history. Replanning thus accounts for the motion actually executed, including tracking errors.

To train the tracker, reference motions are sampled from a dataset \mathcal{D} and executed in simulation [6, 14]. We train a policy via PPO [24] to learn to follow these motions by maximizing the expected tracking return:

J_{\mathrm{track}}(\theta)=\mathbb{E}_{s^{D}\sim\mathcal{D},\,\pi_{\theta}}\!\left[\sum_{j=0}^{T-1}\gamma_{\mathrm{track}}^{j}r_{\mathrm{track},j}\right].

Where the expectation is over sampled references and simulated rollouts over the whole episode length T, and \gamma_{\mathrm{track}} discounts the control steps. The reward r_{\mathrm{track},j} measures how closely the executed robot and object motions follow the reference, with additional contact and regularization terms. During this training stage, the reference comes from the dataset; during planner execution, it comes from the generated chunk.

III-B Conditional Diffusion Models

We use a conditional diffusion model as a planner to learn to generate motion by reversing a prescribed noising process [25, 26]. For training, a clean chunk s^{D,0}_{\tau}=s^{D}_{\tau} and its conditioning z_{\tau} are drawn from the dataset. The forward transitions s^{D,k}_{\tau}\sim q^{k}(\cdot\mid s^{D,k-1}_{\tau},z_{\tau}) progressively corrupt the motion over K diffusion levels. At the final level, the distribution approximately matches a prescribed diffusion prior p(\cdot\mid z_{\tau}). This prior supplies the initial noise for the denoising process.

Learned reverse transitions then progressively denoise it using the conditioning z_{\tau}:

\displaystyle\hat{s}^{D,K}_{\tau} \\ \displaystyle\sim p(\cdot\mid z_{\tau}), \\ \displaystyle\hat{s}^{D,k-1}_{\tau} \\ \displaystyle\sim p_{\phi}\!\left(\cdot\mid\hat{s}^{D,k}_{\tau},z_{\tau}\right),\quad k=K,\ldots,1.

The complete chain defines the planner G_{\phi}; each kernel p_{\phi} describes one denoising step which is parameterized by a neural network which receives the noisy chunk, diffusion level k, and conditioning z_{\tau}. The final sample \hat{s}^{D,0}_{\tau}=\hat{s}^{D}_{\tau} is passed to the tracking policy. Pretraining uses dataset chunks corrupted at sampled noise levels to construct supervised targets for the network. See Section IV-B for more details.

III-C Diffusion-Augmented MDPs and DA:PPO

The planner receives an environment reward when the tracker executes the first m frames of a generated chunk. To improve the planner with RL, this reward must guide the denoising decisions that produced the chunk. A Diffusion-Augmented Markov Decision Process (DA-MDP) [5] makes these decisions explicit: the diffusion augmented state \tilde{s}_{k} contains the planner’s conditioning z, the current noisy chunk \tilde{a}^{k}, and its diffusion level k; the action is the next, less noisy chunk \tilde{a}^{k-1}. Writing \tilde{a}^{k}=\hat{s}^{D,k}_{\tau} and suppressing the window subscript \tau, we obtain

\tilde{s}_{k}=(z,\tilde{a}^{k},k),\qquad\tilde{a}^{k-1}\sim p_{\phi}(\cdot\mid\tilde{s}_{k}).

Each action has the tractable density p_{\phi}(\tilde{a}^{k-1}\mid\tilde{s}_{k}), even though the marginal density of the completed chunk is intractable. For k>1, the environment remains unchanged and the next state contains the updated noisy chunk at level k-1. At k=1, the tracker executes the first m frames, the planner’s conditioning is updated, and a fresh prior sample starts the next denoising chain. For a successor state \tilde{s}^{\prime}=(z^{\prime},a^{\prime},k^{\prime}), these two cases give

\displaystyle p_{\mathrm{DA}}(\tilde{s}^{\prime}\mid\tilde{s}_{k},\tilde{a}^{k-1}) \\ \displaystyle=\begin{cases}\delta_{z}(z^{\prime})\,\delta_{\tilde{a}^{k-1}}(a^{\prime})\,\delta_{k-1}(k^{\prime}),&k>1,\\
P_{m}(z^{\prime}\mid z,\tilde{a}^{0})\,p(a^{\prime}\mid z^{\prime})\,\delta_{K}(k^{\prime}),&k=1.\end{cases}

Here \delta_{x} denotes a point mass at x, and P_{m} is the transition induced by executing the first m frames with a fixed tracker. The resulting observations update the history and goal direction in z^{\prime} and optionally the kinematic reference. Only the final denoising decision receives the reward r accumulated during execution, so \tilde{r}_{k}=\mathbf{1}_{\{k=1\}}r. The discount likewise applies only when execution advances the environment: \bar{\gamma}_{k}=\gamma for k=1 and one otherwise, where \gamma discounts planning steps.

The maximum-entropy formulation also regularizes the denoising decisions. Its action-value function Q_{\mathrm{DA}} and state-value function V_{\mathrm{DA}} satisfy the soft Bellman equations

\displaystyle Q_{\mathrm{DA}}(\tilde{s},\tilde{a}) \\ \displaystyle=\mathbb{E}\!\left[\tilde{r}+\bar{\gamma}_{k}V_{\mathrm{DA}}(\tilde{s}^{\prime})\mid\tilde{s},\tilde{a}\right], \\ \displaystyle V_{\mathrm{DA}}(\tilde{s}) \\ \displaystyle=\mathbb{E}_{\tilde{a}\sim p_{\phi}(\cdot\mid\tilde{s})}\!\left[Q_{\mathrm{DA}}(\tilde{s},\tilde{a})-\mathcal{T}\ell_{\phi}(\tilde{s},\tilde{a})\right],

with zero continuation value at terminal states. \ell_{\phi}(\tilde{s}_{k},\tilde{a}^{k-1})=\log\!\big[p_{\phi}(\tilde{a}^{k-1}\mid\tilde{s}_{k})/q_{\phi}^{k}(\tilde{a}^{k}\mid\tilde{a}^{k-1},z)\big] is a diffusion regularization term and given by the log ratio of the reverse transition density to the corresponding forward density. It replaces the action log-likelihood in the usual soft Bellman equations [27]. Thus, Q_{\mathrm{DA}} combines the current environment reward with the value of the next augmented state, while V_{\mathrm{DA}} also includes the current diffusion regularization term. These recursions propagate the reward from execution through all preceding denoising decisions. -\mathcal{T}\ell_{\phi} encourages entropy in the reverse process while favoring denoising paths that are plausible under forward noising. Retaining this diversity during fine-tuning encourages exploration whose temperature \mathcal{T} controls the strength of the regularization.

DA:PPO applies PPO [24] to these individual denoising decisions. Let i index transitions collected under p_{\phi_{\mathrm{old}}}, and let the critic V_{\psi} approximate V_{\mathrm{DA}}. Generalized advantage estimation [28] computes A_{i}^{\mathrm{old}} from the residuals \delta_{i}=\tilde{r}_{i}-\mathcal{T}\ell_{i}^{\mathrm{old}}+\bar{\gamma}_{i}V_{\psi}(\tilde{s}_{i+1})-V_{\psi}(\tilde{s}_{i}), where \ell_{i}^{\mathrm{old}}=\ell_{\phi_{\mathrm{old}}}(\tilde{s}_{i},\tilde{a}_{i}). The advantage therefore accounts for both execution rewards and the log-ratio contributions along the rollout.

Because the log ratio changes as the policy is updated, DA:PPO replaces its current-step contribution in the stored advantage: \widehat{A}_{i}(\phi)=\operatorname{sg}\!\left[A_{i}^{\mathrm{old}}-\mathcal{T}(\ell_{i}(\phi)-\ell_{i}^{\mathrm{old}})\right]. The stop-gradient operator \operatorname{sg} holds this quantity fixed when differentiating the policy loss. With the reverse-kernel importance ratio \rho_{i}=p_{\phi}(\tilde{a}_{i}\mid\tilde{s}_{i})/p_{\phi_{\mathrm{old}}}(\tilde{a}_{i}\mid\tilde{s}_{i}), the loss is

\displaystyle\mathcal{L}(\phi)={} \\ \displaystyle\mathbb{E}_{i}\big[-\min\!\big(\rho_{i}\widehat{A}_{i},\,\mathrm{clip}(\rho_{i},1-\epsilon,1+\epsilon)\widehat{A}_{i}\big) \\ \displaystyle-\mathcal{T}\operatorname{sg}(\chi_{i}\rho_{i})\log q_{i}(\phi)\big].

The first term is the clipped PPO objective, with clipping range set by \epsilon. The second accounts for optionally learned diffusion coefficients in the forward density q_{i}(\phi), evaluated on transition i; it contributes no gradient if diffusion coefficients are fixed. The mask \chi_{i} applies this correction only where the PPO objective is not clipped: it is zero when \widehat{A}_{i}\geq 0 and \rho_{i}>1+\epsilon, or when \widehat{A}_{i}<0 and \rho_{i}<1-\epsilon, and one otherwise.

IV Method

DiPLO learns a diffusion motion planner and a single generalist tracking policy from dynamically feasible motions constructed from retargeted data. After pretraining, we hold the tracker fixed and improve the planner in two stages. SD:DPO removes its dependence on kinematic reference conditioning; subsequent DA:PPO fine-tuning optimizes towards better task completion through closed-loop execution.

IV-A Robot–Object Motion Representation

The planner represents observed and generated motion through the same robot–object state:

s_{t}=\big(\underbrace{\Delta p^{xy}_{t},\ \Delta\psi_{t},\ h_{t},\ R_{t},\ q_{t}}_{\text{robot motion}},\underbrace{p^{o}_{t},\ R^{o}_{t}}_{\text{object motion}}\big).

The robot states comprise planar root displacement \Delta p^{xy}_{t}\in\mathbb{R}^{2}, heading change \Delta\psi_{t}\in\mathbb{R}, root height h_{t}, heading-invariant root orientation R_{t}\in\mathrm{SO}(3), and joint positions q_{t}\in\mathbb{R}^{29}. The object states are its position p^{o}_{t}\in\mathbb{R}^{3} and orientation R^{o}_{t}\in\mathrm{SO}(3). Planar displacements and object orientations are expressed in the current robot heading frame, while heights remain absolute. The representation therefore removes dependence on global planar position and heading. At the network input, each rotation is encoded in the continuous 6D representation, giving 48 features per time-frame.

IV-B Pretraining from Retargeted Motions

IV-B1 Training motions

Kinematic retargeting provides motion references but does not enforce the robot’s dynamics or contact constraints. We refine these references with sampling-based trajectory optimization (SBTO) [4]. By perturbing the initial conditions and varying the optimization seeds we generate several dynamically feasible variations of each motion. Additionally, we expand this dataset by mirroring robot and object motions across the sagittal plane and applying the box’s quarter-turn rotational symmetries (Fig. 3). Each training pair contains a kinematic reference s^{IK}_{\tau} and a feasible target motion s^{D}_{\tau}.

IV-B2 Motion planner

We use a Diffusion Transformer (DiT) [29] as the denoiser. It receives the noisy chunk, diffusion level k, state history and goal c_{t}, and kinematic reference s^{IK}_{\tau}. During pretraining, we replace the reference with the null token \varnothing in 50% of examples, so the network learns to denoise with either form of conditioning. Appendix -A provides additional architecture details.

We use separate diffusion processes for Euclidean features and rotations. Grouping these features across the chunk, we write s^{D,k}_{\tau}=(x^{k},R^{k}) for noised training data and \hat{s}^{D,k}_{\tau}=(\hat{x}^{k},\hat{R}^{k}) for a generated sample. Conditioning is omitted below for readability.

Euclidean features. After normalization, the Euclidean features follow discrete time variance-preserving (VP) diffusion with a cosine noise schedule [30]. Each forward step scales down the signal and adds Gaussian noise:

q(x^{k}\mid x^{k-1})=\mathcal{N}\!\left(\sqrt{\alpha_{k}}\,x^{k-1},\,\beta_{k}I\right),

where \alpha_{k}=1-\beta_{k} and \bar{\alpha}_{k}=\prod_{j=1}^{k}\alpha_{j}, with \bar{\alpha}_{0}=1. This choice preserves unit marginal variance for initially normalized features. For training, we sample any noise level directly as x^{k}=\sqrt{\bar{\alpha}_{k}}x^{0}+\sqrt{1-\bar{\alpha}_{k}}\epsilon, where \epsilon\sim\mathcal{N}(0,I). As \bar{\alpha}_{K} approaches zero, the terminal distribution approaches the diffusion prior \mathcal{N}(0,I).

Denoising requires the optimal score u_{k}^{*}(s^{D,k}_{\tau})=\nabla_{x^{k}}\log q_{k}(x^{k},R^{k}), which describes how the noisy data density changes with the Euclidean features [31]. A learned estimate u_{\phi,k} determines the mean of the DDPM reverse kernel:

\displaystyle p_{\phi}(\hat{x}^{k-1}\mid\hat{s}^{D,k}_{\tau}) \\ \displaystyle=\mathcal{N}(\mu_{\phi}^{k},\tilde{\beta}_{k}I), \\ \displaystyle\mu_{\phi}^{k} \\ \displaystyle=\frac{\hat{x}^{k}+\beta_{k}u_{\phi,k}(\hat{s}^{D,k}_{\tau})}{\sqrt{\alpha_{k}}},

where \tilde{\beta}_{k}=\beta_{k}(1-\bar{\alpha}_{k-1})/(1-\bar{\alpha}_{k}). The score term provides the denoising correction, and the variance controls the remaining noise. This Gaussian kernel approximates the finite-step reverse distribution.

Although the data score is unknown, the clean sample and sampled noise provide a computable training target. We use \mathbf{v}-prediction [32], with target \mathbf{v}^{k}=\sqrt{\bar{\alpha}_{k}}\epsilon-\sqrt{1-\bar{\alpha}_{k}}x^{0}. The network output \hat{\mathbf{v}}_{\phi}^{k} is converted to a score estimate by

u_{\phi,k}(\hat{s}^{D,k}_{\tau})=-\hat{x}^{k}-\sqrt{\frac{\bar{\alpha}_{k}}{1-\bar{\alpha}_{k}}}\,\hat{\mathbf{v}}_{\phi}^{k}.

Substituting this estimate into (11) gives the mean used for denoising. With squared-error regression, the optimal predictor is the conditional mean of \mathbf{v}^{k}, and this conversion recovers the data score at noise level k. We use smooth-L1 regression in practice (see Appendix -B for more details).

Rotational features. We use a discrete time analogue of VP diffusion on \mathrm{SO}(3), contracting root and object rotations towards a reference orientation A\in\mathrm{SO}(3) while increasing rotational noise. We set A=I for the root and use the current measured object orientation for the object. Each reference remains fixed throughout the chunk’s diffusion process.

To express these operations, let v=(v_{1},v_{2},v_{3})^{\top}\in\mathbb{R}^{3} be a rotation vector whose direction defines the rotation axis and whose norm is the rotation angle. The map v\mapsto[v]_{\times}\in\mathfrak{so}(3) forms the corresponding skew-symmetric matrix, and its corresponding rotation matrix can be computed with \mathrm{Exp}(v)=\exp([v]_{\times})\in\mathrm{SO}(3). Conversely, \mathrm{Log}(R)\in\mathbb{R}^{3} inverts this operation back to its rotation vector and the corresponding matrix logarithm is defined as \log R=[\mathrm{Log}(R)]_{\times}.

We model rotational noise with the isotropic \mathcal{IG}_{\mathrm{SO}(3)} distribution [18, 19]. A perturbation of an orientation M is obtained by sampling a noise rotation E\sim\mathcal{IG}_{\mathrm{SO}(3)}(I,\sigma^{2}) centred at the identity and composing the two rotations as R=ME. The resulting distribution is centred at M, with spread controlled by \sigma. The rotational diffusion prior is given by p(R^{K})=\mathcal{IG}_{\mathrm{SO}(3)}(A,\sigma_{K}^{2}) and it depends on the observed state through the current state reference A.

For training, a clean rotation R^{0} is corrupted directly at level k by first computing the contracted centre M^{k} and then adding a sampled noise rotation:

\displaystyle M^{k} \\ \displaystyle=A\,\mathrm{Exp}\!\left(\sqrt{\bar{\alpha}_{k}}\,\mathrm{Log}(A^{\top}R^{0})\right), \\ \displaystyle E^{k} \\ \displaystyle\sim\mathcal{IG}_{\mathrm{SO}(3)}(I,\sigma_{k}^{2}),\qquad R^{k}=M^{k}E^{k},

where \sigma_{k}^{2}=\sigma_{\max}^{2}(1-\bar{\alpha}_{k}). To interpret the first line, note that R^{0}=A(A^{\top}R^{0}): A^{\top}R^{0} is the rotation relative to A, and \mathrm{Log} expresses it as an axis–angle vector. Multiplication by \sqrt{\bar{\alpha}_{k}} reduces its angle. The exponential map converts the scaled vector back to a rotation matrix, which is composed with A to obtain M^{k}. The second line adds noise by composing M^{k} with E^{k}. As \bar{\alpha}_{k} approaches zero, the centre approaches A and the noise scale approaches \sigma_{\max}, yielding the prescribed prior in the limit.

For each rotation and frame, the network outputs \hat{\epsilon}_{\phi}^{R,k}\in\mathbb{R}^{3}, trained to predict the standardized angular noise \epsilon^{R,k}=\mathrm{Log}((M^{k})^{\top}R^{k})/\sigma_{k}. Scaling the prediction by -\sigma_{k} gives the axis–angle correction that cancels the predicted noise. The exponential map converts this vector, through its skew-symmetric matrix in \mathfrak{so}(3), into a rotation matrix. Applying this correction gives, before clipping,

\hat{M}^{k}=\hat{R}^{k}\,\mathrm{Exp}(-\sigma_{k}\hat{\epsilon}_{\phi}^{R,k}).

This estimates the contracted centre M^{k}. To recover an estimate of the clean rotation relative to A, we divide \mathrm{Log}(A^{\top}\hat{M}^{k}) by \sqrt{\bar{\alpha}_{k}}. We then combine this vector with the current noisy rotation in the same coordinates, using the DDPM coefficients:

\displaystyle p_{\phi}(\hat{R}^{k-1}\mid\hat{s}^{D,k}_{\tau}) \\ \displaystyle=\mathcal{IG}_{\mathrm{SO}(3)}(\mu^{R,k},\sigma_{\max}^{2}\tilde{\beta}_{k}), \\ \displaystyle\mu^{R,k} \\ \displaystyle=A\,\mathrm{Exp}\!\Big(\frac{a_{k}}{\sqrt{\bar{\alpha}_{k}}}\mathrm{Log}(A^{\top}\hat{M}^{k}) \\ \displaystyle+b_{k}\,\mathrm{Log}(A^{\top}\hat{R}^{k})\Big),

where a_{k}=\sqrt{\bar{\alpha}_{k-1}}\beta_{k}/(1-\bar{\alpha}_{k}) and b_{k}=\sqrt{\alpha_{k}}(1-\bar{\alpha}_{k-1})/(1-\bar{\alpha}_{k}). The resulting kernel is an approximation constructed in axis–angle coordinates. Sampling a reverse step amounts to multiplying its centre \mu^{R,k} by a new identity-centred noise rotation. More details are provided in Appendix -B. At deployment, we use the predicted reverse means for both feature groups without adding noise at each step; generation remains stochastic through the initial prior sample.

IV-B3 Generalist tracking policy

We train a single tracking policy on the feasible robot–object motions, optimizing (3) with PPO in mjlab [33]. The tracker operates at 50 Hz using the residual joint targets in (2). Its reference observations include motion features at future offsets of \{1,2,4,8\} frames. Appendix -D specifies the observations and motion-sampling strategy; the rewards and termination conditions are defined in Appendices -F and -E.

IV-C Removing Task Conditioning with SD:DPO

Although pretraining includes examples without reference conditioning through masking, removing the reference substantially reduces closed-loop survival (Section V-B) since planning and tracking errors can lead the system into states poorly represented in the training data. Thus, at such states, the data provides limited guidance on how to continue the motion. To mitigate this problem we propose Self-Distillation Diffusion Policy Optimization (SD:DPO) to supply this supervision, where a student that receives only state history and the object goal learns to match the task-prior-conditioned teacher at states reached during environment rollout.

The student G_{\phi} receives c_{t} and the null reference token \varnothing, whereas the teacher G_{\bar{\phi}} additionally receives the kinematic reference s^{IK}_{\tau}. Both are initialized from the pretrained planner. At each planning step, both generate a chunk. We choose either proposal with probability 1/2 and execute its first m frames with the frozen tracker. The resulting measured states update the history used to generate the next chunk.

At every planning step, we store the student’s proposal, its starting context, and the corresponding kinematic reference in a buffer \mathcal{B}, regardless of which proposal was executed. Failed rollouts terminate and reset, but all previously collected proposals remain in the buffer, including those associated with failure. The tracker supplies the execution contexts used for learning while the teacher supplies the denoising targets.

Let \phi_{\mathrm{old}} denote the student weights held fixed during collection. For each training update, we draw a stored proposal and corrupt it at a uniformly sampled diffusion level:

\displaystyle\hat{s}^{D,0}_{\tau} \\ \displaystyle\sim G_{\phi_{\mathrm{old}}}(\cdot\mid c_{t},\varnothing),\qquad k\sim\mathcal{U}\{1,\ldots,K\}, \\ \displaystyle\hat{s}^{D,k}_{\tau} \\ \displaystyle\sim q_{k}\bigl(\cdot\mid\operatorname{sg}(\hat{s}^{D,0}_{\tau}),c_{t}\bigr).

Here q_{k} denotes direct corruption to level k using the processes in Section IV-B. Teacher and student receive the same noisy chunk, diffusion level, and stored context. The teacher also receives the kinematic reference, which provides guidance for predicting a continuation from that context. We transfer this guidance to the student by matching their Euclidean and rotational denoising predictions:

\displaystyle\mathcal{L}_{\mathrm{distill}}(\phi)={} \\ \displaystyle\mathbb{E}_{\mathcal{B},k,q}\Big[\operatorname{MSE}\!\bigl(\hat{\mathbf{v}}^{k,\varnothing}_{\phi},\operatorname{sg}(\hat{\mathbf{v}}^{k,IK}_{\bar{\phi}})\bigr) \\ \displaystyle+\operatorname{MSE}\!\bigl(\hat{\epsilon}^{R,k,\varnothing}_{\phi},\operatorname{sg}(\hat{\epsilon}^{R,k,IK}_{\bar{\phi}})\bigr)\Big].

The superscripts \varnothing and IK indicate absent and present reference conditioning, respectively and each mean-squared error (MSE) averages the squared prediction differences over frames and features. The expectation is over buffered proposals, diffusion levels, and sampled noise. Importantly, gradients pass only through the student’s predictions whereas teacher targets, buffered proposals, and noisy inputs are held fixed, so learning does not differentiate through collection, simulation, or noise sampling.

We train with (17) alongside the original data-based pretraining loss and an anchor loss. The anchor loss compares the student’s reference-conditioned predictions with those of a frozen pretrained copy on the same noisy chunk. It penalizes deviations from the pretrained model, restricting changes to the reference-conditioned behavior. The teacher is updated through an exponential moving average of the student weights, while the pretrained copy remains fixed. Further details are provided in Appendix -J.

IV-D Improving Task Completion with DA:PPO

SD:DPO improves generation without a kinematic reference, but producing trackable motion does not by itself ensure that the object reaches its goal. We therefore fine-tune the distilled planner with DA:PPO. The frozen tracker executes its plans in simulation, and the resulting task rewards guide the planner’s updates. The planner continues to receive only state history and the object goal, with the reference token set to \varnothing.

We instantiate DA-MDPs as explained in Section III-C with planner state z_{\tau}=c_{t} and noisy action \tilde{a}_{\tau}^{k}=\hat{s}^{D,k}_{\tau}. The reward r_{\tau} collected during execution of the first m frames is assigned to the final denoising decision. The Bellman recursion then propagates this reward through the decisions that produced the chunk. If the episode continues, the planner generates a new chunk over \tau^{\prime}=t+m+1:t+m+n from the updated measured history.

We optimize (8) to increase the execution return while retaining the log-ratio regularization. The tracker remains fixed throughout. Details on hyperparameters and rewards are provided in Appendices -G and -H. IK-retargeted references can optionally be used to compute training rewards, but do not need to be used to condition the planner (see Sec. V-E).

V Experiments and Results

We evaluate our pipeline on humanoid loco-manipulation tasks and organise the evaluations around the design choices. In our experiments, we report the Survival Rate which we define as the fraction of rollouts where the robot neither falls nor drops the object. Additionally we define the Task success as the fraction of survived rollouts whose realized final object pose lies within 15 cm and 30^{\circ} of the goal position the motion generator was conditioned on. We also report MPKPE, which is the mean per-keypoint position error of the executed robot motion against the generated motion from the diffusion model, and Obj. pos. / Obj. ori. the corresponding object position errors aggregated over the whole trajectory. Both metrics measure the trackability of the generated motions of the robot and object. All simulation results are reported as mean \pm standard error of the mean (SEM) over distinct reference motions. Each motion is rolled out 100 times and averaged computing the error over different motions.

V-A Choice of Pretraining Data Augmentations

To identify important design choices during pretraining, we evaluate the individual and combined contributions of SBTO motion variations and symmetry augmentation in our pipeline (see Fig. 3). To test this, we pretrain diffusion models under the same training budget under four different settings: (i) using datasets containing a single motion per demonstration, (ii) additionally using symmetry augmentations, (iii) augmenting each motion with SBTO variations and (iv) combining each motion with both augmentations. SBTO variations are generated by repeating trajectory optimization with slightly perturbed initial conditions, producing dynamically feasible realizations of each demonstration with slight diversity. The resulting pretrained models are then evaluated in closed-loop simulation with the same pretrained tracking policy.

As shown in Table I for the pretrained models, SBTO variations improve survival more than symmetry augmentation alone, while combining both yields the highest survival rate. These results indicate that diversity among dynamically feasible motions yields large individual benefit, with symmetries providing an additional improvement. Table I additionally shows that this advantage carries through post-training: using both augmentations, the survival rate reaches 100\%, while substantially improving tracking performance and task success over not using any augmentations or just using symmetry augmentation. We also observe that, compared to DPPO [23]-style updates, DA:PPO [5] yields a higher survival rate and better-tracking metrics, indicating that the maximum-entropy extension of DPPO to DA:PPO indeed improves performance. Furthermore, the results show that the pretrained planner achieves 51.9% task success, while fine-tuning with DA:PPO improves this result to 96.4%. The trackability of the IK-retargeted/SBTO motions are lower (see last row in Table I) which yield a survival rate of 93.5\% and Task success of 53.0\%. Thus, the same pretrained tracking policy yields substantially higher task success when supplied with improved references from the finetuned planner.

V-B Importance of Task Conditioning During Pretraining

During pretraining, we find that using the IK-retargeted robot and object motion as conditioning for the diffusion planner, which provides a prior on how to solve the task, is important to obtain strong results after pretraining. To ablate this choice, we pretrain seven diffusion models on different task prior choices. The task priors range from using the full IK-retargeted motion down to not using retargeted conditioning at all. Each model is evaluated in closed-loop simulation using the same pretrained tracking policy.

As shown in Fig. 4, the strongest performance is obtained when using the IK-retargeted robot and object motion as task conditioning as derived from OmniRetarget dataset [3]. We also directly compare against using the corresponding human pose and object pose from the OMOMO [2] dataset without any retargeting and observe a roughly 9\% drop in survival rate. Using sparse end-effector and object motion references (only every 5 frames), we do not observe a significant drop in performance, compared to using dense end-effector and object motion conditioning. Removing the remaining priors lowers the closed-loop survival rate further, to {\sim}70\% with only the object motion, and to {\sim}45\% with no task priors. These results suggest that, in our limited data regime, strong task priors are required during pretraining.

Later, in Section V-G, we show that SD:DPO post-training removes the need for a strong task prior at deployment, enabling goal-conditioned motion generation without any performance sacrifice.

V-C SO(3) Diffusion Improves Performance

Standard diffusion models add Gaussian noise in Euclidean space and ignore the geometry of the manifold , whereas the root and object orientations in our motion representation lie on \mathrm{SO}(3). We therefore ablate on the choice of diffusion process for the rotational features. Table II compares two pretrained generators that differ only in this choice: one uses a VP-\mathrm{SO}(3) diffusion process as described in Section IV-B, the other one uses a VP-diffusion process in Euclidean space. Our results show that diffusing on \mathrm{SO}(3) raises the closed-loop survival rate from 85.9\% to 96.5\%, while also improving across other tracking metrics.

V-D Motion-Level vs. Action-Level Diffusion

Our pipeline separates motion generation from feedback control: a diffusion planner proposes whole-body and object trajectories, and an RL-trained tracking policy executes them. Alternatively, diffusion models can predict actions directly, either alone [13] or jointly with future states [34, 14].

We train these action-level baselines by behaviour cloning on state–action data collected by tracking the SBTO- and symmetry-augmented motions. Although this dataset includes perturbed states and corrective actions, the learned feedback behaviour depends on the coverage of a fixed dataset. Our planner also learns from a limited motion corpus, but its tracker learns via RL in simulation, where it can learn to respond to disturbances and errors encountered during its own execution. This separation allows us to learn motion structure from the available data while optimizing feedback control through interaction.

The learning objectives also favour different behaviours. From a maximum-likelihood perspective, generative learning corresponds to forward-KL fitting, which encourages coverage of the data distribution. Under imperfect approximation, a model may cover multiple action modes while also assigning probability to low-density regions between them [35]. For an action-level model, such samples become joint commands and may disrupt balance or contact.

Maximum-entropy RL instead admits a reverse-KL formulation with a reward-weighted target distribution [36]. This objective discourages probability mass on low-return behaviours and can favour concentrating on a subset of successful behaviours. Although this mode-seeking tendency can hinder exploration of alternative behaviours, it favours reliable execution when failures receive low reward. This motivates combining a generative motion planner with an RL-trained tracker: the planner represents motion diversity, while the tracker learns to select actions according to their consequences in the environment.

We therefore compare motion-level and action-level diffusion using the same underlying motion corpus and evaluation tasks to assess whether separating planning from feedback control improves closed-loop survival and task success.

We consider two baselines that output robot joint PD targets directly, with no tracker in the loop: an action-only diffusion policy as in PDP [13], and a state–action diffusion model as in BeyondMimic [14] for humanoid control. Both are trained via behaviour cloning on rollouts collected by our pretrained generalist tracking policy over the SBTO- and symmetry-augmented data corpus, and both replan at every control step (50 Hz), executing only the first action. In this comparison all approaches receive the same task information which consist of states, the goal object pose and the IK-retargeted robot and object motion. We also compare to [14], where a state-action diffusion model is instead an unconditioned prior that is steered at inference. We adopt this by steering the diffusion model towards the IK-retargeted motion with keyframe in-betweening. See Appendix -K for more details.

The results are presented in Table III, where the PDP-style policy is the strongest action-diffusion baseline, with a survival rate of 87.5\% compared to 96.5\% for our pretrained diffusion planner combined with the tracker. Our approach also achieves significantly better task success. State-action diffusion is overall the weakest baseline: the steered prior reaches a survival rate of only 58.2\% and a task success of 8.2\%. On both metrics, our hierarchical approach, a state-only diffusion model for motion generation paired with a high-frequency tracking policy, performs best, even before fine-tuning.

Furthermore, inference of the diffusion model on our hardware setup, with the tracker running in a parallel process, runs at 12.5 Hz. Although action/state-action diffusion models can plan actions over a longer horizon, the generated actions operate open-loop until the next plan arrives, making it extremely challenging for legged robots to stabilise at such low frequencies, compared to having a separate high-frequency low-level tracking policy as we do.

V-E RL fine-tuning reward design

Next, we investigate how the design of the rewards for diffusion RL finetuning with DA:PPO affects the performance of the overall pipeline. A natural choice is a dense reward that encourages resulting rollouts to stay close to the IK-retargeted motion. Such a reward is similar to the whole-body and object-tracking rewards used to train the tracking policy, but it can only be computed where a retargeted demonstration is available. An alternative approach is a simpler goal-driven reward, which does not need a corresponding kinematic trajectory.

In Table IV, we compare finetuning with DA:PPO as described in Section IV-D, starting from the same pretrained diffusion motion generator using the two different rewards described above: r_{\text{kin}}, which rewards staying close to the dense IK-retargeted motion at every step, and r_{\text{goal}}, which rewards approaching the goal combined with a survival bonus (see Appendix -G for details). Both reward sets improve survival and task success relative to the pretrained model. r_{\text{kin}} furthermore reduces all reported tracking errors with respect to generated motions over using r_{\text{goal}}. r_{\text{goal}} achieves 92.9% task success, compared with 96.4% for using r_{\text{kin}}, but increases MPKPE and other tracking errors. Although, we stick to using r_{\text{kin}} for our experiments, r_{\text{goal}} could also be used for finetuning towards unseen goals using a curriculum and where demonstrations are unavailable.

V-F Effect of Maximum-Entropy Fine-Tuning

In Table I, we also compare the finetuning capabilities of DA:PPO against a DPPO-style update, which ablates on using DA:PPO without temperature (see Section IV-D). The original DPPO [23] uses additional tricks, such as injecting more diffusion noise for exploration, which we removed because they led to unstable and untrackable behaviour. While the DPPO-style update improves overall metrics compared to the pretrained model, we find that using the temperature in DA:PPO updates improves the survival rate to 100\% and significantly improves all other metrics when finetuned under identical settings and hyperparameters. These improvements can be attributed to the maximum entropy RL formulation of DA:PPO which encourages exploration while the DPPO-style objective suffers under severe mode collapse [5].

V-G Removing the need for strong task conditioning

We have demonstrated in Sec. V-B that pretraining without any task prior yields significantly lower task performance compared to using the task priors. However, this might limit the applicability of our method as it requires IK retargeted references from human motions, which is not always available. To mitigate this we show in Table V that SD:DPO (see Sec. IV-C) enables us to achieve high survival rates and high task success, without relying on the IK-retargeted task prior.

Table V separates the effects of self-distillation and subsequent finetuning via diffusion-RL. For pick-place, SD:DPO raises survival of the Student model from 57% to 100%. While finetuning the pretrained Student via DA:PPO increases the task success to 92%, further finetuning of the self-distilled Student increases the task success to \sim99%. We also show that the no-prior model cannot improve significantly via finetuning compared to finetuning the self-distilled student, highlighting the importance of using the task conditioning during pretraining.

Additionally, we reproduce our experiments on an alternate set of challenging motions that involve kicking the object or pushing/pulling towards a target goal pose. For each of these, we pretrain a diffusion motion generator only on the corresponding motions, following Section IV-B, then post-train via self-distillation (SD:DPO) to eliminate conditioning on task priors, and finally post-train for fine-tuning to further improve task performance. We observe trends similar to those when using only pick-carry-place task motions, confirming that our method is general and can also be applied to very different tasks.

By removing dependence on task conditioning, we can also plan motions using new goal poses that are unseen in our dataset. To measure this generalisation systematically, in Figure 5, we place goals for the object on a polar grid around the robot’s start at distances ranging from 0.5 to 5 m, and each position has 0^{\circ}/90^{\circ}/180^{\circ}/270^{\circ} rotations of the goal. The farthest goal in our demonstration set is at 1.9 from the origin (marked with dashed circle). We find that we can achieve these goals with \sim100% survival rates and a high task success rate of 88\% on this grid test for generalisation. Further plots in Appendix -M show the same evaluation for the pretrained Student and Student + SD:DPO, along with example robot trajectories for individual new goals.

V-H Planner Finetuning beats Tracker Finetuning

Instead of finetuning the diffusion planner with DA:PPO to close the plan–execution gap, it is also possible to finetune the tracking policy towards better execution of the diffusion plan, as done in [15]. As we show in this section, however, we observe that diffusion planner finetuning leads to better task success and better survival rates. To ablate on this we finetune the pretrained tracking policy while keeping the motion generator frozen (see Appendix -I). We fine-tune the tracking policy with and without the goal term, which yield similar results. We also ablate only finetuning the tracker on the pretrained diffusion motion generator against finetuning the tracker using the finetuned diffusion motion generator.

As shown in Table VI, with the pretrained tracker fixed, planner fine-tuning increases task success from 51% to 96%. Fine-tuning the tracker with the pretrained planner reaches at most 89%. These results show that finetuning the diffusion planner alone already recovers large improvement in task success, with the tracker left unchanged.

V-I Real-World Deployment

We evaluate our diffusion motion generator and tracking policy on a 29-DoF Unitree G1 humanoid performing pick-carry-place tasks as shown in Fig. 1. The motion generator runs on the GPU in a separate process, while the tracking executes on the CPU through ONNX Runtime. Joint encoders and the onboard IMU provide proprioceptive states, whereas a motion-capture system tracks the robot base and object, providing robot position and object pose for closed-loop execution.

The tracking policy runs at 50 Hz and outputs residual joint positions that are added to the generated reference to form joint-level PD targets, which are tracked by a PD controller running at 200 Hz. The generator uses 10 denoising steps to predict a chunk of 64 robot–object motion, corresponding to a 1.28 s horizon. The tracker continues executing the current chunk from the generator during inference, until a new chunk arrives from the generator process. In our setup, the motion generator can run at a maximum frequency of \sim 12.5 Hz, accounting for communication latency and diffusion sampling.

Table VII reports two groups of experiments: (a) ten repeated executions of the same final goal pose, and (b) one execution for distinct final goal poses. Every attempted execution succeeds: ten out of ten times for the repeated motion, and five out of five across the distinct motions.

VI Conclusion and Future Work

We presented DiPLO and established a key finding for limited-data humanoid loco-manipulation: strong task conditioning can be crucial for effective pretraining, while post-training can remove this dependency for deployment. Starting from only 29 retargeted pick-carry-place demonstrations, dynamically feasible data augmentation and strong kinematic conditioning enable the diffusion planner to learn trackable whole-body and object motions. Removing this conditioning directly causes a substantial performance drop, while training from scratch without it cannot recover comparable performance through downstream fine-tuning alone. We address this dependence with SD:DPO, which distills a reference-conditioned self-teacher into a planner conditioned only on state history and the object goal. This recovers goal-conditioned motion generation and enables the planner to operate on goals unseen in pretraining demonstrations. Complementary to this, DA:PPO further optimises the planner through closed-loop interaction with the frozen tracker, aligning generated motions with what the robot can successfully execute. Together, SD:DPO and DA:PPO achieve approximately 99% survival and 99% goal success during pick-carry-place rollouts, while the same training recipe transfers to substantially different skills, including kicking and pushing/pulling. The resulting planner-tracker also transfers directly to the Unitree G1 without any real-world fine-tuning. These results suggest a broader principle for robot learning: privileged or expensive supervision may be most valuable as a scaffold for pretraining rather than as a permanent deployment requirement. Future work will investigate whether this principle extends to a single model trained jointly across multiple loco-manipulation skills, more diverse objects and environments, and onboard perception, thereby removing the current reliance on external motion capture.

References

1. [1] A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karamcheti, S. Nasiriany, M. K. Srirama, L. Y. Chen, K. Ellis, et al. (2024) DROID: a large-scale in-the-wild robot manipulation dataset. In Proceedings of Robotics: Science and Systems (RSS), External Links: Document
2. [2] J. Li, J. Wu, and C. K. Liu (2023) Object motion guided human motion synthesis. ACM Transactions on Graphics 42 (6). Note: Art. no. 197 External Links: Document
3. [3] L. Yang, X. Huang, Z. Wu, A. Kanazawa, P. Abbeel, C. Sferrazza, C. K. Liu, R. Duan, and G. Shi (2026) OmniRetarget: interaction-preserving data generation for humanoid whole-body loco-manipulation and scene interaction. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 11691–11698. External Links: Document
4. [4] V. Dhédin, I. Taouil, S. Omar, D. Yu, K. Tao, A. Dai, and M. Khadiv (2026) DynaRetarget: dynamically-feasible retargeting using sampling-based trajectory optimization. Note: arXiv:2602.06827
5. [5] S. Sanokowski, K. Patil, and M. Khadiv (2026) Diffusion-augmented Markov decision processes for maximum entropy reinforcement learning. Note: arXiv:2512.02019
6. [6] X. B. Peng, P. Abbeel, S. Levine, and M. van de Panne (2018) DeepMimic: example-guided deep reinforcement learning of physics-based character skills. ACM Transactions on Graphics 37 (4). Note: Art. no. 143 External Links: Document
7. [7] Z. Luo, J. Cao, A. Winkler, K. Kitani, and W. Xu (2023) Perpetual humanoid control for real-time simulated avatars. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10861–10870. External Links: Document
8. [8] T. He, Z. Luo, X. He, W. Xiao, C. Zhang, W. Zhang, K. M. Kitani, C. Liu, and G. Shi (2024) OmniH2O: universal and dexterous human-to-humanoid whole-body teleoperation and learning. In Proceedings of the Conference on Robot Learning (CoRL), pp. 1516–1540.
9. [9] Y. Ze, J. P. Araújo, J. Wu, and C. K. Liu (2025) GMR: general motion retargeting. Note: GitHub repository, https://github.com/YanjieZe/GMR
10. [10] K. Lin, A. Mandlekar, C. R. Garrett, N. Chernyadev, Y. Fang, R. Ding, Y. Xie, J. Tran, L. Fan, and Y. Zhu (2026) HumanoidMimicGen: data generation for loco-manipulation via whole-body planning. Note: arXiv:2605.27724
11. [11] C. Chi, Z. Xu, S. Feng, E. Cousineau, Y. Du, B. Burchfiel, R. Tedrake, and S. Song (2025) Diffusion policy: visuomotor policy learning via action diffusion. The International Journal of Robotics Research 44 (10–11), pp. 1684–1704. External Links: Document
12. [12] S. Ha, J. Lee, M. van de Panne, Z. Xie, W. Yu, and M. Khadiv (2025) Learning-based legged locomotion: state of the art and future perspectives. The International Journal of Robotics Research 44 (8), pp. 1396–1427. External Links: Document
13. [13] T. E. Truong, M. Piseno, Z. Xie, and C. K. Liu (2024) PDP: physics-based character animation via diffusion policy. In SIGGRAPH Asia 2024 Conference Papers, Note: Art. no. 86 External Links: Document
14. [14] Q. Liao, T. E. Truong, X. Huang, Y. Gao, G. Tevet, K. Sreenath, and C. K. Liu (2026) BeyondMimic: from motion tracking to versatile humanoid control via guided diffusion. Science Robotics 11 (117). Note: Art. no. eadx8924 External Links: Document
15. [15] Z. Zhang, K. Wen, M. Xu, J. He, C. Li, T. Miki, C. Schwarke, C. Zhang, X. B. Peng, and M. Hutter (2026) Learning whole-body humanoid locomotion via motion generation and motion tracking. IEEE Robotics and Automation Letters 11 (9), pp. 10194–10201. External Links: Document
16. [16] Z. Gu, Y. Chen, Z. Chai, A. Cueva, T. Nguyen, Y. Wu, H. Xue, M. Kim, I. Legene, F. Liu, K. Kim, A. Barula, Y. Chen, and Y. Zhao (2026) REFINE-DP: diffusion policy fine-tuning for humanoid loco-manipulation via reinforcement learning. IEEE Robotics and Automation Letters 11 (10), pp. 11617–11624. External Links: Document
17. [17] P. Debbad, K. Thiagarajan, V. Dhédin, S. Omar, and M. Khadiv (2026) Humanoid-DART: humanoid loco-manipulation using diffusion-guided augmentation through relabeling and tracking. Note: arXiv:2606.26855
18. [18] A. Leach, S. M. Schmon, M. T. Degiacomi, and C. G. Willcocks (2022) Denoising diffusion probabilistic models on SO(3) for rotational alignment. In ICLR Workshop on Geometrical and Topological Representation Learning,
19. [19] Y. Jagvaral, F. Lanusse, and R. Mandelbaum (2024) Unified framework for diffusion generative models in SO(3): applications in computer vision and astrophysics. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38, pp. 12754–12762. External Links: Document
20. [20] G. Corso, H. Stärk, B. Jing, R. Barzilay, and T. Jaakkola (2023) DiffDock: diffusion steps, twists, and turns for molecular docking. In Proceedings of the International Conference on Learning Representations (ICLR),
21. [21] J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter (2020) Learning quadrupedal locomotion over challenging terrain. Science Robotics 5 (47). Note: Art. no. eabc5986 External Links: Document
22. [22] J. Hübotter, F. Lübeck, L. D. Behric, A. Baumann, M. Bagatella, D. Marta, I. Hakimi, I. Shenfeld, T. Kleine Buening, C. Guestrin, and A. Krause (2026) Reinforcement learning via self-distillation. In Proceedings of the International Conference on Machine Learning (ICML), pp. 48677–48721.
23. [23] A. Z. Ren, J. Lidard, L. L. Ankile, A. Simeonov, P. Agrawal, A. Majumdar, B. Burchfiel, H. Dai, and M. Simchowitz (2025) Diffusion policy policy optimization. In Proceedings of the International Conference on Learning Representations (ICLR),
24. [24] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov (2017) Proximal policy optimization algorithms. Note: arXiv:1707.06347
25. [25] J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli (2015) Deep unsupervised learning using nonequilibrium thermodynamics. In Proceedings of the International Conference on Machine Learning (ICML), pp. 2256–2265.
26. [26] J. Ho, A. Jain, and P. Abbeel (2020) Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems (NeurIPS), pp. 6840–6851.
27. [27] T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine (2018) Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. In Proceedings of the International Conference on Machine Learning (ICML), pp. 1861–1870.
28. [28] J. Schulman, P. Moritz, S. Levine, M. I. Jordan, and P. Abbeel (2016) High-dimensional continuous control using generalized advantage estimation. In Proceedings of the International Conference on Learning Representations (ICLR),
29. [29] W. Peebles and S. Xie (2023) Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4172–4182. External Links: Document
30. [30] A. Q. Nichol and P. Dhariwal (2021) Improved denoising diffusion probabilistic models. In Proceedings of the International Conference on Machine Learning (ICML), pp. 8162–8171.
31. [31] Y. Song, C. Durkan, I. Murray, and S. Ermon (2021) Maximum likelihood training of score-based diffusion models. In Advances in Neural Information Processing Systems (NeurIPS), pp. 1415–1428.
32. [32] T. Salimans and J. Ho (2022) Progressive distillation for fast sampling of diffusion models. In Proceedings of the International Conference on Learning Representations (ICLR),
33. [33] K. Zakka, Q. Liao, B. Yi, L. Le Lay, K. Sreenath, and P. Abbeel (2026) mjlab: a lightweight framework for GPU-accelerated robot learning. Note: arXiv:2601.22074
34. [34] X. Huang, T. Truong, Y. Zhang, F. Yu, J. P. Sleiman, J. Hodgins, K. Sreenath, and F. Farshidian (2025) Diffuse-CLoC: guided diffusion for physics-based character look-ahead control. ACM Transactions on Graphics 44 (4). Note: Art. no. 132 External Links: Document
35. [35] T. Minka (2005) Divergence measures and message passing. Technical report Technical Report MSR-TR-2005-173, Microsoft Research, Cambridge, U.K..
36. [36] S. Levine (2018) Reinforcement learning and control as probabilistic inference: tutorial and review. Note: arXiv:1805.00909
37. [37] T. Hang, S. Gu, C. Li, J. Bao, D. Chen, H. Hu, X. Geng, and B. Guo (2023) Efficient diffusion training via Min-SNR weighting strategy. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 7407–7417. External Links: Document

Supplementary Material for DiPLO: Self-distilled Diffusion Planning for Loco-manipulation Anonymous Authors

-A Denoiser Network Architecture

Figure 6 shows the denoiser network of the diffusion motion generator (Section IV-B). Each input is tokenised by embedding to the same dimension using an MLP and accompanied with positional encoding for time series data. The denoising step k enters the network through adaptive layer normalization as in [29].

-B Diffusion Motion Generator Pretraining

Euclidean training loss. Each training example consists of a dataset motion chunk with its conditioning, a uniformly sampled diffusion level k\in\{1,\ldots,K\}, and forward noise drawn from q. The denoiser predicts the \mathbf{v}-target defined in Section III-B. We penalize each scalar prediction error r using smooth-L1 with a unit transition threshold:

\ell_{\mathrm{SL1}}(r)=\begin{cases}\tfrac{1}{2}r^{2},&|r|\leq 1,\\
|r|-\tfrac{1}{2},&|r|>1.\end{cases}

For small errors, the quadratic penalty gives a gradient proportional to the error. For larger errors, the penalty grows linearly and its derivative has magnitude one, limiting the influence of large errors on the scalar loss gradient. Both the penalty and its derivative are continuous at the transition. The threshold is measured in \mathbf{v}-prediction units, computed from normalized features.

Let \mathcal{C}_{\mathrm{E}} denote the Euclidean feature groups and N_{c} the number of scalar entries across all frames and features in group c. For errors r_{c,j}=\hat{v}_{\phi,c,j}^{k}-v_{c,j}^{k}, we average within each group:

\operatorname{SmoothL1}(\hat{\mathbf{v}}_{\phi,c}^{k},\mathbf{v}_{c}^{k})=\frac{1}{N_{c}}\sum_{j=1}^{N_{c}}\ell_{\mathrm{SL1}}(r_{c,j}).

The resulting Euclidean objective is

\mathcal{L}_{\mathrm{E}}(\phi)=\mathbb{E}_{s^{D}_{\tau},k,q}\!\left[\omega_{k}\sum_{c\in\mathcal{C}_{\mathrm{E}}}\lambda_{c}\,\operatorname{SmoothL1}(\hat{\mathbf{v}}_{\phi,c}^{k},\mathbf{v}_{c}^{k})\right].

The expectation averages over dataset chunks, diffusion levels, and forward noise. Averaging within groups makes their relative importance depend on the weights \lambda_{c} in Table VIII, rather than their number of entries.

The diffusion-level weight is \omega_{k}=1 without noise-level reweighting. When Min-SNR weighting [37] is enabled, we use \omega_{k}=\min(\mathrm{SNR}_{k},\gamma_{\mathrm{SNR}})/(1+\mathrm{SNR}_{k}), where \mathrm{SNR}_{k}=\bar{\alpha}_{k}/(1-\bar{\alpha}_{k}) is the signal-to-noise ratio and \gamma_{\mathrm{SNR}} is its clipping threshold. These weights adjust the contribution of different diffusion levels.

Smooth-L1 changes the regression objective: its optimal prediction need not equal the conditional mean of \mathbf{v}^{k}. The exact score-recovery argument for squared-error regression therefore does not apply to this loss.

Rotational training and sampling. To sample the forward noise E^{k} in (13), we draw an axis uniformly on the unit sphere and an angle from a numerically tabulated \mathcal{IG}_{\mathrm{SO}(3)} angle distribution at scale \sigma_{k}. Multiplying the axis by the angle gives a vector whose exponential is E^{k}.

Table VIII lists the full configuration used to pretrain the diffusion motion generator (Section IV-B).

-C Goal Representation

g_{t}=\left[\,\hat{d}_{t},\;\varphi(\Delta R_{t}),\;\min\!\left(1.0,\lVert\Delta p_{t}\rVert\right)\right]\in\mathbb{R}^{10}

Here \Delta p_{t}=p^{o}_{T}-p^{o}_{t} is the displacement of the object towards its final pose, \hat{d}_{t}\in\mathbb{R}^{3} is its unit direction expressed in the robot base frame, \lVert\Delta p_{t}\rVert is clipped at 1 m, and \varphi(\Delta R_{t})\in\mathbb{R}^{6} is the 6D representation of the relative object rotation \Delta R_{t}=(R^{o}_{t})^{-1}R^{o}_{T}, i.e. the rotation that still has to be applied to the object expressed in its own current frame. Expressing the goal as a displacement from the current object pose rather than as an absolute target keeps it on the same relative footing as the motion representation of Section IV-B, and makes it a residual that shrinks to zero as the task is completed.

-D Tracking Controller Training Details

This appendix gives the observation layout and the reset strategy used to pretrain the generalist tracking controller (Section IV-B3).

Observations. The reference observations of the actor consist of reference joint states, body poses, and object-tracking features at future trajectory steps \{1,2,4,8\} frames (20–160 ms). We use an asymmetric actor–critic: the critic additionally receives object velocities, reference contact indicators, and a wider temporal window of reference motion to track.

Reset sampling. Training resets select both a reference motion and a starting phase, then initialize the robot and object from that reference state with perturbations. These two sampling decisions use separate mechanisms. For the starting phase, we use adaptive reference-state initialization as in [14] during the first 25{,}000 learning iterations and uniform phase sampling thereafter. For motion selection, we retain evaluation-based reweighting throughout training, including after the phase-sampling switch. Every 1000 learning iterations, each GPU evaluates the motions in its local shard from the first frame without any domain randomization, distributing its environments across those motions.

A motion is classified as failed if any of its evaluation triggers a non-timeout termination which entails robot and object terminations based on tracking as described in Appendix -E. Let \mathcal{S}_{g} be the motion shard on GPU g, N_{g}=|\mathcal{S}_{g}|, and F_{i}^{(m)} indicate failure at evaluation m. Starting from unit weights, the update is

w_{i}^{(m+1)}=\begin{cases}1,&F_{i}^{(m)}=1,\\[2.0pt]
\max\!\left(\gamma_{s}\,w_{i}^{(m)},\,N_{g}^{-1}\right),&F_{i}^{(m)}=0,\end{cases}

and subsequent resets sample motions within that shard according to

P(i\mid g)=\frac{w_{i}}{\sum_{j\in\mathcal{S}_{g}}w_{j}}.

Thus, repeated success reduces a motion’s weight, while a failure restores its priority. The weight floor preserves a nonzero sampling probability for every motion.

-E Survival Rate and Termination Conditions

The survival rate reported in the results tables is the percentage of evaluation rollouts in which neither a robot fall nor an object drop occurs. As such, the survival rates for kicking and push/pull tasks are high because the object is always on the ground. Task success separately measures whether the final object pose satisfies the goal tolerances. The full set of failure conditions below governs episode termination during tracking controller pretraining (Section IV-B3) and evaluation-based motion reweighting (Appendix -D):

1. Robot fall: the absolute height error of the torso or any wrist or ankle exceeds 0.25 m.
2. Object drop: the absolute object-height error exceeds 1 m.
3. The absolute difference between the z-components of gravity projected into the actual and reference torso frames exceeds 0.8.
4. Any component of the base angular velocity exceeds 8.73 rad/s in magnitude.

The object-height threshold follows a piecewise-linear schedule through 1.0, 0.3, 0.6, and 0.3 m at iterations 5000, 15{,}000, 20{,}000, and 25{,}000, respectively, and is held fixed during each evaluation. Reaching the end of the reference motion or the 10 s episode limit without a failure is treated as a timeout and counts towards the survival rate.

-F Tracking Controller Reward Terms

Table X lists the reward terms used to pretrain the generalist tracking controller (Section IV-B3).

-G Post-Training Reward Terms

Table XI lists the reward terms used to finetune the diffusion motion generator (Section IV-D).

Tracker fine-tuning rewards. The tracker fine-tuning baselines of Table VI (Section V-H) compares tracker finetuning against different sets of rewards: r_{\text{kin}}, r_{\text{kin}}+r_{\text{goal}}, and r_{\text{gen}}. r_{\text{gen}} is a tracking reward of the same form as r_{\text{kin}}, but it measures the error of the realised motion against the reference generated by the diffusion motion generator, \hat{s}^{D}_{\tau}, which is the motion the tracker is commanded to follow, instead of against the IK-retargeted motion s^{IK}_{\tau}.

-H DA:PPO Post-Training Hyperparameters

Table XII lists the configuration used to post-train the diffusion motion generator with DA:PPO (Section IV-D). The DPPO-style baseline uses the same configuration with \mathcal{T}=0 and fixed step variances.

-I Tracker Fine-Tuning Hyperparameters

Table XIII lists the configuration used to fine-tune the tracking policy in Table VI (Section V-H). The motion generator stays frozen, and the rollouts are collected in the same closed loop as the evaluation; the reward sets are those of Appendix -G.

-J SD:DPO Objective and its Weighting

SD:DPO adapts the objective of SDPO [22]. For a prompt x and a response y sampled from the student, SDPO minimises

\mathcal{L}_{\mathrm{SDPO}}(\theta)=\sum_{t}\mathrm{KL}\big(\pi_{\theta}(\cdot\mid x,y_{<t})\,\big\|\,\operatorname{sg}\bigl(\pi_{\theta}(\cdot\mid x,f,y_{<t})\bigr)\big),

the KL divergence at every position t between the student’s next-token distribution and that of a self-teacher, the same model additionally conditioned on feedback f. Mapping tokens to denoising steps, prefixes to noisy chunks with their context, next-token distributions to reverse kernels, and the feedback f to the kinematic reference s^{IK}_{\tau}, and sampling \hat{s}^{D,k}_{\tau} and k as in (16), gives

\mathcal{L}_{\mathrm{KL}}(\phi)=\mathbb{E}_{\mathcal{B},k,q}\Big[\mathrm{KL}\big(p^{\varnothing}_{\phi}\,\big\|\,\operatorname{sg}(p^{IK}_{\phi})\big)\Big],

where p^{\varnothing}_{\phi}=p_{\phi}(\cdot\mid\hat{s}^{D,k}_{\tau},c_{t},\varnothing) and p^{IK}_{\phi}=p_{\phi}(\cdot\mid\hat{s}^{D,k}_{\tau},c_{t},s^{IK}_{\tau}) are the student’s and the teacher’s reverse kernels over (\hat{x}^{k-1},\hat{R}^{k-1}) given by (11) and (15). As in SDPO, the stop-gradient keeps the teacher from drifting towards the student, and in practice the teacher is evaluated with the exponential moving average weights \bar{\phi}, as in (17). Because the Euclidean and rotational parts of each kernel are sampled independently given \hat{s}^{D,k}_{\tau}, the KL splits into a Euclidean and a rotational term. The remainder of this appendix relates (22) to the regression of (17) that is minimised in practice. At denoising step k, we use a noisy motion \hat{s}^{D,k}_{\tau} obtained via the forward kernel from the student’s rollout in closed-loop, where \hat{x}^{k} corresponds to the corresponding noisy Euclidean features. The student and the teacher evaluate the same network at the same (\hat{s}^{D,k}_{\tau},c,k) and differ only in the prior token (\varnothing or s^{IK}_{\tau}); we mark their reverse kernels of \hat{x}^{k-1}, their \mathbf{v}-predictions and their clean estimates with the corresponding superscript. By (11) both kernels are Gaussian with covariance \tilde{\beta}_{k}I and mean a_{k}\,\hat{x}^{0|k}+b_{k}\,\hat{x}^{k}, and \hat{x}^{0|k}=\sqrt{\bar{\alpha}_{k}}\,\hat{x}^{k}-\sqrt{1-\bar{\alpha}_{k}}\,\hat{\mathbf{v}}^{k}, so that

\displaystyle\mathrm{KL}\big(p^{\varnothing}_{\phi}\,\big\|\,p^{IK}_{\phi}\big) \\ \displaystyle=\frac{a_{k}^{2}}{2\tilde{\beta}_{k}}\,\big\|\hat{x}^{0|k,\varnothing}-\hat{x}^{0|k,IK}\big\|^{2} \\ \displaystyle=w_{k}\,\big\|\hat{\mathbf{v}}^{k,\varnothing}-\hat{\mathbf{v}}^{k,IK}\big\|^{2}, \\ \displaystyle w_{k} \\ \displaystyle=\frac{a_{k}^{2}\,(1-\bar{\alpha}_{k})}{2\tilde{\beta}_{k}}=\frac{\beta_{k}}{2}\,\frac{\bar{\alpha}_{k-1}}{1-\bar{\alpha}_{k-1}}.

The KL is therefore the squared difference of the two network predictions, scaled by a weight that depends only on the diffusion step. The same holds for the rotational features to first order in the tangent space, with the tangent-noise predictions in place of \hat{\mathbf{v}}^{k} and another step-dependent weight.

We set w_{k}=1 for two reasons. First, w_{k} cannot be used as it stands. Since \bar{\alpha}_{0}=1, it is infinite at k=1, where the reverse kernel is deterministic. Over the remaining steps in our schedule, the exact weighting would also concentrate the loss on the least-noisy steps. Second, the weights do not change what is learned. For any positive weights, the objective is non-negative and vanishes only when the student’s prediction equals the teacher’s at every step, which is also where (22) attains its minimum; the weights only decide how errors at different noise levels are traded off before that point is reached. Equal weights are the choice of the simplified diffusion objective [26], and they are applied here in the space the network regresses in, where the \mathbf{v}-target has a comparable scale at every noise level [32]. Setting w_{k}=1 for the Euclidean and the rotational features gives the distillation term of (17).

-K Action-Level Diffusion Baselines

This appendix describes the action-level baselines of Section V-D: an action-only diffusion policy in the style of PDP [13], and a state–action diffusion model in the style of BeyondMimic [14]. Both output robot joint PD targets directly, with no tracker in the loop. Table XIV lists their configuration.

Training data. Both baselines are trained via behaviour cloning on rollouts of our pretrained generalist tracking policy over every motion of the SBTO- and symmetry-augmented data corpus, under the randomisation of Table IX. As in PDP [13], the joint targets applied to the robot are perturbed using temporally correlated Ornstein–Uhlenbeck noise as used in [14], while the tracker’s unperturbed output is stored as the label, so that the data pairs off-nominal states with the actions that correct them. We collect full-length rollouts and 5 s rollouts started from random frames of each motion, and discard every rollout that fails, generating 15M frames (85 h) of state–action data.

Models. The state-action diffusion baseline uses the diffusion transformer (DiT) architecture of [34], for a faithful reproduction of the results. Here, a DiT learns to denoise a chunk of joint state–action tokens, each with its own noise level, given the current state and a history of state–action pairs; state tokens attend to all states and to no action, and action tokens attend only to states and actions up to their own step (causal attention). PDP [13] only learns to diffuse the action distribution. For the PDP-style baseline, we therefore drop the state prediction from the same network and exclude it from the loss, leaving an action diffusion policy conditioned on the state–action history and the task. Both models replan at every control step (50 Hz) with 20 denoising steps and execute only the first action.

The network, the diffusion process, the optimizer and the collection of expert rollouts follow Diffuse-CLoC [34] and the first version of BeyondMimic [14], which applies Diffuse-CLoC to the robot we use. Where the two differ (attention heads, horizons, state features) we take the BeyondMimic values. We depart from both works in the following places:

1. The state additionally contains the object’s observations such as its position, orientation and linear velocity.
2. The conditioned models receive the goal and the IK-retargeted motion as additional context tokens.
3. Later versions of BeyondMimic diffuse the latent of an action autoencoder. We diffuse joint targets directly, as Diffuse-CLoC and the first version of BeyondMimic do.

Unconditioned prior. In [34, 14] the state-action diffusion model is not conditioned on the task: it is an unconditioned prior that is steered at inference using a task-specific guidance cost. Hence, we train a separate diffusion model without the conditioning tokens and evaluate it at test time by steering with the IK-retargeted motion. Unlike the conditioned models, it takes its horizons and its sampling of noise levels from Diffuse-CLoC; these levels include zero, so that the clean keyframes of in-betweening are seen in training.

Test-time steering. The guidance cost sums, over the predicted states of the horizon, the squared position error and the orientation error 1-\langle q,q^{IK}\rangle^{2} of the root and of the object with respect to the IK-retargeted motion, with q a unit quaternion. At every denoising step the predicted clean states are moved against the gradient of this cost, scaled by the guidance weight. Keyframe in-betweening instead inpaints a single state ahead in time: the keyframe is the state of the IK-retargeted motion at the given lookahead, expressed relative to the measured planar position and heading of the robot, and the waypoint cost of [14] pulls the planar root position toward that of the keyframe. The steered row of Table III uses in-betweening with the waypoint cost. A lookahead of 0.32 s gave the highest survival rate among 0.08, 0.24, 0.32, 0.64 and 0.96 s.

-L Ablation on Diffusion Prediction Horizon

This appendix examines how far into the future the diffusion motion generator can plan. We pretrain five generators that differ only in the prediction horizon n\in\{20,32,48,64,100\} frames, i.e. 0.4 s to 2.0 s of motion at 50 Hz. The denoiser remains unchanged; only its input sequence grows, since the prior and the generated chunk each contribute n tokens. The tracker reads its reference up to 8 frames ahead (Appendix -D), so the last 8 frames of a chunk are kept as its lookahead, and the first m=n-8 frames are executed, as in our default of n=64 and m=56. A longer horizon, therefore, also means that a longer segment is executed before the generator re-plans from the reached state: the number of generator calls per second of motion falls from 4.2 at n=20 to 0.54 at n=100.

All other settings follow Table VIII, and the absolute numbers below should not be compared with those of the main experiments.

Table XV reports the closed-loop performance with the frozen tracker. Predicting up to 2.0 s and executing up to 1.84 s per re-plan does not measurably degrade any metric in simulation. The survival rate remains above 97\% for every horizon, and task success varies by at most 5.4 points across horizons, which is about one SEM, with no trend in n. The tracking errors do not grow with the horizon either: MPKPE and object position error are lowest for n=64 and n=100 in both settings.

-M Goal Generalization Before DA:PPO Finetuning

Fig. 7 repeats the goal-grid evaluation of Sec. V-G for the two checkpoints that precede the final one in Table V: the pretrained Student and Student + SD:DPO, from which DA:PPO starts. All three checkpoints are evaluated on the same episodes, i.e. the same goal poses, initial object poses and evaluation seed, and the plots share their color scales.

The pretrained student has the highest position errors since the diffusion sampling is ill-conditioned. Self-distillation mitigates the position error a bit, but still struggles with object orientations. However, as shown in Fig. 5, the finetuning on top of the distilled student unlocks the final leg of performance, pushing diffusion sampling towards high reward samples.

In Fig. 8, we vary the goal position from our demonstration by \pm 0.5 m and \pm 1 m along the x and y axes (a,c) and rotate it in place by \pm 90^{\circ} (b,d), and roll out our final post-trained checkpoint, which is crucially only conditioned on the state-history and goal. We find that our self-distilled model can generate consistent motions to achieve the desired object goal pose.