ReCo: 반응 일관 보행과 정책 인지 MPC를 결합한 보행 조작 프레임워크

Abstract

Continuous legged manipulation requires accurate end-effector tracking while the base keeps walking. Combining reinforcement learning (RL) with model predictive control (MPC) suits this task: the learned policy provides robust locomotion, while MPC coordinates the base and arm to compensate for tracking errors. However, MPC can compensate only for base motion that it can predict, and a learned policy’s command response varies with gait phase, contact, and payload. We present ReCo, a framework that couples response-consistent locomotion with policy-aware MPC for legged manipulation. Response shaping trains the policy to respond to commands consistently and repeatably across randomized dynamics. An identified closed-loop response model then lets MPC jointly plan locomotion commands and arm motion. On the simulation benchmark, ReCo reduces position and orientation root-mean-square error (RMSE) by 28.7% and 27.4% relative to the best baseline for each metric. Real-world experiments demonstrate onboard continuous legged manipulation with coordinated base and arm motion.

I Introduction

A quadruped equipped with an arm can bring manipulation capabilities to environments that are difficult for conventional mobile platforms to traverse. Tasks such as inspection, scanning, and tool positioning can require the robot to track a world-frame end-effector (EE) trajectory while walking. Every step moves the arm’s supporting frame: base translation, attitude changes, and gait oscillations all propagate to the EE. Continuous legged manipulation therefore requires coordination throughout locomotion, beyond repositioning the base to extend the arm’s workspace (Fig. 1).

Robust locomotion and precise manipulation place different demands on control. Reinforcement learning (RL) has enabled quadrupeds to traverse challenging terrain under uncertain contact conditions and dynamics [1, 2, 3, 4]. Model-based control explicitly represents task geometry, motion constraints, and future references, supporting coordinated locomotion and manipulation [5, 6]. Learned whole-body controllers also demonstrate effective leg–arm coordination [7, 8]. However, (i) model-based control relies on accurate contact and dynamics models, which are hard to maintain on uneven terrain with uncertain contact, and (ii) learned whole-body control balances task-space accuracy and stability within one reward, which can require retuning across terrains, payloads, and disturbances.

A modular alternative assigns locomotion to RL and base–arm coordination to model predictive control (MPC) [9]. In such architectures the key difficulty lies at the interface: the manipulation planner must predict how the policy-controlled base will move under candidate locomotion commands. To MPC, the base is a closed-loop subsystem whose command response can vary with gait phase, contact conditions, and payload. To the locomotion policy, arm motion changes the dynamics it must stabilize. Assuming that a commanded velocity or posture is realized immediately neglects these interactions. Even an accurate mean-velocity model can miss the periodic height and attitude motion that propagates to the EE.

Our key insight is that a locomotion policy should be trained not only to be robust but also to be predictable by the planner that commands it. Locomotion supplies sustained displacement and disturbance recovery, while the arm supplies fast local corrections within its workspace and actuator limits; a planner can exploit this division of labor only if the base response is repeatable. A consistent delay can be compensated by feedforward, whereas a variable delay cannot. We therefore seek a locomotion response that a low-order model can predict over a useful horizon.

We present ReCo, a framework that couples response-consistent locomotion with policy-aware MPC for legged manipulation. Our contributions are:

• A training method that makes the command response of a locomotion policy repeatable through reference tracking, phase repeatability, steady-state gain, and cross-domain consistency rewards.
• A policy-aware MPC that jointly plans locomotion commands and arm motion for continuous world-frame EE \mathrm{SE}(3) tracking, where an identified closed-loop response model with a gait-periodic component serves as the predictive interface to the learned policy.
• Simulation and hardware experiments in which ReCo reduces position and orientation root-mean-square error (RMSE) by 28.7% and 27.4% relative to the best of five baselines for each metric, response shaping lowers normalized prediction mean squared error (MSE) by 59.6%, and the full system runs onboard.

II Related Work

II-A Model-Based Legged Manipulation

MPC enables legged locomotion through ground-reaction-force optimization [10] and terrain-aware motion planning [11]. For legged manipulation, ALMA combines online planning, torso posture optimization, and hierarchical whole-body control [5]; perceptive MPC incorporates environmental geometry into continuous mobile manipulation [6]. These approaches coordinate base and arm motion through explicit models and constraints. When a learned policy executes locomotion, the manipulation planner additionally needs to predict the command-to-motion response of the policy–robot closed loop; ReCo supplies this prediction with an identified closed-loop response model.

II-B Learning-Based Locomotion and Manipulation

Actuator modeling, domain randomization, and disturbance training support robust sim-to-real locomotion [12, 1]. Proprioceptive and perceptive policies extend locomotion to challenging terrain [2, 3], while online adaptation [13] and parallel training [14] improve robustness and training efficiency. Walk These Ways provides a single policy with gait, posture, and velocity commands [4]. ReCo adopts this command interface and shapes the policy’s transient and periodic responses for predictive planning.

Deep Whole-Body Control learns a unified leg–arm policy [7], and RoboDuet learns cooperating locomotion and manipulation policies [8]. Visual whole-body control combines a visuomotor policy with a controller conditioned on base velocity and EE pose [15]. UMI on Legs learns to track task-frame manipulation trajectories with future-target observations [16]. Portela et al. demonstrate EE pose tracking over a large workspace and rough terrain [17], and learned force control supports compliant interaction [18]. ReCo focuses on the predictive interface between a learned locomotion policy and policy-aware MPC for joint optimization of locomotion commands and arm motion.

II-C Hybrid Control and Policy Response Modeling

Ma et al. condition a learned locomotion policy on predicted external wrenches from model-based manipulation [9]. Their interface lets locomotion anticipate arm-induced disturbances; ReCo lets policy-aware MPC anticipate the base response to candidate commands. RAMBO learns feedback corrections to model-based whole-body control [19]; in ReCo, the learned component is the locomotion policy, and the response model is fitted to it.

Learning-based MPC can improve dynamics models, tune control objectives, or impose predictive constraints on learned behavior [20]. Kim et al. identify lifted linear dynamics of an RL-controlled biped for MPC-based navigation [21]. ReCo extends policy-response modeling by training the policy to produce responses that a low-order model can predict and by representing the gait-periodic height and attitude variations that affect manipulation.

III Method

ReCo trains a locomotion policy with a consistent command response, identifies a closed-loop response model of that policy, and uses the model in a policy-aware MPC that jointly plans locomotion commands and arm motion (Fig. 2).

III-A Problem Formulation

We track a time-indexed world-frame EE pose reference T_{WE}^{\mathrm{ref}}(t)\in\mathrm{SE}(3) during locomotion. Frames W, B, and E denote the world, base, and EE; T_{AB} maps coordinates from frame B to frame A. Let q\in\mathbb{R}^{6} denote arm joint positions. MPC selects locomotion commands and arm joint velocities.

A gait-conditioned policy controls the legs using proprioception, commands, and gait timing. Its command vector is \mathbf{c}=(c_{x},c_{y},c_{\omega},c_{h},c_{\theta})\in\mathbb{R}^{5}, comprising planar velocity, yaw rate, height, and pitch. The base prediction state is

\mathbf{x}_{\mathrm{base}}=(p_{W},\psi,\theta,\rho,v_{x},v_{y},\omega_{z})\in\mathbb{R}^{9},

where p_{W}\in\mathbb{R}^{3} is the modeled base position in W, (\psi,\theta,\rho) are ZYX yaw, pitch, and roll, (v_{x},v_{y}) are heading-frame velocities, and \omega_{z} is body-frame angular velocity about the z axis. The policy has no roll command; roll remains a predicted state because it affects the EE pose. With gait phase \phi, the full state and input are

x=(\mathbf{x}_{\mathrm{base}},\phi,q)\in\mathbb{R}^{16},\qquad u=(\mathbf{c},u_{q})\in\mathbb{R}^{11},

where u_{q}\in\mathbb{R}^{6} denotes arm joint velocity commands. The response model in Sec. III-C predicts the motion produced by the policy–robot closed loop under candidate commands.

III-B Response-Consistent Locomotion Training

We train the locomotion policy with proximal policy optimization [22] and the gait-conditioned interface of Walk These Ways [4]. The gait setting is fixed over each prediction horizon. Response shaping denotes the training terms used to encourage response consistency: repeatable command transients and gait-phase-dependent body motion across randomized dynamics. These terms act on y=(v_{x},v_{y},\omega_{z},h,\theta), where h is body height measured relative to its nominal value. Each of these five command channels drives a critically damped reference generator,

\ddot{\xi}_{i}=\lambda_{i}^{2}(c_{i}-\xi_{i})-2\lambda_{i}\dot{\xi}_{i},\qquad|\dot{\xi}_{i}|\leq r_{i},

where \xi_{i} is the target response and \lambda_{i} and r_{i} specify its bandwidth and rate limit. Unsaturated dynamics use an exact zero-order-hold update; saturation produces a rate-limited ramp. The height command c_{h} uses the same nominal offset as h.

To separate command transients from gait oscillations, we use \tilde{y}=y-\hat{\delta}(\phi,\nu), where \nu is planar speed and \hat{\delta} estimates the gait-periodic component of y during training. This estimate uses running means over phase and speed bins, updated at each environment step from nominal-domain environments. The reference-tracking reward is

r_{\mathrm{ref}}=g\,\frac{\sum_{i}w_{i}\exp[-(\tilde{y}_{i}-\xi_{i})^{2}/\sigma_{i}^{2}]}{\sum_{i}w_{i}},

where w_{i} and \sigma_{i} are channel weights and error scales. The gate g\in\{0,1\} disables this reward during external disturbances and a post-disturbance interval, allowing recovery to take precedence over transient matching.

Table I summarizes the response-shaping terms. Phase regularization penalizes deviations from the repeatable waveform, rather than its amplitude. A separate steady-state term discourages command bias. For cross-domain consistency, groups of environments receive identical command sequences and gait phases. Each group contains a nominal environment and instances with randomized mass, payload, friction, and actuator strength. Instance d incurs

\ell_{\mathrm{domain}}^{d}=m^{d}\|\tilde{y}^{d}-\tilde{y}^{\mathrm{nom}}\|_{W_{d}}^{2},

where \|a\|_{W}^{2}=a^{\top}Wa and W_{d} weights all five response channels, including height. The mask m^{d} excludes nominal, invalid, or desynchronized comparisons. The nominal environment supplies the reference response for the randomized instances, while (4) specifies the target transient.

These terms augment the locomotion reward. Positive terms r_{+}\geq 0 and signed penalties r_{-}\leq 0 are combined as r=r_{+}\exp(c_{-}r_{-}), with c_{-}>0 [4]. Training first establishes locomotion, then introduces reference tracking, consistency penalties, broader randomization, and stronger disturbances.

III-C Closed-Loop Response Model

The response model combines command-response dynamics with gait-periodic base motion. Velocity channels follow

\dot{v}_{i}=(k_{i}c_{i}-v_{i})/\tau_{i},\qquad i\in\{x,y,\omega\},

where v_{\omega}=\omega_{z}, k_{i} is the response gain, and \tau_{i} is the time constant. The second-order training reference in (3) specifies a target transient; it does not impose the order of the realized closed loop. We use one response state per velocity channel to keep horizon optimization compact.

For \eta=(z,\theta,\rho), where z is the world height of the modeled base frame at the arm mounting plane, we add a speed-dependent gait-periodic component, modeled as a harmonic series:

\displaystyle\delta_{j} \\ \displaystyle=\sum_{n=1}^{N_{h}}(a_{0jn}+a_{1jn}\nu)\sin(n\phi+\beta_{jn}), \\ \displaystyle\dot{\eta}_{j} \\ \displaystyle=\frac{b_{j}+k_{j}c_{j}-(\eta_{j}-\delta_{j})}{\tau_{j}}+\dot{\delta}_{j},\qquad\dot{\phi}=2\pi f_{g},

where j\in\{z,\theta,\rho\}, c_{z}=c_{h}, f_{g} is gait frequency, and the roll channel has no command input (k_{\rho}=0). The smoothed planar speed is \nu=\sqrt{v_{x}^{2}+v_{y}^{2}+\epsilon} with a small \epsilon>0. The coefficients a_{0jn} and a_{1jn} set the fixed and speed-dependent amplitudes, and \beta_{jn} sets the phase offset. We use N_{h}=2 for stride and half-stride components. Only b_{z} is nonzero: it sets the nominal mounting-plane height for the policy’s zero height command, c_{h}=0.

The gait-periodic component \delta is identified offline for prediction, separately from the training-time estimate \hat{\delta}. The state \eta includes gait oscillations, so the lag acts on \eta-\delta, and \dot{\delta} accounts for phase and speed variation. Planar world-frame velocity is obtained by yaw-rotating (v_{x},v_{y}), and the ZYX yaw rate satisfies

\dot{\psi}=\frac{\omega_{z}+\sin\rho\,\dot{\theta}}{\cos\theta\cos\rho}.

The model captures the empirical closed-loop response without explicitly modeling arm-induced wrenches.

Parameters are identified offline for each frozen policy by bounded nonlinear least squares on multi-step open-loop prediction errors. Step, ramp, and chirp commands excite the response across randomized domains. Identification data are separate from the prediction and tracking runs in Sec. IV.

III-D Policy-Aware Model Predictive Control

The response model and arm kinematics \dot{q}=u_{q} define \dot{x}=f(x,u). Forward kinematics gives T_{WE}=T_{WB}T_{BE}(q), where T_{BE} includes the arm mounting transform. Let p_{WE} and R_{WE} denote the position and rotation components of T_{WE}. At each update, MPC solves

\displaystyle\min_{x(\cdot),u(\cdot)} \\ \displaystyle\Phi(x(t+H))+\int_{t}^{t+H}\!\left[\ell(x,u,t^{\prime})+\mathcal{P}(x,u)\right]dt^{\prime} \\ \displaystyle\mathrm{s.t.} \\ \displaystyle\dot{x}=f(x,u),\qquad x(t)=\hat{x}(t),

where \hat{x}(t) is the current state estimate, H is the horizon, and \mathcal{P} contains soft penalties for joint bounds, command bounds, and selected self-collision distances. The stage cost is

\begin{split}\ell={}&\tfrac{1}{2}\|e_{p}\|_{Q_{p}}^{2}+\tfrac{1}{2}\|e_{R}\|_{Q_{R}}^{2}+\tfrac{1}{2}\|\dot{p}_{WE}-\dot{p}_{WE}^{\mathrm{ref}}\|_{Q_{v}}^{2}\\
&+\tfrac{1}{2}\|u\|_{R_{u}}^{2}+\ell_{\mathrm{base}}+\ell_{\mathrm{posture}},\end{split}

where Q_{p},Q_{R},Q_{v}, and R_{u} are cost-weight matrices and e_{p}=p_{WE}-p_{WE}^{\mathrm{ref}}. For the predicted and reference EE orientations, represented by unit quaternions (w,\mathbf{v}) and (w_{r},\mathbf{v}_{r}),

e_{R}=w\mathbf{v}_{r}-w_{r}\mathbf{v}+\mathbf{v}\times\mathbf{v}_{r}.

The reference quaternion is sign-aligned with the prediction when w_{r}w+\mathbf{v}_{r}^{\top}\mathbf{v}<0. This quaternion residual is used for optimization; evaluation uses the geodesic rotation angle. Positions are linearly interpolated, orientations use quaternion interpolation, and reference velocities follow the interpolated position slopes. The terminal cost \Phi penalizes EE pose error.

The quadratic term \ell_{\mathrm{base}} penalizes pitch, roll, yaw rate, and command–response discrepancies; \ell_{\mathrm{posture}} penalizes arm deviations from a nominal compact configuration. Base and arm decisions are coupled through the predicted EE pose, allowing the arm to anticipate command lag and gait oscillations.

We solve (9) with OCS2 [23] multiple-shooting sequential quadratic programming (SQP) and automatic differentiation, warm-starting each update from the previous solution. A reference manager samples the plan faster than the MPC update rate and holds locomotion commands between updates. Feedback latency is compensated by propagating the measured state over one sampling interval before optimization. The feedback comprises the base state, arm joint positions, and gait phase from the policy clock. Phase is wrapped to [0,2\pi) in the shooting constraints.

IV Experiments

We first compare ReCo with alternative control architectures, then isolate the effects of response shaping and model identification. Prediction and repeatability experiments examine how shaping affects base responses, a further experiment evaluates the gait-periodic component of the response model, and hardware demonstrations evaluate onboard execution.

IV-A Experimental Setup

Benchmark. We evaluate a Unitree Go2 with a six-DoF X5 arm in MuJoCo [24] at a 2.5 ms integration step. The deterministic 16\times 16 m rolling heightfield has elevations between -0.10 and 0.10 m and RMS height 0.0438 m. The benchmark contains 50 world-frame EE pose trajectories generated from randomly sketched paths with varied geometry and timing. Targets can lie outside the arm workspace at a fixed base pose, requiring locomotion to make them reachable. These trajectories test sustained base–arm coordination across diverse motion demands. Each controller executes every trajectory nominally and with an 80 N world-+X force applied at the base center of mass for 0.1 s (8 N s impulse), yielding 100 runs per controller. References and their timing are fixed across controllers.

Baselines. We compare representatives of three control paradigms. RL+MPC: Ma et al. [9] combine learned locomotion with model-based manipulation. We reimplement their approach with predicted wrench inputs and the corresponding locomotion training, using the same OCS2 infrastructure, horizon, and arm model as ReCo. This provides a closely related hybrid comparison with a different locomotion–manipulation interface. Model-based: Pure MPC [25] coordinates the whole body without a learned locomotion policy. Learned whole-body control: RoboDuet [8], UMI on Legs [16], and Visual Whole-Body Control (Visual WBC) [15] are retrained from their public implementations for Go2–X5, retaining their reward and observation designs and matching our training budget.

Command interfaces. RoboDuet and Visual WBC require externally supplied base velocities; their evaluated low-level controllers do not generate these commands directly from a world-frame EE trajectory. We therefore pair both with a classical omnidirectional pure-pursuit (PP) tracker and convert EE targets to their native body-frame interfaces, denoting the results RoboDuet+PP and Visual WBC+PP. Table II groups these two adaptations separately. UMI on Legs receives task-frame trajectory chunks and coordinates base and arm motion through its learned controller. All methods use the same reference clock; imposed trajectories replace high-level visual or task policies.

Training and metrics. All learned controllers are trained in Isaac Gym [26] and evaluated in MuJoCo. The ReCo locomotion policy converges in approximately 10 h on one RTX 4090 with about 12 GB GPU memory. Position error is \|p_{WE}-p_{WE}^{\mathrm{ref}}\|_{2}; orientation error is the geodesic angle of (R_{WE}^{\mathrm{ref}})^{\top}R_{WE}. We report means and sample standard deviations of per-run RMSE over all 100 runs. For early failures, RMSE covers the recorded portion of the run and must be read together with falls and final progress. Success requires at least 97% final progress without a fall, numerical fault, or early termination, endpoint position and orientation errors within 0.15 m and 40^{\circ} held continuously for at least 0.5 s, and at least 40% of samples within both tolerances.

Implementation. MPC runs at 100 Hz and uses a 1.0 s horizon with 67 shooting nodes at approximately 15 ms spacing. Each update performs three SQP iterations with RK2 integration on three threads. Base-state feedback comes from the simulator in simulation and from the leg-kinematics odometry on hardware.

IV-B Trajectory Tracking Performance

Figure 3 illustrates the spatial and temporal demands of the benchmark. ReCo achieves the lowest mean position and orientation RMSE in Table II, 28.7% and 27.4% below the best baseline on each metric (Pure MPC and Ma et al., respectively). Paired Wilcoxon signed-rank tests over the 100 runs favor ReCo against every baseline on both metrics (p<10^{-3} after Holm correction). Decomposed into lateral-path and arc-length timing components, the mean position errors of ReCo are 0.1261 and 0.1701 m, compared with 0.2270 and 0.3608 m for Ma et al.

ReCo also completes more of each reference: its mean progress is 11.4 percentage points above the next-best controller, at a mean base speed (0.205 m/s) comparable to Ma et al. (0.217 m/s) and above Pure MPC (0.158 m/s). It succeeds in 75/100 runs (39/50 nominal and 36/50 push), compared with 42/100 for RoboDuet+PP, the strongest baseline by success count, and records the fewest falls. Thus, the tracking improvement accompanies greater completion and fewer falls.

Baseline behavior. RoboDuet+PP and Visual WBC+PP complete at most 65.7% of the reference on average, with position RMSE 5.0 and 10.8 times that of ReCo. Their externally supplied base paths are not jointly optimized with arm motion for the imposed EE reference. These results are consistent with a benefit from planning base displacement and arm motion together, although the comparison does not isolate planning from differences in the underlying controllers.

UMI on Legs is trained to track task-frame trajectories drawn from manipulation demonstrations [16]. On in-distribution trajectories, our retrained policy achieves position and orientation RMSEs of 0.035 m and 4.7^{\circ}, respectively. On the benchmark trajectories, it falls in 80/100 runs. This contrast highlights the challenge of generalizing from demonstration-derived motions to the diverse geometry and timing of independently generated references.

Disturbance response. ReCo’s mean position RMSE increases from 0.1326 m in nominal runs to 0.1371 m under the applied push (3.4%), compared with a 6.4% increase for Ma et al. ReCo records two falls in nominal runs and one in push runs. Aggregate tracking error thus changes little under the tested impulse, which covers one direction, duration, and application schedule.

Figure 4 shows a selected 3.43 m trajectory with an 8.92 s reference duration. ReCo achieves 0.0695 m position and 8.20^{\circ} orientation RMSE, less than half the error of either Pure MPC or Ma et al. on both metrics. Both ReCo and Ma et al. succeed. Pure MPC reaches the endpoint without satisfying the hold requirement; UMI on Legs falls at 12.2% progress.

IV-C Response Shaping and Model Identification

Table III reports a 2\times 2 factorial ablation of response shaping and model identification on the same benchmark. We refer to policies trained without and with response shaping as unshaped and shaped, respectively. Without identification, MPC assumes instantaneous, exact command execution.

Both response shaping and model identification reduce position error, and their combination achieves the lowest error. Shaping alone reduces position RMSE from 0.257 to 0.166 m (35.4%). For the shaped policy, identification further reduces the error to 0.135 m (18.7%), compared with a 9.7% reduction for the unshaped policy. The prediction experiments in Sec. IV-D examine how shaping changes model accuracy.

Model identification improves position accuracy but increases orientation RMSE from 21.7^{\circ} to 24.3^{\circ} for the unshaped policy and from 21.5^{\circ} to 23.8^{\circ} for the shaped policy. This trade-off may reflect attitude dynamics and translation–rotation coupling omitted by the per-channel response model. All four configurations have lower mean orientation error than the best baseline in Table II. ReCo has the lowest EE jerk and leg torque RMS among the ablations, but not the lowest base jerk. Identification therefore primarily improves position accuracy in these runs.

IV-D Response Prediction and Repeatability

We compare the unshaped and shaped policies on 36 command trajectories held out from identification. A response model with the same structure is fitted independently to each frozen policy’s response. Predictions start from measured states and are propagated under recorded commands without intermediate measurement correction. We evaluate forward, lateral, and yaw-velocity errors at 0.25, 0.50, and 1.00 s. MSE is normalized by each channel’s variance and averaged equally across trajectories, channels, and horizons.

Shaping reduces normalized prediction MSE from 0.105 to 0.042 (59.6%). Table IV shows improvement at all three horizons, with little change between 0.50 and 1.00 s. Figure 5 illustrates the command transient and corresponding forecasts.

We assess repeatability under 12 matched speed and initial-phase conditions, with the arm held stationary or executing periodic six-joint motion. We compute residual RMS after subtracting phase-conditioned means over [4,11) s using 16 gait-phase bins. Under arm motion, forward-velocity residual RMS decreases from 0.0345 to 0.0139 m/s with shaping (Fig. 6). Moving the arm increases residual RMS by 0.0040 m/s for the shaped policy, compared with 0.0210 m/s for the unshaped policy. These results indicate reduced sensitivity of forward response to the tested arm motions without an explicit wrench model. Height residuals do not show the same improvement, despite height being included in the cross-domain objective.

IV-E Gait-Periodic Prediction

With the policy and the non-periodic part of the response model fixed, we evaluate the gait-periodic component in (7). Phase is extrapolated from information available before each forecast starts, and an initial offset matches the measured state. Across 36 trajectories and horizons of 0.10, 0.25, and 0.34 s, height RMSE decreases from 6.49 to 3.58 mm (44.8%). Squared errors are averaged across trajectories and horizons before taking the square root. Figure 7 shows closer alignment with the measured gait cycle; a deliberately shifted phase degrades prediction, supporting the importance of phase alignment.

In the closed-loop component comparison, adding the gait-periodic component reduces position RMSE by 5.5%, orientation RMSE by 6.9%, and vertical tracking-error RMS in the 1.5–8 Hz trot-excitation band by 6.5%.

IV-F Hardware Experiments

The Go2–X5 platform integrates a Unitree Go2, an ARX X5 six-DoF arm, and a Jetson Orin NX onboard computer. The control stack uses ROS 2 Foxy [27] and runs entirely onboard. The complete software stack, including concurrent programs, uses an average of 75% CPU and 43% RAM.

Motion-capture tracking. To bound the hardware accuracy without state-estimation drift, motion capture supplies base-state feedback and EE ground truth on eight trajectories (Fig. 8). Mean position and orientation RMSEs are 0.0760 m and 9.93^{\circ}; the shaped policy lowers them by 29.1% and 7.4% relative to the unshaped policy.

Outside the motion-capture arena, world-frame EE trajectories for object organization, cabinet-door opening, and pick-and-place are prerecorded using VR teleoperation (Fig. 1). ReCo completes these tasks during locomotion, with mean position and orientation RMSEs of 0.153 m and 24.9^{\circ} over three runs per task. EE poses are reconstructed from the onboard odometry and arm forward kinematics, so these metrics characterize tracking in the estimated world frame, without an independent measure of odometry drift. The experiments demonstrate onboard execution of ReCo on physical manipulation tasks.

V Conclusion

We presented ReCo, a framework that couples response-consistent locomotion with policy-aware MPC for legged manipulation. Response shaping and an identified closed-loop response model allowed MPC to jointly plan locomotion commands and arm motion. On the simulation benchmark, ReCo reduced position and orientation RMSE by 28.7% and 27.4% relative to the best baseline for each metric and succeeded in 75 of 100 runs, compared with 42 for the strongest baseline. Response shaping lowered normalized prediction MSE by 59.6%, and hardware experiments demonstrated onboard manipulation during locomotion. Model identification reduced position error but increased orientation error, and shaping did not improve height repeatability. Future work could address both by modeling coupled arm–base dynamics in the response model and by shaping the height response, and could extend the predictive interface to perceptive locomotion.

References

1. [1] J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V. Tsounis, V. Koltun, and M. Hutter (2019) Learning agile and dynamic motor skills for legged robots. Science Robotics 4 (26), pp. eaau5872. External Links: Document
2. [2] J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter (2020) Learning quadrupedal locomotion over challenging terrain. Science Robotics 5 (47), pp. eabc5986. External Links: Document
3. [3] T. Miki, J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter (2022) Learning robust perceptive locomotion for quadrupedal robots in the wild. Science Robotics 7 (62), pp. eabk2822. External Links: Document
4. [4] G. B. Margolis and P. Agrawal (2023) Walk These Ways: tuning robot control for generalization with multiplicity of behavior. In Proceedings of the 6th Conference on Robot Learning, Proceedings of Machine Learning Research, Vol. 205, pp. 22–31.
5. [5] C. D. Bellicoso, K. Krämer, M. Stäuble, D. Sako, F. Jenelten, M. Bjelonic, and M. Hutter (2019) ALMA—articulated locomotion and manipulation for a torque-controllable robot. In IEEE International Conference on Robotics and Automation, pp. 8477–8483. External Links: Document
6. [6] J. Pankert and M. Hutter (2020) Perceptive model predictive control for continuous mobile manipulation. IEEE Robotics and Automation Letters 5 (4), pp. 6177–6184. External Links: Document
7. [7] Z. Fu, X. Cheng, and D. Pathak (2023) Deep whole-body control: learning a unified policy for manipulation and locomotion. In Proceedings of the 6th Conference on Robot Learning, Proceedings of Machine Learning Research, Vol. 205, pp. 138–149.
8. [8] G. Pan, Q. Ben, Z. Yuan, G. Jiang, Y. Ji, S. Li, J. Pang, H. Liu, and H. Xu (2025) RoboDuet: learning a cooperative policy for whole-body legged loco-manipulation. IEEE Robotics and Automation Letters 10 (5), pp. 4564–4571. External Links: Document
9. [9] Y. Ma, F. Farshidian, T. Miki, J. Lee, and M. Hutter (2022) Combining learning-based locomotion policy with model-based manipulation for legged mobile manipulators. IEEE Robotics and Automation Letters 7 (2), pp. 2377–2384. External Links: Document
10. [10] J. Di Carlo, P. M. Wensing, B. Katz, G. Bledt, and S. Kim (2018) Dynamic locomotion in the MIT Cheetah 3 through convex model-predictive control. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1–9. External Links: Document
11. [11] R. Grandia, F. Jenelten, S. Yang, F. Farshidian, and M. Hutter (2023) Perceptive locomotion through nonlinear model-predictive control. IEEE Transactions on Robotics 39 (5), pp. 3402–3421. External Links: Document
12. [12] J. Tan, T. Zhang, E. Coumans, A. Iscen, Y. Bai, D. Hafner, S. Bohez, and V. Vanhoucke (2018) Sim-to-real: learning agile locomotion for quadruped robots. In Proceedings of Robotics: Science and Systems, External Links: Document
13. [13] A. Kumar, Z. Fu, D. Pathak, and J. Malik (2021) RMA: rapid motor adaptation for legged robots. In Proceedings of Robotics: Science and Systems, External Links: Document
14. [14] N. Rudin, D. Hoeller, P. Reist, and M. Hutter (2022) Learning to walk in minutes using massively parallel deep reinforcement learning. In Proceedings of the 5th Conference on Robot Learning, Proceedings of Machine Learning Research, Vol. 164, pp. 91–100.
15. [15] M. Liu, Z. Chen, X. Cheng, Y. Ji, R. Qiu, R. Yang, and X. Wang (2025) Visual whole-body control for legged loco-manipulation. In Proceedings of the 8th Conference on Robot Learning, Proceedings of Machine Learning Research, Vol. 270, pp. 234–257.
16. [16] H. Ha, Y. Gao, Z. Fu, J. Tan, and S. Song (2025) UMI-on-Legs: making manipulation policies mobile with manipulation-centric whole-body controllers. In Proceedings of the 8th Conference on Robot Learning, Proceedings of Machine Learning Research, Vol. 270, pp. 5254–5270.
17. [17] T. Portela, A. Cramariuc, M. Mittal, and M. Hutter (2025) Whole-body end-effector pose tracking. In IEEE International Conference on Robotics and Automation, pp. 11205–11211. External Links: Document
18. [18] T. Portela, G. B. Margolis, Y. Ji, and P. Agrawal (2024) Learning force control for legged manipulation. In IEEE International Conference on Robotics and Automation, External Links: Document
19. [19] J. Cheng, D. Kang, G. Fadini, G. Shi, and S. Coros (2025) RAMBO: RL-augmented model-based whole-body control for loco-manipulation. IEEE Robotics and Automation Letters 10 (9), pp. 9462–9469. External Links: Document
20. [20] L. Hewing, K. P. Wabersich, M. Menner, and M. N. Zeilinger (2020) Learning-based model predictive control: toward safe learning in control. Annual Review of Control, Robotics, and Autonomous Systems 3, pp. 269–296. External Links: Document
21. [21] J. Kim, Y. Han, H. Ravichandar, and S. Ha (2026) Safe navigation of bipedal robots via Koopman operator-based model predictive control. Note: arXiv:2409.14736v3Version 3; first submitted in 2024
22. [22] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov (2017) Proximal policy optimization algorithms. Note: arXiv:1707.06347
23. [23] F. Farshidian et al. OCS2: an open source library for optimal control of switched systems. Note: [Online]. Available: https://github.com/leggedrobotics/ocs2
24. [24] E. Todorov, T. Erez, and Y. Tassa (2012) MuJoCo: a physics engine for model-based control. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5026–5033.
25. [25] skywoodsz qm_control: quadruped manipulator controller using model predictive control and whole-body control. Note: Code: https://github.com/skywoodsz/qm_controlAccessed: September 16, 2026
26. [26] V. Makoviychuk, L. Wawrzyniak, Y. Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, and G. State (2021) Isaac Gym: high performance GPU-based physics simulation for robot learning. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks,
27. [27] S. Macenski, T. Foote, B. Gerkey, C. Lalancette, and W. Woodall (2022) Robot operating system 2: design, architecture, and uses in the wild. Science Robotics 7 (66), pp. eabm6074. External Links: Document