KungfuAthleteBot, 영상에서 고난도 휴머노이드 동작과 0.7초 낙상 복구를 하나의 정책으로 학습

Abstract

Video is an abundant, inexpensive source of human motion data that is rich in extreme athletic behaviours. Making it usable for humanoid robots, however, is not a matter of simply retargeting a reconstructed trajectory: video-derived motion is physically inconsistent, devoid of actuation information, and says nothing about failure or recovery. We present KungfuAthleteBot (KAB), a framework that treats “learning high-dynamic motion from video” as the central problem and resolves each of these three failure modes in turn. (C1) We build the KungfuAthlete dataset from videos of national-level martial artists and introduce a physics-guided parabolic trajectory correction that removes height floating, ground penetration, and high-frequency jitter from reconstructed aerial and landing phases. (C2) Because video carries no force information, strict tracking of a reconstructed trajectory is dynamically infeasible, and error-driven initialisation keeps re-launching the policy from infeasible aerial poses. We introduce physics-driven pseudo-low-kinetic-energy (LKE) sampling, our central mechanism for making such references learnable: it biases initialisation towards dynamically feasible states, letting the policy discover feasible actuation patterns instead of imitating infeasible ones. (C3) Finally, we introduce a direct training paradigm in which disturbance rejection and fall recovery are learned inside the same policy that tracks the video motion, requiring no recovery reference data and no manual mode switching. On a real humanoid robot, KAB learns highly dynamic skills from video and recovers from arbitrary falls in \approx 0.7 s, to our knowledge the fastest reported recovery for a unified policy. Ablations on the unified policy confirm the necessity of its components, supporting the view that repairing and compensating video data – rather than only collecting more of it – is what unlocks high-dynamic humanoid skills. Code and data: https://kungfuathletebot.github.io/.

A Preprint

Keywords Humanoid Robots, Learning from Video, Motion Tracking, Reinforcement Learning, Fall Recovery, Video-based Motion Reconstruction

1 Introduction

Humanoid robots that can execute highly dynamic, athletic skills – aerial spins, flips, and explosive strikes – are a long-standing goal of the field (Fig. 1). The dominant bottleneck is no longer the control algorithm but the data that drives it Liao et al. (2025); Wang et al. (2026); Sun et al. (2025). Motion-capture pipelines require instrumented studios, wearable hardware, and bespoke collection procedures: they are expensive, slow to scale, and restricted to controlled environments. This paper starts from a different premise: compared with motion capture and wearable systems, video is abundant, inexpensive, and able to capture elite athleticism outside the laboratory.

Learning high-dynamic motion from video, however, is harder than learning from curated motion-capture data, and this is the problem we address. Retargeting a reconstructed human trajectory to a humanoid and running reinforcement learning on it does not work out of the box, for reasons that are specific and diagnosable:

• (C1) Video reconstructions are physically inconsistent. Monocular reconstruction methods such as GVHMR Shen et al. (2024) and generative models such as GENMO Li et al. (2025a) produce motion that is globally smooth but locally unphysical: during flight the root height “floats” instead of following a ballistic arc, the feet penetrate the ground on landing, and frame-wise noise appears as joint jitter. This is not a cosmetic issue – a floating root makes the aerial phase dynamically impossible for any robot.
• (C2) Video contains no actuation information. A trajectory reconstructed from pixels describes where the body goes, not how the joints are driven to get there. In explosive motions such as a backflip, the human and the robot realise the same joint trajectory with radically different torque profiles (e.g. at the knee); demanding strict tracking therefore asks the robot to reproduce a physically infeasible joint trajectory, and learning stalls.
• (C3) Video says nothing about failure. A video ends when the motion ends; it provides no model of disturbance, loss of balance, or recovery. A policy that faithfully tracks video motion is therefore also a policy that has no idea what to do once the real world pushes it off the reference trajectory.

Our central claim is that these three failure modes are not incidental implementation details but the actual obstacles to learning high-dynamic motion from video, and that they can only be overcome jointly. Accordingly, the paper is organised as a single pipeline, where each technical contribution is precisely the remedy for one failure mode, and where an ablation shows that removing any one of them breaks the pipeline.

The video-to-robot pathway. Videos are segmented into clips, reconstructed to a kinematic sequence \mathbf{q}^{\mathrm{hum}}_{1:T}, retargeted Araujo et al. (2025) to the robot to give a reference \mathbf{q}^{\mathrm{ref}}_{1:T}, and tracked by a policy \pi. The last three steps are where the pathway breaks, and they require two sub-systems: a repair stage that restores physical validity, and a compensation stage that supplies, through learning, the dynamics and the failure model that video never contained. These are not independently swappable: without the compensation, strict single-stage tracking of an infeasible reference collapses (Fig. 4a), and without the unified recovery the robot cannot return to the motion once disturbed (Sec. 6.5).

Contributions.

• A video-derived professional high-dynamic dataset (addresses C1). In collaboration with national-level martial artists, we construct KungfuAthlete from 197 training videos, segmented into 1,726 sub-clips, reconstructed and retargeted to a Unitree G1. It covers both sustained ground skills and highly non-stationary aerial skills, with higher body linear and angular velocities than the corpora compared in Table 1. Dataset, licence and released artefacts are detailed in Sec. 3.1.
• A physics-aware repair of video artifacts (addresses C1). We replace the floating root trajectory with a physically consistent one: ground alignment from the lowest body point, event-aware propagation with contact detection, ballistic (constant-acceleration) interpolation of flight phases, penetration removal, and Savitzky–Golay smoothing. Unlike prior work that simply discards jump segments as unusable, our correction recovers them, preventing a large-scale waste of the most valuable video data.
• Physics-driven sampling for missing actuation (addresses C2). We introduce pseudo-low-kinetic-energy (LKE) sampling, our central mechanism for making unreliable video references learnable: rather than initialising wherever the current tracking error is largest, it derives the initial-state distribution from a kinetic-energy proxy of the reference and therefore never concentrates capacity on dynamically infeasible aerial poses. A standard progressive-tolerance curriculum complements it (Sec. 4.3); Sec. 4.4 contrasts our setting – unreliable video-derived references – with prior high-dynamic tracking work that assumes faithful references.
• A unified robust policy over video data (addresses C3). We propose a direct formulation in which one policy tracks the video motion, rejects disturbances, and recovers from arbitrary gravity-induced falls, with no recovery-specific reference data and no manual switching. It attains \approx 0.7 s recovery – to our knowledge the fastest reported for a unified policy – validated on real hardware.

Sec. 2 positions this work against prior art. The remainder of the paper is organised around the three failure modes: Sec. 3 presents the dataset and the repair of reconstructed motion (C1); Sec. 4 addresses missing actuation (C2); Sec. 5 presents the unified policy (C3); Sec. 6 evaluates tracking, recovery and ablations, with additional material in the appendix.

2 Related Work

Data and capture.

Humanoid motion tracking has largely been driven by motion-capture corpora such as AMASS Mahmood et al. (2019) and LAFAN1 Harvey et al. (2020), which are natural and continuous but confined to low-to-medium dynamics, and by hardware-based collection such as exoskeletons (HOMIE, HumanoidExo) Ben et al. (2025); Zhong et al. (2025) or VR teleoperation (TWIST2, SONIC) Ze et al. (2025b); Luo et al. (2025), which are physically consistent but require specialist hardware and scale poorly. Video is therefore attractive: monocular reconstruction such as GVHMR Shen et al. (2024) and PromptHMR Wang et al. (2025) recovers complex motion from arbitrary footage, at the cost of global drift, temporal jitter and error accumulation that make the result unusable for control without repair.

Learning motion from video.

FARM Jing et al. (2025), HDMI Weng et al. (2025) and SONIC Luo et al. (2025) retarget GVHMR or GENMO Li et al. (2025a) output to a humanoid; ZEST Sleiman et al. (2026) combines MegaSaM Li et al. (2025c) and TRAM Wang et al. (2024). None of them treats reconstruction artifacts systematically. Exbody2 Ji et al. (2024) filters out hard motions with a tuned threshold, while ASAP He et al. (2025a) and KungfuBot Xie et al. (2025) mask or offset physically invalid frames; all of them discard what cannot be made compliant, which is precisely the high-dynamic data we want. Our correction instead repairs the aerial phase.

High-dynamic tracking.

ASAP adds a delta-action model and fine-tuning; KungfuBot filters via centre of mass and adapts the tracking tolerance to the current error; HuB Zhang et al. (2025) and Agility Meets Stability Pan et al. (2025) regularise foot and centre-of-mass behaviour. BeyondMimic Liao et al. (2025) uses diffusion models and dynamic sampling, and Robot Dancing Sun et al. (2025) uses residual actions and domain randomisation, but neither sustains long high-dynamic sequences nor handles a fall. OmniXtreme Wang et al. (2026) scales policy capacity and refines for actuation, assuming high-fidelity references.

Recovery.

HumanUP He et al. (2025b) and HoST Huang et al. (2025) learn standing-up policies without data, and Unified Humanoid Get-Up Spraggett (2025) generalises across morphologies, but all treat recovery as an isolated task. SafeFall Meng et al. (2025) switches to a protective policy, and FIRM Xu et al. (2025) unifies prevention, mitigation and get-up from limited demonstrations; BFM-Zero Li et al. (2025b) unifies tracking and recovery but needs auxiliary data. StableMimic Wu et al. (2026) also unifies tracking and recovery in one policy, and reports smoother, more human-like post-fall motion; it obtains that style by shaping recovery toward a library of retargeted human get-up references. Our recovery is shaped by no human reference at all: it is learned from the same tracking objective as the motion itself (Sec. 5), which is what makes it applicable when no paired get-up demonstration exists. None of the works above learns recovery jointly with tracking from video, and none does so without human reference data.

3 KungfuAthlete: Repairing Video-Derived Motion

3.1 From raw videos to robot trajectories

The dataset is collected from daily training videos of national-level martial arts athletes: 197 videos, automatically segmented into 1,726 sub-clips, each reconstructed with GVHMR Shen et al. (2024) and retargeted with GMR Araujo et al. (2025). The corpus mixes ground actions (822 instances) and jumping actions (170 instances); the full per-genre breakdown is given in Appendix C.

We split the dataset into KungfuAthlete (Ground) (no jumping) and KungfuAthlete (Jump) (aerial motions). Table 1 shows that the Jump subset attains the highest body linear and angular velocities of all compared datasets, while the Ground subset still exceeds low-intensity corpora such as AMASS and PHUMA and sits just below LAFAN1. The corpus therefore spans a wide dynamic range, from sustained ground force production and rapid posture adjustment to highly non-stationary aerial behaviour.

Dataset release.

KungfuAthlete is released as follows. Scale: 992 annotated instances (Ground 822 / Jump 170), of which 848 screened samples are publicly released, stored as 30 fps robot qpos (3 position, 4 quaternion, 29 DoF). Artefacts: the retargeted G1 reference trajectories used in all experiments; the reconstructed motion as GVHMR predictions in SMPL-H; the segmentation indices and per-clip metadata; the height-correction code and training scripts needed to reproduce every result; and a trained fall-recovery checkpoint. Access: these artefacts are available at the project page given in the abstract; the full dataset will be released publicly under an MIT licence on acceptance. Raw video: the original recordings are not redistributed, as they contain personally identifiable footage; the derived motion is sufficient to use or check the dataset.

3.2 Repairing root height floating and jitter

As shown in Fig. 2(a), existing methods Araujo et al. (2025); Li et al. (2025a); Shen et al. (2024); Lee et al. (2025); Zhao et al. (2026) often produce a “floating” effect during the landing phase when processing jumping motions; as a consequence, prior work (e.g., PHUMA) tends to discard such segments entirely, wasting precisely the most dynamic samples a video corpus offers. We instead repair them. Given a retargeted pose sequence \mathcal{Q}=\{\mathbf{q}_{t}\}_{t=1}^{T}, where \mathbf{q}_{t}=[\mathbf{p}_{t},\mathbf{r}_{t},\boldsymbol{\theta}_{t}], let P(t)=p_{t}^{(z)} denote the root height. We enforce ground consistency while preserving temporal trends.

For each frame t, let z_{\min}(\mathbf{q}_{t})=\min_{i\in\mathcal{B}}z_{i}(\mathbf{q}_{t}) be the world-space height of the lowest body part, so that physically valid poses satisfy z_{\min}\geq 0. We then (1) align the first frame to the ground, \hat{P}(1)=P(1)-z_{\min}(\mathbf{q}_{1}); (2) propagate events: local extrema of P(t) are found by finite differences, local minima are treated as candidate contacts at which we re-align to the ground provided this introduces no penetration, and elsewhere the height is propagated by truncated velocity integration \hat{P}(t)=\hat{P}(t-1)+\max(P(t)-P(t-1),0), which preserves upward motion and suppresses downward drift; (3) interpolate ballistically: between each local maximum t_{M} and the following minimum t_{m} with \hat{P}(t_{M})>\hat{P}(t_{m}), the trajectory follows a constant-acceleration model \hat{P}(t)=\hat{P}(t_{M})-\tfrac{1}{2}g(t-t_{M})^{2}, whose duration is set by enforcing \hat{P}(t_{m}); (4) remove residual penetration wherever z_{\min}(\mathbf{q}_{t})<0; and (5) smooth with a Savitzky–Golay filter Savitzky and Golay (1964) to suppress high-frequency jitter while preserving low-order motion structure.

Implementation.

The ambiguous local minima of step (2) are the only manual component; the released code includes the annotation tooling and the height-correction script. Unlike discard-based filtering, step (3) retains the descending and ascending phases of every detected jump.

4 Compensating for Missing Actuation with Low-Kinetic-Energy Sampling

4.1 Why strict tracking of video motion fails

High-dynamic motions reconstructed from videos lack consistent dynamical information and thus violate the robot’s intrinsic dynamics constraints. For instance, in highly explosive motions such as backflips, the joint torque distributions (e.g., knee actuation patterns) differ substantially between humans and robots, making strict motion tracking insufficient for learning physically plausible force-generation strategies. This is the concrete manifestation of C2: the reference describes kinematics, whereas the policy must output actuation.

4.2 Pseudo-low-kinetic-energy (LKE) sampling

Because video-captured jumping motions carry no actuation information, the resulting aerial poses may violate the physical feasibility of real robots. To mitigate this, we bias initialisation towards dynamically plausible states while still covering both pre- and post-jump phases. We define a pseudo kinetic energy at frame t as E_{t}=\sum_{i}(v_{t,i}^{\text{joint}})^{2}\;+\;\sum_{j}\|\mathbf{v}_{t,j}^{\text{lin}}\|^{2}\;+\;\sum_{k}\|\boldsymbol{\omega}_{t,k}^{\text{ang}}\|^{2}, aggregating joint velocities, body linear velocities, and angular velocities. Let E_{0} denote the energy of the first frame, and enforce a lower bound to avoid degenerate low-energy bias: E^{\prime}_{t}=\max(E_{t},E_{0}). Sampling probabilities are then computed via a softmin distribution:

p_{t}=\frac{\exp(-E^{\prime}_{t})}{\sum_{j=0}^{T-1}\exp(-E^{\prime}_{j})}.

This formulation prioritizes low-energy (i.e., dynamically stable) states while preventing over-concentration on near-static frames, enabling effective learning of jumping behaviours from both take-off and landing contexts.

Relation to error-driven sampling. LKE sampling is the component specific to the video setting, and its defining property is that it is physics-driven rather than error-driven: the initial-state distribution is derived from the reference trajectory alone and does not depend on the current policy, so it never concentrates capacity on frames that are dynamically impossible for the robot, however large the tracking error there becomes. Adaptive sampling (as used in BeyondMimic Liao et al. (2025)) is instead error-driven: it re-weights initialisation by a kernel whose width follows the tracking error, and can therefore focus on frames that are infeasible precisely because the reference is infeasible. LKE also introduces no kernel-width or decay hyper-parameters to tune. Fig. 4(b) compares the two.

4.3 Training pipeline

The complete reward specification is given in Appendix A: a stage-agnostic tracking reward, terms added in the second stage, and the recovery terms used by the unified policy of Sec. 5.

The training schedule is a standard three-stage curriculum that progressively tightens the tracking-termination tolerances and ends with aggressive domain randomization for sim-to-real transfer; progressive-tolerance stages followed by domain randomization are established practice in high-dynamic humanoid control Xie et al. (2025); Wang et al. (2026); Ji et al. (2024), and we follow that practice here. We adopt the following progressive-tolerance curriculum as a standard training schedule; it is not claimed as a contribution of the paper. High-dynamic motions reconstructed from videos often lack consistent dynamical information and thus violate the intrinsic dynamics constraints of the robot. The curriculum progressively tightens termination conditions to encourage the discovery of force-generation patterns that enable full-motion tracking, rather than mere imitation.

Stage I (Exploration). This stage facilitates comprehensive exploration of the motion space by leveraging diverse sampling strategies (e.g., LKE, adaptive, and uniform), without enforcing strict adherence to the reference motion. Deviations in executing high-dynamic motions (such as jumps and rotations) are permitted, avoiding over-constraining the policy and hindering the discovery of effective behaviours. The reward terms for this stage, adapted from BeyondMimic Liao et al. (2025) and augmented with additional DoF terms to encourage more realistic motion, are listed in the “Stage-agnostic” block of Table 7.

Stage II (Learning the details). Building on the framework learned in Stage I, the policy now learns the finer details of the motion under stricter termination conditions and lower tracking-error tolerances. Balance-related and motor-power-related rewards are added on top of the Stage-agnostic terms (Table 7, “Stage II additional”). The corresponding termination thresholds are given in Table 2.

Stage III (Aggressive domain randomization). After Stage II the robot can track the full motion and recover from falls. To close the sim-to-real gap, inspired by OmniXtreme Wang et al. (2026), we apply more aggressive domain randomization: typical parameters are increased by 50%, mainly affecting push, initial joint configuration, velocity, gravity, and friction (Table 3). Unlike OmniXtreme, and because of the high dynamic range of our dataset and the difficulty of the jumping task, we additionally randomize robot mass and gravity. Under this aggressive randomization the termination thresholds are relaxed accordingly, allowing the robot to recover from significant deviations and preventing premature termination.

The component specific to our setting is the initial-state distribution of Sec. 4.2.

4.4 Positioning: learning high-dynamic skills from unreliable video data

Table 4 contrasts KAB with the closest prior work along three binary axes: whether the reference motion is video-derived, whether a physically invalid aerial phase is repaired rather than discarded, and whether initialisation is derived from a kinetic-energy proxy of the reference rather than from the current tracking error. The relevant difference is not the schedule but the reference: because the reference is unreliable, a physics-driven initial-state distribution (Sec. 4.2) is the appropriate choice.

Both prior systems assume that, after processing, the reference is essentially learnable: KungfuBot removes whatever cannot be made physically compliant, and OmniXtreme intervenes on the policy rather than on the data. Neither targets the case where the reference is irreparably wrong in its most dynamic phase – a reconstructed jump that floats and encodes no actuation, which filtering would simply discard. We therefore intervene at three levels at once: repair the reference (Sec. 3.2), compensate for the missing actuation through physics-driven initialisation (Sec. 4.2) and a standard curriculum (Sec. 4.3), and unify recovery – with no get-up reference data of any kind – so the learned skill survives deployment (Sec. 5).

5 A Unified Policy for Tracking and Recovery

5.1 Problem formulation

We model video-conditioned control as a reference-conditioned Markov decision process \mathcal{M}=(\mathcal{S},\mathcal{A},P,r,\gamma). At step k, the state s_{k}\in\mathcal{S} concatenates the robot’s proprioceptive state (base angular velocity, joint positions and velocities, and the previous action) with the reference state \mathbf{q}^{\mathrm{ref}}_{k} and its phase index; the action a_{k}\in\mathcal{A}\subset\mathbb{R}^{29} targets the 29 actuated DoF through joint-level PD controllers; P(s_{k+1}\mid s_{k},a_{k}) is the simulator transition under the domain randomization of Sec. 4.3; and \gamma is the discount factor. The observation and action terms, including the privileged terms available to the critic only, are listed in Appendix B.

Unlike a motion-capture reference, which records feasible behaviour and makes tracking error a meaningful learning signal, a video-derived reference encodes kinematics without actuation (C2) and may be dynamically impossible in its most dynamic phase (C1). Minimising tracking error alone is thus not well posed, and the remainder of this section replaces it with an objective that rewards matching the reference where possible and returning to a standing, trackable state where it is not.

5.2 Unified objective

To make a video-learned skill survive the real world, we formulate autonomous fall recovery and motion tracking as a single task: recovering from arbitrary initial states to a desired target state. We define an end-to-end (one-policy) reward objective:

\max_{\pi}\quad\mathbb{E}_{\pi}\!\left[\sum_{k=0}^{\infty}\gamma^{k}\left(r_{\mathrm{mt}}+\mathbb{I}_{\mathrm{rc}}(k)\,r_{\mathrm{rc}}\right)\right],

where r_{\mathrm{mt}} is the physics-based motion-tracking reward and r_{\mathrm{rc}} is the recovery reward (specified in Appendix A), both reference-conditioned functions of the robot state \mathbf{q}_{k} and reference state \mathbf{q}^{\mathrm{ref}}_{k}. The recovery indicator is

\mathbb{I}_{\mathrm{rc}}(k)=\mathbb{I}\!\Big(\big|h_{\mathrm{ref}}(\mathbf{q}_{k}^{\mathrm{ref}})-h_{\mathrm{robot}}(\mathbf{q}_{k})\big|>\tau\Big),

where h_{\mathrm{ref}} and h_{\mathrm{robot}} are the shoulder heights of the reference and the robot at step k, and \tau is a threshold on that deviation. Intuitively, when the robot’s upper body deviates too far in height from the reference, the task is interpreted as “recover first”, and the recovery reward activates.

5.3 State initialization

The initial state is sampled from a mixture of tracking and recovery conditions:

z\sim\mathrm{Bernoulli}(p),\qquad\mathbf{q}_{0}=\begin{cases}\mathbf{q}_{0}^{\mathrm{ref}},&z=1,\\[2.0pt]
\sim\mathcal{D}_{\mathrm{GRSI}},&z=0,\end{cases}

where p is the probability of starting from motion tracking and 1-p the probability of starting from a fall. The reference is drawn from the dataset, \mathbf{q}^{\mathrm{ref}}_{0:\infty}\sim\operatorname{Sample}_{\mathrm{LKE}}(\mathcal{D}_{\mathrm{ref}}), using the LKE sampler of Sec. 4.2. This decoupling of the objective (Eq. 1) from the state initialization (Eq. 3) is what allows a single policy to own both behaviours. We optimise Eq. 1 with FastSAC Seo et al. (2025), the algorithm used for every result reported here.

5.4 Gravity-based randomized state initialization (\mathcal{D}_{\mathrm{GRSI}})

To enable the robot to recover from various fall poses and resume its tracked motion, we propose a gravity-based method for generating a variety of fall poses. The robot is initialized in a zero-torque mode and released under gravity with randomized contact friction, allowing the system dynamics to naturally generate a wide range of fallen configurations characterised by base position \mathbf{p}, base orientation \mathbf{r}, and joint configuration \boldsymbol{\theta}. The resulting samples form the fall-recovery dataset \mathcal{D}_{\mathrm{GRSI}}. No reference motion is associated with these samples; the recovery behaviour is therefore learned entirely from the tracking objective plus the recovery reward.

5.5 Termination conditions

At each time step k, we define a bad-tracking indicator as the logical disjunction (OR) of all failure conditions of the tracking-termination criterion (listed in Table 2), yielding \mathbb{I}_{\mathrm{bad}}(k). To avoid premature termination during recovery, we track the length of consecutive bad-tracking steps:

L(k)=\begin{cases}L(k-1)+1,&\text{if }\mathbb{I}_{\mathrm{bad}}(k)=1,\\
0,&\text{otherwise}.\end{cases}

The termination condition is

\mathbb{I}_{\mathrm{bad\_tracking\_terminate}}(k)=\begin{cases}\mathbb{I}\!\left(L(k)\geq\tau_{\mathrm{bad}}\right),&\text{if }\mathbb{I}_{\mathrm{rc}}=1,\\
\mathbb{I}_{\mathrm{bad}}(k),&\text{otherwise},\end{cases}

where \tau_{\mathrm{bad}} is the number of consecutive bad-tracking steps tolerated before termination, set to 150 steps (3 s at 50 Hz).

6 Experiments

We evaluate KAB in simulation and on real hardware to answer: (Q1) is the KungfuAthlete dataset sufficiently challenging for representative whole-body controllers? (Q2) can KAB itself learn high-dynamic motion from this video-derived data, and how does low-kinetic-energy sampling compare with adaptive sampling? (Q3) does the unified policy provide disturbance rejection and rapid recovery on hardware?

Setup.

Experiments run on Ubuntu 24.04 with an NVIDIA RTX 4090. Every result in this paper uses FastSAC Seo et al. (2025), a distributional off-policy actor–critic, with 8{,}192 parallel environments; the robot is the 29-DoF Unitree G1, and its observation and action terms are listed in Appendix B.

6.1 Is the dataset challenging? Stress testing whole-body controllers

We select seven representative high-dynamic and complex motions from the dataset: 213 (striking during running), 1480 (hand-supported flip), 278 (backflip), 969 (stands on one leg), 468 (360° side flip), 1307 (long-duration Tai Chi), and 1080 (combining high leg lifts and sweeping kicks). We deploy them on three recent whole-body teleoperation systems – TWIST Ze et al. (2025a), GMT Chen et al. (2025), and SONIC Luo et al. (2025) – in MuJoCo. We report mean per-joint position error (E_{\text{mpjpe}}, 10^{-3} rad), mean per-joint velocity error (E_{\text{mpjve}}, 10^{-3} rad/frame), and success rate over six trials (Table 5).

Existing unified teleoperation systems show clear limitations in handling highly dynamic motions, particularly those involving rapid pose transitions and aerial movement. While SONIC achieves partial success on relatively stable motions, this advantage does not generalise to high-dynamic scenarios, revealing poor robustness under drastic dynamic variations. These findings establish the dataset as a meaningful benchmark and, by the same token, establish that solving it requires the repair-and-compensation pipeline of Secs. 3–5: KAB, which uses exactly that pipeline, succeeds on all seven motions.

6.2 Can KAB learn high-dynamic motion from video?

To test whether the pipeline actually learns high-dynamic tracking from video-derived data – and not merely that the data is hard – we compare against a single-stage BeyondMimic Liao et al. (2025) baseline and quantify accuracy on the full benchmark.

Single-stage baseline.

The single-stage baseline uses identical environment, reward, and termination settings as our Stage II. As shown in Fig. 4(a), it cannot track challenging motions such as 468 (360° side flip): early strict tracking constraints cause premature termination and insufficient exploration, trapping the policy in a local optimum with reward stuck at 7.8. Our three-stage curriculum lets the agent first learn the motion’s structure and then refine precision, reaching a final reward of 54.6.

6.3 Sampling strategy: LKE vs. adaptive

Fig. 4(b) compares the two on Motion 278. Error-driven adaptive sampling places its restarts where the current tracking error is largest, which keeps re-initialising the policy from the same infeasible aerial poses it cannot yet realise, so episodes stay short. LKE instead selects states from the physics of the reference and is therefore independent of the policy; this is also why it needs no kernel-width or decay hyper-parameter tuned against a moving error signal. Enlarging BeyondMimic’s adaptive_kernel_size or adaptive_lambda Liao et al. (2025) does not remove the dependence, since both remain functions of the error of a policy that is still failing.

6.4 Robustness and rapid recovery

Protocol.

To make the comparison verifiable we fix one protocol. Fall states: initial states are drawn from \mathcal{D}_{\mathrm{GRSI}} (Sec. 5.4) and grouped into four families – prone, supine, lateral and twisted configurations – obtained by releasing the zero-torque robot under gravity with randomised contact friction. Identical initialisation: every method starts from the same sampled states with a fixed seed per trial. Metric: recovery time is the interval from a fallen state to a stable upright posture (shoulder height within \epsilon of standing and root orientation within tolerance, sustained for 0.5 s). Reported quantities: recovery time, averaged over the trials in which the method recovers; a method that fails to stand within the horizon is counted as a failure rather than credited with a time, since time alone can favour policies that trade stability for speed. Results average three repeated trials, in simulation and on hardware where a hardware baseline exists.

Results.

We deploy the learned policy in MuJoCo and on a Unitree G1, using Motion 1307 (a five-minute Tai Chi sequence) as the long-horizon benchmark. Under mild perturbations the robot rapidly stabilises and resumes motion (Fig. 1(d)); under large disturbances causing falls (Fig. 1(c)) the policy promptly recovers, stands up, and returns to the target trajectory. Table 6 compares against existing methods: HumanUP and HoST are reference-free but are standing-only policies and cannot track motion; FIRM supports fall buffering and standing but not tracking; BFM-Zero and Heracles unify the two but require recovery data and are slower; StableMimic also unifies the two, but shapes recovery with retargeted human get-up references and does not report a recovery time. KAB unifies tracking and recovery without recovery-specific data at \approx 0.7 s.

Real-world evaluation and diverse postures.

On the physical Unitree G1 the policy recovers from all four fall families and resumes the tracked motion, including the highly distorted twisted configurations whose asymmetric limb arrangements are far from any upright pose; recovery is reproduced in simulation and on hardware (Fig. 5), so the policy learns a generalised recovery primitive rather than a fixed one.

Recovery style, without reference data.

StableMimic Wu et al. (2026) reports smoother, more human-like post-fall motion by shaping recovery toward human get-up references, at the cost of requiring a retargeted get-up library. Our recovery uses no human reference at all: it emerges from the unified objective of Sec. 5 and is shaped by the robot’s own dynamics, which is what makes it applicable when no paired get-up demonstration exists. This is the property reported as reference-free in Table 6.

Is faster always better?

Minimising recovery time is not unconditionally desirable: an aggressive return to standing increases motor wear, mechanical stress, and the risk to nearby people or objects. What matters is whether the robot becomes usable again within the task’s time budget, and staged settings make that budget explicit. In the boxing, kickboxing and dance events of the World Humanoid Robot Games World Humanoid Robot Games Organizing Committee (2026) and in RoboCup humanoid soccer RoboCup Humanoid League Technical Committee (2025), a robot that is down cannot strike, defend or continue its routine, so time on the ground is time in which the round is lost. Such settings reward a recovery that completes inside the budget, not the fastest possible scramble. Our objective therefore does not optimise time directly: the recovery reward (Appendix A) penalises root motion before standing and rewards smooth, upright completion, so the reported time is an outcome rather than a target.

6.5 Ablations

All ablations use Motion 1307, which requires long-horizon stability and precise tracking.

(i) Recovery rewards do not harm tracking.

Adding the height and root-orientation rewards leaves the episode-length and error curves essentially unchanged, saturating at the full 500 steps (Fig. 6a,b).

(ii) Recovery components alone are insufficient.

With random initial falls and no termination tolerance the episode length stays at 1: the policy is terminated immediately at the start of every episode. With a 3 s tolerance window (150 steps) it converges to only \sim150, i.e. the policy survives but neither stands up nor tracks the full motion (Fig. 6e).

(iii) Joint design works.

The full configuration reaches the same episode length and a comparable tracking error to the tracking-only baseline, converging at \sim16k vs. \sim2.5k iterations (Fig. 6c,d).

(iv) Each reward term is necessary.

Removing the height reward gives partial tracking (episode length 300–400) but no recovery; removing the root-orientation reward destabilises training, which collapses at \sim5k iterations (Fig. 6f).

7 Conclusion

We have shown that three failure modes of video-derived motion – physically inconsistent reconstruction, missing actuation information, and the absence of any failure model – must be addressed together for the video-to-robot pathway to work. KungfuAthleteBot repairs reconstructed motion with a physics-guided parabolic correction, compensates for missing actuation with LKE sampling and a three-stage curriculum, and unifies tracking with disturbance rejection and fall recovery in a single policy that recovers in \approx 0.7 s and succeeds on every motion in our benchmark, validated on real hardware. Ablations confirm that removing any one component breaks the pipeline. The pathway is not yet end-to-end, since correction still requires manual annotation of ambiguous local minima; scaling to a larger and more diverse motion set, and broadening the evaluation protocols, are natural next steps.

References

1. Araujo et al. (2025) J. P. Araujo, Y. Ze, P. Xu, J. Wu, and C. K. Liu Retargeting Matters: General Motion Retargeting for Humanoid Motion Tracking. arXiv. External Links: 2510.02252, Document
2. Ben et al. (2025) Q. Ben, F. Jia, J. Zeng, J. Dong, D. Lin, and J. Pang HOMIE: Humanoid loco-manipulation with isomorphic exoskeleton cockpit. arXiv. External Links: 2502.13013, Document
3. Chen et al. (2025) Z. Chen, M. Ji, X. Cheng, X. Peng, X. B. Peng, and X. Wang GMT: General motion tracking for humanoid whole-body control. arXiv. External Links: 2506.14770, Document
4. Harvey et al. (2020) F. G. Harvey, M. Yurick, D. Nowrouzezahrai, and C. Pal Robust motion in-betweening. ACM Transactions on Graphics 39 (4), pp. 60:1–60:12. External Links: Document
5. He et al. (2025a) T. He, J. Gao, W. Xiao, Y. Zhang, Z. Wang, J. Wang, Z. Luo, G. He, N. Sobanbab, C. Pan, Z. Yi, G. Qu, K. Kitani, J. Hodgins, L. ". Fan, Y. Zhu, C. Liu, and G. Shi ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills. arXiv. External Links: 2502.01143, Document
6. He et al. (2025b) X. He, R. Dong, Z. Chen, and S. Gupta Learning getting-up policies for real-world humanoid robots. arXiv. External Links: 2502.12152, Document
7. Huang et al. (2025) T. Huang, J. Ren, H. Wang, Z. Wang, Q. Ben, M. Wen, X. Chen, J. Li, and J. Pang Learning humanoid standing-up control across diverse postures. arXiv. External Links: 2502.08378, Document
8. Ji et al. (2024) M. Ji, X. Peng, F. Liu, J. Li, G. Yang, X. Cheng, and X. Wang ExBody2: Advanced expressive humanoid whole-body control. arXiv. External Links: 2412.13196, Document
9. Jing et al. (2025) T. Jing, S. Chen, Y. Li, W. Xu, and R. Xu FARM: Frame-Accelerated Augmentation and Residual Mixture-of-Experts for Physics-Based High-Dynamic Humanoid Control. arXiv. External Links: 2508.19926, Document
10. Lee et al. (2025) K. Lee, S. Kim, M. Park, H. Kim, D. Hwang, H. Lee, and J. Choo PHUMA: Physically-grounded humanoid locomotion dataset. arXiv. External Links: 2510.26236, Document
11. Li et al. (2025a) J. Li, J. Cao, H. Zhang, D. Rempe, J. Kautz, U. Iqbal, and Y. Yuan Genmo: a generalist model for human motion. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Piscataway, NJ, USA, pp. 11766–11776.
12. Li et al. (2025b) Y. Li, Z. Luo, T. Zhang, C. Dai, A. Kanervisto, A. Tirinzoni, H. Weng, K. Kitani, M. Guzek, A. Touati, et al. BFM-Zero: A promptable behavioral foundation model for humanoid control using unsupervised reinforcement learning. arXiv. External Links: 2511.04131, Document
13. Li et al. (2025c) Z. Li, R. Tucker, F. Cole, Q. Wang, L. Jin, V. Ye, A. Kanazawa, A. Holynski, and N. Snavely Megasam: accurate, fast and robust structure and motion from casual dynamic videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Piscataway, NJ, USA, pp. 10486–10496.
14. Liao et al. (2025) Q. Liao, T. E. Truong, X. Huang, Y. Gao, G. Tevet, K. Sreenath, and C. K. Liu BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusion. arXiv. External Links: 2508.08241, Document
15. Luo et al. (2025) Z. Luo, Y. Yuan, T. Wang, C. Li, S. Chen, F. Castañeda, Z. Cao, J. Li, D. Minor, Q. Ben, et al. SONIC: Supersizing motion tracking for natural humanoid whole-body control. arXiv. External Links: 2511.07820, Document
16. Mahmood et al. (2019) N. Mahmood, N. Ghorbani, N. F. Troje, G. Pons-Moll, and M. J. Black AMASS: Archive of Motion Capture as Surface Shapes. arXiv. External Links: 1904.03278, Document
17. Meng et al. (2025) Z. Meng, T. Liu, L. Ma, Y. Wu, R. Song, W. Zhang, and S. Huang SafeFall: Learning protective control for humanoid robots. arXiv. External Links: 2511.18509, Document
18. Pan et al. (2025) Y. Pan, R. Qiao, L. Chen, K. Chitta, L. Pan, H. Mai, Q. Bu, H. Zhao, C. Zheng, P. Luo, and H. Li Agility Meets Stability: Versatile Humanoid Control with Heterogeneous Data. arXiv. External Links: 2511.17373, Document
19. RoboCup Humanoid League Technical Committee (2025) RoboCup Humanoid League Technical Committee RoboCup Humanoid League rules. Note: https://humanoid.robocup.org/materials/rules/Accessed 26 September 2026
20. Savitzky and Golay (1964) A. Savitzky and M. J. Golay Smoothing and differentiation of data by simplified least squares procedures.. Analytical chemistry 36 (8), pp. 1627–1639.
21. Seo et al. (2025) Y. Seo, C. Sferrazza, J. Chen, G. Shi, R. Duan, and P. Abbeel Learning Sim-to-Real Humanoid Locomotion in 15 Minutes. arXiv. External Links: 2512.01996, Document
22. Shen et al. (2024) Z. Shen, H. Pi, Y. Xia, Z. Cen, S. Peng, Z. Hu, H. Bao, R. Hu, and X. Zhou World-Grounded Human Motion Recovery via Gravity-View Coordinates. In SIGGRAPH Asia 2024 Conference Papers, New York, NY, USA, pp. 1–11. External Links: 2409.06662, Document
23. Sleiman et al. (2026) J. P. Sleiman, H. Li, A. Adu-Bredu, R. Deits, A. Kumar, K. Bergamin, M. Bhardwaj, S. Biddlestone, N. Burger, M. A. Estrada, et al. ZEST: Zero-shot embodied skill transfer for athletic robot control. arXiv. External Links: 2602.00401, Document
24. Spraggett (2025) J. Spraggett Learning to get up across morphologies: Zero-shot recovery with a unified humanoid policy. arXiv. External Links: 2512.12230, Document
25. Sun et al. (2025) Z. Sun, Y. Peng, Y. Meng, X. Li, B. Huang, Z. Bing, X. Wang, and A. Knoll RobotDancing: Residual-Action Reinforcement Learning Enables Robust Long-Horizon Humanoid Motion Tracking. arXiv. External Links: 2509.20717, Document
26. Tao et al. (2026) Z. Tao, Z. Su, P. Liu, J. Sun, W. Que, J. Ma, J. Yu, J. Cao, P. Sun, H. Liang, et al. Heracles: Bridging precise tracking and generative synthesis for general humanoid control. arXiv. External Links: 2603.27756, Document
27. Wang et al. (2025) Y. Wang, Y. Sun, P. Patel, K. Daniilidis, M. J. Black, and M. Kocabas PromptHMR: Promptable Human Mesh Recovery. arXiv. External Links: 2504.06397, Document
28. Wang et al. (2024) Y. Wang, Z. Wang, L. Liu, and K. Daniilidis Tram: global trajectory and motion of 3d humans from in-the-wild videos. In European Conference on Computer Vision, Cham, Switzerland, pp. 467–487.
29. Wang et al. (2026) Y. Wang, S. Zhu, P. Zhi, Y. Li, J. Li, Y. Li, Y. Xiao, X. Wang, B. Jia, and S. Huang OmniXtreme: Breaking the generality barrier in high-dynamic humanoid control. arXiv. External Links: 2602.23843, Document
30. Weng et al. (2025) H. Weng, Y. Li, N. Sobanbabu, Z. Wang, Z. Luo, T. He, D. Ramanan, and G. Shi HDMI: Learning interactive humanoid whole-body control from human videos. arXiv. External Links: 2509.16757, Document
31. World Humanoid Robot Games Organizing Committee (2026) World Humanoid Robot Games Organizing Committee World Humanoid Robot Games. Note: https://www.whrgoc.com/Humanoid competitive events include boxing, kickboxing, dance and fully autonomous sprint and relay races. Accessed 26 September 2026
32. Wu et al. (2026) W. Wu, M. Huang, R. Liu, J. Nie, S. Guo, and C. Li StableMimic: Smooth Human-Like Recovery for Humanoid Motion Tracking – Learning Beyond the Tracking Distribution for Structured Post-Fall Behavior. arXiv. External Links: 2608.02385, Document
33. Xie et al. (2025) W. Xie, J. Han, J. Zheng, H. Li, X. Liu, J. Shi, W. Zhang, C. Bai, and X. Li KungfuBot: Physics-based humanoid whole-body control for learning highly-dynamic skills. arXiv. External Links: 2506.12851, Document
34. Xu et al. (2025) Z. Xu, Y. Li, K. Lin, and S. X. Yu Unified humanoid fall-safety policy from a few demonstrations. arXiv. External Links: 2511.07407, Document
35. Ze et al. (2025a) Y. Ze, Z. Chen, J. P. Araújo, Z. Cao, X. B. Peng, J. Wu, and C. K. Liu TWIST: Teleoperated whole-body imitation system. arXiv. External Links: 2505.02833, Document
36. Ze et al. (2025b) Y. Ze, S. Zhao, W. Wang, A. Kanazawa, R. Duan, P. Abbeel, G. Shi, J. Wu, and C. K. Liu TWIST2: Scalable, portable, and holistic humanoid data collection system. arXiv. External Links: 2511.02832, Document
37. Zhang et al. (2025) T. Zhang, B. Zheng, R. Nai, Y. Hu, Y. Wang, G. Chen, F. Lin, J. Li, C. Hong, K. Sreenath, and Y. Gao HuB: Learning Extreme Humanoid Balance. arXiv. External Links: 2505.07294, Document
38. Zhao et al. (2026) Q. Zhao, K. Yang, X. Wang, S. Zhao, Y. Lu, X. Zhang, W. Yin, Q. Shen, X. Long, and X. Cao Make tracking easy: Neural motion retargeting for humanoid whole-body control. arXiv. External Links: 2603.22201, Document
39. Zhong et al. (2025) R. Zhong, Y. Sun, J. Wen, J. Li, C. Cheng, W. Dai, Z. Zeng, H. Lu, Y. Zhu, and Y. Xu HumanoidExo: Scalable whole-body humanoid manipulation via wearable exoskeleton. arXiv. External Links: 2510.03022, Document

Appendix A Reward Specification

As shown in Table 8, the reward design largely follows that of BeyondMimic, with one key distinction: a stronger emphasis on angular velocity tracking. This encourages the policy to prioritize stable tracking of the robot’s global motion during highly dynamic behaviours, rather than overfitting to precise tracking of individual joints.

Appendix B Observation and Action Specification

The released configuration uses the observation terms listed below. Actions are target joint positions for the 29 actuated DoF; actor and critic are three-layer ELU MLPs with hidden sizes (512,256,128), following the unitree_rl_mjlab defaults.

Appendix C Dataset Composition

The corpus comprises ground actions (822 instances: 550 daily training, 93 fist, 99 saber/sword, 80 staff) and jumping actions (170 instances: 74 daily training, 5 fist, 12 saber/sword, 10 staff, 69 skills).

Appendix D Additional Simulation Results

We provide additional MuJoCo simulation results to demonstrate the diversity and fidelity of the learned high-dynamic motions. Fig. 7 shows Motion 78 (a 720° aerial spin followed by a 360° kick), Fig. 8 shows Motion 1080 (a kick followed by a roundhouse kick) and Fig. 9 shows Motion 1480 (a side roll); together they cover strong rotational dynamics, compound striking, and robust ground interaction under highly dynamic transitions.

Appendix E Code and Data

All artefacts are available at the project page https://kungfuathletebot.github.io/. It contains (i) the retargeting and height-correction code, (ii) the training configuration files for all three curriculum stages, (iii) the trained fall-recovery checkpoint used in the paper, and (iv) the released KungfuAthlete motion data (848 screened samples as 30 fps robot qpos: 3 position, 4 quaternion, 29 DoF). The dataset, the code and the checkpoint will be published under an MIT licence on acceptance. The raw athlete videos are not redistributed, as they contain personally identifiable footage; the derived motion is sufficient to use or to check the dataset.