HiFun: a Hierarchical Framework for Efficient Functional Dexterous Manipulation Learning

Abstract

While multi-fingered dexterous robot hands have the potential to provide human-level dexterity, using them for functional manipulation remains difficult because task success depends on precise contact, coordinated arm-hand motion, and reactive recovery from execution errors. Precise contact depends on complex contact dynamics that are difficult to model accurately in simulation, motivating real-world learning. However, real-world demonstrations in the full arm-hand control space are costly and noisy because each trajectory must coordinate arm motion with high-DoF multi-finger contact. Small execution deviations can further cause slipped contact or premature force application, making recovery from off-nominal contact states important for robust execution. To deal with these challenges, we propose HiFun, a hierarchical real-world RL framework for functional dexterous manipulation tasks. Our key insight is to decouple the control complexity into a functional hand skill for precise contact execution and an arm-hand coordination policy for arm control and hand-skill activation. This design enables effective real-time human intervention for reactive recovery learning and efficient data collection with Dynamic Movement Primitive (DMP) augmentation. Across six real-world tasks on two dexterous hands, HiFun achieves 98.3% average success within one hour of online training and remains robust to dynamic disturbances. The project page is available at https://hly-123.github.io/HiFun/.

Keywords: Dexterous Manipulation, Real-world Reinforcement Learning

1 Introduction

Functional dexterous manipulation is central for robot applications from factories to homes. We study this setting through representative tasks in precise tool use, contact-rich rotation, and constrained-space manipulation. For example, Thin-Handle Tool Retrieval (Fig. 1C) requires extracting a foam-embedded handle through a narrow opening where direct grasping is blocked. The robot must place fingertip contact on the exposed handle, coordinate arm motion with multi-finger levering to create access, and form a precision grasp without losing contact. Small contact errors can cause slipping, trapping, or collisions with the opening. These stages require precise contact, high-DoF arm-hand coordination, and recovery from contact errors, making functional dexterous manipulation challenging to learn and execute.

Specifically, precise contact requires the robot to place fingertips on task-relevant regions and regulate force as local contact changes. High-DoF arm-hand coordination requires arm motion, multi-finger contact, and contact timing to be synchronized within one execution sequence. Because fine-grained contact tasks are sensitive to small deviations, policies must recover from slipped contact, missed insertion, partial progress, or premature force application rather than only reproduce nominal successful trajectories.

Existing learning routes have made substantial progress, but these challenges remain only partially addressed for functional dexterity. Precise contact can be explored at scale in simulation, enabling robust dexterous manipulation performance [30, 5, 52, 35, 51, 32]. However, sim-to-real transfer is difficult for functional manipulation due to its intricate contact dynamics. High-DoF arm-hand behaviors can be represented by imitation learning (IL) and pretrained visuomotor policies from demonstrations and robot datasets [8, 56, 3, 31, 18, 2, 48]. However, collecting real-world demonstrations for functional dexterity remains costly and noisy because each trajectory must jointly specify arm motion, multi-finger contact, and contact timing, which are difficult for an operator to perform at the same time, especially considering the embodiment gap between human and robotic hands [13, 36]. Furthermore, these demonstrations usually focus on successful cases, providing limited evidence for recovery after failures. Recovery from execution errors can be improved by real-world RL from online physical outcomes [21, 24, 26], but direct exploration and human correction in the full arm-hand action space are inefficient for learning reactive recovery.

To address these limitations, we propose HiFun, a hierarchical real-world RL framework for functional dexterous manipulation. For fine-grained finger control and precise contact execution, HiFun adapts kinesthetic hand-motion priors with residual RL and real contact feedback, producing functional hand skills grounded in physical interaction. To improve exploration efficiency in high-DoF arm-hand coordination, HiFun freezes the hand skills as action primitives and trains a coordination policy over arm motion and skill activation, with DMP-augmented rollouts providing initial coverage. During online fine-tuning, human corrections are provided through the hand-skill interface to make interventions more consistent and effective, while a multi-head critic improves temporal coordination and intervention advantage weighting (IAW) prioritizes useful corrections for reactive recovery. Together, this factorization reduces the burden of full arm-hand demonstrations, exploration, and correction, enabling reliable functional dexterous manipulation with efficient online training. We evaluate HiFun on six functional dexterous tasks across two dexterous hands, as shown in Fig. 1. HiFun achieves 98.3% average success within one hour of online training, outperforms IL and flat human-in-the-loop RL baselines, and remains robust to disturbances, while ablations verify the contributions of the hand-skill and coordination components.

In summary, our contributions are: (1) a hierarchical real-world RL framework that decouples high-DoF arm-hand control into functional hand-skill learning and skill-conditioned coordination, enabling efficient real-world learning of functional dexterous tasks; (2) residual-RL hand-skill learning that adapts kinesthetic priors with real contact feedback for fine-grained contact execution; (3) skill-conditioned coordination that uses hand skills as action primitives, DMP-augmented rollouts for efficient initial coverage, a multi-head critic for temporal coordination of arm motion and hand activation, and IAW for sample-efficient online recovery learning.

2 Related Work

Dexterous Manipulation. Model-based control plans stable dexterous grasps from explicit object and contact models [27, 37], but functional manipulation depends on contact-rich dynamics that are difficult to model accurately. Large-scale simulation has enabled robust dexterous grasping [55, 7, 15, 23, 41, 54] and in-hand rotation [30, 5, 34, 22, 53], yet functional contact dynamics involving deformation, friction variation, and fine-grained force feedback remain difficult to simulate [44, 14], causing precise contact behaviors to transfer unreliably. IL and pretrained visuomotor policies can represent high-DoF arm-hand behaviors from demonstrations via diffusion-based policies [8, 49], transformer policies with action chunking [56, 4], and vision-language-action models [3, 2, 18]. For dexterous arm-hand systems, however, such demonstrations require specialized teleoperation or capture interfaces [13, 36, 43, 46, 10], and human-to-robot retargeting can introduce contact noise under morphology mismatch [47]. Thus, demonstrations for functional dexterity remain costly and noisy, struggle to cover fine-grained finger contacts and dynamic arm-hand coordination, and provide limited recovery evidence. HiFun instead obtains hand-motion priors through kinesthetic teaching [50], adapts them into contact-aware skills with residual RL, and uses these skills to collect downstream coordination data, reducing direct teleoperation of all dexterous-hand DoFs.

Real-World Reinforcement Learning. Real-world RL learns directly from physical interactions, capturing hardware dynamics and contact physics that are difficult to simulate [21, 25, 1]. Human-in-the-loop (HIL) methods improve sample efficiency by incorporating online corrections during training [26, 16, 39], and HIL-SERL achieves near-perfect success on precise gripper tasks with demonstrations and real-time guidance [26]. For functional dexterous manipulation, however, the high-dimensional arm-hand system degrades data precision and efficiency while increasing exploration burden, and coupled finger joints make consistent real-time correction across all DoFs difficult [57]. Hierarchical and skill-based methods reduce exploration complexity by reusing lower-level behaviors or guiding policy search. Prior work chains dexterous sub-policies for long-horizon manipulation [6], reuses learned rotation skills for harder in-hand reorientation [35], and guides RL exploration with simple sub-skill controllers [17]. We adopt a hierarchical framework that decouples hand primitive skills from arm-hand coordination, shifting online learning from full arm-hand exploration to skill-conditioned coordination and recovery.

3 Method

To efficiently learn reliable contact-rich dexterous manipulation policies for high-DoF arm-hand systems, HiFun uses a hierarchical real-world RL framework (Fig. 2). It learns (1) a residual-RL hand skill for contact-aware execution and (2) a coordination policy for arm motion and skill activation. Coordination uses DMP rollouts for warm start, a multi-head critic to separate arm-alignment and skill-activation Q-values, and IAW for advantage-weighted intervention imitation. We introduce formulation (Sec. 3.1), hand skill training (Sec. 3.2), and coordination training (Sec. 3.3).

3.1 Problem Formulation

We formulate dexterous arm-hand manipulation as Markov Decision Processes with observation s_{t}=(I_{t},q_{t},f_{t}), where I_{t} contains wrist and external RGB images, q_{t} contains arm-hand proprioception, and f_{t} contains fingertip contact forces. The hand skill policy \pi_{\text{skill}}(s_{t},\theta_{t}^{\text{ref}}) takes a reference hand pose from a kinesthetic demonstration and outputs a residual correction \delta\theta_{t}\in\mathbb{R}^{12}:

\theta_{t}^{\text{skill}}=\theta_{t}^{\text{ref}}+\lambda_{\text{res}}\delta\theta_{t},

where \lambda_{\text{res}} limits residual exploration around the demonstrated motion. Here, \theta_{t}^{\text{ref}} denotes the reference hand pose selected at execution step t, allowing an earlier reference state to be revisited during contact recovery (Sec. 3.2). The coordination policy \pi_{\text{coord}}(s_{t}) outputs a_{t}^{\text{coord}}=[\Delta x_{t},\alpha_{t}], where \Delta x_{t}\in\mathbb{R}^{6} is the arm end-effector increment and \alpha_{t}\in[0,1] modulates the hand skill from initial neutral hand configuration \theta^{\text{reset}}:

\theta_{t}^{\text{hand}}=\alpha_{t}\theta_{t}^{\text{skill}}+(1-\alpha_{t})\theta^{\text{reset}}.

The robot executes (\Delta x_{t},\theta_{t}^{\text{hand}}), so \pi_{\text{skill}} determines how the fingers form contact and \pi_{\text{coord}} determines where and when that contact is used.

3.2 Contact-Aware Hand Skill Learning

The hand-skill policy abstracts high-dimensional finger articulation into robust, contact-aware functional skills. We acquire a kinesthetic prior using the hand-over-hand teaching interface of [50], recording reference hand poses \xi_{\text{ref}}=\{\theta_{0}^{\text{ref}},\ldots,\theta_{T}^{\text{ref}}\} together with fingertip-force evolution and collecting per-finger force thresholds F_{\max}^{(i)} for reward clipping. To recover from disrupted contact, the reference can rewind to the most recent contact-consistent state when measured contact deviates substantially from the expected state. Given the current observation s_{t} and the selected reference pose \theta_{t}^{\text{ref}}, the residual policy outputs \delta\theta_{t} and executes Eq. (1), adjusting finger configuration and contact force to re-establish the desired contact. We train the critic on the executed hand command \theta_{t}^{\text{skill}} so value estimates correspond to the actual finger configuration.

The hand skill reward combines trajectory tracking, contact incentive, and sparse skill success:

\begin{split}r_{\text{skill}}=&-\omega_{\text{track}}\|\theta_{t}^{\text{cur}}-\theta_{t}^{\text{ref}}\|^{2}+\omega_{\text{force}}\mathcal{F}(f_{t})+\mathbb{I}_{\text{skill-success}}R_{\text{skill}},\end{split}

where \mathcal{F}(f_{t})=\sum_{i=1}^{N_{f}}\min(\|f_{t}^{(i)}\|,F_{\max}^{(i)}) sums clipped fingertip-force magnitudes. The tracking term keeps exploration near contact geometry, the force term encourages task-relevant contact, and the sparse bonus marks operator-labeled success. We train \pi_{\text{skill}} using SAC [12] on the real robot and initialize replay with kinesthetic demonstrations. Once converged, \pi_{\text{skill}} is frozen as (i) the skill primitive invoked by \pi_{\text{coord}} and (ii) a reliable intervention interface for human operators.

3.3 Skill-Conditioned Arm-Hand Coordination

Given the frozen hand skill, \pi_{\text{coord}} maps s_{t} to [\Delta x_{t},\alpha_{t}], determining where to move the arm and when to activate the hand skill. We use RLPD [1] for sample-efficient learning from offline initialization data and online real-world interaction. Task completion is detected by a binary classifier over ResNet-10 visual features, fingertip forces, and joint torques, producing the terminal reward R_{\text{task}}\in\{0,1\}.

Since the sparse reward R_{\text{task}} guides approach but provides no feedback on activation timing, the modulation reward r_{\text{mod}} penalizes premature activation and rewards successful activation:

r_{\text{mod}}=\begin{cases}r_{\text{bonus}}&\text{if }\alpha_{t}>\tau\text{ and }R_{\text{task}}=1\\
r_{\text{penalty}}&\text{if }\alpha_{t}>\tau\text{ and }R_{\text{task}}=0\\
0&\text{otherwise},\end{cases}\quad(r_{\text{penalty}}<0<r_{\text{bonus}})

Multi-head critic. Naively training one Q-function on R_{\text{task}}+r_{\text{mod}} can make the policy avoid activation: early high-\alpha trials often fail, so accumulated penalties dominate the values of high-\alpha actions. To decouple spatial approach from activation timing, we use two value heads trained with separate TD targets: a navigation head Q_{\phi}^{\text{nav}} whose target y^{\text{nav}} is built from R_{\text{task}}, and a modulation head Q_{\phi}^{\text{mod}} whose target y^{\text{mod}} is built from r_{\text{mod}}:

\mathcal{L}_{Q}(\phi)=\mathbb{E}\!\left[(Q_{\phi}^{\text{nav}}-y^{\text{nav}})^{2}+(Q_{\phi}^{\text{mod}}-y^{\text{mod}})^{2}\right].

Then the actor is trained with the combined value: \hat{Q}_{\phi}(s,a)=Q_{\phi}^{\text{nav}}(s,a)+Q_{\phi}^{\text{mod}}(s,a). During actor optimization, we evaluate Q^{\mathrm{nav}} with gradients stopped through \alpha_{t}, and evaluate Q^{\mathrm{mod}} with gradients stopped through \Delta x_{t}. This encourages the navigation head to shape arm alignment and the modulation head to shape activation timing, while both heads score the same physical action. With this multi-head critic, coordination training follows an offline-to-online scheme.

Offline pre-training. To avoid the prohibitive sample complexity of learning from scratch, we bootstrap coordination from one demonstration containing an arm trajectory and skill-activation schedule. Cartesian Dynamic Movement Primitives (DMP) [42] generate diverse arm trajectories by randomizing initial poses while fixing the goal. DMP action labels use clipped waypoint differences for \Delta x_{t} and align \alpha_{t} with the demonstration by task progress. The resulting \mathcal{D}_{\text{dmp}} provides initial-state coverage. We pre-train \pi_{\text{coord}} with SAC plus a global behavior-cloning term:

\displaystyle\mathcal{L}^{\text{off}}_{\pi}= \\ \displaystyle\mathbb{E}_{s\sim\mathcal{D}_{\text{dmp}},a\sim\pi}[\eta\log\pi(a|s)-\hat{Q}_{\phi}(s,a)] \\ \displaystyle+\lambda_{\text{gbc}}\mathbb{E}_{(s,a^{\text{dmp}})\sim\mathcal{D}_{\text{dmp}}}[\|\bar{a}_{\pi}(s)-a^{\text{dmp}}\|^{2}],

where a^{\text{dmp}} and \bar{a}_{\pi} denote the DMP label and deterministic policy action. This keeps the policy near generated approach-and-activation behavior while allowing critic improvement.

Online fine-tuning. Online training uses real policy rollouts and skill-level corrections to refine \pi_{\text{coord}} for better target alignment, skill timing, and recovery under real contact dynamics. Arm interventions use relative SE(3) retargeting through Gello [45] for smooth takeover, while hand interventions trigger the frozen \pi_{\text{skill}} by button press rather than teleoperating all hand DoFs. Rollouts and intervention segments are stored for RLPD updates, and IAW weights an intervention-imitation loss by the critic-estimated advantage of each correction. For an intervention action a^{I}, we estimate its advantage over the current deterministic action using the target critic ensemble:

A^{I}(s)=\min_{i}\hat{Q}_{\bar{\phi},i}(s,a^{I})-\min_{i}\hat{Q}_{\bar{\phi},i}(s,\bar{a}_{\pi}(s)).

The imitation weight and loss are

w(s)=\sigma(\mathrm{Norm}(A^{I}(s))/\tau_{I}),\qquad\mathcal{L}_{\text{IAW}}=\mathbb{E}_{(s,a^{I})\sim\mathcal{D}_{I}}[w(s)\|\bar{a}_{\pi}(s)-a^{I}\|^{2}].

The online actor objective augments the standard SAC actor loss with the IAW term, \mathcal{L}^{\text{on}}_{\pi}=\mathcal{L}_{\text{SAC}}+\lambda_{I}\mathcal{L}_{\text{IAW}}. IAW prevents blind cloning: corrections estimated to improve over the current policy receive larger weights, while low-advantage corrections have limited influence.

4 Experiments

4.1 Experimental Setup

We evaluate HiFun on six real-world tasks spanning Power-Drill Actuation, Angle-Spreader Retrieval, Thin-Handle Retrieval, Power-Drill Bit Removal, Pipette Dispensing, and Thin-Shaft Tool Retrieval, corresponding to Fig. 1A–F in order. These tasks cover precise tool-use, contact-rich rotation, and constrained-space manipulation. A trial is counted as successful only when the task-specific functional outcome is completed: tightening the target screw (A), fully extracting the target tools (B,C,F), removing the drill bit (D), or dispensing into the vial (E). All tasks use Franka Research 3 arms [11] with either a Sharpa hand [40] or an XHand [38], wrist RGB observations from an Intel RealSense D405, and a side-view D415 when needed.

Metrics. For final evaluation, each task randomizes object poses within the training workspace while using the same robot reset strategy as training. We report IL data-collection time for recording 200 demonstrations, RL data-collection time for generating 100 automated DMP rollouts without human labor, RL online training time for HIL fine-tuning of the coordination policy, and success rate, the percentage of evaluation trials that meet the task criterion. Full evaluation protocol details are provided in the Appendix.

Baselines and Protocol. To evaluate HiFun, we compare against HIL-SERL [26], a human-in-the-loop RL method that combines offline data, online RL, and real-time interventions for real-world manipulation. The key difference is that HiFun packages contact execution into a frozen hand skill and trains a hierarchical coordination policy over arm motion and skill activation, whereas HIL-SERL keeps a flat policy architecture that directly outputs full-DoF arm-hand actions.

HIL-SERL (Glove+Gello) uses AnyTeleop [36] for hand-mapped teleoperation and Gello [45] for arm control, testing the benefit of the hand-skill intervention interface over full arm-hand correction. HIL-SERL (Ours Interv.) uses the same learned intervention interface as HiFun but keeps a flat full-DoF policy, isolating the benefit of the hierarchical architecture beyond the intervention mechanism. All RL methods in Tab. 1 are initialized from an offline demo buffer containing 100 DMP-generated trajectories and trained online for up to one hour, a cap that pilot runs found sufficient while limiting variability from operator attention and intervention timing. Three operators trained and evaluated the six tasks. Final success rates differed by approximately 2–3 percentage points across operators, although their intervention styles produced modest behavioral variation, suggesting that performance does not depend strongly on a single operator.

We additionally evaluate Skill-HIL, which equips standard HIL-SERL with the same frozen hand skill for robot execution, to test whether skill reuse alone explains HiFun’s gains. Although the skill reduces the burden of finger-level exploration and correction, coordination learning still requires credit assignment between arm alignment and skill activation, as well as effective learning from heterogeneous human interventions. The controlled comparison in Appendix D.3 shows that sharing the hand skill alone does not recover HiFun’s performance under matched task settings.

For IL baselines, Behavior Cloning (BC) [33] and Diffusion Policy (DP) [8] are trained with 200 successful teleoperated demonstrations per task to test whether offline imitation from human demonstrations can handle functional dexterous tasks, where small contact or timing deviations often cause failure. To distinguish training-data coverage from the learning paradigm, the Appendix reports additional IL baselines trained on the same successful trajectory sources as HiFun.

4.2 Main Results

We evaluate the performance of HiFun by comparing it against representative baselines that include both RL and IL paradigms. Tab. 1 shows the strongest performance: 98.3% average success across six tasks and two hands with 18.8 minutes of RL data collection and 46.2 minutes of online training. To further validate the autonomy of HiFun, we monitor the intervention rate over time, defined as the ratio of intervention steps to total steps per episode. In the Appendix, intervention-rate curves show human guidance approaching zero by the end of training, indicating autonomous execution.

Despite sharing the same RL backbone and initial data, HiFun substantially outperforms the HIL-SERL baselines, which we attribute to intervention quality and exploration efficiency. Specifically, standard teleoperation methods (Glove+Gello) generate noisy interventions due to morphology mismatches and retargeting errors, leading to inconsistent demonstrations (for instance, wrist shaking when operating a power drill) and a degraded replay buffer. For tool retrieval tasks with complex hand motions, obtaining successful corrections becomes exceedingly challenging, leading to complete failures marked “–” in Tab. 1. Using our hand-skill intervention interface improves correction quality, as HIL-SERL (Ours Interv.) rises to 13.7% average success, but its flat full-DoF policy remains far below HiFun. The staged success results marked with \ddagger, such as 40 — 0, 28 — 0, and 30 — 0, show that this flat policy often approaches the target but fails to complete reliable hand contact. This indicates that the absence of hierarchical decomposition still limits exploration efficiency, particularly for tasks with complex hand motions. For example, a flat policy for Thin-Handle Retrieval must discover sustained non-prehensile levering, precision grasping, and collision avoidance as one multi-stage contact sequence, which requires many full arm-hand interactions, producing many noisy rollouts before the correct contact sequence is found. Overall, the results indicate that our hierarchical structure effectively improves intervention quality by leveraging fine-grained hand skill primitives and enables more efficient learning by simplifying the exploration space.

Furthermore, even with 200 expert demonstrations per task, IL baselines struggle with precise and contact-rich tasks. BC suffers from compounding error in long-horizon execution, reaching only 15% on Power-Drill Actuation and 12% on Pipette Dispensing. DP improves by modeling multimodal action distributions, achieving 60% on Angle-Spreader and 58% on Thin-Shaft Tool, but still fails on the Bit Removal (36%) and Pipette (26%) tasks. These failures highlight deficiencies in fine-grained alignment and reactive behavior, indicating that successful offline demonstrations weakly cover contact deviations and provide limited recovery signal. To separate data coverage from the learning paradigm, we additionally report IL baselines in the Appendix trained with successful trajectories from HiFun. These policies remain brittle under precise alignment requirements and off-nominal contact states, indicating that purely offline training makes them sensitive to distribution shifts during inference. Our method addresses this issue by incorporating online exploration and human interventions to refine closed-loop behaviors. In summary, the results show that our method achieves both more efficient and effective learning of contact-rich functional dexterous manipulation tasks compared with the baselines.

Robustness Evaluation. We evaluate recovery from task-level perturbations and finger-level contact disturbances. For the full system, we conduct 20 perturbation trials per task, covering randomized object poses across all tasks, approach-phase displacements in alignment-sensitive tasks, and drill-chuck re-tightening during Power-Drill Bit Removal. HiFun succeeds in 20/20 trials on four of the six tasks and 18/20 on both Power-Drill Actuation and Pipette Dispensing. The remaining failures involve screw–drill angular misalignment or vial motion after alignment. Successful trials show that HiFun re-aligns before contact and suspends hand-skill activation until contact becomes reachable (Fig. 1), consistent with recovery through adaptive activation timing rather than fixed replay. Additional recovery examples are provided in the videos on the project website.

To isolate finger-level adaptation, we additionally fix the arm near the execution region and evaluate the learned hand skills under unseen object orientations and external force disturbances. Under orientation changes, Angle-Spreader Retrieval achieves 19/20 success with toolbox yaw of 0–30^{\circ} and tool pitch/roll of 0–20^{\circ}, while Power-Drill Bit Removal achieves 18/20 with drill yaw up to 30^{\circ}. Under external force disturbances, the fixed-arm Angle-Spreader Retrieval skill achieves 18/20 success. Fig. 3 illustrates this recovery mechanism: after an unexpected collision reorients the key and disrupts thumb contact, reference-state rewind and residual finger corrections restore contact without arm re-alignment. The supplementary video also shows the same hand skill operating on 6.5 cm and 5.5 cm tool variants after learning with a 7.5 cm tool. Together, these results support local finger-level contact recovery and transfer across orientation and size variations around the demonstrated contact structure.

4.3 Ablation and Analysis

1) Arm-Hand Coordination: Tab. 2 analyzes coordination policy training from two aspects: (i) the effect of warm-start data sources (DMP rollouts vs. human demonstrations) and (ii) the contributions of the multi-head critic and intervention advantage weighting (IAW) through component ablations.

Data Source Analysis. Compared with human demonstrations, DMP rollouts achieve the same or higher final success (100%) with about 4\times less collection time (Tab. 2), indicating that automatically collected DMP data can serve as a practical alternative to labor-intensive human demonstrations. Additional trajectory-distribution analysis is provided in the Appendix.

Effect of Multi-head Critic. Tab. 2 shows that the multi-head critic is important for complex arm-skill coordination: removing it drops success from 100% to 85% on Angle-Spreader and to 75% on Bit Removal. To explain this drop, we measure how close the arm end-effector is to the target when the trained policy triggers the hand skill. The multi-head critic keeps 95% of activations within the 2.00\,\mathrm{cm} pre-contact region, while the single-head variant increases the 95th-percentile error to 4.58\,\mathrm{cm}. This indicates that a single-head critic activates the skill before the arm is sufficiently aligned, because it conflates sparse terminal success rewards with smaller modulation penalties. Simply increasing the penalties does not resolve the conflict, as it can over-suppress activation to avoid penalty accumulation. By decoupling arm-alignment and skill-activation value learning, the multi-head critic enables efficient arm movement with better-timed hand-skill activation.

Effect of IAW. During online fine-tuning, intervention actions provide an additional imitation loss for the actor objective. IAW weights each intervention-imitation loss by the critic-estimated advantage in Eq. (7), comparing the human correction with the current policy action. We compare it with two variants: Uniform Interv. BC, which keeps the imitation loss but assigns all interventions equal weight, and w/o Interv. BC, which removes this imitation loss from the actor objective. Uniform Interv. BC drops success to 60% and w/o Interv. BC to 40%, while HiFun attains lower cycle time (6.69 s) than both variants (9.21 s and 10.03 s). These results show that intervention imitation helps, but uniform cloning can copy noisy or lower-value corrections and pull the actor away from Q maximization. The Appendix further shows that IAW assigns separated weights to weak and strong corrections (0.175 vs. 0.884 mean), with high-weight corrections occurring where the current policy action has lower estimated value. Thus, IAW concentrates imitation on states where the policy needs guidance while limiting low-advantage corrections that could conflict with the critic objective.

2) Hand Motion Control Comparison: To isolate contact execution, Tab. 3 compares the learned residual hand skill \pi_{\text{skill}} with open-loop kinesthetic replay and retargeted teleoperation, with DexPilot∗ and AnyTeleop∗ reported after task-specific tuning. Ours succeeds in all 20 trials on all tasks, while Kinesthetic Replay succeeds only partially on Power-Drill Bit Removal (5/20) and fails elsewhere because geometric trajectories without force feedback cannot sustain precision contact or constrained extraction; retargeted teleoperation reaches 20/20 on Pipette but drops on constrained retrieval and drill-bit removal due to fine-grained thumb-finger retargeting errors. Thus, residual contact modulation is necessary for reliable functional hand skills across both embodiments.

5 Conclusion

Functional dexterous manipulation on real robots is difficult because precise contact, arm-hand coordination, and recovery are tightly coupled in a high-dimensional control space. We present HiFun, a hierarchical real-world RL framework that learns contact-aware hand skills separately and freezes them for coordination and human intervention. The resulting skill-conditioned policy optimizes arm motion, skill activation, and recovery without directly exploring or correcting all arm-hand DoFs. DMP-augmented rollouts warm-start coordination, the multi-head critic improves temporal arm-hand coordination, and IAW prioritizes useful interventions for recovery learning. Across six real-world tasks and two dexterous hands, HiFun achieves 98.3% average success within one hour of online training and remains robust to dynamic disturbances. These results suggest that structuring dexterous learning around functional hand skills is a practical path toward efficient, reliable real-world manipulation with multi-fingered hands.

6 Limitations

HiFun assumes task-relevant hand skills learned from kinesthetic priors and real contact feedback. This improves fine-grained contact execution and reduces full arm-hand exploration, but tasks demanding contact behaviors beyond the learned skill set may fail and require additional priors, online adaptation, or reusable hand-skill libraries. Our method also targets reliable low-level execution rather than open-ended planning or language-conditioned generalization. This scope makes it complementary to VLA policies and world models: HiFun could provide contact-rich low-level controllers and consistent autonomous rollouts, while pretrained models provide task selection and semantic grounding. Combining pretrained reasoning with HiFun controllers remains future work.

References

1. [1] P. J. Ball, L. Smith, I. Kostrikov, and S. Levine (2023) Efficient online reinforcement learning with offline data. In International Conference on Machine Learning, pp. 1577–1594.
2. [2] K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, S. Jakubczak, T. Jones, L. Ke, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, L. X. Shi, J. Tanner, Q. Vuong, A. Walling, H. Wang, and U. Zhilinsky (2026) \pi_{0}: A vision-language-action flow model for general robot control. External Links: 2410.24164, Link
3. [3] A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, X. Chen, K. Choromanski, T. Ding, D. Driess, A. Dubey, C. Finn, P. Florence, C. Fu, M. G. Arenas, K. Gopalakrishnan, K. Han, K. Hausman, A. Herzog, J. Hsu, B. Ichter, A. Irpan, N. Joshi, R. Julian, D. Kalashnikov, Y. Kuang, I. Leal, L. Lee, T. E. Lee, S. Levine, Y. Lu, H. Michalewski, I. Mordatch, K. Pertsch, K. Rao, K. Reymann, M. Ryoo, G. Salazar, P. Sanketi, P. Sermanet, J. Singh, A. Singh, R. Soricut, H. Tran, V. Vanhoucke, Q. Vuong, A. Wahid, S. Welker, P. Wohlhart, J. Wu, F. Xia, T. Xiao, P. Xu, S. Xu, T. Yu, and B. Zitkovich (2023) RT-2: vision-language-action models transfer web knowledge to robotic control. External Links: 2307.15818, Link
4. [4] T. Buamanee, M. Kobayashi, Y. Uranishi, and H. Takemura (2024) Bi-act: bilateral control-based imitation learning via action chunking with transformer. External Links: 2401.17698, Link
5. [5] T. Chen, M. Tippur, S. Wu, V. Kumar, E. Adelson, and P. Agrawal (2023) Visual dexterity: in-hand reorientation of novel and complex object shapes. Science Robotics. Note: arXiv:2211.11744 External Links: Document
6. [6] Y. Chen, C. Wang, L. Fei-Fei, and C. K. Liu (2023) Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulation. Conference on Robot Learning (CoRL).
7. [7] Z. Chen, Q. Yan, Y. Chen, T. Wu, J. Zhang, Z. Ding, J. Li, Y. Yang, and H. Dong (2025) ClutterDexGrasp: a sim-to-real system for general dexterous grasping in cluttered scenes. In Conference on Robot Learning,
8. [8] C. Chi, Z. Xu, S. Feng, E. Cousineau, Y. Du, B. Burchfiel, R. Tedrake, and S. Song (2025) Diffusion policy: visuomotor policy learning via action diffusion. The International Journal of Robotics Research 44 (10-11), pp. 1684–1704.
9. [9] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby (2020) An image is worth 16x16 words: transformers for image recognition at scale. ArXiv abs/2010.11929. External Links: Link
10. [10] H. Fang, B. Romero, Y. Xie, A. Hu, B. Huang, J. Alvarez, M. Kim, G. Margolis, K. Anbarasu, M. Tomizuka, E. Adelson, and P. Agrawal (2025) DEXOP: a device for robotic transfer of dexterous human manipulation. arXiv preprint arXiv:2509.04441.
11. [11] FRANKA ROBOTICS FRANKA-RESEARCH-3. Note: https://franka.de/franka-research-3
12. [12] T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine (2018) Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pp. 1861–1870.
13. [13] A. Handa, K. V. Wyk, W. Yang, J. Liang, Y. Chao, Q. Wan, S. Birchfield, N. Ratliff, and D. Fox (2019) DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm System. arXiv. External Links: 1910.03135, Document
14. [14] E. Hsieh, W. Hsieh, Y. Wang, T. Lin, J. Malik, K. Sreenath, and H. Qi (2025) Learning dexterous manipulation skills from imperfect simulations. arXiv preprint arXiv:2512.02011.
15. [15] L. Huang, H. Zhang, Z. Wu, S. Christen, and J. Song (2025) FunGrasp: functional grasping for diverse dexterous hands. IEEE Robotics and Automation Letters.
16. [16] M. Kelly, C. Sidrane, K. Driggs-Campbell, and M. J. Kochenderfer (2019) Hg-dagger: interactive imitation learning with human experts. In 2019 International Conference on Robotics and Automation (ICRA), pp. 8077–8083.
17. [17] G. Khandate, C. P. Mehlman, X. Wei, and M. Ciocarlie (2024) Dexterous in-hand manipulation by guiding exploration with simple sub-skill controllers. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 6551–6557.
18. [18] M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, Q. Vuong, T. Kollar, B. Burchfiel, R. Tedrake, D. Sadigh, S. Levine, P. Liang, and C. Finn (2024) OpenVLA: an open-source vision-language-action model. External Links: 2406.09246, Link
19. [19] I. Kostrikov, A. Nair, and S. Levine (2022) Offline reinforcement learning with implicit q-learning. In International Conference on Learning Representations (ICLR),
20. [20] A. Kumar, A. Zhou, G. Tucker, and S. Levine (2020) Conservative q-learning for offline reinforcement learning. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 33.
21. [21] S. Levine, C. Finn, T. Darrell, and P. Abbeel (2016) End-to-end training of deep visuomotor policies. Journal of Machine Learning Research 17 (39), pp. 1–40.
22. [22] X. Liu, H. Wang, and L. Yi (2025) DexNDM: closing the reality gap for dexterous in-hand rotation via joint-wise neural dynamics model. arXiv preprint arXiv:2510.08556.
23. [23] T. G. W. Lum, M. Matak, V. Makoviychuk, A. Handa, A. Allshire, T. Hermans, N. D. Ratliff, and K. V. Wyk (2024) DextrAH-G: Pixels-to-Action Dexterous Arm-Hand Grasping with Geometric Fabrics. In Conference on Robot Learning (CoRL),
24. [24] J. Luo, Z. Hu, C. Xu, Y. L. Tan, J. Berg, A. Sharma, S. Schaal, C. Finn, A. Gupta, and S. Levine (2024) Serl: a software suite for sample-efficient robotic reinforcement learning. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 16961–16969.
25. [25] J. Luo, O. Sushkov, R. Pevceviciute, W. Lian, C. Su, M. Vecerik, N. Ye, S. Schaal, and J. Scholz (2021) Robust multi-modal policies for industrial assembly via reinforcement learning and demonstrations: a large-scale study. arXiv preprint arXiv:2103.11512.
26. [26] J. Luo, C. Xu, J. Wu, and S. Levine (2025) Precise and dexterous robotic manipulation via human-in-the-loop reinforcement learning. Science Robotics 10 (105), pp. eads5033.
27. [27] A.T. Miller and P.K. Allen (2004) Graspit! A versatile simulator for robotic grasping. IEEE Robotics & Automation Magazine.
28. [28] A. Nair, M. Dalal, A. Gupta, and S. Levine (2020) Accelerating online reinforcement learning with offline datasets. In arXiv preprint arXiv:2006.09359,
29. [29] M. Nakamoto, Y. Zhai, A. Singh, M. S. Mark, Y. Ma, C. Finn, A. Kumar, and S. Levine (2023) Cal-ql: calibrated offline rl pre-training for efficient online fine-tuning. In Advances in Neural Information Processing Systems (NeurIPS),
30. [30] OpenAI, M. Andrychowicz, B. Baker, M. Chociej, R. Jozefowicz, B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, J. Schneider, S. Sidor, J. Tobin, P. Welinder, L. Weng, and W. Zaremba (2019) Learning dexterous in-hand manipulation. arXiv preprint arXiv:1808.00177.
31. [31] A. O’Neill, A. Rehman, A. Gupta, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, et al. (2023) Open x-embodiment: robotic learning datasets and rt-x models. External Links: 2310.08864, Link
32. [32] L. Pei, T. Wu, H. Zhang, P. Luo, and J. Song (2026) Assembling two parts in one hand. External Links: 2609.10137, Link
33. [33] D. A. Pomerleau (1988) Alvinn: an autonomous land vehicle in a neural network. Advances in neural information processing systems 1.
34. [34] H. Qi, A. Kumar, R. Calandra, Y. Ma, and J. Malik (2023) In-hand object rotation via rapid motor adaptation. In Conference on Robot Learning, pp. 1722–1732.
35. [35] H. Qi, B. Yi, M. Lambeta, Y. Ma, R. Calandra, and J. Malik (2025) From simple to complex skills: the case of in-hand object reorientation. arXiv preprint arXiv:2501.05439.
36. [36] Y. Qin, W. Yang, B. Huang, K. V. Wyk, H. Su, X. Wang, Y. Chao, and D. Fox (2024) AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System. arXiv. External Links: 2307.04577
37. [37] M. A. Roa and R. Suárez (2015) Grasp quality measures: review and performance. Auton. Robots.
38. [38] ROBOTERA XHAND1. Note: https://www.robotera.com/goods/2.html
39. [39] S. Ross, G. J. Gordon, and J. A. Bagnell (2011) A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. arXiv. External Links: 1011.0686, Document
40. [40] SHARPA SHARPA WAVE. Note: https://www.sharpa.com/pages/wave
41. [41] R. Singh, A. Allshire, A. Handa, N. Ratliff, and K. Van Wyk (2024) Dextrah-rgb: visuomotor policies to grasp anything with dexterous hands. arXiv preprint arXiv:2412.01791.
42. [42] A. Ude, B. Nemec, T. Petrić, and J. Morimoto (2014) Orientation in cartesian space dynamic movement primitives. In 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 2997–3004.
43. [43] C. Wang, H. Shi, W. Wang, R. Zhang, L. Fei-Fei, and K. Liu (2024) DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation. In Robotics: Science and Systems XX, External Links: Document, ISBN 979-8-9902848-0-7
44. [44] J. Wang, Y. Yuan, H. Che, H. Qi, Y. Ma, J. Malik, and X. Wang (2024) Lessons from learning to spin” pens”. arXiv preprint arXiv:2407.18902.
45. [45] P. Wu, Y. Shentu, Z. Yi, X. Lin, and P. Abbeel (2024) Gello: a general, low-cost, and intuitive teleoperation framework for robot manipulators. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 12156–12163.
46. [46] M. Xu, H. Zhang, Y. Hou, Z. Xu, L. Fan, M. Veloso, and S. Song (2025) DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulation. arXiv. External Links: 2505.21864, Document
47. [47] Z. Yin, C. Wang, L. Pineda, K. Bodduluri, T. Wu, P. Abbeel, and M. Mukadam (2025) Geometric Retargeting: A Principled, Ultrafast Neural Hand Retargeting Algorithm. arXiv. External Links: 2503.07541, Document
48. [48] C. Yuan, Z. Zhang, M. Zhou, W. Chen, Y. Wang, Z. Liu, D. Niu, S. Wang, H. Zhang, W. Zhang, et al. (2026) FTP-1: a generalist foundation tactile policy across tactile sensors for contact-rich manipulation. arXiv preprint arXiv:2606.13102.
49. [49] Y. Ze, G. Zhang, K. Zhang, C. Hu, M. Wang, and H. Xu (2024) 3D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations. In Robotics: Science and Systems (RSS),
50. [50] D. Zhang, C. Yuan, C. Wen, H. Zhang, J. Zhao, and Y. Gao (2025) KineDex: learning tactile-informed visuomotor policies via kinesthetic teaching for dexterous manipulation. arXiv preprint arXiv:2505.01974.
51. [51] H. Zhang, S. Christen, Z. Fan, O. Hilliges, and J. Song (2024) GraspXL: generating grasping motions for diverse objects at scale. In European Conference on Computer Vision (ECCV),
52. [52] H. Zhang, S. Christen, Z. Fan, L. Zheng, J. Hwangbo, J. Song, and O. Hilliges (2024) ArtiGrasp: Physically Plausible Synthesis of Bi-Manual Dexterous Grasping and Articulation. In International Conference on 3D Vision (3DV),
53. [53] H. Zhang, J. Ferchow, J. Song, and M. Meboldt (2026) UniCross: unified cross-skill dexterous manipulation synthesis. arXiv preprint arXiv:2607.28198.
54. [54] H. Zhang, M. Meboldt, and J. Song (2026) ConGraspXL: controllable constraint-conditioned dexterous grasping motion synthesis. arXiv preprint arXiv:2609.16319.
55. [55] H. Zhang, Z. Wu, L. Huang, S. Christen, and J. Song (2025) RobustDexGrasp: robust dexterous grasping of general objects. arXiv preprint arXiv:2504.05287.
56. [56] T. Z. Zhao, V. Kumar, S. Levine, and C. Finn (2023) Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware. arXiv. External Links: 2304.13705, Document
57. [57] H. Zhu, J. Yu, A. Gupta, D. Shah, K. Hartikainen, A. Singh, V. Kumar, and S. Levine (2020) The ingredients of real-world robotic reinforcement learning. arXiv preprint arXiv:2004.12570.

Appendix A Method Details

Trajectory augmentation via Cartesian DMP. To provide offline data for coordination pre-training, we generate rollouts from a single reference demonstration instead of collecting full arm-hand demonstrations for every rollout. The reference contains an arm end-effector trajectory \{[\mathbf{p}^{ref}_{t},\mathbf{q}^{ref}_{t}]\}_{t=0}^{T} and a hand-skill activation schedule \{\alpha^{ref}_{t}\}_{t=0}^{T} collected with a Gello interface [45]. We fit Cartesian Dynamic Movement Primitives (DMP) [42] to the demonstrated arm trajectory. The translational dynamics follow a second-order spring-damper system with a learned forcing term \Phi_{p}(s), while the rotational dynamics evolve on the quaternion manifold:

\displaystyle\lambda^{2}\ddot{\mathbf{p}} \\ \displaystyle=K_{p}(\mathbf{g}_{p}-\mathbf{p})-\lambda D_{p}\dot{\mathbf{p}}+\Phi_{p}(s), \\ \displaystyle\lambda\dot{\bm{\eta}} \\ \displaystyle=K_{o}\big(2\log(\mathbf{g}_{o}*\bar{\mathbf{q}})\big)-D_{o}\bm{\eta}+\Phi_{o}(s), \\ \displaystyle\lambda\dot{\mathbf{q}} \\ \displaystyle=\tfrac{1}{2}\,\bm{\eta}*\mathbf{q}.

Here \lambda is the temporal scaling factor, s is a canonical phase variable, \mathbf{g}_{\{p,o\}} are goal configurations, and K_{\{p,o\}},D_{\{p,o\}} are stiffness and damping gains. The forcing terms \Phi_{p,o}(s) encode the demonstrated motion profile. To augment the data, we fix the goal and randomize the initial pose [\mathbf{p}_{0},\mathbf{q}_{0}] within the task workspace before integrating Eq. (9)–(11). Since DMP modifies only the arm motion, the hand activation follows the demonstration by phase alignment, \alpha_{t}=\alpha^{ref}(s_{t}).

We execute the generated trajectories on the physical robot to collect state-action transitions. The arm action label is the clipped displacement between the next DMP waypoint x^{*}_{t+1} and the current robot state x_{t}:

\Delta x_{t}=\mathrm{clip}\big(x^{*}_{t+1}-x_{t},\;-1,\;1\big).

When \alpha_{t}>\tau, the frozen hand-skill policy \pi_{\text{skill}} generates finger commands following Eq. (1); otherwise the hand remains at \theta^{\text{reset}}. The collected transitions form \mathcal{D}_{\text{dmp}}, which is used for offline pre-training and as seed data in the online replay buffer.

DMP rollout coverage analysis. The comparison illustrates why DMP augmentation is used as the offline data source for coordination pre-training. Human demonstrations contain realistic task executions but are time-consuming and vary with operator timing, while DMP rollouts systematically cover different approach states from the same task goal. This gives the coordination policy a varied warm start without repeated human arm-hand demonstrations.

SAC-based offline pre-training. We initialize the coordination policy \pi_{\text{coord}} with Soft Actor-Critic (SAC) [12] rather than specialized offline RL objectives such as CQL [20] or IQL [19]. This choice follows from the structure of \mathcal{D}_{\text{dmp}}: unlike standard offline RL datasets with mixed-quality behavior, the DMP dataset is generated from successful reference motions and mainly provides coarse coverage for later online fine-tuning. In this setting, we empirically find that IQL behaves similarly to behavior cloning and CQL adds little benefit when the data already covers successful state regions.

Figure 5 validates this choice on a representative disassembly task. The total target Q-value remains stable during offline pre-training, and the separated critic heads reveal a clear scale gap between arm-related and hand-activation values. Using SAC for both offline pre-training and online fine-tuning also avoids a value-calibration gap between pessimistic offline objectives and real online returns [29, 28].

Appendix B Experiment Details

B.1 Hardware Setup

All tasks use Franka Research 3 arms with either a Sharpa hand or an XHand. The setup includes wrist RGB observations from an Intel RealSense D405 and a side-view D415 when needed. Figure 6 summarizes the task setup and camera views across all six tasks.

B.2 Task Descriptions

The descriptions below define the task sequence, success condition, and main difficulty for each task, with (A)–(D) serving as Sharpa main validation tasks and (E)–(F) as XHand portability tasks.

(A) Power-Drill Actuation (Precise Tool-Use) requires maintaining a stable functional grasp on the drill, aligning the screwdriver bit with the target screw, and actuating the trigger to tighten the screw into the fixture. Success requires the target screw to be tightened into the fixture. The main difficulty is that axial alignment must be established before activation, since premature trigger engagement or excessive contact force can shift the hand pose and cause misalignment.

(B) Angle-Spreader Retrieval (Constrained-Space Manipulation) requires extracting the target tool from a constrained tray where surrounding slots limit direct grasping. Success requires the target tool to be fully extracted. The main difficulty is creating enough fingertip contact to expose or lift the tool before forming a stable grasp.

(C) Thin-Handle Retrieval (Constrained-Space Manipulation) requires extracting a thin-handle tool from a narrow opening where limited clearance prevents direct grasping. Success requires the target tool to be fully extracted. The main difficulty is executing non-prehensile levering to create access, followed by a precision grasp while avoiding collisions with the surrounding tray.

(D) Power-Drill Bit Removal (Contact-Rich Rotation) requires grasping the drill chuck, applying sustained rotational motion to loosen it, and removing the drill bit. Success requires the drill bit to be removed from the chuck. The main difficulty is maintaining stable multi-finger contact under friction and changing contact forces during rotation.

(E) Pipette Dispensing (Precise Tool-Use) requires aligning the pipette tip with a vial and depressing the plunger. Success requires dispensing liquid into the vial without collision or leakage. The main difficulty is applying thumb force while preserving grasp stability and keeping the tip aligned with the vial opening.

(F) Thin-Shaft Tool Retrieval (Constrained-Space Manipulation) requires extracting a thin-shaft tool embedded in foam with limited clearance. Success requires the target tool to be fully extracted. By using a contact sequence similar to Thin-Handle Retrieval with the XHand embodiment, this task tests cross-hand deployment for constrained-space retrieval.

B.3 Evaluation Protocol

Main Evaluation. For main evaluation, each task is tested over 50 trials. For every task, object poses are randomized within the red-box workspace shown in Fig. 6, and robot reset poses follow the train-time reset protocol. This evaluates repeatable functional execution within the trained workspace rather than zero-shot workspace generalization. Full success requires the task-specific functional outcome to be completed: tightening the target screw, fully extracting the target tool, removing the drill bit, or dispensing into the vial. Reported success rate is the mean success credit over 50 trials: full success receives credit 1 and failure receives credit 0. For rare baseline trials that reach the task-specific final contact stage but terminate at the evaluation horizon before completing the functional outcome, we assign credit 0.5. This distinguishes horizon-limited near-completion from failures that never establish the required contact.

Perturbation Evaluation. The robustness evaluation uses 20 perturbation trials per task. The perturbations include randomized object poses across tasks, approach-phase displacements for alignment-sensitive tasks, and drill-chuck re-tightening for Power-Drill Bit Removal. Because these perturbations place the policy in off-nominal contact states, success requires recovery through re-alignment or delayed hand-skill activation rather than replaying a nominal trajectory.

Appendix C Implementation Details

C.1 Policy Training Details

Across Sharpa and XHand, coordination policies share the same high-level action interface and RL pipeline. Task-dependent differences are the active camera streams and online interaction required during fine-tuning, with Thin-Shaft Tool Retrieval using the wrist camera and Pipette Dispensing using both wrist and side cameras.

Table 4 summarizes common policy hyperparameters, including control frequency, episode length, network size, optimizer, discount factor, and image augmentation.

For hand skill acquisition, kinesthetic prior collection is typically the time needed to record one reference trajectory and usually takes less than five minutes per task. Because the hand skill reward uses fingertip force feedback, we avoid rapid or jerky contact motions during teaching, which can introduce noisy force and contact measurements. In practice, we teach the contact sequence slowly to obtain consistent tactile signals, and temporally resample the recorded prior when faster reference execution is desired. During early residual-RL training, we initialize the residual action near zero and use a small residual action scale so the policy stays close to the kinesthetic prior while collecting initial contact data. The residual scale is then increased to allow reward-driven adaptation to real contact dynamics.

For arm-hand coordination learning, Table 5 reports the wall-clock time spent on reward-classifier data collection, automated DMP rollout collection, and online coordination-policy fine-tuning. For the reward-classifier column, the reported time includes both positive and negative frame collection. Across the six task datasets, collection takes 4–13 minutes per task and includes 10–35 positive frames with roughly 1.9k–7.9k negative frames. The imbalance mainly comes from sampling failure frames from rollout videos, while positive frames are short clips of completed outcomes. In practice, classifier quality depends on whether the wrist or side camera can observe the task-completion state and distinguish it from near-success failures. When this visibility condition holds, classifier data collection is inexpensive.

C.2 Baseline Details

HIL-SERL variants. The two HIL-SERL baselines in Tab. 1 use the same RLPD backbone and offline DMP buffer as HiFun but keep a flat policy that directly outputs full arm-hand actions. HIL-SERL (Glove+Gello) provides full arm-hand corrections using AnyTeleop [36] for hand-mapped teleoperation and Gello [45] for arm control. HIL-SERL (Ours Interv.) uses the same learned hand-skill intervention interface as HiFun while retaining the flat full-DoF policy. This separates the effect of the intervention interface from the effect of the hierarchical architecture.

Behavior Cloning. BC is trained on the same 200 human demonstrations and uses the same frozen pre-trained ResNet-10 visual encoder as our method for fair comparison. Visual features from each camera view and proprioceptive inputs are concatenated and processed by an MLP that outputs a diagonal Gaussian policy. Training minimizes the negative log-likelihood loss \mathcal{L}_{\text{BC}}=-\mathbb{E}[\log\pi_{\theta}(a\mid o)] using Adam with learning rate 3\times 10^{-4} and batch size 256 for 20,000 steps. Random cropping is applied as image augmentation, and zero-action transitions are filtered from demonstrations.

Diffusion Policy. DP [8] is trained on the same demonstrations as BC and uses a 1D temporal U-Net noise prediction network, with fully fine-tuned ViT-B/16 [9] encoders for each camera view without weight sharing. The observation and action prediction horizons are set to 2 and 16, respectively. Training uses DDIM with 50 diffusion steps and a squared cosine noise schedule, optimizing the MSE loss on predicted noise with AdamW (learning rate 1\times 10^{-4}, batch size 256) for 100 epochs. EMA with decay 0.995 is applied.

Additional IL training protocol. The main BC/DP comparison tests whether offline imitation from successful human demonstrations is sufficient for functional dexterous tasks. To separate data coverage from the learning paradigm, we additionally train BC and DP on successful trajectories collected during HiFun training, including DMP rollouts, online policy rollouts, and task-completing human-intervention segments. Additional results for these same-source IL policies are reported in Sec. D.2.

Appendix D Additional Experimental Results

Project videos. The project website shows all six tasks, position and fingertip-contact recovery, tool-size transfer, and a long-horizon multi-skill sequence.

D.1 Online Training Curves and Autonomy

Figure 7 reports intervention rate and cycle time during online training across the six tasks for HiFun and HIL-SERL (Ours Interv.). Since each successful human correction completes the task during training, success rate alone does not reveal whether the learned policy becomes autonomous. The intervention rate therefore measures how often human guidance is still needed, and cycle time measures execution efficiency. For HiFun, the intervention rate decreases toward zero within one hour of online training on every task (and in as little as {\sim}15 minutes on the fastest tasks), and its cycle time at convergence is consistently lower than HIL-SERL (Ours Interv.). In contrast, the baseline keeps a high intervention rate and never becomes autonomous within the same one-hour training budget, despite using the same intervention interface. This supports the role of HiFun in efficient online learning.

D.2 Additional IL Baseline Results

Table 6 shows that replacing human demonstrations with HiFun training trajectories does not materially improve offline imitation: BC changes from 27.5% to 26.0% average success, and DP changes from 41.5% to 42.5%. Policies trained on HiFun data often approach objects faster and hesitate less near intermediate states, consistent with exposure to rollouts and intervention segments from more varied contact states. However, they still fail to reliably complete precise alignment and contact recovery. BC and DP are trained by supervised action prediction over the collected trajectories, which does not provide a value-based mechanism for selecting which recovery-like actions should be reinforced for downstream task completion. This explains why recovery-rich offline data alone does not yield reliable recovery behavior without value-guided online RL.

D.3 Skill-HIL Comparison

We examine whether providing a learned hand skill to standard HIL-SERL is sufficient to reproduce HiFun’s performance. The skill reduces the burden of exploring and correcting high-dimensional finger actions, but two challenges remain in coordination learning. First, sparse task outcomes do not directly distinguish errors in arm alignment from errors in skill activation, making credit assignment between these action components difficult. Second, successful human interventions differ in recovery efficiency and consistency, and therefore provide supervision of varying usefulness. HiFun addresses these challenges with separate value learning for arm alignment and skill activation, plus intervention advantage weighting that scores each correction against the policy action.

To test whether HiFun’s gains come from giving HIL-SERL the learned hand skill, we evaluate Skill-HIL, which combines that frozen skill with standard HIL-SERL coordination learning. Across four Sharpa tasks under matched settings and training budgets, Skill-HIL achieves 46% average success, compared with 100% for HiFun. Most failures involve incorrect contact regions or activation timing, followed by collisions, consistent with unresolved coordination errors.

On Angle-Spreader Retrieval, doubling Skill-HIL’s training data yields 35/50 successful trials, compared with 50/50 for HiFun. Thus, Skill-HIL’s average remains lower across the four tasks, and doubling its data does not close the gap on Angle-Spreader Retrieval. This comparison evaluates HiFun’s coordination design as a whole; Tab. 2 separately ablates the multi-head critic and intervention advantage weighting.

D.4 Coordination Component Analysis

Figure 8 visualizes the mechanisms behind the coordination ablations in the main paper. In Fig. 8A, the multi-head critic reduces the 95th-percentile activation error from 4.58\,\mathrm{cm} to 2.00\,\mathrm{cm} and reduces activations outside the 2\,\mathrm{cm} pre-contact band from 6.0\% to 3.7\%. This indicates that separating arm-alignment and skill-activation value learning localizes hand-skill activation closer to the target region, supporting better temporal coordination between arm approach and hand activation. In Fig. 8B, IAW assigns clearly separated imitation weights to weak and strong corrections (0.175 vs. 0.884 mean). The matched policy-value comparison is negative, showing that high-weight corrections occur at states where the current policy action has lower estimated value. This supports the main-text claim that IAW avoids uniform cloning by focusing imitation on useful corrections while limiting low-advantage corrections that could conflict with critic optimization.

D.5 Hand Motion Control Failure Cases

Figure 9 compares hand-control methods on three contact-rich tasks: Thin-Shaft Tool Retrieval with XHand, Angle-Spreader Retrieval with Sharpa, and Thin-Handle Retrieval with Sharpa. After placing the arm near the task execution region and focusing on the task-relevant fingers, the failures mainly reflect functional finger execution rather than arm placement or full-hand teleoperation noise. Kinesthetic replay preserves the demonstrated motion pattern but executes it open-loop, losing the contact-force information needed to lift or lever objects with the task-relevant fingers. DexPilot∗, despite fingertip-based retargeting and per-task tuning, can produce coarse task-relevant contacts but remains sensitive to the human-robot morphology gap in constrained settings: in Angle-Spreader Retrieval, it can lift the tool but forms an unstable grasp, while in Thin-Shaft Tool Retrieval and Thin-Handle Retrieval, it struggles with precise pinch alignment and little-finger levering. In contrast, our residual hand skill adapts the kinesthetic prior with real contact feedback, enabling closed-loop force modulation for reliable functional hand execution.