Embodiment-Aware Planning for Heterogeneous Bimanual Manipulation

Abstract

Many manipulation tasks that are effortless for dexterous human hands remain challenging for robotic systems equipped with simple, task-specialized grippers. Industrial platforms commonly rely on end-effectors such as suction cups or parallel-jaw grippers, yet no single mode can reliably satisfy the diverse constraints encountered across different phases of complex manipulation tasks, including pickup, reorientation, insertion, and constrained placement. In this work, we investigate whether complementary simple grippers within a heterogeneous bimanual setup can collectively realize tasks that exceed the capability of any individual gripper alone. Given a single human demonstration specifying the desired object motion, we formulate embodiment-aware execution as a finite-horizon optimization over arm–trajectory assignments. At each segment, the planner determines whether single-arm manipulation or a coordinated dual-arm handover is required, jointly reasoning over heterogeneous grasp feasibility, motion constraints, manipulability, and cross-gripper compatibility. Exploiting this formulation, we compute globally consistent assignments with explicit switching regularization. Experiments on a dual Franka FR3 platform equipped with suction and parallel-jaw grippers demonstrate improved feasibility and reliability in tasks that are infeasible under single-gripper execution. WEBSITE

I Introduction

Humans perform complex manipulation tasks using highly dexterous hands capable of continuously varying contact modes, regrasping strategies, and in-hand adjustments. In contrast, most robotic systems rely on simple, task-specialized end-effectors such as suction cups or parallel-jaw grippers. While these grippers are robust, affordable, and widely deployed in industrial environments, each offers limited affordances. A suction gripper excels at lifting flat surfaces but struggles in confined insertions; a parallel-jaw gripper provides stable force-closure grasps yet often fails on thin or flush objects. Consequently, many tasks that are effortless for human hands cannot be reliably executed using a single simple robotic gripper.

In modern industrial settings such as warehouse automation, packaging lines, and retail fulfillment, objects vary widely in geometry, thickness, surface properties, and accessibility. During a single transport task, different phases may demand different grasp modalities. Rather than relying on complex anthropomorphic hands, a practical alternative is to combine complementary simple grippers within a heterogeneous bimanual system. Such a configuration has the potential to approximate the functional diversity of human manipulation while retaining the robustness and simplicity of industrial hardware. However, enabling two specialized grippers to collectively realize tasks that exceed the capability of either alone requires systematic coordination and planning.

Figure illustrates this principle. A thin plate must be transferred from a narrow shelf to a table. The suction gripper cannot safely execute the pickup due to geometric interference during approach, whereas the parallel jaw gripper successfully performs the initial pick and transport. However, it cannot complete the placement due to collision constraints with table. The task becomes feasible only when the system performs a coordinated dual-arm handover, allowing the suction gripper to execute the final placement. Importantly, neither gripper alone can solve the task, yet their coordinated use enables seamless execution. This example captures the central premise of our work: heterogeneous coordination can elevate simple industrial grippers’ performance, provided that switching and compatibility are planned systematically.

In this work, we study embodiment-aware realization of demonstrated manipulation tasks using a heterogeneous dual-arm robotic system. A single human demonstration provides the desired object motion as a task-level specification. Rather than learning a policy, our objective is to determine how the demonstrated task can be executed under the constraints of heterogeneous robotic hardware. Given a discretized object trajectory, we determine at each segment which arm should manipulate the object and when coordinated handovers are necessary to maintain feasibility under grasp, kinematic, and collision constraints.

Heterogeneous execution introduces unique planning challenges. At different phases of a task, only specific gripper modalities may satisfy feasibility requirements. Coordinated handovers must respect cross-gripper grasp compatibility, and excessive switching increases execution time and failure risk creating the need for a global reasoning. We formulate this problem as a finite-horizon optimization over arm–trajectory assignments. Each state encodes both the active robot and its associated grasp-induced motion. Segment costs capture feasibility and manipulability quality, while transition costs encode heterogeneous grasp-pair compatibility and switching penalties. Exploiting the optimal substructure of this formulation, we compute a globally consistent execution plan via dynamic programming, equivalent to a structured shortest-path computation over a time-expanded graph.

Our contributions are threefold:

1. Heterogeneous gripper-level coordination: We formalize how a complementary simple gripper system can collectively realize manipulation tasks that are infeasible for any single modality alone, extending beyond simple pick-only tasks.
2. Globally consistent embodiment-aware assignment: We introduce a structured planning formulation that integrates heterogeneous grasp affordances and handover constraints into a globally consistent assignment.
3. Sim-to-real transfer: We demonstrate the framework on a dual Franka FR3 platform equipped with suction and parallel-jaw grippers on a task requiring modality transitions and coordinated handovers.

II Related Work

II-A Dexterous Manipulation vs. Simple Grippers

Robotic dexterous hands aim to replicate the versatility of human manipulation through multi-fingered contact, in-hand regrasping, and rich contact reasoning [1, 2]. While highly expressive, they introduce substantial mechanical and control complexity. Classical multi-fingered manipulation established many of the analytical foundations for contact kinematics and grasp planning [3, 4, 5]. Industrial systems instead rely on simple, task-specific end-effectors such as suction cups or parallel-jaw grippers for their robustness and cost efficiency. However, each supports only limited contact modalities. This leaves the challenge of expanding manipulation capability without complex anthropomorphic hardware largely open. Recent reinforcement learning has enabled impressive dexterous manipulation on anthropomorphic hands, including large-scale sim-to-real policies and reorientation tasks [6, 7, 8]. We instead explore extending capability by coordinating complementary simple grippers in a heterogeneous bimanual system. Rather than increasing single-hand dexterity, we study whether structured coordination between specialized modalities can approximate the functional diversity.

II-B Bimanual Manipulation and Grasp Planning

Bimanual manipulation has been widely studied for coordinated transport, reorientation, and in-hand manipulation [9]. Early work largely focused on symmetric dual-arm systems with identical end-effectors, emphasizing synchronized motion planning and closed-chain control [10, 11]. Several works also address grasp generation and object handover between two arms. GraspGen [12] generates grasps for parallel-jaw grippers, while sampling-based and graph-based methods search over candidate grasp pairs and intermediate transfer poses [13] [14].

More recently, learning-based approaches have demonstrated coordinated dual-arm behaviors, including end-to-end Vision–Language–Action models [15], learned bimanual coordination [16], and reactive self-handover in confined environments [17]. Manipulability-aware dual-arm grasp planning has also been explored in redundant systems [18], along with physics-aware saliency learning for bimanual grasp point selection [19]. However, most existing methods assume identical end-effectors or treat grasp feasibility separately for each arm. Coordination with heterogeneous grippers and explicit reasoning over grasp compatibility and switching across task phases still remains a challenge. Our approach instead performs global planning that jointly models grasp compatibility, motion feasibility, and modality transitions.

II-C Heterogeneous Gripper Systems

There is limited prior work on planning with heterogeneous grippers to extend the manipulation capability of bimanual robots. HeterBot [20] introduces a mobile platform with two different grippers but focuses mainly on teleoperation rather than autonomous coordination. VIKI-R [21] assigns task primitives across heterogeneous platforms on high-level, but does not explicitly reason about cross-gripper low-level manipulation. [22],[23] and [24] employs heterogeneous grippers; however, its applicability is restricted to object picking tasks. Overall, existing work focuses on gripper selection for individual pick actions, while holistic task planning and feasibility across heterogeneous embodiments remain understudied. Positioning: Most existing methods assume identical end-effectors, treat grasp selection separately from task planning, or rely on heuristic or manual switching between arms. In contrast, our method jointly reasons about grasp feasibility, grasp-pair compatibility, and arm switching in a single global planning framework.

III Problem Formulation

We consider a heterogeneous bimanual manipulation system with two robot arms \mathcal{R}:=\{R_{L},R_{R}\} operating in a shared workspace. Each arm R_{j} is equipped with a possibly different end-effector G_{j}\in\{\text{parallel-jaw},\text{suction},\dots\}, leading to asymmetric grasp affordances.

Task Setup: An object o must be moved from a start pose X_{start}\in SE(3) to a goal pose X_{goal}\in SE(3). A single human demonstration provides an object-centric trajectory \xi:[0,T]\rightarrow SE(3) describing the desired object motion. The trajectory is discretized into N ordered segments \{\xi_{1},\dots,\xi_{N}\}. Due to heterogeneous gripper affordances and kinematic limits, different segments may only be executable by specific arms.

For each segment \xi_{i}, we construct a finite set of feasible arm–trajectory pairs s_{i}=(r_{i},\tau_{i}), where r_{i}\in\mathcal{R} is the active robot and \tau_{i} is a grasp-conditioned, collision-free trajectory that executes \xi_{i}. Let \mathcal{S} denote the set of all feasible states. A plan is a sequence \mathbf{s}_{1:N}=(s_{1},\dots,s_{N}) such that exactly one arm manipulates the object at each segment. The objective is to select a sequence that remains feasible while minimizing motion and switching effort over the full task.

IV Embodiment-Aware Planning Framework

Given the feasible state set \mathcal{S} for each segment, task execution is formulated as a finite-horizon discrete optimization problem. Candidate grasps on the object generate multiple trajectory instances for each arm. These candidates are evaluated for grasp validity, kinematic reachability, and collision-free motion. Each feasible state is assigned a segment cost, while transitions between states encode switching penalties and grasp compatibility. A dynamic programming procedure then computes the minimum-cost sequence of states over the task horizon, producing a feasible and smooth bimanual execution plan such as \{(r_{1},\tau_{1}),(r_{1},\tau_{1}),(r_{2},\tau_{8}),(r_{2},\tau_{8}),\dots\}.

The individual components of the framework are described in the following subsections.

IV-A Trajectory Extraction from Human Demonstration

We extract the object trajectory via model-based 6D pose tracking using FoundationPose [25], leveraging the known CAD mesh to enforce geometric consistency during manipulation (See Figure 2). While the mesh is required for planning, recent work enables reliable mesh reconstruction of novel objects from multiple RGB-D images [26]. The pipeline initializes an SE(3) pose from RGB-D observations and then tracks the object by conditioning each estimate on the previous pose to maintain temporal coherence. At each timestep, pose alignment is optimized between observed depth and the rendered mesh. The resulting poses are temporally filtered to suppress high-frequency jitter while preserving rigid-body consistency, yielding smooth trajectories robust to partial occlusion. From these trajectory points, we define motion segments \xi_{i}=(p_{i},p_{i+1}) between consecutive poses.

While Foundation Pose and trajectory extraction from Human Demonstration serve as a general approach, our method works with any method that can provide an object trajectory; thus, we also tested our approach with the RRT* algorithm for object trajectory generation.

IV-B Grasp Generation, Ranking and End-Effector Trajectory Formation

A feasible plan requires at least one robot to have a valid grasp pose g for gripper G_{j} at object pose p_{i}. Let \mathcal{G}_{j}=\texttt{SampleGrasps}(o,G_{j}) denote the candidate grasp set; feasibility requires \mathcal{G}_{j}\neq\emptyset. This captures gripper-dependent affordances (e.g., suction on flat surfaces and parallel-jaw grasping on accessible edges), modeled over the coupled state of both grippers and the object rather than any single gripper in isolation. We use a geometry-based grasp generator [12]. Scene-conditioned generation is avoided because a grasp infeasible in one segment may become feasible later due to changes in arm assignment or handover configuration, complicating consistent reasoning across segments.

To encourage grasp diversity, we bypass the generator’s internal ranking and instead perform entropy-based subset selection. Given candidate grasps \{g_{1},\dots,g_{M}\} with positive quality scores w_{i}, we construct a Determinantal Point Process (DPP) using an L-ensemble. The L-ensemble matrix is defined as

L_{ij}=w_{i}w_{j}K_{ij},\qquad K_{ij}=\exp\!\left(-\tfrac{1}{2}d(g_{i},g_{j})^{2}\right).

The grasp distance metric is given by

d(g_{i},g_{j})^{2}=\frac{\|p_{g,i}-p_{g,j}\|^{2}}{\sigma_{p}^{2}}+\frac{\bigl(1-|\langle o_{g,i},o_{g,j}\rangle|\bigr)^{2}}{\sigma_{r}^{2}}.

where p_{g,i} and o_{g,i} denote the position and orientation (quaternion) of grasp g_{i}, respectively. We select a subset S of size k by greedily maximizing \log\det(L_{S}), which jointly promotes grasp quality (via w_{i}) and spatial–rotational diversity (via K_{ij}). Each selected grasp is used to generate the corresponding end-effector trajectory. These trajectories share the same time parameterization as the original object trajectory.

IV-C Segment-wise Feasibility and Cost Formulation

For each transport segment \xi_{i}, we determine which arm R_{j}\in\mathcal{R}=\{R_{L},R_{R}\} can execute the motion. An action is defined as A_{i,j}=(o,p_{i},p_{i+1},R_{j}), representing transport of object o from p_{i} to p_{i+1} using arm R_{j}. Its feasibility F(A_{i,j})\in\{\text{True},\text{False}\} depends on grasp and motion feasibility.

IV-C1 Grasp Feasibility

A valid grasp must exist for gripper G_{j} at pose p_{i}. Let \mathcal{G}_{i,j}=\texttt{SampleGrasps}(o,p_{i},G_{j}) denote the candidate grasp set; feasibility requires \mathcal{G}_{i,j}\neq\emptyset (already enforced by the filtering step in Section IV-B). We additionally compute mutually feasible grasp pairs across robots, i.e., collision-free grasp pairs that can simultaneously hold the object, which are required for handovers.

IV-C2 Motion Planning Feasibility

A collision-free trajectory must exist for arm R_{j} to transport the object from p_{i} to p_{i+1}. Feasible motions must also exist for pick at p_{0}, release at p_{T}, and handover execution. If any motion primitive is infeasible, the action is rejected.

An analytical Inverse Kinematics (EAIK) [27] solver evaluates exact endpoint reachability in microseconds, acting as a rapid feasibility filter. If analytical reachability is confirmed, we then evaluate the required motion primitives.

IV-C3 Segment Cost

In order to avoid awkward or near-singular configurations, we define a task cost based on the Yoshikawa manipulability measure and smoothness of the transitions between points. For a configuration q_{i,j} corresponding to action A_{i,j}, we define two cost components.

Smoothness Cost.

The smoothness term penalizes large joint variations between consecutive configurations:

C_{1}(A_{i,j})=\ \left\lVert q_{i+1,j}-q_{i,j}\right\rVert^{2}.

Manipulability Cost.

Given the Jacobian J(q), the Yoshikawa manipulability index [28] for a joint configuration q is

w(q)=\sqrt{\det\!\left(J(q)J(q)^{\top}\right)}.

To penalize near-singular configurations along the transport segment, we evaluate the index at both the start (q_{i,j}) and end (q_{i+1,j}) configurations of the action. We define the cost as:

C_{2}(A_{i,j})=\frac{1}{w(q_{i,j})+\epsilon}+\frac{1}{w(q_{i+1,j})+\epsilon},

where \epsilon>0 is a small constant for numerical stability.

Combined Task Cost.

Let F(A_{i,j})\in\{\mathrm{True},\mathrm{False}\} denote the feasibility of the action. The overall task cost is

C(A_{i,j})=\begin{cases}\alpha\,C_{1}(A_{i,j})+\beta\,C_{2}(A_{i,j}),&\text{if }F(A_{i,j})=\mathrm{True},\\[8.0pt]
\infty,&\text{otherwise},\end{cases}

where \alpha,\beta>0 balance smoothness and manipulability. Through a grid search for \alpha\in\{2.5,\allowbreak 5.0,\allowbreak 10.0,\allowbreak 20.0,\allowbreak 40.0,\allowbreak 50.0\} and \beta\in\{0.0,\allowbreak 0.25,\allowbreak 0.5,\allowbreak 1.0,\allowbreak 2.0,\allowbreak 4.0\}, these weights were set to \alpha=40 and \beta=0.5 to yield the best performance. For kinematically redundant manipulators, we include the manipulability term directly in the inverse kinematics (IK) optimization too. By doing so, the solver naturally prefers well-conditioned configurations, resulting in smooth and highly manipulable robot motions.

IV-D Transition Design for Heterogeneous Handover

We define the handover primitive at a point p_{i} as a dual-arm exchange between a transfer and a receiver arm using predefined grasp transforms. Each end-effector pose is computed by combining the object pose with the inverse of its grasp frame. The primitive consists of three stages: Pre-alignment, where the object is slightly offset while the receiver aligns to this intermediate pose; Receiver approach, where the receiver interpolates smoothly to the final grasp along the grasp normal; and Transfer retreat, where the original arm retreats along a fixed axis after handoff. This sequence ensures smooth, coordinated, and geometrically consistent dual-arm handovers.

Transitions between identical robot states are disallowed by assigning infinite cost to such edges. Although counterintuitive, this is necessary for feasibility: switching planners mid-trajectory requires an instantaneous release and regrasp, which may cause the object to fall. If a grasp change is required, it can be achieved through two handovers, but handover costs discourage unnecessary frequent switching unless required for feasibility.

Following Section IV-C, we also enforce paired heterogeneous grasp feasibility by assigning infinite cost to infeasible simultaneous grasp configurations, thereby eliminating transitions corresponding to infeasible dual-arm trajectories. For feasible handover primitives, the cost is defined analogously to the segment cost in Section IV-C. Specifically, the total cost of a handover primitive \mathcal{H} is given by

C_{\text{switch}}(\mathcal{H})=\sum_{(i,j)\in\mathcal{H}}C(A_{i,j}),

where C(A_{i,j}) is the combined smoothness–manipulability cost defined previously.

The transition cost between states s and s^{\prime} is defined as

\Phi(s^{\prime},s)=\begin{cases}C_{\text{switch}},&r(s^{\prime})\neq r(s);\ \text{grasp pair feasible},\\[6.0pt]
\infty,&r(s^{\prime})\neq r(s);\ \text{grasp pair infeasible},\\[6.0pt]
\infty,&r(s^{\prime})=r(s);\ \tau(s^{\prime})\neq\tau(s).\end{cases}

Here, r(s) denotes the active robot, \tau(s) the associated trajectory index, and C_{\text{switch}}>0 is a switching penalty that regularizes inter-robot handovers.

IV-E Dynamic Programming and Optimal Sequence Recovery

The problem admits optimal substructure and is solved via dynamic programming. The value function is defined as,

V_{i}(s)=\min_{s_{1:i-1}}\left(\sum_{k=1}^{i}C_{k}(s_{k})+\sum_{k=2}^{i}\Phi(s_{k-1},s_{k})\right),

which represents the minimum cumulative cost of reaching segment i and terminating in state s.

Initialization.

V_{1}(s)=\begin{cases}C_{1}(s),&s\in\mathcal{S}_{0},\\
\infty,&\text{otherwise}.\end{cases}

Bellman Recursion.

For i=2,\dots,N, recursion is

V_{i}(s)=C_{i}(s)+\min_{s^{\prime}\in\mathcal{S}}\left(V_{i-1}(s^{\prime})+\Phi(s^{\prime},s)\right).

Infeasible transitions are excluded by setting \Phi(s^{\prime},s)=\infty. The recursion is equivalent to a shortest-path computation on a layered directed acyclic graph. For policy recovery, we store the predecessor mapping

\pi_{i}(s)=\arg\min_{s^{\prime}\in\mathcal{S}}\left(V_{i-1}(s^{\prime})+\Phi(s^{\prime},s)\right).

Each optimal state encodes both the selected robot and its associated trajectory, i.e., s_{i}^{\star}=(r_{i}^{\star},\tau_{i}^{\star}). The resulting assignment sequence is \mathbf{s}_{1:N}^{\star}=((r_{1}^{\star},\tau_{1}^{\star}),\dots,(r_{N}^{\star},\tau_{N}^{\star})), from which the robot and trajectory sequences can be extracted as \mathbf{r}_{1:N}^{\star}=(r_{1}^{\star},\dots,r_{N}^{\star}) and \boldsymbol{\tau}_{1:N}^{\star}=(\tau_{1}^{\star},\dots,\tau_{N}^{\star}). Switching events are defined as \mathcal{H}=\{\,i\in\{2,\dots,N\}\mid r(s_{i}^{\star})\neq r(s_{i-1}^{\star})\,\}.

IV-F Complexity Reduction via Beam Search and Stride

To mitigate the \mathcal{O}(N|\mathcal{S}|^{2}) worst-case time complexity of the dynamic programming pass and ensure scalability to longer horizons, we introduce a beam search with a stride parameter. We evaluate potential dual-arm handovers only every k steps (stride) rather than at every discretized trajectory waypoint. Furthermore, during the forward pass, we retain only the top B lowest-cost states (beam width) for evaluation at the next step. This significantly prunes the state space and branching factor, preserving performance while drastically reducing planning times.

V Experiments and Results

Our experiments are designed to systematically evaluate the proposed system framework. Specifically, we aim to answer the following questions:

• To what extent does the proposed dynamic programming formulation improve task completion rates in scenarios involving heterogeneous grippers?
• What is the impact of incorporating diversity-aware grasp selection and manipulability-based costs on overall system performance?
• Which components are most crucial to the method’s performance, and what are the dominant failure modes observed in practice?

V-A Experimental Setup

All experiments are conducted on a heterogeneous dual-arm platform with two 7-DoF Franka FR3 manipulators: one equipped with a parallel-jaw gripper and the other with a suction gripper. The arms are arranged with overlapping workspaces to enable coordinated manipulation and object handovers, and their relative configuration is varied to induce different reachability and feasibility constraints. This setup allows evaluation of grasp-dependent affordances and asymmetric reachability. All experiments are conducted on a workstation equipped with an AMD Ryzen Threadripper PRO 7975WX processor and an NVIDIA RTX 6000 Ada Generation GPU.

Grasp Generation, Motion Planning and Execution:

• Inverse Kinematics: An EAIK [27] based IK solver with multi-joint enumeration (using the redundancy of 7 DOF) to check for the most continuous and manipulable joint pose.
• Grasp Generation: 6-DoF grasp candidates are sampled via GraspGen[12] for both parallel-jaw and suction grippers, followed by collision-based feasibility filtering. 40 grasps are sampled per gripper type, except in Task A: Plate from Table to Shelf (as described below) due to its restrictive environment, thus requiring more diverse sampling (60 grasps).
• Collision modeling: Full robot self-collision, object and environment meshes, and dual-arm interactions checked using MuJoCo’s default collision engine.

Simultaneous dual-arm motions use an asymmetric leader-follower setup: Arm A follows the planned, while Arm B tracks a reference via a reactive mink QP with collision-avoidance constraints. We have used mink here due to its natural incoporation with mujoco and being a QP solver it can incorporate posture tasks (for warm starts), collision handling and velocity/joint limits in its optimization while finding optimal control steps to control the arm (during planning we wanted speed in order to iterate fast upon a lot of possible configuration and solutions, thus used EAIK). Arm A’s joints are pinned, and transport segments are executed as constrained joint tracking to faithfully follow the object trajectory.

V-B Task Design

The tasks are designed with two objectives: (1) to enforce heterogeneous gripper use by making at least one trajectory segment infeasible for a single arm, requiring a handover; and (2) to introduce reachability-based infeasibility, demonstrating that the framework generalizes beyond collision constraints. We design both single-switch and two-switch tasks, described below (see Figure 3 for task examples).

• Task A: Plate from Table to Shelf A flat plate is positioned flush on the table to mandate a suction pick, while the destination shelf’s narrow clearance prohibits suction entry, deliberately forcing a handover for final placement.
• Task B: YCB Plate from Shelf to Table The recessed YCB plate blocks suction access, requiring a parallel-jaw grasp; however, tight boundaries at the target cause gripper collisions, necessitating a handover to suction for top-down placement on the table.
• Task C: Mug Inversion An inverted mug prevents parallel-jaw rim grasping, requiring suction pickup. A handover to the parallel jaw then enables flipping and upright placement in a container.
• Task D: Plate from box to box A flat plate is transferred between two containers. Only the suction gripper can pick and place at the containers due to collision constraints for the parallel jaw. However, intermediate regions are unreachable by suction due to a wall, along the constraint that the plate can’t go over it, thereby requiring two handovers.

V-C Baselines

Exisiting object manipulation research with heterogeneous grippers has mainly focused on grasp generation and picking. We therefore compare our method against the following configurations:

V-C1 Greedy Reachability

At each segment, the robot with a feasible grasp and higher instantaneous manipulability is greedily selected (equivalent to a beam width of 1).

V-C2 No-Diversity Grasp Selection

The top-k grasps are selected purely based on grasp quality scores, without applying DPP-based diversity filtering.

We also evaluated single-robot execution, which achieved 0% success across all scenarios, confirming that the tasks require heterogeneous bimanual coordination.

V-D Quantitative Results

Table II summarizes performance across 520 simulated runs to address Q1, evaluating success rates, switches, and timing. Our exact DP method achieves 72.5% task success compared to the greedy baseline’s 20.0%, demonstrating the critical need for global look-ahead over myopic reachability decisions in complex transport tasks. Furthermore, excluding DPP-based grasp diversity (67.5% success) heavily penalizes planning time (1015.87 s), as concentrating grasps in similar poses redundantly expands the dynamic programming search space without improving workspace reachability.

While global optimization is vital, the Beam 10, Stride 3 configuration offers a practical computational trade-off. It drastically cuts average planning time from 532.96 s (Exact DP) to 91.05 s, while still maintaining a 65.0% task success rate and a 100.0% handover success rate.

V-E Per-Experiment Details and Evaluation

Table II details performance across the four tasks using the practical Beam 10, Stride 3 configuration, maintaining a robust 100.0% handover success globally. Task C: Mug Inversion achieved the highest success (80.0%) and rapid execution (63.8 s).

Conversely, Task A: Plate from Table to Shelf proved most challenging (50.0% success, 129.87 s planning time) due to environmental complexity and increased grasp sampling (60 vs. 40). Failures primarily stemmed from the inability to find valid robot assignments or confined insertion tolerances that forced suboptimal grasps, leading to transport slippage. Tasks B and D maintained balanced rates of 60.0% and 70.0%.

V-F Ablation Studies

We evaluate the contribution of key components via ablation studies, maintaining the rest of the pipeline unchanged (Table II).

V-F1 Switching Penalty and Frequency (C_{\text{switch}}=0)

Removing the switching penalty leads to excessive, unsystematic handovers, increasing the average switches from 1.3 to 10.7. Consequently, task success plummets to 25.0%, handover success drops to 81.6%, and execution time increases by 211.4%, demonstrating that unregularized handovers compound failure points. Conversely, systematic switching remains highly effective: valid executed plans requiring two switches maintain a robust 87.0% task success rate, compared to 90.3% for one-switch plans. This confirms that deliberate handovers safely navigate restricted workspaces without the severe performance drops of unregularized switching.

V-F2 Manipulability Cost (\beta=0)

Disabling the manipulability term drops overall task success to 52.5% and handover success to 89.1%. Without this objective, the solver frequently selects poorly conditioned kinematic states, resulting in jerky trajectories and unstable grasps. Prioritizing manipulability is therefore vital to smoothly and safely execute the complex, contact-rich motions required during physical dual-arm handovers.

V-G Real-World Validation via Sim and Real Digital Twin

To validate our simulated results, we deployed the framework on a physical dual Franka FR3 setup using Task B: YCB Plate from Shelf to Table and Task C: Mug Inversion. We bridged the sim-to-real gap via a ROS 2 and MuJoCo-based digital twin, computing dynamically verified trajectories with the Beam=10, Stride=3 configuration (which demonstrated a 60% and 80.0% task success rate). During physical deployment, impedance control safely absorbed minor calibration offsets during contact-rich handovers, successfully replicating the digital twin’s precise and safe bimanual behavior.

VI Limitations and Conclusion

Limitations We rely on pre-computed CAD meshes for tracking and DPP-based grasp generation, which limits zero-shot deployment on novel objects. Recent advances such as FoundationPose [25] have enabled on-the-fly mesh reconstruction from RGB-D observations and reduce this dependency. The framework also assumes a fixed, human-provided object trajectory. Future work could allow embodiment-aware trajectory refinement, where the planner slightly adjusts the demonstrated trajectory to achieve more stable handovers.

In conclusion, we present an embodiment-aware planning framework for heterogeneous bimanual manipulation. By jointly optimizing segment reachability, asymmetric grasp compatibility, and transition penalties, our method translates a single human demonstration into a globally optimal execution plan. Supported by extensive simulation sweeps and validated via a real-world digital twin, this approach systematically elevates simple industrial grippers toward the versatile execution of complex manipulation tasks.

References

1. [1] W. Lin, S. Yuan, C. Gong, and J. Wang (2025) A comprehensive review of dexterous manipulation and learning methods for robotic hands. Biomimetics 10 (6), pp. 400. External Links: Document
2. [2] A. Bicchi and V. Kumar (2000) Robotic grasping and contact: a review. Proceedings of the IEEE International Conference on Robotics and Automation, pp. 348–353. External Links: Document
3. [3] J. Kerr and B. Roth (1986) Analysis of multifingered hands. The International Journal of Robotics Research 4 (4), pp. 3–17. External Links: Document
4. [4] D. J. Montana (1988) The kinematics of contact and grasp. The International Journal of Robotics Research 7 (3), pp. 17–32. External Links: Document
5. [5] M. R. Cutkosky (1989) On grasp choice, grasp models, and the design of hands for manufacturing tasks. IEEE Transactions on Robotics and Automation 5 (3), pp. 269–279. External Links: Document
6. [6] OpenAI, M. Andrychowicz, B. Baker, M. Chociej, R. Józefowicz, B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, J. Schneider, S. Sidor, J. Tobin, P. Welinder, L. Weng, and W. Zaremba (2020) Learning dexterous in-hand manipulation. The International Journal of Robotics Research 39 (1), pp. 3–20. External Links: Document
7. [7] A. Rajeswaran, V. Kumar, A. Gupta, G. Vezzani, J. Schulman, E. Todorov, and S. Levine (2018) Learning complex dexterous manipulation with deep reinforcement learning and demonstrations. In Robotics: Science and Systems (RSS),
8. [8] I. Akkaya, M. Andrychowicz, M. Chociej, M. Litwin, B. McGrew, A. Petron, A. Paino, M. Plappert, G. Powell, R. Ribas, J. Schneider, N. Tezak, J. Tworek, P. Welinder, L. Weng, Q. Yuan, and W. Zaremba (2019) Solving rubik’s cube with a robot hand. arXiv preprint arXiv:1910.07113.
9. [9] M. S. Islam, Z. Najdovski, M. A. Hosen, and V. T. Huynh (2026) An in-depth review of dual-arm robot learning methods with advances and challenges in variable compliance-aware control for uncertain tasks. Annual Reviews in Control 61, pp. 101050. External Links: ISSN 1367-5788, Document
10. [10] R. Shome, K. Solovey, J. Yu, K. Bekris, and D. Halperin (2018) Fast, high-quality dual-arm rearrangement in synchronous, monotone tabletop setups. In International Workshop on the Algorithmic Foundations of Robotics, pp. 778–795.
11. [11] F. J. Martínez-Peral, J. B. Méndez, D. Mronga, J. V. Segura-Heras, and C. Perez-Vidal (2025) Trajectory planning system for bimanual robots: achieving efficient collision-free manipulation. Robotics and Autonomous Systems 194, pp. 105118. External Links: ISSN 0921-8890, Document
12. [12] A. Murali, S. Fidler, A. Garg, S. James, A. Khazatsky, A. Kurenkov, Y. Li, T. Zhang, Y. Zhang, R. Zhong, et al. (2025) GraspGen: a diffusion-based approach for 6-dof grasp generation on unseen objects. arXiv preprint arXiv:2507.13097.
13. [13] M. Djeha, A. Dallard, A. Zermane, P. Gergondet, and A. Kheddar (2022) Human-robot handovers using task-space quadratic programming. In 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Vol. , pp. 518–523. External Links: Document
14. [14] B. Cohen, M. Phillips, and M. Likhachev (2021) Planning single-arm manipulations with n-arm robots. Proceedings of the International Symposium on Combinatorial Search 6, pp. 226–227. External Links: Document
15. [15] K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al. (2024) pi\_0: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164.
16. [16] H. Chen, J. Xu, L. Sheng, T. Ji, S. Liu, Y. Li, and K. Driggs-Campbell (2025) Learning coordinated bimanual manipulation policies using state diffusion and inverse dynamics models. In 2025 IEEE International Conference on Robotics and Automation (ICRA),
17. [17] Z. Geng, Z. Yang, W. Xu, W. Guo, and X. Sheng (2024) A globally guided dual-arm reactive motion controller for coordinated self-handover in a confined domestic environment. Biomimetics 9 (10), pp. 629. External Links: Document
18. [18] C. Wong, C. Tsai, Y. Lai, and S. Wong (2024) Manipulability-aware task-oriented grasp planning and motion control with application in a seven-dof redundant dual-arm robot. Electronics 13 (24), pp. 5025. External Links: Document
19. [19] S. Wang, X. Liu, C. C. L. Wang, and J. Liu (2024) Physics-aware iterative learning and prediction of saliency map for bimanual grasp planning. Computer Aided Geometric Design 111, pp. 102298. External Links: Document
20. [20] Y. Wang, X. Wang, C. Cao, X. Wang, M. Ding, F. Li, J. Duan, J. Liu, and J. Wang (2023) HeterBot: a heterogeneous mobile manipulation robot for versatile operation. IET Cyber-Systems and Robotics 5 (1), pp. e12068. External Links: Document
21. [21] Y. Kang, J. Frey, G. Yang, and J. Peters (2025) VIKI-r: coordinating embodiment and task for cross-platform generalization in robot manipulation. arXiv preprint arXiv:2506.09049.
22. [22] N. Nechyporenko, A. Morales, E. Cervera, and A. P. del Pobil (2021) A practical approach for picking items in an online shopping warehouse. Applied Sciences 11 (13). External Links: Link, ISSN 2076-3417, Document
23. [23] A. Wu and D. Kruse (2025) In the wild ungraspable object picking with bimanual nonprehensile manipulation. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pp. 5669–5676.
24. [24] J. Mahler, M. Matl, V. Satish, M. Danielczuk, B. DeRose, S. McKinley, and K. Goldberg (2019) Learning ambidextrous robot grasping policies. Science Robotics 4 (26), pp. eaau4984. External Links: Document, Link
25. [25] B. Wen, W. Yang, J. Kautz, and S. Birchfield (2024) Foundationpose: unified 6d pose estimation and tracking of novel objects. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 17868–17879.
26. [26] B. Wen, J. Tremblay, V. Blukis, S. Tyree, T. Müller, A. Evans, D. Fox, J. Kautz, and S. Birchfield (2023) Bundlesdf: neural 6-dof tracking and 3d reconstruction of unknown objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 606–617.
27. [27] D. Ostermeier, J. Külz, and M. Althoff (2025) Automatic geometric decomposition for analytical inverse kinematics. IEEE Robotics and Automation Letters.
28. [28] T. Yoshikawa (1985) Manipulability of robotic mechanisms. The international journal of Robotics Research 4 (2), pp. 3–9.