BiPOP: Partial-Order Planning for Long-Horizon Bimanual Robotic Manipulation
The BiPOP preprint combines GPT-4.1 with temporal symbolic planning to coordinate two robot arms on long tasks. In simulation, it achieved 100%, 100% and 95% success on ServeFruit, ServeWater and StackBlock, respectively, showing how explicit coordination constraints can improve reliability over direct LLM planning. An end-to-end ServeWater demonstration on NICOL establishes feasibility on hardware, without a reported real-robot success rate.
Can an LLM coupled with temporal symbolic planning reliably assign, order and parallelize actions for long-horizon bimanual manipulation?
Two arms must respect reachability, shared objects, occupied locations and synchronization requirements across many actions. A locally plausible action can invalidate later steps or conflict with the other arm. Direct LLM planning can overlook these dependencies, especially when trying to execute actions concurrently.
SayCan, Code as Policies and VoxPoser connect language to executable skills or motion but do not solve the same long-horizon coordination problem. LLM+P and related systems translate instructions into symbolic problems, usually for sequential execution. DAG-Plan and RoboPARA exploit dependency graphs but rely on heuristic coordination or limited checks. LLM+MAP produces partial-order plans without the same explicit temporal makespan objective.
BiPOP uses a hand-authored temporal PDDL domain covering left-arm, right-arm and shared workspace regions. Preconditions and resource locks constrain arm access, object use and synchronized skills. GPT-4.1 generates Python code through Unified-Planning to encode the scene and goal; interpreter errors and unsuccessful planning trigger revisions. Temporal Fast Downward searches alternative schedules, VAL rejects invalid candidates, and the shortest validated symbolic schedule found is selected. LocateAnything supplies object detections for scene grounding.
CoppeliaSim evaluation used 100 randomized instances per domain. In ServeFruit/ServeWater/StackBlock order, success rates were BiPOP: 100/100/95%; I/O o3: 98/87/86%; I/O R1: 66/82/52%; I/O GPT-4.1: 25/0/0%; ReAct GPT-4.1: 100/45/36%. BiPOP exceeded o3 by 2/13/9 percentage points. Planning times in seconds were BiPOP: 6.56 ±1.49 / 5.95 ±1.83 / 35.56 ±9.02; o3: 32.74 ±15.39 / 43.76 ±34.86 / 73.20 ±37.85; ReAct: 16.56 ±5.47 / 24.74 ±9.70 / 42.46 ±22.60. I/O GPT-4.1 was faster at 3.42 ±0.47 / 2.31 ±0.95 / 4.59 ±2.04, with much lower success. Separately, perception was assessed on 100 real images per domain: all relevant objects were localized correctly for ServeFruit and ServeWater, while StackBlock showed tray and stacked-block errors. Real NICOL execution demonstrated ServeWater; its trial count and success rate were not reported.
Quantitative task-success comparisons are simulation results; hardware execution covers one task on one platform. The system requires a manually specified domain and hand-crafted atomic skills. Unit action durations measure symbolic parallel steps, not elapsed execution time. The optimality gap references the best plan found across evaluated methods, not a proven optimum. Symbolic validation does not guarantee collision-free trajectories; the motion layer may serialize actions. Specification errors and planner timeouts remain on StackBlock. Execution monitoring, recovery and motion-based duration estimates are future work. The implementation is publicly available.
Bimanual service-robot developers and manipulation-software suppliers could use this now as a research planning layer when they already have reliable skills and a bounded task vocabulary. Product deployment would require robust scene grounding, geometric motion checks, execution monitoring and recovery, plus scheduling based on actual skill durations. Commercial deployment timing: Not reported in the paper.
Full paper
BiPOP: Partial-Order Planning for Long-Horizon Bimanual Robotic Manipulation
Published under CC BY 4.0. Reproduced with attribution; original at arXiv:2611.15317 (PDF).
Abstract
Bimanual robotic manipulation offers substantial versatility, yet its central challenge lies not only in executing dexterous motion with each hand, but also in coordinating the two arms for long-horizon manipulation under coupled spatial and temporal constraints. Prior work has largely focused on endowing robots with human-level manipulation skills, paying comparatively less attention to the structure of bimanual behaviour itself: many practical tasks are best understood as a coordinated composition of reusable unimanual skills together with a small set of synchronous bimanual skills. This makes high-level spatio-temporal coordination, rather than skill acquisition alone, a key bottleneck for efficient bimanual execution. Therefore, we propose BiPOP, a framework that couples Large Language Models (LLMs) with Temporal Partial-Order Planning for long-horizon bimanual manipulation. At its core is an explicit spatio-temporal model that partitions the workspace by arm reachability and captures inter-arm dependencies, arm assignment, synchronization requirements, and opportunities for safe concurrent execution. Building on this model, an LLM translates natural-language instructions into a formal PDDL specification, while a temporal planner enforces the constraints and schedules actions, yielding plans that are logically consistent at the symbolic level and efficient through parallel execution. On simulated long-horizon tasks of varying complexity, BiPOP outperforms open-loop planning with strong reasoning models and closed-loop ReAct-style planning in success rate, planning time, and plan efficiency, and the full pipeline runs end-to-end on a real bimanual robot NICOL. A failure analysis further shows that LLM-only planners mainly violate spatial and temporal coordination constraints, which motivates delegating these constraints to a symbolic planner. Our implementation is publicly available at https://github.com/Kchu/BiPOP.
I Introduction
Humans effortlessly perform bimanual tasks, from tying shoelaces to flipping book pages while holding a coffee cup, orchestrating complex spatial and temporal synchronization without conscious thought. Robots, in contrast, struggle even with apparently simple bimanual tasks, which demand planning, synchronization, and coordination. Bridging this gap remains an inherent challenge in robotics [1, 2].
Deep learning techniques ease the acquisition of dexterous bimanual manipulation, such as folding shirts [3], cutting vegetables [4], and scooping food [5]. However, most approaches focus on learning human-level skills as a whole and overlook high-level task planning, which demands task decomposition, subtask assignment between the arms, execution sequencing and parallelization over long horizons.
Large Language Models (LLMs) have shown strong reasoning and planning abilities across domains [6], and recent work applies them to robotic task planning [7, 8, 9, 10]. Yet generating embodiment-aware, executable plans is difficult for complex manipulation, where LLMs remain prone to long-horizon reasoning errors and hallucinations [11, 12], especially in spatially constrained scenarios. To mitigate these limitations, LLMs have been combined with classical symbolic planners [13]: by translating natural-language instructions into declarative representations such as PDDL [14], LLMs enable plans with logical-correctness guarantees. However, these approaches generally do not address the spatio-temporal coordination intrinsic to bimanual manipulation, in particular inter-arm dependencies, task allocation, synchronization, and parallelization. Our research question is therefore: how can LLMs be integrated with symbolic planning to achieve efficient spatial and temporal coordination for long-horizon bimanual robotic tasks?
To address this, we propose BiPOP (Bimanual Partial-Order Planning), a framework (Fig. 1) for long-horizon bimanual task planning that integrates semantic reasoning with explicit spatio-temporal coordination. Our contributions are threefold. First, we introduce a spatio-temporal model that partitions the workspace by arm reachability and encodes inter-arm dependencies, synchronization, and safe concurrency as durative PDDL actions with mutual-exclusion predicates. Second, we couple LLM-based problem generation with temporal planning: an LLM translates the task description and scene information into a PDDL problem, and a temporal planner computes a validated partial-order plan with a short symbolic makespan. Third, we evaluate BiPOP against open-loop and closed-loop LLM planners on three simulated bimanual task domains and deploy it on a real bimanual robot NICOL [15]. The results show that BiPOP outperforms open-loop planning with recent OpenAI and DeepSeek reasoning models and closed-loop ReAct-style planning in success rate, planning time, and plan efficiency.
II Related Work
II-A Task Planning in Robotics
Planning with Symbolic Planners. Task planning generates action sequences that achieve specified goals [16, 17]. Classical planners formalize domains and problems in declarative languages, most prominently PDDL [14], and generate plans with guaranteed logical correctness through domain-independent search [18, 19]. Task and motion planning couples such a symbolic planner in discrete space with a motion planner in continuous space [20, 21, 22]. These planners provide formal validity guarantees but require expert effort to author the symbolic representations, motivating the use of LLMs to automate this step.
Planning with LLMs. Prompting LLMs to generate robot behaviour directly is now a common technique [7, 9, 23, 24]. SayCan [7] lets an LLM propose feasible skills and grounds them to the current scenario through learned value functions, while language-to-code methods ground language more directly in executable robot behaviour: Code as Policies [8] writes policy code over perception and control primitives, and VoxPoser [25] composes LLM/VLM-generated 3D value maps for zero-shot trajectory synthesis. These methods turn short-horizon instructions into executable, mostly single-arm motion, but presuppose a short or given task-level sequence and execute the LLM output unverified. Although iteratively querying LLMs with feedback or error messages mitigates these issues [9, 26, 27], such strategies still offer no formal guarantee of plan validity and rely on hand-crafted, domain-specific feedback with considerable expert effort.
To pair language understanding with formal guarantees, a growing line of work uses LLMs not as planners but as translators of natural-language tasks into PDDL solved by classical planners. LLM+P [13] established this paradigm; ViLaIn [28] adds visual grounding, DELTA [29] decomposes long-horizon goals into sub-goals, and AutoGPT+P [30] handles incomplete information via affordance-based scene representations. However, these methods plan for a single agent and produce strictly sequential plans. Although AutoGPT+P is validated on bimanual humanoid platforms, it models neither the two arms as coordinated agents nor the action durations and concurrency constraints required for bimanual manipulation. LLM+MAP [31] combines LLM reasoning with PDDL-based multi-agent planning for bimanual manipulation. Although it also produces partial-order plans, it frames coordination primarily as task and resource allocation between the two arm agents, rather than explicitly optimizing makespan through temporal planning.
A and P denote area and point, respectively. Light-blue and red boxes represent left- and right-arm durative actions, while horizontally aligned boxes indicate actions executed in parallel.II-B Bimanual Manipulation and Planning
Bimanual Manipulation. Deep learning has enabled dexterous bimanual skills [2] with parallel grippers [4] or anthropomorphic hands [32], from scooping food [5] to folding clothes [33] and zipping zippers [4]. As these works focus on skill acquisition, they do not address planning for complex, long-horizon tasks; the diversity of coordination patterns they exhibit suggests that high-level planning should be considered jointly in an integrated control-system design [1, 34].
Bimanual Planning. Recent bimanual planning methods fall into two main categories. (i) Multi-agent formulations [35, 36] model each arm as an autonomous agent and use LLMs to decompose and allocate subtasks. This design is flexible, but typically lacks explicit concurrency scheduling, so coordination rests on heuristic LLM reasoning rather than verified constraints. (ii) Graph-centric pipelines [10, 37] induce a task-dependency graph with an LLM and dispatch its nodes to the two arms, exploiting parallelism where precedence permits. DAG-Plan [10] assigns nodes to arms reactively at run-time via heuristic feasibility and cost estimates, leaving the LLM-generated dependencies unverified. RoboPARA [37] validates its graph only against a few hand-crafted structural error patterns and schedules it greedily with deadlock rollback, without a correctness guarantee or an explicit makespan objective. In both, the dependency semantics depend substantially on LLM assertions. In contrast, we ground concurrency in a verified symbolic model: the workspace partition and skills are formalized as temporal PDDL, and a temporal planner determines which actions may remain partially ordered and which require coordination, minimizing makespan with correctness guaranteed at the symbolic level.
III Method
Given a task description in natural language, we aim to generate a valid and efficient plan for a bimanual robot based on the initial scene configurations, e.g., spatial information about the objects with respect to the robot. To this end, this section introduces the three elements of BiPOP: (A) Spatio-temporal modelling, (B) Bimanual PDDL definition with the LLM, and (C) Partial-Order Planning with the temporal planner.
III-A Spatio-temporal Modelling
To enable efficient bimanual manipulation, we first develop a spatio-temporal model that characterizes the reachable workspace of each arm and the coordination relations between them, as illustrated in Fig. 3. The workspace is partitioned into two uncoordinated regions, each reachable by a single arm, and one coordinated region (the overlap area) reachable by both. Arms operating in separate uncoordinated regions on distinct objects are spatially independent and can execute concurrently as parallel unimanual skills. Within the coordinated region, the arms become spatially and temporally coupled, requiring either asynchronous coordination, in which one arm establishes the preconditions for the other, or synchronous coordination, in which both arms execute a tightly coupled bimanual action.
This workspace partitioning operates at the task-planning level: it determines whether two actions may proceed concurrently based on the arms, objects, and regions they involve, rather than on the geometric feasibility of the underlying trajectories. Two actions are scheduled concurrently only when they share no symbolic resource (neither arm, object, nor region), while all remaining cases are serialized through explicit coordination (ordering) constraints. Consequently, the resulting schedules are sound at the task level. Collision-free execution of concurrent motions, including cross-arm trajectory interference near the coordinated region, is delegated to the downstream motion planner. When a concurrent assignment proves geometrically infeasible, the motion planner may serialize it without affecting the validity of the produced plan.
Planning for a bimanual task thus begins with identifying the regions in which the task-relevant objects and target poses lie: an open-vocabulary object detector (e.g., LocateAnything [38]) localizes the objects, and the resulting bounding boxes are mapped onto the workspace partition.
III-B Bimanual PDDL Definition with the LLM
III-B1 PDDL Formalization
The Planning Domain Definition Language (PDDL) [14] is a declarative language for formalizing planning problems in a standardized form. A planning problem is defined as a tuple \mathcal{T}=\langle\mathcal{O},\mathcal{P},\mathcal{A},\mathcal{S},\mathcal{G}\rangle, where \mathcal{O} is the set of objects, \mathcal{P} the set of predicates, \mathcal{A} the set of actions, \mathcal{S} the initial state, and \mathcal{G} the goal. A state is a set of grounded predicates, instantiated over objects with Boolean values, and the goal is a logical expression over such predicates; each action is specified by typed parameters together with preconditions and effects expressed as predicates over these parameters. A PDDL specification comprises a domain, declaring the object types, predicates, and parameterized actions of the task world, and a problem, instantiating a concrete set of objects, an initial state, and a goal.
III-B2 Domain Definition
To encode the proposed spatio-temporal model and the underlying manipulation logic, we define four object types, arm, area, object (obj), and point, together with predicates describing their properties and relations. To support the durative actions introduced below, we additionally define predicates that represent the execution state. The predicate available indicates whether an arm is unoccupied, e.g., not holding an object, while arm_free indicates whether it is currently idle, which prevents a single arm from executing multiple actions at once. The predicate is_static indicates whether an object is at rest rather than being moved. These predicates encode mutual-exclusion conditions that prevent infeasible concurrent actions, such as two actions manipulating the same object simultaneously, while still permitting independent actions to run in parallel. For temporal planning, we model each action as a durative action, distinguishing predicates that must hold at the start of an action, throughout its execution, and at its end, as shown in Listing 1. This temporally annotated form is essential for expressing the overlap and ordering constraints that arise when two arms act concurrently.
We design a set of atomic skills for the two identical robotic arms, covering both independent single-arm operations and joint dual-arm operations. Each symbolic action is grounded in a robotic controller through its arm parameters: a single-arm action takes a specific arm as a parameter and maps directly to that arm’s controller, whereas a joint operation is not parameterized by an individual arm. Instead, its precondition requires both arms to be in the available and arm_free states, and the skill is bound to both arms simultaneously, coupling the two controllers into a single synchronized execution. Region membership enters the domain directly through preconditions such as arm_access(?arm, ?area) (cf. Listing 1), so reachability constrains action applicability rather than being checked post hoc. Location-dependent skills such as pour carry no explicit target parameter: they act through the above relation established by a preceding move_above action, as visible in the plans of Fig. 2.
III-B3 LLM as a PDDL Writer
Following the formalization above, we prompt the LLM to generate the problem specification: the initial state is derived from the scene description obtained from the camera image, and the goal is derived from the natural-language task instruction. We employ the LLM for this step because, paired with open-vocabulary detection, it provides autonomous visual grounding of the task configuration: free-form object references in the instruction are resolved directly against detected labels. Rather than generating raw PDDL text, we instruct the LLM to produce Python code that constructs the problem through the Unified-Planning library [39], which standardizes problem definition and provides a uniform interface to built-in symbolic solvers; this exploits the code-generation ability of LLMs while constraining their output to a well-defined API. To handle generation errors, we re-prompt the LLM with feedback from two sources. When code execution raises an exception, the Python interpreter error message is fed back for regeneration, which corrects syntactic and API-level mistakes. When the code executes successfully but the planner returns no plan within the fixed timeout, this no-plan outcome is likewise returned to the LLM, prompting it to revise the initial state or goal specification. At our task scale, a no-plan outcome almost always stems from an erroneous specification rather than from search timeout, making it a reliable re-prompting signal.
III-C Temporal Partial-Order Planning
We adopt Temporal Fast Downward (TFD) [19] as the temporal planner. TFD supports durative actions with start and end conditions and effects, and it schedules independent actions concurrently, which shortens the makespan while preserving temporal consistency. Applied to our bimanual domain, TFD yields a temporal partial-order plan in which independent unimanual actions run in parallel and coordination-dependent actions are ordered accordingly (cf. Fig. 2). To exploit the planning budget, we run TFD under several parameter configurations and search strategies, yielding multiple candidate plans of varying quality. While the problem-generation stage uses interpreter and planner feedback to obtain a solvable specification, this stage uses the VAL plan validator1 as the safeguard of the planner itself: TFD occasionally emits schedules that violate durative-action conditions or the temporal semantics of the domain, most often among the shorter candidates, and VAL rejects these before execution. Among the validated candidates, we dispatch the one with the shortest makespan to the robot controllers.
Our framework operates at the task-planning level, reasoning about the ordering and concurrency of symbolic actions rather than the kinematics of their execution. We therefore assign all actions a uniform duration, since exact skill execution time depends on trajectory length, robot dynamics, and low-level control resolved by the motion-planning layer. This abstraction isolates the effect of concurrency structure from low-level execution variability. Under unit durations, the makespan of a plan equals its number of parallel steps, with concurrent actions counted once. We refer to this quantity as the symbolic makespan and use it as the single efficiency objective throughout the paper. As TFD runs under a fixed time budget, the returned plan has the best symbolic makespan found within this budget rather than a certified optimum. The uniform-duration choice does not restrict the formulation itself: TFD can directly incorporate per-action or state-dependent durations, and integrating motion-layer duration estimates is left for future work.
IV Experiments
We compare BiPOP, built on GPT-4.1, against an open-loop and a closed-loop LLM-based planning baseline, I/O and ReAct. I/O generates the entire plan in a single pass and then executes it, whereas ReAct interleaves reasoning and acting, emitting one action at a time and receiving environmental feedback before deciding the next. As the two baselines place different demands on the underlying model, we pair each with an appropriate choice. Open-loop synthesis is reasoning-intensive, since the full plan must be produced at once; we therefore equip I/O with two strong reasoning models, OpenAI o3 (denoted o3) and DeepSeek-R1-0528 (denoted R1), and additionally report I/O with GPT-4.1, the backbone of BiPOP. ReAct, by contrast, re-queries the model at every step, making per-step inference latency critical for a usable closed-loop system; we therefore use GPT-4.1, which offers low-latency responses and matches the backbone of BiPOP.
This configuration isolates the planning mechanism along two axes: against I/O, BiPOP is evaluated against reasoning models and its own backbone, so no advantage can be attributed to model capability; against ReAct, the backbone is held fixed at GPT-4.1, so the remaining difference reflects the mechanism alone. We further align the prompts across methods. All receive the same task description, scene information, and skill set, differing only in how these are encoded:
- I/O: the task description (with spatial scene understanding), the spatio-temporal bimanual modelling paradigm, and the skill specifications as natural-language preconditions and effects.
- ReAct: the same content as I/O, but generated step by step with an observation fed back after each action; if a step for two arms cannot be fully executed, the executable portion is carried out while the remainder and its error message are returned to the agent.
- BiPOP: the same task description with the general PDDL domain definition and predicate specifications used by the temporal partial-order planner.
For I/O and ReAct, the LLM is prompted to generate a valid plan with the fewest parallel steps, i.e., the smallest symbolic makespan, matching the objective of BiPOP. ReAct is additionally given a fixed step budget per task domain. We do not compare against language-to-code policies such as Code as Policies or VoxPoser, which, as discussed in Sec. II, synthesize motion-level behaviour and presuppose a short or given task-level sequence, precisely the capability our baselines are designed to probe.
IV-A Environment Setup
For quantitative evaluation, the experiments are conducted in the CoppeliaSim simulator on a bimanual robot with two arms of eight degrees of freedom each, terminated by adult-sized, five-fingered hands for precise handling of everyday objects. Visual perception uses one of the two cameras at each eye’s position. We evaluate BiPOP against the baselines across three task domains within a human-robot environment: ServeFruit, ServeWater, and StackBlock (cf. Fig. 4), which resemble real-world interactions among the robot, a human, and the task objects. ServeFruit contains a banana, an apple, and a large red bowl; the robot must place the fruit into the bowl and serve the bowl to the human. As the bowl is too large to be grasped by one hand, it requires simultaneous two-hand manipulation, which is only possible in the overlap area. ServeWater contains an empty yellow cup and a water-filled blue cup on the table; the robot must serve the water in the yellow cup to the human while placing the blue cup at a specified store point on the brown box. StackBlock contains six coloured blocks, randomly located or initially stacked on the table. The goal is to stack them into one or two piles in a specified order. Each configuration varies in two aspects: the pile locations, one in front of the human, the other on the brown box, and the block order and count per pile in the initial and goal stacks.
Each domain is evaluated on 100 randomized instances, over which Tables I and II report statistics. For every instance, the object configuration is sampled as described above, and the natural-language instruction is instantiated from a per-domain template with the sampled goal (e.g., the object stacking order). Instructions state only the desired outcome, leaving arm assignment, hand-over strategy, intermediate placements, and action ordering unspecified. They are thus not surface paraphrases of the PDDL goal, and the LLM must ground them jointly with the scene description.
IV-B Metrics
To assess the efficacy and efficiency of the generated plans, we compare our method with the baselines along three metrics:
- Planning Time (PT). For the I/O and ReAct agents, the planning time depends solely on LLM inference, whereas for our framework it comprises both the LLM inference time and the TFD planning time. TFD is run as an anytime planner under several search configurations: within a per-domain optimization budget of
1s,1s, and25s for ServeFruit, ServeWater, and StackBlock respectively (scaled to search-space size), it keeps searching for shorter plans, and the best validated plan is returned. Reported TFD times 2 include grounding and validation in addition to search. - Success Rate (SR). The proportion of runs whose generated plan achieves the task goal upon execution.
- Optimality Gap (OG). A measure of planning efficiency relative to the best plan found by any evaluated method. Each run
rdenotes one task instance evaluated by all methods. Lets_{m}(r)be the symbolic makespan of methodm’s plan onr, measured in unit-duration steps (concurrent actions count once), and let\mathcal{M}(r)be the set of methods that solvedr, withs^{*}(r)=\min_{j\in\mathcal{M}(r)}s_{j}(r). Form\in\mathcal{M}(r),\mathrm{OG}_{m}(r)=s_{m}(r)-s^{*}(r)\geq 0,(1) where\mathrm{OG}_{m}(r)=0marks the most efficient method(s) onrand larger values indicate less efficient plans. The resulting\mathrm{OG}_{m}distributions of all methods are reported in Fig. 5.
IV-C Results
Planning Time. As shown in Table I: (1) GPT-4.1 and BiPOP require substantially less planning time than the reasoning models o3 and R1; (2) inference time grows with task complexity across all models; and (3) for BiPOP, the LLM inference time grows only moderately with task complexity, while the TFD planning time is negligible on simpler tasks but dominates on StackBlock, where the larger object and predicate sets expand the search space. Even so, BiPOP remains faster than both reasoning models on every task.
- I/O
- R1
- 227.07 ±51.19
- 245.71 ±54.10
- 367.57 ±76.35
- o3
- 32.74 ±15.39
- 43.76 ±34.86
- 73.20 ±37.85
- No data
- GPT-4.1
- 3.42 ±0.47
- 2.31 ±0.95
- 4.59 ±2.04
- No data
- ReAct
- GPT-4.1
- 16.56 ±5.47
- 24.74 ±9.70
- 42.46 ±22.60
- BiPOP
- Total
- 6.56 ±1.49
- 5.95 ±1.83
- 35.56 ±9.02
- GPT-4.1
- 4.95 ±1.44
- 4.98 ±1.73
- 6.65 ±1.78
- No data
- TFD
- 1.60 ±0.29
- 0.97 ±0.34
- 28.91 ±7.46
- No data
- I/O
- R1
- 66
- 82
- 52
- o3
- 98
- 87
- 86
- No data
- GPT-4.1
- 25
- 0
- 0
- No data
- ReAct
- GPT-4.1
- 100
- 45
- 36
- BiPOP
- GPT-4.1
- 100
- 100
- 95
0, +1,\dots,\geq 5; dark = optimal), and failures are coloured by failure class as shown in the legend.Success Rate and Failure Modes. Table II gives the per-domain success rate, and Fig. 5 splits all instances into successes (graded by optimality gap) and failures by class. (1) BiPOP matches or exceeds every baseline in all domains and is the only method to reach at least 95% success throughout; although built on the non-reasoning GPT-4.1, it far surpasses I/O planning with the same model, thanks to coupling LLM coding and temporal planning with pre-execution validation. (2) These long-horizon bimanual tasks demand strong reasoning: the reasoning I/O models (o3, R1) recover much of GPT-4.1’s gap, yet no LLM-only baseline matches the planner-coupled approach. (3) We observe diverse failure modes. I/O agents exhibit Spatial errors (out-of-reach targets, untracked staging/serve-point occupancy), Temporal errors (both hands to one point, grasping a buried block, releasing before placement, wrong stacking order), and Skill errors (mis-called primitives, e.g., a single-hand grasp of the co-hold bowl, or wrong arguments). BiPOP shows the same Spatial errors but far more rarely, and these arise from the LLM-authored PDDL specification rather than execution (mis-specified serve point), plus Planning failures where the TFD planner times out, all confined to StackBlock and easily auditable. ReAct failures are Misjudged (declares completion while the goal is unmet, repeating the I/O mistakes undetected), Aborted (detects a deadlock and gives up), and Incomplete (exhausts its step budget3). Overall, LLM-only planning methods fail to ensure physical-action feasibility, whereas coupling the LLM with a planner shifts the small residual error to the transparent specification and planning stages.
Optimality Gap. The green portion of Fig. 5 grades each success by its OG value (darkest = optimal). (1) BiPOP concentrates at the minimum on ServeFruit and ServeWater, almost always producing the most efficient plan. (2) On StackBlock the larger search space prevents the TFD planner from always converging within the time budget, spreading its OG beyond zero, yet it stays far more reliable than the baselines. (3) The I/O agents occasionally beat BiPOP on StackBlock, but only at a much lower overall success rate. (4) ReAct, even when successful, uses markedly longer symbolic makespans, confirming that closed-loop retrying buys coverage at the cost of parsimony.
IV-D Discussion
Our design separates the contribution of the planner from raw LLM capability: the I/O baselines receive the same spatio-temporal knowledge as BiPOP, in natural language rather than as a PDDL domain. The gap to BiPOP thus reflects delegating constraint enforcement and concurrency scheduling to the planner rather than model differences, and the residual gap of o3 and R1 shows that scaling reasoning alone does not close it. We accordingly build BiPOP on the non-reasoning GPT-4.1: problem generation is a constrained translation task at which it already attains near-ceiling success, whereas a reasoning backbone would add an order of magnitude of latency with little room for improvement.
To ensure comparability, we prompt I/O and ReAct for a valid plan with the fewest parallel steps. Under this objective, the reasoning models attempt aggressive parallelization, and most of their failures are spatial or temporal violations of the coordination constraints. BiPOP avoids this failure mode by construction: with the constraints in the PDDL domain, the LLM only grounds objects to reachability regions and emits a well-formed problem, while the planner handles arm assignment and makespan-minimizing concurrency, shifting the principal challenge to robust natural-language-to-PDDL alignment.
The reasoning baselines show that LLMs can exploit this spatio-temporal knowledge, reaching near-optimal plans in favourable configurations, yet hallucination and opacity still yield errors that go undetected without external verification. Symbolic planners can aid reasoning models: once a bimanual task is expressed in PDDL, plans proposed by reasoning models can be pre-verified with VAL, screening efficient but error-prone candidates before execution.
IV-E Real-World Validation
Since BiPOP operates at the task-planning level, the sim-to-real gap relevant to our contribution lies in perception: the pipeline must recover correct spatial relations and produce valid plans accordingly. The perception front-end is lightweight and detector-agnostic: LocateAnything [38] localizes the task-relevant objects, and a single camera-to-table calibration maps bounding boxes to metric coordinates, assigning each object unambiguously to the robot’s workspace region. In the real-world setup, ServeWater uses a glass and a bottle instead of the two cups, and StackBlock uses a grey tray instead of the brown box. Evaluated on 100 real images per task domain with varied object configurations, the front-end localizes all task-relevant objects correctly on ServeFruit and ServeWater. On StackBlock, errors are limited to occasional mislocalizations of the grey tray and, more often, of vertically stacked blocks in the initial configuration. They stem from the detector and the fixed eye-camera viewpoint rather than from the planning framework, and they are method-agnostic: BiPOP and all baselines consume the same scene description, so perception quality shifts the shared input rather than the comparison. The decoupling also works in our favour: a stronger open-vocabulary detector or an added viewpoint upgrades the whole pipeline without touching the planner. With this front-end, BiPOP completes the ServeWater task end to end on the real robot: across varied task configurations, it produces correct partial-order plans, and executes them in live human-robot interaction through hand-crafted atomic skills. Transfer thus requires minimal additional engineering, with low-level execution left to the motion layer.
V Conclusion
We present BiPOP, a task-planning framework for bimanual manipulation that combines the coding and reasoning abilities of LLMs with temporal partial-order planning. An LLM translates scene information and natural-language task descriptions into PDDL, and a temporal planner computes efficient concurrent plans whose correctness is guaranteed at the symbolic level. Across three task domains, BiPOP attains higher success rates and more efficient plans at shorter planning time than strong reasoning models, and the full pipeline transfers to the real bimanual robot NICOL.
Limitations and Future Work. BiPOP builds on low-level motion primitives, and its decomposition of tasks into reusable unimanual skills and a few synchronous bimanual ones facilitates learning them. Because action effects are explicit in PDDL, execution monitoring and replanning can be added to recover from failures such as unsuccessful grasps. Motion-layer duration estimates can replace the uniform durations to obtain tighter schedules. Finally, a single domain file already covers all three tasks in simulation and on the real robot, and inducing it from skill libraries or demonstrations would support open-ended skill repertoires.