Encore:编码智能体用少量演示发现机械臂操作策略,LIBERO-PRO扰动集成功率96.3%
这篇arXiv预印本提出Encore框架,将遥操作演示蒸馏为编码智能体可查询的证据包,让智能体编写、运行并迭代修复策略程序,运行时不调用模型。在LIBERO-PRO的60个扰动单元上,3条演示的冻结程序在3000个密封回合中成功2889次(96.3%),无演示为2846次(94.9%);在RoboDojo上无演示全部失败,3条演示在前10个任务取得95/450成功。双臂真机上完成手递手和杯子倒扣,各9/10成功。
少量人类演示能否作为编码智能体构建机器人操作策略的搜索先验,让智能体在无成功信号下发现并转移操作策略?
从少量演示学习策略困难,遥操作数据是密集多模态流,关键帧占比很小;现有方法要么预先压缩、可能丢弃意外但必要的线索,要么把完整轨迹塞给智能体使其被细节淹没。代码智能体从语言出发搜索时,也可能漏掉演示中的关键操作细节,例如夹爪姿态或拖拽方向。
此前方法包括代码即策略(如Code-as-Policies)和智能体系统(如ASPIRE、CaP-Agent0)从语言生成并调试程序;少样本模仿直接转移轨迹;演示增强合成训练数据。它们要么固定压缩方式、单次生成,要么忽略演示,或产出不可审计的网络策略。ASPIRE还依赖人工编写的诊断规则,例如判断夹爪是否抓空的宽度阈值。
Encore用确定性蒸馏器把每条演示转成证据包:意图句、关键帧(夹爪开合和速度方向突变)、夹爪事件表、状态路径和完整动作流,不解释其含义。全新opus-5编码智能体先用自写分析代码探索证据包,再针对统一感知/动作API编写策略程序,并在15个开发状态上迭代修复;部署程序冻结、不调用模型。仿真与真机使用同一API,策略只能通过固定的感知和动作调用与环境交互。
在LIBERO-PRO的30个未扰动单元上,Encore达到98.6%密封成功率。在60个扰动单元(Pos和Task各30个)上,K=3程序密封成功2889/3000(96.3%),K=0为2846/3000(94.9%),K=1为2791/3000(93.0%);重跑ASPIRE(opus-5)为89.3%,原论文ASPIRE(opus-4.6)为71.7%。Pos轴最终分别为1454/1500和1437/1500;Task轴为1435/1500和1409/1500。开发阶段:Pos轴K=3首个程序在11/30个单元成功,K=0为0;Task轴K=3在18/30个单元成功,K=0仅1个。抽屉任务K=3为50/50,K=0为0/50;盘子任务相反,K=3为0/50,K=0为50/50。RoboDojo的22个任务中,无演示全部失败,K=3在前10个任务取得95/450,K=1为40/450,六个接触精细任务所有配置均为0。双臂真机Trossen WidowX AI上,手递手和杯子倒扣各10次人类评判,均为9/10。
作者承认演示不保证搜索方向正确,例如盘子任务K=3冻结为0/50而K=0为50/50;RoboDojo六个接触精细任务所有配置均为0,部分任务(整理桌子、打包物体等)全零。早期实验因共享笔记文件高估K=3优势,正式协议未报告该差距。真机仅每个任务10次人类评判,规模较小。论文未报告代码是否开源及单任务算力成本细节。
适合需要快速为新布局或新指令编写可审计策略的机器人集成商,尤其是双臂操作、实验室自动化等任务。由于策略冻结且不依赖运行时模型,短期(1至3年)可用于仿真到真机迁移的原型;但距离工业部署需解决接触精细任务、更大规模真机验证,并降低对前沿LLM的依赖。前提条件包括高质量的遥操作演示、统一API的仿真或硬件环境,以及足够的开发轮次。
论文全文
Encore: Few-Shot Agentic Discovery of Manipulation Strategies
本文依据 CC BY 4.0 许可发布,经署名转载,原文见 arXiv:2609.37359(PDF)。
\tl_set:Ne\codefile
codefile
1 Introduction
Consider how an engineer would program the bimanual cube handover in Figure 1 from five demonstrations. They would notice that the giving arm presents the cube from the side, that the receiving gripper rolls a quarter turn so the finger planes cross, and that it closes before the giving gripper opens. The task sentence mentions none of this, and each detail is visible in only a few frames. The engineer would then write a program that reproduces the sequence in a new layout, run it, and repair it. Learning a policy from the same five trajectories remains difficult [1, 2]: they cover a tiny part of the state space, and without synthetic augmentation [3, 4] a policy trained on them fails as soon as it leaves the demonstrated path.
The engineer’s workflow of writing, running, and repairing a program is exactly what coding agents now do well. Given tools, an execution environment, and a budget of attempts, a frontier language model can write a program, run it, and repair it from the result. Code-as-policies systems have already shown that robot policies can be written as programs over perception and control APIs [5, 6], while more recent agents can debug failed rollouts and accumulate reusable skills across tasks [7, 8, 9].
Coding agents suggest another use for demonstrations. Modern agents interpret images and text and write analysis code to inspect unfamiliar data, so we convert each teleoperated demonstration into a structured record an agent can query: keyframes at the demonstrator’s gripper actions, frame strips around each contact event, and the complete underlying trajectory. Such a record states both the intended outcome and the strategy that reached it: where the demonstrator grasped, at what orientation, and along which path.
Existing approaches do not provide demonstrations to coding agents in a form they can use effectively. A teleoperated episode is a dense multimodal stream containing hundreds of timesteps, several camera views, and many joint-state channels. The moments that determine the outcome occupy a small fraction of the frames. Compressing the episode into a textual summary [10] or a compact token sequence [11] requires deciding in advance which details matter and may discard unexpected but essential cues, such as a change in grasp orientation or a wobble before a regrasp. Placing the entire episode in context creates the opposite problem: it overwhelms the agent with incidental details. In our experiments, handing even the strongest prior agentic pipeline [8] the same demonstrations as raw trajectory files brought it no measurable gain.
We introduce Encore, an agentic framework for few-shot robot manipulation. A deterministic builder first transforms each demonstration into the evidence described above. A fresh coding agent then explores this evidence using analysis code that it writes itself, infers the task, and implements a policy against a narrow perception and action API. The agent then refines the policy iteratively over a small number of development rollouts. To measure the demonstrations’ contribution, we compare three demonstrations (K{=}3) with none (K{=}0) under the same model, API, development budget, and evaluation protocol. The design places the boundary between agent and policy at development time: the language model writes, tests, and repairs the program, and the deployed policy makes no model calls, so it can be evaluated on thousands of sealed episodes with no agent in the loop.
Our central claim is that demonstrations act as a search prior for agentic policy construction: they expose manipulation strategies that the agent can transfer to new layouts, compose under new instructions, and turn into executable programs. On LIBERO-PRO [12], the first program written with demonstrations already succeeds during development in about half of the perturbed cells, and the first program written without them almost never does; given enough attempts, the agent without demonstrations reaches a similar final rate (Figure 2). Where the agent’s own search fails, the demonstrations can supply the missing strategy, and in one cell the demonstrated grasp decides every episode (Section 4.3). On RoboDojo, whose sentences leave the goal or the contact strategy unstated, no program succeeds without demonstrations. Our contributions are: (i) a distiller that turns teleoperated demonstrations into evidence a coding agent can query without being told what it means; (ii) a harness that develops, verifies, and freezes policy programs with no access to the success signal, whose programs succeed on 96.3% of sealed LIBERO-PRO perturbation episodes; and (iii) a controlled comparison on LIBERO-PRO, 22 RoboDojo tasks, and a real bimanual robot that shows where demonstrations change what the agent discovers.
2 Related work
2.1 Code as policies and agentic robotics
Language models can express robot policies as code over perception and control primitives [5, 6], and successors structure that interface with spatial value maps [13] or relational keypoint constraints [14]. A second line of work makes the model an agent: it executes its programs, reads the outcome, and revises them, accumulating reusable skills across tasks [8, 15, 9, 16]. ASPIRE [8] is the strongest published system of this kind on LIBERO-PRO [12] and our primary comparison. It iterates from language alone, and its execution traces are annotated with human-written diagnostic rules, including the gripper width that counts as a grasp and the width that counts as closing on air. Our harness gives no such aids. The brief states only the task, traces are passed on uninterpreted, and each agent builds its own diagnostics.
2.2 Learning from few demonstrations
Demonstration-conditioned code generation compresses trajectories into prompts, by recursive summarization [10] or keypoint action tokens [11]. The compression fixes in advance what the model may see, and generation is single-pass. Few-shot imitation instead transfers the demonstrated trajectories directly [1, 17, 18], and demonstration-augmentation systems synthesize training data from a few seeds [3, 19, 4]; in both cases the resulting artifact is a network whose behavior cannot be read, audited, or edited. Unlike these methods, Encore exposes distilled but uninterpreted demonstration data to an iterative coding agent and evaluates the resulting frozen program on held-out states.
3 Method
Problem statement.
A task is specified by one sentence of intent and K recorded human demonstrations on the same embodiment: in simulation three episodes drawn from the benchmark’s own demonstration files, on hardware five that we teleoperate. The output is a policy program: a module that runs in a process holding no simulator and no robot handle, and that acts on the world through a fixed API. Evaluation uses held-out initial states, and the success predicate is never exposed to the agent or to the program. The agent may run a bounded number of development episodes on a disjoint band of states and has no access to asset files or object poses.
Encore has three components (Figure 1): a demonstration distiller (§3.1), a perception-and-action API identical in simulation and on hardware (§3.2), and a development loop (§3.3) that ends with the program frozen for the sealed evaluation of Section 4. Tasks share no code.
3.1 Demonstration distiller
A demonstration is a stream D=\{(I_{t},s_{t},a_{t})\}_{t=1}^{T}, where T is the number of timesteps and I_{t} collects every camera view. The state s_{t}=(p_{t},q_{t},g_{t}) carries each arm’s 6-DoF end-effector pose, joints, and gripper width. A handful of the T steps decide the task, so distillation must select. Our distiller selects only on signals the demonstrator produced and does not say what they mean: a builder that tags a frame release has already done part of the work we want the agent to do. The pack of a demonstration is
\mathcal{P}(D)\;=\;\big(\ell,\;K,\;E,\;\tau,\;A\big).\ell is the intent sentence. K=\{(t,I_{t},s_{t}):t\in T_{K}\} is the keyframe set: every camera view and the full state at each selected instant. The indices T_{K} are the demonstrator’s own annotations, gripper sign transitions \{t:\operatorname{sign}g_{t}\neq\operatorname{sign}g_{t-1}\} joined with the sharpest heading breaks of the smoothed end-effector velocity. A multi-stage task therefore arrives pre-segmented, with no video understanding in the pipeline. E is the gripper-event table: each open and close, with its time, arm, and width signal. The labels describe the width signal only, and the agent decides which opening is a release. On hardware each event also carries a frame strip, the tiled frames spanning roughly 1.5 s around it. \tau is the state path at stride five, units declared. A is the complete action stream. A whitelist validator removes all other fields, including simulator state, object poses, and file paths, so the pack cannot lead the agent to benchmark files.
3.2 Perception-and-action API
The API draws the line between what we give the agent and what it must work out. It supplies sensing and motion, the same in simulation and on hardware (Table 1). Everything specific to the task, including which object to take, how to grasp it, where it goes, and how to tell that it got there, must come from the pack or from what the agent measures during development. Perception and motion calls draw on a per-episode budget, so a program cannot buy success with unlimited queries. On the robot the rig also serves SAM segmentation and a guarded top-down pick and place; the handover program calls neither.
- sense
- capture(cam)
- RGB-D frame with intrinsics and extrinsics
- —
- deproject(u, v)
- 3-D point in the robot frame
- —
- eef(), tool_rotation(), gripper()
- pose, tool rotation, jaw width and effort
- perceive
- sam3(q), ground(q), vqa(q)
- masks, boxes, or an answer for a text prompt
- act
- move(xyz, R), move_path(pts, R)
- Cartesian motion; returns the position residual
- —
- grip(w)
- sets the jaw width
Example. The frozen drawer program of Section 4.3 (Appendix G) turns a capture into a point cloud, finds the cabinet face, and takes the lowest handle bar standing off it. It tilts the open gripper 20^{\circ} below horizontal, the attitude it read from the demonstrated pulls, seats the fingertips on top of the bar, and drags with a move target far out and low, as the demonstrated actions do. It then captures again and drags a second time if the drawer front has come out less than 12 cm.
3.3 Development loop
Each task gets one fresh agent, which explores the pack with analysis code it writes itself, implements a program, and revises it over development episodes on a disjoint band of initial states. Because benchmark success is unavailable at runtime, and evaluation statically refuses any program that reads the episode termination flag, programs implement their own grasp and placement checks using proprioception and RGB-D. When development ends the agent selects one version and freezes it.
4 Experiments
4.1 Setup
Benchmark. LIBERO-PRO [12] perturbs LIBERO [20] along a layout axis (Pos), which relocates objects and the fixtures they sit on, and an instruction axis (Task), which re-authors the sentence and with it the rewarded end state. Neither perturbed axis carries demonstrations of its own. We evaluate all 30 tasks of the spatial, object, and goal suites on all three axes: 90 cells, 50 sealed episodes each. On the 30 unperturbed cells Encore reaches 98.6% (Table 4); Sections 4.2 and 4.3 cover the two perturbed axes. Every cell runs the loop of Section 3.3 once, with a fresh opus-5 agent that shares no files with any other cell. It develops on 15 initial states, and one frozen program is evaluated once on 50 sealed states and scored by the benchmark’s own predicate (Appendix D).
Baselines. We compare against the results Lu et al. [8] report for the VLA baselines (\pi_{0.5}; OpenVLA and \pi_{0} score zero and are omitted), for their code-as-policies agent without a skill library (CaP-Agent0), and for ASPIRE on opus-4.6. Because these come from an older model, we also rerun ASPIRE’s released code at a pinned commit on opus-5 over the same 60 perturbation cells (Appendix D). The two harnesses give their agents different tools (Appendix E).
4.2 Transferring a demonstrated strategy to a new layout
On the Pos axis the sentence is unchanged and the objects and their fixtures have moved, so the question is whether the agent can carry a demonstrated strategy to a layout it has not seen. It usually can, and from its first program. The first program written with three demonstrations already succeeds on some development state in 11 of 30 cells, and on average on 29% of its development episodes; the first program written without them succeeds nowhere. The first success arrives at a median of version 2 against 4, earlier in 21 cells and later in 5 (Figure 2). The final rates are close, 1454 and 1437 of 1500 sealed episodes: with 15 development episodes the K{=}0 agent measures most of what the pack states, including the grasp geometry and the goal site.
k (60 cells; a frozen cell keeps its final version). (b) Paired per cell, how often the K{=}3 agent needs less or more of each resource. The demonstrations reduce the number of attempts, while agent time and cost stay similar (0.51 against 0.55 hours per cell), because the K{=}3 agent spends part of the saved iterations reading the pack. Same harness, budget, model, and sealed evaluation.4.3 Composing demonstrated skills under a new instruction
The Task axis rewrites the sentence and the rewarded end state and keeps the scene. The rewrites are recombinations: each cell receives two packs, neither of which demonstrates its goal, and the agent must decide from the sentence which part of each pack to reuse (Figure 3). Here the demonstrations help more. The first program already succeeds in 18 of 30 cells, on average on 53% of its development episodes, against one cell and 2% without demonstrations, and the median first success arrives at version 1 against 5. The frozen programs differ as well: paired per cell, K{=}3 wins 10, loses 2, and ties 18, and succeeds on 1435 of 1500 sealed episodes against 1409.
50/50 with the two packs.The drawer cell. On open_middle_drawer-Task the sentence asks for the bottom drawer, while the demonstrations open the middle one. The cell scores 50/50 with demonstrations and 0/50 without, in both acquisitions. The K{=}0 agent could open the other two drawers, but in 33 program versions no grasp it tried engaged the bottom handle: tilted pinches, a hook from below the bar, and pressing its closed jaws on top of the bar and dragging outward, which did not move the drawer. It froze a program that opens the middle and top drawers and concluded that the bottom handle cannot be engaged through this API. The demonstrations of the middle drawer show the missing piece: throughout the pull the demonstrator’s actions drive downward as well as outward, with the wrist tilted 15 to 36^{\circ} below horizontal. The K{=}3 program seats its open fingertips on the bar and reproduces that drag, and the bottom drawer opens (Appendices F and G). A failed search during development does not show that a task is infeasible, and a demonstration can supply the strategy the search missed.
The plate cell. The largest loss runs the other way. On put_bowl_top_cabinet-Task, which asks for the plate on top of the cabinet, the K{=}3 agent never held the plate long enough to carry it, recorded that no grasp its jaws could keep would reach the cabinet, and froze after 11 versions at 0/50. The K{=}0 agent kept searching for 23 versions, found a grasp that lifts the plate along an arc, and scored 50/50. The demonstrations shorten the search; they do not guarantee that it ends in the right place.
\times 50 episodes). \pi_{0.5}, CaP-Agent0, and ASPIRE on opus-4.6 as reported by Lu et al. [8]; ASPIRE on opus-5 is our rerun of its released code. K{=}1 is in Appendix A.4.4 Comparison with prior systems
Over the 60 perturbation cells the K{=}3 programs succeed on 2889 of 3000 sealed episodes (96.3%) and the K{=}0 programs on 2846 (94.9%), against 89.3% for ASPIRE rerun on the same model and 71.7% as published; most of the margin lies in the spatial suite (Figure 4; per task in Appendix A). With one demonstration the programs succeed on 2791 (93.0%). An earlier acquisition of the same cells, run before we removed a shared note file from the brief, showed a K{=}3 advantage of 252 episodes. The clean protocol reproduces the K{=}3 number but not the gap, so we do not report the gap as an effect. Removing the verification loop while keeping the demonstrations drops the goal suite from 440 to 117 of 450 (Appendix C).
4.5 Underspecified tasks: RoboDojo
K{=}3 success each (start, middle, end). The first row states its goal only through the demonstrations; the others need a delicate contact or a visual cue.RoboDojo [21] is a bimanual Isaac Sim benchmark. Its instructions often leave the goal or the contact strategy unstated (“Build a tower using the wooden blocks and wooden boards”, “put the bottles into the dustbin, using handover when needed”), while its judge scores a specific structure or sequence. We ran the unchanged loop on 22 of its tasks with three teleoperated demonstrations per task, a development band of 15 layouts, and a sealed band drawn from the official evaluation layouts and scored by the benchmark’s own judge (Figure 5, Table 2). On 12 of them, chosen to separate goal-underspecified from contact-sensitive tasks, we add an arm that receives the same three demonstrations reduced to their keyframe images, with every pose, gripper reading, and action removed.
Without demonstrations, no program succeeds. On the first 10 tasks K{=}3 succeeds on 95 of 450 sealed episodes and K{=}0 on none. The 95 come from four tasks, and on each of them the K{=}0 agent solved the manipulation but could not tell what to build. One wrote: “the manipulation is solved; the target structure is not known.” The K{=}3 agents read the structure from the keyframe images. With one demonstration the 10 tasks reach 40 of 450.
Images alone are not enough when the contact is delicate. On stacking, swapping, inserting tubes, and pushing the T, the images-only arm falls well below the full pack, and on stacking and swapping it scores below no pack at all: the agent saw a picture of the demonstrated grasp without its measurements and spent its budget reconstructing them. Only on pressing buttons by number, where the goal is visual, do the images carry the whole effect; on tic-tac-toe they do best, because the full-pack agent kept the demonstrations’ pauses between turns and ran out of steps. Six contact-precise tasks (plugging a charger, inserting a key, fastening screws, making toast, storing a laptop, hanging mugs) score zero in every arm, which marks the current limit of these programs.
- build tower
- \mathbf{36}
- 0
- 0
- make kong
- \mathbf{36}
- 0
- 0
- bottles to dustbin
- \mathbf{15}
- 2
- 0
- classify objects
- \mathbf{8}
- 7
- 0
- arrange by number
- 0
- \mathbf{31}
- 0
- 10 tasks
- \mathbf{95}
- 40
- 0
4.6 Real-robot experiments
Setup.
We run the loop of Section 3.3 and the API of Section 3.2 unchanged on a bimanual Trossen WidowX AI: two 6-DoF arms with parallel jaws, observed by four RealSense D405 cameras, the overhead one calibrated to the table plane to 1.1 mm. We evaluate a two-arm cube handover and cup inversion (Figure 7), five teleoperated demonstrations each, distilled by the same builder as in simulation. A fresh agent develops from the pack and freezes one program; evaluation initial states are operator-varied and recorded by the harness, and a human who did not develop the program judges each trial.
Tasks.
The handover demonstrations present the cube from the side, roll the receiving gripper a quarter turn so the finger planes cross, and close the receiving grip before the giving one opens. The cup demonstrations pinch the standing cup’s wall, lay it on its side, regrasp it near the base, and stand the wrist up so the opening lands on the table, four gripper events rather than two (Figure 7; Appendix F). On sealed 10-episode human-judged bands, both tasks score 9/10.
References
Appendix A Main result tables
Table 3 reports the exact values visualized in Figure 4, and Table 4 the per-task breakdown behind them. All Encore numbers are 50-seed sealed evaluations (seeds 1–50) of one frozen program per cell; baselines are as published by Lu et al. [8], except the opus-5 rerun of ASPIRE, which we ran at a pinned commit (Section 4). The Encore K{=}3 and K{=}0 columns are the clean acquisition of 2026-09-13 and the K{=}1 column its counterpart of 2026-09-25; the earlier acquisition that included a shared note file is not reported.
- OpenVLA
- 0.00
- 0.00
- 0.00
- 0.00
- 0.00
- 0.00
- 0.00
- 0.00
- \pi_{0}
- 0.00
- 0.00
- 0.00
- 0.00
- 0.00
- 0.00
- 0.00
- 0.00
- \pi_{0.5}
- 0.17
- 0.01
- 0.38
- 0.00
- 0.20
- 0.01
- 0.25
- 0.01
- CaP-Agent0
- 0.22
- 0.18
- 0.26
- 0.17
- 0.12
- 0.14
- 0.20
- 0.16
- ASPIRE (opus-4.6)
- 0.98
- 0.95
- 0.81
- 0.45
- 0.51
- 0.60
- 0.77
- 0.67
- ASPIRE (opus-5, our rerun)
- 1.00
- 1.00
- 0.86
- 0.86
- 0.84
- 0.80
- 0.90
- 0.89
- Encore (K{=}0)
- 1.00
- 1.00
- 0.92
- 0.87
- 0.96
- 0.95
- 0.96
- 0.94
- Encore (K{=}1)
- 0.99
- 1.00
- 0.89
- 0.85
- 0.90
- 0.94
- 0.93
- 0.93
- Encore (K{=}3)
- 1.00
- 1.00
- 0.93
- 0.90
- 0.98
- 0.97
- 0.97
- 0.96
- Task
- K{=}3
- K{=}3
- K{=}1
- K{=}0
- K{=}3
- K{=}1
- K{=}0
- libero-object
- 无数据
- 无数据
- 无数据
- 无数据
- 无数据
- 无数据
- 无数据
- alphabet soup
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- bbq sauce
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- butter
- 1.00
- 1.00
- 0.98
- 1.00
- 1.00
- 1.00
- 1.00
- chocolate pudding
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- cream cheese
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- ketchup
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- milk
- 1.00
- 1.00
- 0.96
- 0.96
- 1.00
- 1.00
- 1.00
- orange juice
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- salad dressing
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- tomato sauce
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- 0.98
- 1.00
- libero-goal
- 无数据
- 无数据
- 无数据
- 无数据
- 无数据
- 无数据
- 无数据
- open middle drawer
- 0.96
- 0.70
- 0.06
- 0.44
- 1.00
- 0.00
- 0.00
- open top drawer put bowl
- 0.98
- 1.00
- 0.98
- 1.00
- 1.00
- 1.00
- 0.98
- push plate front stove
- 0.92
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- 0.98
- put bowl on plate
- 1.00
- 0.94
- 0.92
- 0.96
- 1.00
- 1.00
- 1.00
- put bowl on stove
- 1.00
- 0.98
- 1.00
- 1.00
- 1.00
- 0.86
- 0.84
- put bowl top cabinet
- 1.00
- 1.00
- 1.00
- 1.00
- 0.00
- 0.82
- 1.00
- put cream cheese in bowl
- 1.00
- 0.92
- 0.98
- 1.00
- 1.00
- 0.94
- 0.88
- put wine on rack
- 0.94
- 0.76
- 1.00
- 0.80
- 1.00
- 1.00
- 0.98
- put wine top cabinet
- 0.98
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- turn on stove
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- 0.88
- 1.00
- libero-spatial
- 无数据
- 无数据
- 无数据
- 无数据
- 无数据
- 无数据
- 无数据
- bowl between
- 0.96
- 0.98
- 1.00
- 1.00
- 0.92
- 0.80
- 0.92
- bowl cookie box
- 1.00
- 0.92
- 0.38
- 1.00
- 1.00
- 1.00
- 1.00
- bowl next to plate
- 0.96
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- bowl next to ramekin
- 1.00
- 0.98
- 0.98
- 1.00
- 0.98
- 0.96
- 0.92
- bowl on cookie box
- 1.00
- 1.00
- 0.90
- 0.94
- 1.00
- 0.78
- 0.88
- bowl on ramekin
- 1.00
- 1.00
- 1.00
- 0.90
- 1.00
- 1.00
- 1.00
- bowl on stove
- 1.00
- 1.00
- 1.00
- 1.00
- 1.00
- 0.96
- 0.98
- bowl on wooden cabinet
- 1.00
- 0.90
- 0.78
- 0.78
- 1.00
- 1.00
- 1.00
- bowl table center
- 1.00
- 1.00
- 1.00
- 1.00
- 0.84
- 0.94
- 0.94
- bowl top drawer cabinet
- 0.88
- 1.00
- 0.98
- 0.96
- 0.96
- 1.00
- 0.88
Appendix B Development cost
Per cell, median over the sixty perturbation cells, clean acquisition: the K{=}3 agent writes 4 program versions, runs 46.5 development episodes, and spends 0.51 hours and 66k output tokens before freezing; the K{=}0 agent 6 versions, 53 episodes, 0.55 hours, and 69k tokens. Totals over the sixty cells are 37 against 44 agent hours. The first version already succeeds on some development state in 29 cells with demonstrations and in 1 without, and the version of first success has median 2 against 4 (Figure 2). By comparison, ASPIRE’s released loop on the same model spends a median 0.99 hours and 91k output tokens per cell over the sixty cells. A frozen program then costs nothing at evaluation time beyond the simulator: a sealed LIBERO episode takes eleven seconds and no model call.
Appendix C Additional ablations
Verification loop and raw demonstrations. Removing the verification loop while keeping the demonstrations drops the goal suite from 440 to 117 of 450 (Table 5; both arms under the earlier brief with the shared note file). Without the loop, the agent freezes programs it has not seen fail. Giving ASPIRE’s loop the same three demonstrations as raw trajectory files gains 6 episodes in 500, within single-run variance. The demonstrations pay off when they are distilled into a pack and the agent can test what it reads from them.
When is the action channel necessary? On RoboDojo the images-only arm loses to the full pack where contact is delicate. To test whether this holds for a class of fixtures and not only for particular cells, we ran the three arms, under the earlier brief, on eight LIBERO-90 tasks with articulated fixtures (microwave, three drawers, stove, opening and closing each). The other seven score 50/50 in every arm but one, which scores 42/50. On open_bottom_drawer the full pack scores 50/50 and both no-action arms 0/50. Two measurable conditions separate this fixture from the seven: the bar protrudes less than the jaw’s open half-width, so the default pinch is geometrically impossible, and the pinch fails silently, closing on air without a contact signature. On the microwave the pinch is also impossible but fails loudly, the door prises the jaws open, and the K{=}0 agent diagnosed its way to the hook. The demonstration was necessary only where the failure left no signal the agent could observe.
- verification ablation, Encore
- 无数据
- full system (K{=}3)
- 440/450
- verification loop removed
- 117/450
- ASPIRE’s loop
- 无数据
- as released
- 431/500
- plus three demonstrations as raw trajectories
- 437/500
Appendix D Evaluation protocol
Every cell runs the development loop once, headless, with a fresh agent that has not seen any other task; the inner model is opus-5 for our arms and for the rerun baseline. All agents receive the same brief. In the LIBERO-PRO and RoboDojo experiments each cell is developed in isolation: agents share no files with one another, and each agent derives the constants in its program from its own pack and development runs. Development touches a disjoint band of 15 initial states (seeds 51–65 on LIBERO-PRO). The selected program is frozen, its hash recorded, and evaluated once by a coordinator on 50 sealed states (seeds 1–50), with no per-episode result reaching any agent before the band completes; the benchmark’s own success predicate is the only judge. A static check refuses programs that branch on the episode termination flag, so each program must carry its own verification. Comparisons between arms are paired per cell over the 60 matched perturbation cells. The ASPIRE rerun uses a wrapper that restarts a session if it delivers a program before its development sweep has finished; it reaches 89.3%, against the 71.7% reported in the original paper. The two ablations of Appendix C were acquired earlier, under a brief that shared a note file across cells.
Appendix E Inputs available to each system
The two contracts differ, and the difference favors the baseline. ASPIRE’s agents call a grasp planner and a top-down grasp selector, fit oriented bounding boxes with a library routine, read a coordinator-curated library of reusable code at the start of every session, and are handed diagnostic thresholds in the brief, down to the gripper width that counts as a grasp and the width that counts as closing on air. Encore’s agents receive the demonstrations and none of these tools or hints, and every threshold in a Encore program was set by the agent that wrote it.
Appendix F Case studies
A demonstration is an existence proof (Task).
(Section 4.3.) On open_middle_drawer-Task the K{=}0 agent resolved the re-authored target correctly: it opened the middle and top drawers on its development seeds, saw the benchmark reject both, and turned to the bottom bar. Its notes record why every grasp failed. A side pinch needs a horizontal wrist, and the horizontal hand extends 96 mm below the end effector while the bar stands 48 mm above the table; rolling the wrist reduces the overhang to 41 mm, still too much. A hook must sit in the slot behind the bar, which is 11 mm deep, and the closed finger pair is 15 mm thick. Pressing the closed jaws on top of the bar and dragging it 10 cm outward left the drawer face where it was. Across 20 probe versions and four wrist families, the deepest low pose it reached was a bar thickness short of a grip. It wrote this down as a falsifiable claim that the handle cannot be engaged, froze a program that opens the two drawers it can reach, and scored 0/50.
The K{=}3 agent hit the same wall with pinches and then read the pack: the demonstrated pull keeps the wrist 15 to 36^{\circ} below horizontal, and its actions stay saturated outward and downward (a commanded height change near the maximum throughout the pull while the measured height does not change). Seating the open fingertips on top of the bar and commanding a target far out and low reproduces that pull; on its development seeds it dragged the bottom drawer 19 cm. The K{=}0 agent turned its failed attempts into a claim about the robot, and the demonstrations show that the claim is false.
Appendix G Frozen program listings
The two frozen programs of Section 4.3’s drawer cell, exactly as evaluated (md5 011b5af6 and c2fcb7de in the reproduction bundle), less the PROVENANCE dictionary that our audit reads and the policy never does. First the K{=}0 program (0/50), then the K{=}3 one (50/50).
G.1 Without demonstrations
\codefile
program_k0.py \cdot frozen, K{=}0, 0/50 sealedlistings/program_k0.py
G.2 With demonstrations
\codefile
program_k3.py \cdot frozen, K{=}3, 50/50 sealedlistings/program_k3.py