Skill2Real: Hierarchische Agenten-Skills für Zero-Shot-Sim-to-Real-Robotermanipulation

Abstract

Sim-to-real learning commonly adapts policies or representations to domain variation, often directing the learning budget toward task-agnostic low-level representations at the expense of task-level knowledge. Skill2Real builds on cross-domain grounding from foundation vision-language models (VLMs) to acquire validated, reusable skills in simulation through a shared code-based policy interface. In its asymmetric Proposer–Verifier–Governor (PVG) loop, the Proposer acts from public observations and API returns, the Verifier diagnoses outcomes with privileged simulation evidence, and the Governor admits updates supported by validation rollouts. A Cerebellum learns reusable local manipulation skills; a Brain then composes the frozen library into long-horizon programs with planning and recovery. At deployment, this hierarchy is grounded in real observations through the same interface. On unseen LIBERO-Pro Long, frozen LIBERO-90 skills raise success from 2.0% to 56.3% with Astra and from 0.5% to 49.0% with Opus 5. Robosuite success reaches 85.1–89.4% across seven tasks. On a real UR5e, the same agent’s mean completion rises from 27.50% to 78.75% across four tasks with Skill2Real, a controlled gain that, with cross-model improvements, shows Skill2Real adds a capability orthogonal to the foundation model by learning transferable task knowledge in simulation. Project page: skill2real.github.io.

1 Introduction

Zero-shot sim-to-real robot manipulation aims to convert experience acquired in simulation into behavior that can be executed by a physical robot without real-world task training. Simulation provides a scalable environment for learning such behavior, but discrepancies in appearance, dynamics, contact, and embodiment can prevent policies learned in simulation from transferring reliably to reality (Tobin et al., 2017; Chebotar et al., 2019). For long-horizon manipulation, the challenge extends beyond transferring individual actions: a robot must reliably execute local manipulation skills, compose them into task-level behavior (Gupta et al., 2020; Shi et al., 2025b), and recover when intermediate actions fail.

Existing methods leave a central bottleneck unresolved: learning reusable, composable task knowledge from privileged simulation evidence without spending the learning budget on task-agnostic low-level representations. Randomization and asymmetric actor–critic methods improve transfer by adapting neural policies or representations (Tobin et al., 2017; Pinto et al., 2018). Appearance-heavy domain randomization can therefore spend much of the learning budget modeling task-irrelevant variation in lighting and texture rather than acquiring reusable manipulation knowledge. Code-generation methods make behavior composable, but assemble supplied primitives rather than acquire local skills and task strategies from experience (Liang et al., 2022). Representative skill-library systems accumulate programs through human guidance or real-world execution feedback (Tziafas & Kasaei, 2024; Lu et al., 2026). Skill2Real instead relies on foundation VLMs for cross-domain grounding and directs simulation learning toward validated, hierarchical executable knowledge.

We introduce Skill2Real, an agentic policy framework that learns executable robot skills through a shared code-based policy interface. The key idea is to separate the information used to supervise skill learning from the information required to execute learned skills: simulator privilege can evaluate and improve behavior, while persistent skill knowledge is expressed entirely through public observations and API semantics. The shared interface defines the transfer boundary: programs invoke operations with common semantics, while each backend provides its own perception, calibration, and low-level execution. This separation lets task knowledge be reused across simulation and reality without incorporating simulator-only state into the deployed policy.

Skill2Real realizes this idea with an asymmetric Proposer–Verifier–Governor (PVG) loop that learns a two-level skill hierarchy. The Proposer executes programs from public observations and proposes reusable updates; the Verifier turns privileged simulation outcomes into publicly grounded diagnoses; and the Governor aggregates evidence across validation rollouts to admit supported updates into memory. Stage I learns Cerebellum skills for reusable local manipulation. Stage II freezes this library and learns Brain skills that compose it into long-horizon task programs with planning and recovery. At deployment, the Proposer grounds both frozen memories in current observations through the real code-based policy interface.

Our experiments test whether frozen hierarchical knowledge generalizes across task suites, Proposers, and the real robot. As frozen LIBERO-90 libraries (Liu et al., 2023) progress from no memory through the Cerebellum to the full hierarchy, success on unseen LIBERO-Pro Long tasks (Zhou et al., 2025) rises from 2.0% to 35.3% to 56.3% with Astra and from 0.5% to 29.3% to 49.0% with Opus 5, showing that both skill levels transfer across Proposers beyond the source tasks. Across seven single-arm and bimanual Robosuite tasks (Zhu et al., 2020), independently learned libraries raise mean success from 16.6% to 85.1% with Sol and from 19.4% to 89.4% with Opus 5, demonstrating consistent gains across Proposers. On the real UR5e, the full Brain–Cerebellum hierarchy reaches 78.75% mean task completion versus 56.25% with local skills alone, confirming the value of task-level composition for long-horizon behavior.

Our main contributions are:

• We introduce Skill2Real, an agentic policy framework that learns executable robot programs in simulation and transfers frozen skill knowledge through a shared code-based policy interface using only public observations at deployment, without real-world task-policy adaptation.
• We develop an asymmetric Proposer–Verifier–Governor scheme: the Proposer acts on public observations, the Verifier translates privileged execution evidence into public feedback, and the Governor admits updates supported by validation rollouts.
• We develop a hierarchical policy learning strategy in which the Cerebellum learns reusable local manipulation skills and the Brain learns long-horizon compositions conditioned on the frozen Cerebellum library, enabling task-level planning and recovery.

2 Related Work

Sim-to-Real Robot Learning.

Classical sim-to-real methods primarily bridge visual, dynamics, and execution gaps by learning policies or representations that are invariant to domain shift. Domain randomization (Tobin et al., 2017) and dynamics randomization (Peng et al., 2018) expose the learner to controlled variation, while privileged critics (Pinto et al., 2018), real-world adaptation (Chebotar et al., 2019), and active randomization (Mehta et al., 2020) provide additional training signals or target-domain experience when the gap is difficult to model. Foundation vision-language-action models instead bring broadly pretrained representations that can reason across heterogeneous observations, tasks, and embodiments: RT-2 (Zitkovich et al., 2023) and Open X (Collaboration, 2024) scale such learning, while Octo (Ghosh et al., 2024) and OpenVLA (Kim et al., 2025) make generalist policies more accessible. Building on the cross-domain reasoning capability of foundation VLMs, Skill2Real learns transferable task-level knowledge in the form of validated, executable skills and task programs, without learning a new representation from scratch. The frozen skills are deployed through a shared code-based policy interface: privileged state and evaluation are used only during simulation training, while the real backend supplies perception, calibration, and low-level execution.

Language Agents and Code Policies.

Language-conditioned policies provide a natural interface for composing robot behavior. Code as Policies (Liang et al., 2022) and ProgPrompt (Singh et al., 2023) generate executable programs, while SayCan (Ichter et al., 2023) grounds language plans in physically feasible affordances. Recent agentic systems expand beyond fixed skill sets: Agentic Skill Discovery (Zhao et al., 2024) bootstraps skills and success criteria, CaP-X (Fu et al., 2026) studies feedback-driven coding-agent improvement, and ASPIRE (Lu et al., 2026) executes, diagnoses, repairs, and validates robot programs. Playful Agentic Robot Learning (Zhang et al., 2026) and ENPIRE (Xiao et al., 2026) further investigate autonomous exploration and real-world policy evolution. Zetta (Ding et al., 2026) evolves code-based runtime critics and recovery skills around frozen base policies, combining action-level monitoring with validation-gated harness updates. It also demonstrates zero-shot transfer of learned recovery skills. Skill2Real is a multi-agent decision-making system in which the Proposer generates executable robot programs, the Verifier diagnoses their outcomes using privileged simulation evidence, and the Governor decides which validated updates enter persistent skill memory. Together, the agents learn a two-level hierarchy of local skills and task programs that is deployed as frozen, publicly grounded knowledge through the shared interface.

Robot Skill Learning and Hierarchical Policies.

Hierarchical reinforcement learning separates short-horizon control from long-horizon decisions. Options (Sutton et al., 1999), MAXQ (Dietterich, 2000), and HIRO (Nachum et al., 2018) establish temporal abstraction through subpolicies, value decompositions, and subgoal-conditioned control. Relay Policy Learning (Gupta et al., 2020) and SPiRL (Pertsch et al., 2021) show that reusable stages or skill priors can improve long-horizon manipulation. More recent systems build semantic or persistent skill libraries, including Lifelong Robot Library Learning (Tziafas & Kasaei, 2024), AtomSkill (Zhu et al., 2025), and Hi Robot (Shi et al., 2025b); MemoryVLA (Shi et al., 2025a), MemoryVAM (Jiang et al., 2026), RoboHarness (Huang et al., 2026), RoboHiMan (Chen et al., 2025), RoboPARA (Duan et al., 2026), and RigPI (He et al., 2026) study memory, policy routing, hierarchical evaluation, task planning, or dynamics identification for manipulation. Their transfer units are typically policies, latent skills, fixed stages, or episodic records. Skill2Real instead learns a two-level hierarchy of reusable skills. The Cerebellum acquires and freezes validated local manipulation skills, while the Brain learns task-level skills that compositionally recombine the local library into long-horizon programs. Reuse and composition at both levels support generalization across tasks, scenes, and deployment domains through the shared interface.

3 Method

Skill2Real transfers a frozen hierarchy of executable Cerebellum and Brain skills learned in simulation, rather than model parameters. This transfer builds on the premise that foundation VLMs provide representations capable of zero-shot generalization from simulated to real-world visual domains, while the shared code-based policy interface gives learned skills a common executable form in both. The Cerebellum learns reusable local manipulation skills, while the Brain learns task-level skills that compose them into long-horizon programs.

3.1 Shared Code-Based Policy Interface

The central intuition is to make learned robot knowledge executable and reusable across domains by representing it as programs over a public interface with common semantics.

Let \mathcal{E}^{\mathrm{sim}} and \mathcal{E}^{\mathrm{real}} denote a simulated and a real robot environment. Both expose an API contract \mathcal{A}=\{a_{k}\}_{k=1}^{K}, where K is the number of documented operations and a_{k} is the kth operation. The contract provides public observations and robot operations with common semantics. Its implementation is environment-specific, but the policy reasons only through the shared interface. This API therefore serves as the transfer boundary between high-level task knowledge and embodiment-specific execution. Superscripts \mathrm{sim} and \mathrm{real} identify the simulation and real-world domains, respectively.

At decision stage t, let g be the language goal, o_{\leq t} the history of public observations and API returns through stage t, and D_{\mathcal{A}} the documentation for \mathcal{A}. We use S_{\mathrm{cer}} and S_{\mathrm{brain}} for the active cerebellum and brain memories. The Proposer, instantiated by a foundation model M, samples an executable program c_{t} according to

c_{t}\sim\pi_{M}\!\left(c\mid g,o_{\leq t},D_{\mathcal{A}},S_{\mathrm{cer}},S_{\mathrm{brain}}\right).

Here, \pi_{M} is the conditional distribution over candidate programs c induced by M, and \sim denotes sampling from that distribution. Executing c_{t} produces documented API outcomes and subsequent public observations, which are appended to the interaction history. Consequently, the policy is grounded in the same information channel during simulation training and real deployment.

This interface is the concrete transfer boundary: the policy transfers executable behavior and task knowledge, while each backend implements embodiment-specific perception, calibration, and low-level control.

3.2 Multi-Agent Learning Framework

The multi-agent design separates roles by their information access and learning responsibilities. To remain deployable in the real world, the Proposer generates programs and skill updates using only information grounded in public observations and API outcomes available in both domains. The Verifier may instead inspect privileged simulation evidence to provide stronger rollout-level supervision, while the Governor aggregates and organizes multiple Verifier judgments across repeated rollouts, admitting only updates with consistent empirical support. These three large language model (LLM) agents form the Proposer–Verifier–Governor (PVG) loop used at both hierarchy levels. A deterministic execution evaluator and skill store support learning but are not additional agents. Deployment retains only the Proposer, shared interface, and frozen skill memories.

For a rollout trace \tau, let T be its final decision stage and x_{0:T}^{\mathrm{priv}} the corresponding simulator-only state trajectory. Let \mathcal{E}_{\mathrm{exec}} denote the deterministic execution-evaluator mapping, and let \mathcal{F}_{\mathrm{pub}} be the set of feedback messages expressible only through public observations and API semantics. The asymmetric flow is

e=\mathcal{E}_{\mathrm{exec}}(\tau,x_{0:T}^{\mathrm{priv}}),\qquad f=\operatorname{Verifier}(\tau,e),\qquad f\in\mathcal{F}_{\mathrm{pub}}.

Here, e is the evaluator’s summary of the realized physical outcome, \operatorname{Verifier}(\tau,e) denotes the Verifier’s feedback mapping, and f is the resulting feedback message. The superscripts \mathrm{priv} and subscript \mathrm{pub} denote privileged and public information, respectively. The Verifier is constrained to express f only through public observations and API semantics, so simulator-only quantities are excluded from the feedback supplied to the Proposer. Privileged information thus supervises learning without being required at deployment.

3.3 Hierarchical Skill Training

Long-horizon manipulation benefits from separating knowledge by abstraction: local interaction skills recur across tasks, while task-level knowledge specifies how to compose them for a goal. The cerebellum memory S_{\mathrm{cer}} contains primitive-level manipulation skills for reliable local interaction. Each skill combines natural-language guidance, an API-level code template, and a lightweight structured state description. The brain memory S_{\mathrm{brain}} instead contains task-level skills that compose cerebellum skills into long-horizon behavior. Thus, the cerebellum captures how to realize local effects, whereas the brain captures how those effects should be organized to solve a task.

The two levels share the same PVG training mechanism but differ in representation granularity and training context. Their hierarchical dependency is introduced by sequential training: Stage II learns task-level composition conditioned on the frozen Stage-I cerebellum library. Let \mathcal{Q}_{\mathrm{cer}} and \mathcal{Q}_{\mathrm{brain}} denote the primitive-level and task-level training-task distributions. We write \operatorname{TrainStage}(\mathcal{Q},C) for the shared PVG procedure that learns a new memory from tasks sampled from \mathcal{Q} while receiving fixed cross-level context C; C=\emptyset means that no such context is supplied. Stage I runs this procedure from an empty memory on \mathcal{Q}_{\mathrm{cer}} and freezes the resulting cerebellum library. Stage II starts a new empty brain memory, runs the same procedure on \mathcal{Q}_{\mathrm{brain}}, and receives the frozen cerebellum library only as context. Brain training therefore learns composition without modifying the local skills on which it depends. This dependency is

S_{\mathrm{cer}}^{*}=\operatorname{TrainStage}(\mathcal{Q}_{\mathrm{cer}},\emptyset),\qquad S_{\mathrm{brain}}^{*}=\operatorname{TrainStage}(\mathcal{Q}_{\mathrm{brain}},S_{\mathrm{cer}}^{*}).

The superscript * marks the validated memory returned and frozen at the end of a training stage.

To distinguish the evidence used to form a proposal from the evidence used to admit it, we denote by e_{\delta} the validation evidence associated with a candidate update \delta. After \delta is synthesized, the updated memory is evaluated on subsequent rollouts, and e_{\delta} aggregates their execution outcomes, such as task completion, failure, and regression relative to the pre-update memory. The Governor decides whether to accept or reject the proposal based on (\delta,e_{\delta}), so it is admitted only when its observed validation behavior supports the proposed improvement. As with e, this evidence is used by the learning-time Governor and is not exposed as simulator-state input to the deployed policy.

The complete PVG training procedure is provided in Algorithm 1 (Appendix G).

3.4 Zero-Shot Sim-to-Real Deployment

After training, Skill2Real transfers the learned skill memories through the shared interface between simulation and reality. The Proposer conditions on real observations together with S_{\mathrm{cer}}^{*} and S_{\mathrm{brain}}^{*}. Because both memories are expressed through public observations and API semantics, the same hierarchy can be grounded by the real robot.

4 Experimental Setup

4.1 Simulation Setup

To test whether one learning framework can acquire reusable hierarchical task knowledge across diverse tasks and robot configurations, we train separate Cerebellum and Brain libraries from scratch on LIBERO-90 (Liu et al., 2023) and seven Robosuite tasks (Zhu et al., 2020), using the same two-stage PVG procedure. We evaluate GPT-5.6 Sol (OpenAI, 2026a) and Claude Opus 5 (Anthropic, 2026b), learning a separate hierarchy per model and task set and assigning the same model to all three PVG roles. LIBERO-90 uses a Franka Panda; Robosuite spans single- and two-arm environments. In both experiments, the Proposer acquires RGB-D observations on demand by calling the shared code-based policy interface to request configured camera views, including top-down views. Task definitions, robot configurations, and observation settings are detailed in Appendices B.2 and B.3.

4.2 Training Setup

Each benchmark uses three Cerebellum iterations (C1–C3), then four Brain iterations (B1–B4) with C3 frozen; C0 is evaluation before training. Across both benchmarks, every iteration covers every task with five seeded training rollouts per task. This yields 450 episodes per iteration and 3,150 across C1–B4 on LIBERO-90, and 35 per iteration and 245 total across the seven Robosuite tasks. Governor-based runs evaluate each candidate over 25 validation rollouts using seeds disjoint from training. Sol, Opus 5, and the role ablations share the same per-task training-rollout budget; no-Governor omits admission validation. Figure 6 reports frozen LIBERO-90 checkpoints on LIBERO-Pro Long and independently trained Robosuite checkpoints. Additional evaluations and ablations on the source benchmarks appear in Appendix B.

4.3 Real-World Setup

All real-world tasks use a single UR5e arm with a Pika gripper and scene and wrist RGB-D cameras (Appendix B.5). A robot-specific backend implements the shared API. The real-world benchmark comprises pick-and-place, attribute-based sorting, equation assembly, and drawer manipulation (R1–R4; Appendix A). Deployment uses no real-world task demonstrations, task-policy fine-tuning, or online skill-memory updates.

4.4 Evaluation Setup

Evaluation isolates the contribution of hierarchical knowledge by comparing the base Proposer, Cerebellum-only skills, and the full hierarchy, with an additional Brain-only control on the real robot. On Pro Long, Astra and Opus 5 evaluate each LIBERO-90 checkpoint across 20 tasks and 10 seeds; the Astra series and training-role ablations use Sol-trained libraries. Real-world and cross-Proposer comparisons use the Sol-trained C3/B3 checkpoints (Appendix B.1). Success denotes complete task execution, and aggregate scores average per-task success rates. Real-world results use 20 trials per method for each task (R1–R4); the fixed C3/B3 Pro Long comparison uses 20 trials per task under each position and instruction perturbation setting (Appendix B.4).

5 Results

The evaluation addresses three questions: (1) What do local skill acquisition in the Cerebellum and task-level composition in the Brain each contribute to performance? (2) How broadly does the learned executable knowledge transfer through the shared code-based policy interface across unseen tasks, foundation VLM Proposers, and real deployment? (3) How much do the Verifier’s privileged diagnosis and the Governor’s evidence-based admission contribute to the learned skill library? We first analyze LIBERO-90-to-Pro Long checkpoint transfer and independent Robosuite learning for (1), then use real-world and fixed-library cross-Proposer evaluation for (2), and finally ablate the Verifier and Governor for (3).

5.1 Skill Transfer Across Training Checkpoints

Figure 6(a) tests whether continued skill learning on LIBERO-90 improves transfer to unseen long-horizon tasks. After each GPT-5.6 Sol training iteration on LIBERO-90, we freeze the resulting skill library and evaluate it on LIBERO-Pro Long with GPT-6 Astra as the Proposer. Opus 5 is evaluated under the same 20-task, 10-seed target protocol. Panel (a) therefore traces transfer from successive LIBERO-90 checkpoints to Pro Long.

LIBERO-90 to LIBERO-Pro Long. Both stages improve transfer across Proposers. Cerebellum learning raises Overall success from 2.0% to 35.3% for Astra and from 0.5% to 29.3% for Opus 5. With the C3 Cerebellum fixed, Brain learning further raises success to 56.3% and 49.0%, adding 21.0 and 19.8 percentage points. These gains show that the Brain can reuse local skills learned on LIBERO-90 to construct stronger long-horizon programs for unseen tasks. At B4, instruction/position success is 76.0%/36.5% for Astra and 66.5%/31.5% for Opus, indicating that object placement remains more challenging for both. The source-suite LIBERO-90 curves and six-family analysis are retained in Appendix B.2; the checkpoint transfer protocol is in Appendix B.4.

Robosuite. The same two-stage benefit appears when each model learns its own hierarchy from scratch across single-arm and bimanual Robosuite tasks (Zhu et al., 2020) (Figure 6(b)). Cerebellum learning raises seven-task mean success from 16.6% to 45.6% for Sol and from 19.4% to 49.6% for Opus. Brain learning then composes the fixed C3 skills to reach 85.1% and 89.4%, adding 39.6 and 39.9 percentage points and improving all seven tasks.

Across both benchmarks and all three Proposers, the two stages play complementary roles: the Cerebellum acquires reusable manipulation skills, and the Brain composes those fixed skills into more capable task programs. Concrete Cerebellum and Brain memory examples are shown in Appendix E.

5.2 Zero-Shot Real-World Transfer

With the Astra Proposer and API held fixed, the full hierarchy raises mean real-world completion from 27.50% without learned skills to 78.75% (Table 1), outperforming both the Brain-only (37.50%) and Cerebellum-only (56.25%) variants. Adding the Brain to the Cerebellum improves R1–R3 by 15, 5, and 20 percentage points and is the only configuration to complete drawer manipulation (50%). The hierarchy therefore contributes beyond reliable local execution: task-level composition turns reusable operations into stronger multi-step behavior.

Figures 4 and 4 show the learned local skills and their composition in recorded real executions. Appendix A provides additional examples under varied appearance and lighting; Appendix B.5 gives the evaluation protocol.

5.3 Zero-Shot Long-Horizon Generalization

Figure 7 compares Skill2Real with CaP-Agent0 (Fu et al., 2026), ASPIRE (Lu et al., 2026), and Zetta (Ding et al., 2026) on LIBERO-Pro Long. Using Astra with the fixed Sol-trained C3/B3 hierarchy, Skill2Real attains the best Overall (55.3%) and Task (75.0%) success, exceeding Zetta’s 51.5% and 63.0%. The larger Task margin reflects the value of hierarchical executable knowledge: local manipulation skills remain reusable while Brain programs rebind and compose them for redirected instructions. The same hierarchy reaches 35.3% Overall with Opus 4.6, above ASPIRE’s 30.5%, showing that the learned knowledge also transfers across test-time Proposers.

Zetta leads under position perturbations (40.0% versus 35.5%). This gap is driven by cases such as bowl-in-drawer and book-in-caddy, where an end-to-end VLA policy provides tighter reactive control for narrow-space pick-and-place than the current code-based skill interface. Appendix B.4 provides the per-task breakdown.

5.4 Verifier and Governor Ablations

PVG separates two complementary forms of learning-time supervision: the Verifier diagnoses execution from privileged simulation evidence, and the Governor admits validated updates. We ablate each role during LIBERO-90 training to test how these functions shape transfer to Pro Long (Figure 8).

Full Skill2Real leads at every iteration and reaches 56.3% Overall, versus 39.0% without the Verifier and 43.0% without the Governor—losses of 17.3 and 13.3 percentage points. The gap appears during Cerebellum learning and persists as the Brain composes the fixed C3 library, showing that grounded feedback and selective memory admission jointly improve transferable skill learning at both hierarchy levels. Appendix B.6 gives the ablation protocol and source-suite LIBERO-90 results.

6 Conclusion and Discussion

Skill2Real defines sim-to-real transfer at the level of executable task knowledge. Its multi-agent learning system uses privileged simulation evidence to build a two-level hierarchy: the Cerebellum captures reusable local manipulation skills, and the Brain composes them into long-horizon task programs. Expressing both levels through a shared code-based policy interface allows the frozen hierarchy to be grounded in real observations by a foundation VLM.

Across LIBERO-90, Robosuite, LIBERO-Pro Long, and UR5e experiments, the two levels provide complementary capabilities: local skills improve reliable interaction, while task-level composition supports reuse on unseen and long-horizon tasks. In particular, successive frozen LIBERO-90 checkpoints raise Astra’s success on unseen LIBERO-Pro Long from 2.0% to 56.3%, while the full hierarchy reaches 78.75% mean completion on UR5e versus 56.25% for the Cerebellum alone. The Proposer–Verifier–Governor ablations further show that privileged diagnosis and evidence-based memory admission materially improve skill acquisition. Together, these results demonstrate that simulation-learned programs can generalize across tasks, scenes, and the real robot while their learned hierarchy remains fixed.

Skill2Real currently operates through a shared policy interface whose robot backend provides perception and low-level control. Future work will extend the current pure code-as-policy interface by exposing end-to-end policies as callable tools, and reduce VLM inference latency for faster closed-loop execution.

AI Disclosure

We used generative AI tools to assist with manuscript drafting and language polishing. The LLMs used within our proposed method and their roles are described in the main text. The authors reviewed and revised all AI-assisted text to ensure accuracy, clarity, and consistency with the research conducted. We take responsibility for the final content of this work, including text, claims or artifacts produced with the aid of generative AI.

References

1. Anthropic (2026a) Anthropic. Introducing Claude Opus 4.6, February 2026a. URL https://www.anthropic.com/news/claude-opus-4-6. Accessed September 26, 2026.
2. Anthropic (2026b) Anthropic. Introducing Claude Opus 5, July 2026b. URL https://www.anthropic.com/news/claude-opus-5. Accessed September 26, 2026.
3. Chebotar et al. (2019) Yevgen Chebotar, Ankur Handa, Viktor Makoviychuk, Miles Macklin, Jan Issac, Nathan Ratliff, and Dieter Fox. Closing the sim-to-real loop: Adapting simulation randomization with real world experience. In 2019 International Conference on Robotics and Automation (ICRA), pp. 8973–8979, 2019. doi: 10.1109/ICRA.2019.8793789.
4. Chen et al. (2025) Yangtao Chen, Zixuan Chen, Nga Teng Chan, Junting Chen, Junhui Yin, Jieqi Shi, Yang Gao, Yong-Lu Li, and Jing Huo. Robohiman: A hierarchical evaluation paradigm for compositional generalization in long-horizon manipulation. arXiv preprint arXiv:2510.13149, 2025.
5. Collaboration (2024) Open X-Embodiment Collaboration. Open x-embodiment: Robotic learning datasets and rt-x models. In IEEE International Conference on Robotics and Automation (ICRA), pp. 6892–6903, May 2024. doi: 10.1109/ICRA57147.2024.10611477. URL https://ieeexplore.ieee.org/document/10611477.
6. Dietterich (2000) Thomas G. Dietterich. Hierarchical reinforcement learning with the maxq value function decomposition. Journal of Artificial Intelligence Research, 13:227–303, 2000. doi: 10.1613/JAIR.639.
7. Ding et al. (2026) Xin Ding, Liang Mi, Mingzhe Huang, Zixuan Wang, Chao Zhang, Zixu Hao, Fu Chen, Xiangyu Li, Yikai Zheng, Yaoyu Guo, Weijun Wang, Kun Li, Hao Wu, Yunxin Liu, and Ting Cao. Zetta \zeta: An efficient closed-loop embodied harness for self-evolving physical intelligence. arXiv preprint arXiv:2608.16590, 2026. URL https://arxiv.org/abs/2608.16590.
8. Duan et al. (2026) Shiying Duan, Pei Ren, Nanxiang Jiang, Zhengping Che, Jian Tang, Zhaoxin Fan, Yifan Sun, and Wenjun Wu. RoboPARA: Dual-arm robot planning with parallel allocation and recomposition across tasks, 2026. URL https://arxiv.org/abs/2506.06683.
9. Fu et al. (2026) Letian Fu, Justin Yu, Karim El-Refai, Ethan Kou, Haoru Xue, Huang Huang, Wenli Xiao, Guanzhi Wang, Dantong Niu, Fei-Fei Li, Guanya Shi, Jiajun Wu, Shankar Sastry, Yuke Zhu, Ken Goldberg, and Linxi Fan. CaP-X: A framework for benchmarking and improving coding agents for robot manipulation. arXiv preprint arXiv:2603.22435, 2026. URL https://arxiv.org/abs/2603.22435.
10. Ghosh et al. (2024) Dibya Ghosh, Homer Rich Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, Jianlan Luo, You Liang Tan, Lawrence Yunliang Chen, Quan Vuong, Ted Xiao, Pannag R Sanketi, Dorsa Sadigh, Chelsea Finn, and Sergey Levine. Octo: An Open-Source Generalist Robot Policy. In Proceedings of Robotics: Science and Systems, Delft, Netherlands, July 2024. doi: 10.15607/RSS.2024.XX.090.
11. Gupta et al. (2020) Abhishek Gupta, Vikash Kumar, Corey Lynch, Sergey Levine, and Karol Hausman. Relay policy learning: Solving long-horizon tasks via imitation and reinforcement learning. In Leslie Pack Kaelbling, Danica Kragic, and Komei Sugiura (eds.), Proceedings of the Conference on Robot Learning, volume 100 of Proceedings of Machine Learning Research, pp. 1025–1037. PMLR, 30 Oct–01 Nov 2020. URL https://proceedings.mlr.press/v100/gupta20a.html.
12. He et al. (2026) Xincheng He, Rongrong Zhang, Wei Jiang, and Wenqiang Xu. RigPI: Dynamic parameter identification of rigid body via VLM-seeded differentiable simulation, 2026. URL https://arxiv.org/abs/2606.25212.
13. Huang et al. (2026) Jinbang Huang, Yuanzhao Hu, Zhiyuan Li, Ran Qi, Yixin Xiao, Zhanguang Zhang, Mark Coates, Tongtong Cao, and Yingxue Zhang. Roboharness: Memory-driven orchestration of heterogeneous robot policies for long-horizon planning. arXiv preprint arXiv:2607.18060, 2026.
14. Ichter et al. (2023) Brian Ichter, Anthony Brohan, Yevgen Chebotar, Chelsea Finn, Karol Hausman, Alexander Herzog, Daniel Ho, Julian Ibarz, Alex Irpan, Eric Jang, Ryan Julian, Dmitry Kalashnikov, Sergey Levine, Yao Lu, Carolina Parada, Kanishka Rao, Pierre Sermanet, Alexander T Toshev, Vincent Vanhoucke, Fei Xia, Ted Xiao, Peng Xu, Mengyuan Yan, Noah Brown, Michael Ahn, Omar Cortes, Nicolas Sievers, Clayton Tan, Sichun Xu, Diego Reyes, Jarek Rettinghouse, Jornell Quiambao, Peter Pastor, Linda Luu, Kuang-Huei Lee, Yuheng Kuang, Sally Jesmonth, Nikhil J. Joshi, Kyle Jeffrey, Rosario Jauregui Ruano, Jasmine Hsu, Keerthana Gopalakrishnan, Byron David, Andy Zeng, and Chuyuan Kelly Fu. Do as i can, not as i say: Grounding language in robotic affordances. In Karen Liu, Dana Kulic, and Jeff Ichnowski (eds.), Proceedings of The 6th Conference on Robot Learning, volume 205 of Proceedings of Machine Learning Research, pp. 287–318. PMLR, 14–18 Dec 2023. URL https://proceedings.mlr.press/v205/ichter23a.html.
15. Jiang et al. (2026) Yuxin Jiang, Chang Yu, Yunuo Chen, Xiang Feng, Yin Yang, Nishank Gite, and Chenfanfu Jiang. Memoryvam: Integrating memory into video action model for robot manipulation. arXiv preprint arXiv:2606.20679, 2026.
16. Kim et al. (2025) Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan P Foster, Pannag R Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Openvla: An open-source vision-language-action model. In Pulkit Agrawal, Oliver Kroemer, and Wolfram Burgard (eds.), Proceedings of The 8th Conference on Robot Learning, volume 270 of Proceedings of Machine Learning Research, pp. 2679–2713. PMLR, 06–09 Nov 2025.
17. Liang et al. (2022) Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodied control. In arXiv preprint arXiv:2209.07753, 2022.
18. Liu et al. (2023) Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 44776–44791. Curran Associates, Inc., 2023. doi: 10.52202/075280-1939. URL https://proceedings.neurips.cc/paper_files/paper/2023/file/8c3c666820ea055a77726d66fc7d447f-Paper-Datasets_and_Benchmarks.pdf.
19. Lu et al. (2026) Runyu Lu, Yubo Wu, Ethan Kou, Letian Fu, Wenli Xiao, Ajay Mandlekar, Yinzhen Xu, Guanya Shi, Ken Goldberg, Ang Chen, Mosharaf Chowdhury, Yuke Zhu, Linxi "Jim" Fan, and Guanzhi Wang. ASPIRE: Agentic /Skills Discovery for Robotics. arXiv preprint arXiv:2607.00272, 2026. URL https://arxiv.org/abs/2607.00272.
20. Mehta et al. (2020) Bhairav Mehta, Manfred Diaz, Florian Golemo, Christopher J. Pal, and Liam Paull. Active domain randomization. In Leslie Pack Kaelbling, Danica Kragic, and Komei Sugiura (eds.), Proceedings of the Conference on Robot Learning, volume 100 of Proceedings of Machine Learning Research, pp. 1162–1176. PMLR, 30 Oct–01 Nov 2020. URL https://proceedings.mlr.press/v100/mehta20a.html.
21. Nachum et al. (2018) Ofir Nachum, Shixiang (Shane) Gu, Honglak Lee, and Sergey Levine. Data-efficient hierarchical reinforcement learning. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018. URL https://proceedings.neurips.cc/paper_files/paper/2018/file/e6384711491713d29bc63fc5eeb5ba4f-Paper.pdf.
22. OpenAI (2026a) OpenAI. GPT-5.6 Sol. OpenAI API model documentation, 2026a. URL https://developers.openai.com/api/docs/models/gpt-5.6-sol. Accessed September 26, 2026.
23. OpenAI (2026b) OpenAI. GPT-6 Astra. OpenAI API model documentation, 2026b. URL https://developers.openai.com/api/docs/models/gpt-6-astra. Accessed September 26, 2026.
24. Peng et al. (2018) Xue Bin Peng, Marcin Andrychowicz, Wojciech Zaremba, and Pieter Abbeel. Sim-to-real transfer of robotic control with dynamics randomization. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 1–8, May 2018. doi: 10.1109/ICRA.2018.8460528.
25. Pertsch et al. (2021) Karl Pertsch, Youngwoon Lee, and Joseph Lim. Accelerating reinforcement learning with learned skill priors. In Jens Kober, Fabio Ramos, and Claire Tomlin (eds.), Proceedings of the 2020 Conference on Robot Learning, volume 155 of Proceedings of Machine Learning Research, pp. 188–204. PMLR, 16–18 Nov 2021. URL https://proceedings.mlr.press/v155/pertsch21a.html.
26. Pinto et al. (2018) Lerrel Pinto, Marcin Andrychowicz, Peter Welinder, Wojciech Zaremba, and Pieter Abbeel. Asymmetric actor critic for image-based robot learning. In Proceedings of Robotics: Science and Systems, Pittsburgh, Pennsylvania, June 2018. doi: 10.15607/RSS.2018.XIV.008.
27. Shi et al. (2025a) Hao Shi, Bin Xie, Yingfei Liu, Lin Sun, Fengrong Liu, Tiancai Wang, Erjin Zhou, Haoqiang Fan, Xiangyu Zhang, and Gao Huang. Memoryvla: Perceptual-cognitive memory in vision-language-action models for robotic manipulation. arXiv preprint arXiv:2508.19236, 2025a.
28. Shi et al. (2025b) Lucy Xiaoyang Shi, Brian Ichter, Michael Robert Equi, Liyiming Ke, Karl Pertsch, Quan Vuong, James Tanner, Anna Walling, Haohuan Wang, Niccolo Fusai, Adrian Li-Bell, Danny Driess, Lachy Groom, Sergey Levine, and Chelsea Finn. Hi robot: Open-ended instruction following with hierarchical vision-language-action models. In Aarti Singh, Maryam Fazel, Daniel Hsu, Simon Lacoste-Julien, Felix Berkenkamp, Tegan Maharaj, Kiri Wagstaff, and Jerry Zhu (eds.), Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pp. 54919–54933. PMLR, 13–19 Jul 2025b. URL https://proceedings.mlr.press/v267/shi25d.html.
29. Singh et al. (2023) Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and Animesh Garg. Progprompt: Generating situated robot task plans using large language models. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 11523–11530, 2023. doi: 10.1109/ICRA48891.2023.10161317.
30. Sutton et al. (1999) Richard S. Sutton, Doina Precup, and Satinder Singh. Between mdps and semi-mdps: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence, 112(1–2):181–211, 1999. doi: 10.1016/S0004-3702(99)00052-1.
31. Tobin et al. (2017) Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 23–30, 2017. doi: 10.1109/IROS.2017.8202133.
32. Tziafas & Kasaei (2024) Georgios Tziafas and Hamidreza Kasaei. Lifelong robot library learning: Bootstrapping composable and generalizable skills for embodied control with language models. In 2024 IEEE International Conference on Robotics and Automation, ICRA 2024, Proceedings - IEEE International Conference on Robotics and Automation, pp. 515–522. Institute of Electrical and Electronics Engineers Inc., 2024. doi: 10.1109/ICRA57147.2024.10611448.
33. Xiao et al. (2026) Wenli Xiao, Jia Xie, Tonghe Zhang, Haotian Lin, Letian "Max" Fu, Haoru Xue, Jalen Lu, Yi Yang, Cunxi Dai, Zi Wang, Jimmy Wu, Guanzhi Wang, S. Shankar Sastry, Ken Goldberg, Linxi "Jim" Fan, Yuke Zhu, and Guanya Shi. Enpire: Agentic robot policy self-improvement in the real world, 2026. URL https://arxiv.org/abs/2606.19980.
34. Zhang et al. (2026) Junyi Zhang, Jiaxin Ge, Hanjun Yoo, Letian Fu, Zihan Yang, Yaowei Liu, Raj Saravanan, Shaofeng Yin, Justin Yu, Dantong Niu, Zirui Wang, Roei Herzig, Ken Goldberg, Yutong Bai, David M. Chan, Ion Stoica, Angjoo Kanazawa, Jiahui Lei, Haiwen Feng, and Trevor Darrell. Playful agentic robot learning. arXiv preprint arXiv:2606.19419, 2026. URL https://arxiv.org/abs/2606.19419.
35. Zhao et al. (2024) Xufeng Zhao, Cornelius Weber, and Stefan Wermter. Agentic skill discovery. arXiv preprint arXiv:2405.15019, 2024.
36. Zhou et al. (2025) Xueyang Zhou, Yangming Xu, Guiyao Tie, Yongchao Chen, Guowen Zhang, Duanfeng Chu, Pan Zhou, and Lichao Sun. LIBERO-PRO: Towards robust and fair evaluation of vision-language-action models beyond memorization. arXiv preprint arXiv:2510.03827, 2025.
37. Zhu et al. (2025) Yihang Zhu, Weiqing Wang, Shijie Wu, Ye Shi, and Jingya Wang. Learning semantic atomic skills for multi-task robotic manipulation. arXiv preprint arXiv:2512.18368, 2025.
38. Zhu et al. (2020) Yuke Zhu, Josiah Wong, Ajay Mandlekar, and Roberto Martín-Martín. Robosuite: A modular simulation framework and benchmark for robot learning. arXiv preprint arXiv:2009.12293, 2020. URL https://arxiv.org/abs/2009.12293v1.
39. Zitkovich et al. (2023) Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, Quan Vuong, Vincent Vanhoucke, Huong Tran, Radu Soricut, Anikait Singh, Jaspiar Singh, Pierre Sermanet, Pannag R. Sanketi, Grecia Salazar, Michael S. Ryoo, Krista Reymann, Kanishka Rao, Karl Pertsch, Igor Mordatch, Henryk Michalewski, Yao Lu, Sergey Levine, Lisa Lee, Tsang-Wei Edward Lee, Isabel Leal, Yuheng Kuang, Dmitry Kalashnikov, Ryan Julian, Nikhil J. Joshi, Alex Irpan, Brian Ichter, Jasmine Hsu, Alexander Herzog, Karol Hausman, Keerthana Gopalakrishnan, Chuyuan Fu, Pete Florence, Chelsea Finn, Kumar Avinava Dubey, Danny Driess, Tianli Ding, Krzysztof Marcin Choromanski, Xi Chen, Yevgen Chebotar, Justice Carbajal, Noah Brown, Anthony Brohan, Montserrat Gonzalez Arenas, and Kehang Han. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Jie Tan, Marc Toussaint, and Kourosh Darvish (eds.), Proceedings of The 7th Conference on Robot Learning, volume 229 of Proceedings of Machine Learning Research, pp. 2165–2183. PMLR, 06–09 Nov 2023. URL https://proceedings.mlr.press/v229/zitkovich23a.html.

Appendix A Real-World Tasks and Visual Generalization

A.1 Tabletop Appearance and Lighting Variation

Generalization across tabletop appearance.

The recorded deployments provide qualitative evidence that the learned manipulation skills can be reused across changes in tabletop color and texture (Figure A.2). The white-table and floral-cloth examples show the same potato-to-plate operation, with task completion confirmed in the archived reviews for both backgrounds. On the red gingham cloth, the robot executes three object transfers into color-labeled boxes; an external-video review confirms all three placements. Together, these examples support reuse across visually distinct real scenes. They complement the aggregate task-completion results in Table 1.

Execution under changed lighting.

Under dim, uneven side lighting, the robot executes two answer-cube pick-and-place sequences, including both releases and the return motion (panel (d)); the record ends before final equation acceptance. This qualitative example does not supply a separate condition-wise success rate or add trials to Table 1.

Appendix B Additional Experimental Details

B.1 Shared Configuration and Evaluation Protocol

Models and training iterations.

We train separate LIBERO-90 libraries with GPT-5.6 Sol (OpenAI, 2026a) and Claude Opus 5 (Anthropic, 2026b). Within each training run, the Proposer, Verifier, and Governor use the same model, with reasoning effort set to medium. C0 denotes the no-skill baseline. C1–C3 denote the first through third cerebellum training iterations; B1–B4 denote the first through fourth brain training iterations with the C3 cerebellum library held fixed. For the Astra Pro Long checkpoint series and role ablations, GPT-5.6 Sol trains the skills on LIBERO-90 and GPT-6 Astra evaluates each frozen checkpoint on LIBERO-Pro Long. The checkpoint curves also include Opus 5 under the same 20-task, 10-seed evaluation protocol. No Pro Long task is used for training or skill updates. The separate real-world and cross-Proposer comparisons in Table 1 and Figure 7 retain Sol’s LIBERO-90 cerebellum library at C3 and brain library at B3. These fixed libraries are shared across the test-time Proposers in those comparisons. Robosuite libraries are trained independently and are excluded from these transfer evaluations.

Controlled evaluation.

Both memories are frozen before each held-out evaluation; the Proposer receives public visual observations and API returns, without privileged simulator state or feedback from the Verifier and Governor. Within our memory-stage and role comparisons for a fixed Proposer, the API documentation, task definitions, and evaluation instances are held fixed. Comparisons across Proposers and with published methods use the models and protocols specified in the corresponding subsections. Success refers to complete tasks. Aggregate scores average per-task success rates, and missing results are marked with dashes rather than treated as zero success.

Reporting scope.

Success rates are point estimates for the recorded evaluations of fixed libraries. Repeated episodes assess a fixed library’s performance; they do not measure variability across independent training runs. Learning curves are indexed by training iteration, not cumulative rollout cost. Stage comparisons track the combined effect of additional training and the second memory level, rather than isolating hierarchy at a matched training budget.

B.2 LIBERO-90 Families and Stage Comparisons

Robot and observations.

LIBERO uses a seven-degree-of-freedom Franka Panda arm with a gripper and joint-position control at 20 Hz, with a maximum episode horizon of 4,000 environment steps. The Proposer obtains visual observations on demand by calling the shared code-based policy interface. The configured views include a fixed scene camera and a wrist-mounted camera, each providing RGB images at 800\times 512 resolution; top-down views can also be requested through the same API. Metric depth is reconstructed from calibrated top-down stereo RGB images through the documented depth operation. An image-path return alone does not provide depth or calibration; the wrist operation includes these only where documented and available. End-effector pose is available through its public query. Joint-position control does not imply a public joint-state query. Privileged simulator evidence is restricted to execution evaluation and the learning-time PVG loop.

Evaluation instances and aggregation.

The LIBERO-90 analysis asks which types of manipulation benefit from local skills and which improve further through task-level composition. Cerebellum and brain training have been completed with Sol and Opus as the Proposer. We organize evaluation by six task families and compare three memory states: no learned skills, the frozen Stage-I cerebellum, and the full hierarchy after Stage II. Evaluation at C0 and after each training iteration uses the same five held-out initial states per task, giving 450 trials per evaluation. Overall success is the number of successful trials divided by 450. Family success is the unweighted mean of its task success rates; the suite-level score averages over all 90 tasks, rather than giving differently sized families equal weight.

Table B.1 describes the manipulation scope of each family. These are analysis groups within LIBERO-90 (Liu et al., 2023), rather than separate benchmark suites. The row order matches the panels in Figure B.2.

Figure B.1 shows overall performance; Figure B.2 reports the six families using independent vertical scales. Sol and Opus are distinguished by color; the cerebellum and brain background regions meet at C3. The additional brain stage produces larger gains in caddy storage, basket/tray storage, and surface placement than in stacking for both models. Opus reaches 100.0% in caddy storage at B4, up from 56.7% at C3, while its stacking score remains at 40.0%. Thus, the aggregate gain does not imply equal improvement across manipulation types. These are independently trained libraries for each Proposer; reuse of the same frozen Sol-trained libraries across Proposers is evaluated separately in Appendix B.4.

Overall source-suite learning.

Sol improves from 14.2% at C0 to 38.2% at C3 and 62.0% at B4; Opus improves from 18.7% to 43.8% and 73.3%, respectively. Brain learning adds 23.8 and 29.6 percentage points with the C3 Cerebellum fixed. These curves evaluate the independently trained models on LIBERO-90; the main-text curves instead evaluate frozen checkpoints on Pro Long using Astra and Opus 5.

B.3 Independent Robosuite Training

Tasks, robots, and observations.

We report seven Robosuite tasks: cube lifting, stacking, and restacking; nut assembly; spill wiping; and two-arm lifting and handover. The configured task set also includes pick-and-place with a can, milk carton, or cereal box; these three tasks have no reported results and are excluded from the mean. Single-arm tasks use one manipulator, while lifting and handover use two arms. Robosuite allows the robot model to be selected through the environment configuration. Both configurations use joint-position control through the shared code-based policy interface. The Proposer acquires RGB-D observations through this interface to request the configured camera views. Task-facing images are rendered at 512\times 512 resolution, using a robot-view camera for single-arm tasks and a scene camera for bimanual tasks. Configured top-down views are accessible through the same observation API.

Independent acquisition and reporting.

The Robosuite experiment examines whether the same two-stage learning procedure is effective on another set of manipulation tasks (Zhu et al., 2020). Skills are trained anew on Robosuite, using its backend for the shared interface, rather than initialized from the LIBERO-90 memories. We report separate Sol and Opus runs at C0 and after each of three cerebellum iterations and four brain iterations, holding the C3 cerebellum fixed throughout brain learning. Both memories are frozen during evaluation. Mean success gives equal weight to the seven reported tasks. Since LIBERO itself uses Robosuite, this experiment broadens task-suite coverage within the simulation framework; it does not constitute transfer to a different physics engine.

Task-level progress.

Brain-stage gains occur on every reported task (Figure B.3). From C3 to B4, nut assembly improves by 62 and 50 percentage points for Sol and Opus, respectively; two-arm lifting improves by 48 and 46 points, and handover by 52 and 60 points. Cube lifting improves from 94% to 98% for Sol and from 77% to 95% for Opus. Thus, the aggregate progress includes substantial gains in assembly and bimanual manipulation.

Final checkpoint results.

At B4, Opus leads on five of seven tasks, while Sol scores higher on cube lifting (98% versus 95%) and spill wiping (100% versus 99%).

B.4 Long-Horizon Transfer Protocol and Per-Task Results

LIBERO-Pro Long tests whether skills learned on shorter source tasks can be composed on unseen long-horizon tasks. For this experiment, Skill2Real memories are learned only on LIBERO-90 and frozen before evaluation; the independently trained Robosuite memories are not used. Neither LIBERO-Long nor LIBERO-Pro Long tasks are used for skill construction, prompt examples, debugging, bootstrapping, or memory updates. Overall success is the equal mean of the position-perturbation (Pos) and instruction-perturbation (Task) success rates.

Evaluation of source-training checkpoints.

Figure 6(a) evaluates GPT-6 Astra and Opus 5 on LIBERO-Pro Long with 20 tasks and 10 seeds per task. For the Astra series, GPT-5.6 Sol performs skill training on LIBERO-90; after each iteration, the resulting library is frozen for target evaluation. C0 has no learned skills; C1–C3 use the corresponding Cerebellum library, and B1–B4 pair the frozen C3 Cerebellum with each successive Brain library. The target evaluations do not contribute training rollouts, feedback, or updates to later checkpoints. Figure 6(a) shows Overall success for both evaluation models across this series.

Separate fixed-library cross-Proposer comparison.

The comparison below retains the previously recorded C3/B3 evaluation, with the same Sol-trained libraries shared by Astra and Opus 4.6. It is a separate evaluation record from the C0–B4 checkpoint series; its aggregate and per-task results are not substituted into that curve. This earlier record uses ten tasks with 20 trials per task under each perturbation setting, with success macro-averaged over tasks.

Figure 7 compares Skill2Real using GPT-6 Astra (OpenAI, 2026b) and Claude Opus 4.6 (Anthropic, 2026a) against ASPIRE (Lu et al., 2026) and CaP-Agent0 from CaP-X (Fu et al., 2026) using Claude Opus 4.6, and Zetta (Ding et al., 2026). The ASPIRE and CaP-Agent0 scores are reported by Lu et al. (2026). ASPIRE uses its full N{=}90 LIBERO-90 skill library and generates one program per target task, evaluated on seeds 1–50 without additional debugging, retries, or task-specific library updates. CaP-Agent0 uses per-seed program generation with test-time reasoning and retries. For Skill2Real, evaluation excludes target-task skill updates and access to the privileged Verifier or Governor.

Zetta (Ding et al., 2026) evolves a harness around a frozen \pi_{0.5} action policy using 50 development seeds per target task, excluding test seeds 1–20. Its final evaluation uses those 20 held-out seeds. Its LIBERO-10 (Long) position-swap setting S and instruction-redirection setting T correspond to Pos and Task, respectively. Zetta’s 40.0% Pos and 63.0% Task scores give 51.5% Overall.

Tables B.2 and B.3 provide the per-task breakdown for Skill2Real, ASPIRE, and Zetta. CaP-Agent0 is omitted because only its aggregate scores are available.

Skill2Real’s success depends on the perturbation and test-time Proposer. For example, Astra achieves 90.0% on both the two-mug and mug-plus-pudding tasks under instruction perturbations, but 35.0% and 25.0% under position perturbations. Neither Proposer succeeds on the soup-plus-cream-cheese task under position perturbations. The Pos–Task gap also varies across tasks: on soup-plus-tomato-sauce, Astra instead scores 80.0% on Pos and 30.0% on Task. Position perturbations are therefore harder in aggregate, with substantial variation across individual tasks.

B.5 Real-Robot Evaluation Details

Hardware and observation interface.

All real-world tasks use a single UR5e arm equipped with a Pika gripper. The robot executes joint-space trajectories through ROS Noetic and the UR trajectory controller. Visual observations come from a fixed overhead Intel RealSense D435 and a wrist-mounted RealSense D405; the wrist camera provides 640\times 480 images. Both views provide RGB images and aligned metric depth from the RealSense sensors. Hand–eye calibration relates the wrist camera to the robot’s end effector. As in LIBERO, the public observation interface exposes both scene and wrist RGB-D views. Robot-specific calibration and low-level control are part of the backend implementation.

Tasks and deployment.

We evaluate real-robot transfer on specified-object pick-and-place (R1), attribute-based sorting (R2), equation assembly with cubes (R3), and drawer manipulation (R4). Appendix A defines their instructions and completion conditions. Both memories are learned in simulation and frozen for real evaluation. Deployment uses no real-world task demonstrations, task-policy fine-tuning, or online skill-memory updates. The real backend implements the same documented API, and the Proposer uses camera observations and public API returns without a real-world Verifier, Governor, or skill update.

Table 1 compares CaP-Agent0 from CaP-X (Fu et al., 2026), a GPT-6 Astra Proposer without learned skills, the two single-memory conditions, and the full hierarchy. All Astra conditions share the test-time model and robot interface. Cerebellum-only evaluation uses the Stage-I memory. Brain-only training runs Stage II with empty cross-level context, so its memory does not depend on unavailable cerebellum skills. Each method has 20 trials on each of R1–R4. A completed trial must satisfy its task-specific terminal condition, and Mean averages the four task success rates without weighting.

Qualitative image selection.

Figure 4 pairs recorded LIBERO-90 and real-robot frames by operation type. The top row of Figure 4 shows a complete citrus-retrieval task, from grasp in the open drawer to release on the table. The bottom row shows the initial equation and two answer-cube placements for R3, with completion confirmed by an archived final-state review. These images illustrate selected execution stages rather than the aggregate evaluation in Table 1.

B.6 PVG Training Ablations and Transfer Evaluation

All variants train with GPT-5.6 Sol on LIBERO-90 and share the API and source task stream. Without the Verifier, the Proposer consolidates from its public rollout record without privileged diagnosis. Without the Governor, updates pass deterministic skill-store checks but bypass evidence-based admission. The evaluator and skill store remain present. A public-observation-only Verifier would be needed to isolate privilege from the Verifier’s other contributions.

Transfer to LIBERO-Pro Long.

Figure 8 evaluates each frozen checkpoint with GPT-6 Astra on LIBERO-Pro Long. The removed roles affect only LIBERO-90 training; neither the Verifier nor the Governor participates in target evaluation. All variants begin at 2.0% Overall; at B4, Full Skill2Real achieves 56.25%, compared with 39.0% without the Verifier and 43.0% without the Governor. Losses are calculated from unrounded values, yielding 17.25 and 13.25 percentage points; the main text reports one decimal place.

Source-suite LIBERO-90 evaluation.

Figure B.4 evaluates the learned libraries on LIBERO-90 using Sol, reporting the C0 baseline and all seven training iterations. Success is the total score divided by 450. All three variants start from the same C0 score of 64/450 (14.2%). C1–C3 track cerebellum learning, followed by brain learning at B1–B4. Each evaluation uses five initial states per task, as specified in Appendix B.2. All variants are compared at B4, the final brain iteration of the same two-stage schedule.

Full Skill2Real has the highest success after every training iteration. Without the Governor, success stays near 40% from B2 through B4, while Full Skill2Real increases from 53.3% to 62.0%. Full Skill2Real reaches 62.0%, versus 38.0% without the Verifier and 40.0% without the Governor. The 24.0- and 22.0-point final gaps show lower success when either role is removed in these recorded runs. The curves do not measure variability across independent training runs.

Appendix C Shared Robot API and Execution Contract

The shared API in Section 3.1 separates the Proposer’s manipulation decisions from backend-specific perception and control. This appendix describes the 18 operations in the supplied standard_api/v1 simulation contract and the information passed between them. The operation inventory instantiates \mathcal{A}; its documentation supplies D_{\mathcal{A}}. Cerebellum and brain skills guide how the Proposer composes these operations, without extending the callable interface. The descriptions below summarize operation semantics rather than reproduce complete signatures or backend implementations. All edited examples in the appendices use this documented operation inventory.

C.1 Public Perception and Measured Geometry

Perception operations expose observations and measurements from which the Proposer selects task entities and action parameters. In the simulation contract, metric depth is reconstructed from calibrated stereo RGB images; the simulator depth buffer is not a public input. This specifies how depth is obtained, while RGB-D in the experimental setup describes the resulting observation modality. The real backend uses calibrated RGB-D sensors (Appendix B.5); sharing API semantics does not require identical sensing implementations.

The public interface excludes simulator object handles, ground-truth object poses, hidden scene structure, and task-success predicates. Visible candidates therefore remain hypotheses to bind from observations, rather than privileged object identities. Likewise, a principal axis or a valid point cloud describes measured geometry; the Proposer must still choose the contact, closing axis, destination, and intended effect. For a box-based segmentation query, the Proposer selects the box from the public image. Multiple candidate regions require separate supplied boxes; the contract does not promise text-prompted mask enumeration or a segmentation score. Geometry confidence describes the measurement returned by measure_visible_geometry, not semantic target correctness.

C.2 Execution State, Pose Construction, and Control

State and pose operations connect public observations to executable commands. The runtime associates calls with the current execution step, so programs do not supply an internal step index. The two image-path operations in Table C.1 return strings; the remaining operations return structured results with status information. A returned status must be interpreted at the level of the operation that produced it.

Execution state is distinct from learned skill memory. Here, “previous stage” means an earlier decision stage t, not Stage I or Stage II of training. The runtime’s held-object state carries geometric information needed to construct placement poses; it is not a sensor assertion that the object remains held. Refreshing this state or rebinding a target during deployment does not modify S_{\mathrm{cer}}^{*} or S_{\mathrm{brain}}^{*}. The placement compiler consumes the runtime’s carry geometry; a skill’s explanatory contact offset is not a replacement for that state. Motion quaternions use the documented xyzw order.

C.3 From an Observed Target to an Executed Placement

Placement combines a policy decision, geometric construction, and physical execution. The following sequence illustrates their dependencies under the contract; it is not an additional API operation or a recorded rollout.

1. Acquire a calibrated observation. get_topdown_stereo_view supplies an RGB pair, and get_foundation_stereo_depth produces metric depth with its observation_ref/depth_ref binding.
2. Choose and bind the target. The Proposer selects the destination and target pixel from the corresponding public frame, conditioned on the task and skill memories. It supplies this binding and the runtime’s carry geometry to compile_pose in place mode. The compiler uses them to determine support-relative release and clearance poses.
3. Execute the constructed poses. After inspecting compilation status, the program composes goto_pose and gripper_goto calls for transport, release, and retreat, checking each returned result.
4. Observe the physical result. Check public retention evidence during loaded motion and inspect fresh observations after release for the requested placement relation. A completed command alone does not establish either outcome.

The observation and depth references bind a chosen pixel to its measured frame and calibration; programs do not construct or edit that binding. These consistency checks prevent mixing incompatible inputs, but they do not establish that a previously observed object has remained stationary. The Proposer must obtain fresh evidence when scene changes invalidate a binding. Similarly, successful compilation establishes that poses were constructed under the contract, not that execution achieved the task.

C.4 Relation to PVG Learning and Frozen Deployment

The API defines the Proposer’s information and action channel, while PVG defines how execution evidence changes persistent knowledge. In Algorithm 1, the deterministic evaluator can inspect privileged simulation state to produce e. The Verifier converts that evidence into f\in\mathcal{F}_{\mathrm{pub}}, expressed through public observations and API semantics. Thus, excluding a success oracle from the robot API does not remove privileged evaluation from simulation training. Neither the evaluator nor the Verifier is a callable robot operation.

The Proposer uses the feedback to author candidate update \delta. Subsequent validation produces e_{\delta}, on which the Governor bases admission. API input checks and pose-compilation status are not substitutes for this validation, and candidate admission is distinct from the stage-level stopping criterion. The same three roles learn cerebellum skills in Stage I and brain skills in Stage II, with the cerebellum fixed during Stage II. At deployment, the Proposer executes through the real backend with both memories frozen and without the Verifier or Governor.

The supplied interface specification also records checks on the operation inventory, dispatch surface, documentation, and implementation digests. Such checks track a particular contract version; they do not establish physical task success or equivalence of perception and control across backends. The transfer requirement in Section 3.4 is compatible public semantics, with calibration and low-level execution implemented for the target robot.

Appendix D Recorded PVG Learning and Admission

This recorded bowl-transfer case illustrates how execution feedback informs program revision and how the Governor selects knowledge for memory. The account condenses supplied programs, feedback, candidate entries, and admission decisions into a qualitative example.1

D.1 From Execution Failure to a Candidate Skill

Failure and diagnosis.

The instruction asks the robot to place the left black bowl in a tray. Both visible bowls receive similar segmentation confidence, so the Proposer uses their current image positions to select the intended instance. Early attempts identify the correct bowl but fail to retain it during lifting. The Verifier localizes the failed transition to contact and retention: touching the bowl or completing a gripper-close command does not establish a grasp. It also notes that interpreting left depends on the current viewpoint.

Revision and candidate submission.

The Proposer revises contact construction toward the bowl’s narrower lower profile and checks the object after a short lift. Subsequent executions provide successful task outcomes and evidence for candidate review. The Proposer then submits entries about target selection, contact binding, destination geometry, and progress checking. The Governor evaluates each entry’s content against the actual program and execution record.

D.2 Why One Candidate Was Admitted and Three Were Rejected

The four proposals share a successful source program, but differ in their abstraction level and evidential support (Table D.1).

What enters memory.

The admitted Brain entry resolves the relative referent: merge duplicate bowl masks and select the distinct bowl with the leftmost center in the current image. It calls for reconsideration when viewpoint, visibility, or occlusion changes that ordering. The local contact revision remains outside the admitted Brain entry, and the untested progress-checking clause is not promoted.

What the case illustrates.

The Verifier diagnoses a failed physical transition, the Proposer authors the revision and proposed knowledge, and the Governor checks both layer boundaries and agreement with execution evidence. Whole-program success supports review but does not validate every associated proposal or isolate the causal contribution of one admitted rule.

Appendix E PVG Skill-Memory Templates and Examples

This appendix expands the skill representation in Section 3.3 through an initial brain template and edited cerebellum and brain examples. Every API call in these examples belongs to the final 18-operation interface in Appendix C. The examples illustrate skill content; they are not additional evaluated rollouts. The initial template is based on the preserved source snapshot. The PVG roles and admission procedure follow Algorithm 1.

E.1 Initial Brain Template

The initial task_policy.md supplies a writing scaffold with no learned task-level entries. Thus, an empty brain memory means an absence of learned task knowledge, not an absence of API documentation or a generic instruction scaffold. During Stage II, the Proposer develops task-level composition while the Stage-I cerebellum remains frozen.

⬇

Task: <public task instruction>

Learned task-level entries: none

Fixed context in Stage II: frozen cerebellum skills; documented robot API

## Task Understanding

Describe the required outcome and the public evidence that distinguishes

completion from partial progress. Bind objects and destinations from fresh

observations. Separate observed facts, assumptions, and unresolved questions.

Do not retain hidden scene facts or initial-state-specific coordinates.

## Starting Procedure

Read the public goal and observe the scene.

Determine the remaining objectives and the uncertainties that affect them.

Choose the next objective and an applicable frozen cerebellum skill.

Bind public inputs; specify the intended effect and its evidence check.

Execute through the documented API and inspect the returned evidence.

Continue on supported progress; request observations when a distinction is

missing; revise the remaining plan after an unsupported transition.

Preserve achieved effects and reason from the actual post-action scene.

Report completion only when supported by evidence under the task protocol.

## Learned Task-Level Knowledge

<Initially empty.>

For each proposed method, describe the public situation, the change in task

decisions, evidence for continuing, and failure or untested conditions.

Organize supported knowledge as procedures, branches, or short sketches.

The layout is editable; no fixed number or taxonomy of skills is required.

## Open Questions And PVG Update

<Initially empty.>

Use filtered Verifier feedback to identify an unresolved decision and a

discriminating trial. The Proposer authors the program and candidate update.

The Governor decides admission from validation evidence; an untested guess

or a successful episode alone does not establish a reusable improvement.

## Hierarchy Boundary

The brain selects objectives, their order, skills, and public input bindings.

Local geometry, contact, motion, retention checks, and local recovery belong

to the cerebellum. Do not modify that library during brain training.

Record a missing capability and its evidence when it limits composition.

E.2 Illustrative Skill Representation under the Shared API

Section 3.3 represents a skill through natural-language guidance, an API-level call sequence, and a compact state description. The edited example below uses the final shared API to illustrate this format. It has no associated rollout evaluation and is not used as evidence for the reported results. Its compact fields describe episode state rather than an API return schema.

⬇

## Scope

Use when the public scene supports the rim-contact trigger. The task objective

and destination are provided by the Proposer’s task-level composition.

## Natural-Language Guidance

Use measured visible body/rim geometry and a visibly clear side. Engage the wall

below the rim, with explicit image-selected closing and downward axes.

Inspect object–hand contact after closure and co-motion after a short lift.

Use runtime carry geometry when the next operation compiles a placement.

## Public Inputs

Object/handle images; local clearance; mask-based measured geometry;

fresh post-action images.

Missing geometry, calibration, or visual evidence remains unresolved.

## Compact State (Explanatory Fields)

contact: selected rim side, contact position, closing axis, approach axis

phase: contact selected / closed / lifted

retention: unresolved / supported / contradicted

evidence: public observations and API returns supporting the current phase

carry: runtime-owned held_object_state when available; never synthesized

## Decision And Execution Boundary

1. Read a public image; select the object box and a clear rim side.

2. Use get_sam_mask_from_box and measure_visible_geometry with the calibrated

depth belonging to that observation. Inspect the returned measurements.

3. Choose grasp_center_xyz and closing_axis_world from this public evidence.

Supply the documented request to compile_pose in grasp mode.

4. After checking compilation, use goto_pose for the constructed approach

and contact; inspect each result before issuing a dependent command.

5. Use gripper_goto for closure. Reobserve contact, then execute the short

lift with goto_pose and inspect fresh images for continued retention.

6. If retention is unresolved, obtain another public view. If contradicted,

revise local recovery. Otherwise return progress for task-level reasoning.

## Expected Effect And Limits

Supported acquisition means the vessel remains retained after the lift.

A constructed pose or completed command alone does not establish that effect.

An unsupported local effect calls for observation or local recovery, without

claiming task completion or changing the frozen skill text during deployment.

Memory and execution state.

In this illustration, the contact rule is persistent guidance. Contact coordinates, selected objects, measured offsets, and current retention evidence are episode-specific bindings refreshed from public observations. Updating those bindings during execution does not update S_{\mathrm{cer}}^{*} or S_{\mathrm{brain}}^{*}. This distinction is essential to the frozen deployment policy in Section 3.4.

What the representation establishes.

The illustrative template specifies when to act, how to construct a local action, and what evidence to inspect afterwards. Its compact state supports composition with later skills. The representation alone does not supply a new success metric or prove that the rule works outside its supported scope.

E.3 Documented Calls and the Placement Handoff

The sequence below names only operations in the final 18-operation inventory. The Proposer supplies arguments from the bound public observation and inspects each result before a dependent call.

⬇

1. get_topdown_stereo_view; get_foundation_stereo_depth.

2. get_sam_mask_from_box for each selected region;

measure_visible_geometry for a supported mask.

3. compile_pose in grasp mode; goto_pose for approach and contact;

gripper_goto for closure; reobserve retention after a short lift.

4. get_previous_proposer_state for runtime carry geometry;

compile_pose in place mode for the selected destination.

5. goto_pose for carry, release, and retreat; gripper_goto for release;

reobserve the placement relation.

Ground the inputs.

Metric depth and calibration come from get_topdown_stereo_view followed by get_foundation_stereo_depth. The Proposer chooses box_xyxy on the source image identified by that depth result; the mask and measurements must refer to this bound observation. A separately retrieved top-down image can check the physical result but cannot replace the bound source frame. The measurement options are only those documented for the backend. For grasp construction, the request contains the chosen grasp_center_xyz and closing_axis_world. Motion positions come from public measurements or constructed poses, and quaternions use xyzw order.

Inspect before continuing.

The Proposer groups native mask results under caller-supplied region labels; these are episode bindings, not verified identities. Using D_{\mathcal{A}}, it checks status and extracts a supported mask before requesting geometry with matching depth and calibration. Failed compilation blocks motion; completed closure calls for fresh visual retention checks.

Pass the placement binding.

After an executed compiled grasp, the previous public state may contain held_object_state. The Proposer passes that runtime-owned geometry, the selected destination pixel, and its immutable observation_ref/depth_ref binding in the documented place request. The compiler constructs carry, release, and retreat poses. Missing carry geometry or an invalid binding leaves placement unresolved; the program does not invent replacement state. Execution uses the returned poses, and fresh post-release observations check the intended relation.

E.4 Cerebellum Skills Beyond Grasping

The following cerebellum examples address two local decisions: identifying a receptacle’s actionable opening and selecting an interior placement target. Their decision rules are recorded in entry chunk47-task52-pattern0 of inspect.md and entry chunk35-task48-pattern1 of transport.md. The API sequence separates region segmentation from geometric measurement: named boxes supply candidate masks through get_sam_mask_from_box, and the Proposer inspects those results before calling measure_visible_geometry for a selected region (Appendix E.3). Task-entity selection and the intended relation remain brain-level decisions.

⬇

Source: chunk47-task52-pattern0; memory: cerebellum / inspect

## Public Trigger

The parent receptacle is visible, but an edge or wall patch can be mistaken

for its broad actionable opening. The intended region needs geometric checks.

## Learned Rule

Mark the parent and candidate opening regions on the bound public image.

Call get_sam_mask_from_box for each named box and inspect each returned mask.

Require sufficient region area and parent-relative overlap or containment.

Keep the distinction unresolved when the visible evidence is ambiguous.

## Procedure And State

Choose a supported opening from the inspected region results. Extract its

mask artifact according to the API schema, then call measure_visible_geometry with

the depth and calibration belonging to the same observation. Inspect the

native geometry result before using it. Carry the region label, selected

mask, parent relation, and observation/depth binding as episode state.

## Expected Effect And Boundary

Return geometry for the actionable opening, rather than an unsupported sliver.

If no candidate has adequate public support, keep localization unresolved;

no text-prompted mask enumeration or segmentation-score field is assumed.

⬇

Source: chunk35-task48-pattern1; memory: cerebellum / transport

## Public Trigger

An open receptacle is viewed obliquely. A visible wall or an incomplete view

of the opening makes its apparent center an unreliable placement target.

## Learned Rule

Compare the observed receptacle bounds with the visible opening to choose

a supported interior target pixel. Reobserve if occlusion prevents that

choice. Preserve the source rule’s support-relative placement intent through

compile_pose, which owns support height, carry geometry, and release poses.

## Procedure And State

Bind the target pixel to its observation_ref and depth_ref. Read the runtime’s

held_object_state through get_previous_proposer_state; never fabricate it.

Pass these inputs in the documented compile_pose place request. Inspect the

constructed carry, release, and retreat poses before executing them through

goto_pose and gripper_goto. Check each result before the next dependent move.

## Expected Effect And Boundary

The released object settles in the intended interior. Reobserve this relation;

a completed descent or opening command alone does not establish containment.

E.5 A Final Brain Skill for Relational Composition

The task-63 Brain document in the completed third round of the stacking training group contains two learned methods: chunk1-task63-brain0 and chunk1-task63-brain1. The source document records task-level ordering and referent binding while using the frozen cerebellum. The example below combines those two methods for presentation; it does not replace them with a new experimentally evaluated program.

⬇

Sources: chunk1-task63-brain0 (sequence), chunk1-task63-brain1 (binding)

Memory: brain; context: frozen cerebellum

## Task And Public Trigger

The requested final relations are the designated left bowl on the designated

right/support bowl, with the pair in the tray. Public evidence provides no

verified way to carry the nested pair together.

## Learned Task-Level Order

When the instruction allows this order, place the support bowl in the tray

first. Verify and preserve that relation, then execute only the remaining

upper-bowl-on-support part of the task.

## Learned Rebinding Rule

After moving the support bowl, inspect a fresh tray image. Select boxes for

the visible bowls, obtain their masks with get_sam_mask_from_box, and measure

them with matching depth and calibration. Bind the support only when current

image and geometry evidence distinguish the bowl inside the tray. An old

relational description can instead refer to the untouched bowl; measurement

confidence alone neither resolves identity nor proves placement success.

## Compact Task State

Goal relations: support in tray; upper bowl on support.

Progress: only relations supported by current public evidence.

Bindings: upper bowl, tray, and support bowl rebound after its movement.

Remaining objective: the unsatisfied relation, with prior progress preserved.

## Composition Sketch

Bind the two task objects and tray from public observations.

Use applicable frozen cerebellum skills to place the support bowl in the tray.

Observe the resulting relation and rebind the support by current containment.

If exactly one support is grounded, compose the upper-bowl placement suffix.

Reobserve both requested relations before reporting completion.

## Applicability Boundary

Do not apply this reordering when the instruction requires stacking first.

If the support is outside the tray, has moved, or cannot be identified

unambiguously, withhold the dependent suffix and preserve the observed state.

Initially occupied trays and other object classes were not established by

the recorded evidence for these methods.

Why this knowledge is brain-level.

The learned decisions concern which relation to establish first and how to identify an entity after its role in the scene changes. They do not change grasp geometry, gripper control, or local placement code. This is the Stage-II dependency in Equation 3: task composition is learned with S_{\mathrm{cer}}^{*} fixed.

The source reports whole-program development evidence and explicitly notes that it is not a matched causal evaluation of either decision alone. We use the record to illustrate learned content, without converting those annotations into a new held-out score or an isolated effect estimate.

E.6 Task Progress and Local Recovery in Drawer Manipulation

Drawer manipulation illustrates how the two memory levels handle different decisions. The Brain entry below is retained in the task-5 document from a completed third round of articulated-storage training. The cerebellum entry comes from the final manipulate.md library. Their juxtaposition explains the division of responsibility; it is not a claim that this exact pair was jointly executed in a recorded rollout.

⬇

Source: chunk9-task5-brain2; memory: brain / sequence

## Public Trigger And Decision

The requested carton has been released near the bound open drawer, and

closure remains unexecuted. Preserve the place-before-close order.

Allow closure only when a fresh observation supports the carton above the

measured drawer floor and within the originally observed drawer footprint.

## State And Boundary

Carry the intended drawer identity, its observed footprint, and evidence for

the released object’s relation to it. If the carton is outside or below the

drawer, or the destination is unresolved, withhold closure and preserve state.

A post-close handle detection that jumps to a different cabinet does not

justify another push. Do not treat a completed release as proved placement.

⬇

Source: chunk51-task0-pattern0; memory: cerebellum / manipulate

## Public Trigger And Local Policy

A drawer or slider initially moves, then deflects or stalls despite later

commands completing. Preserve the continuous, useful part of the stroke.

Disengage without an outward motion that reverses progress, reobserve the

shifted part, and relocalize it using relative geometry and continuity.

Use fresh short monotonic pusher strokes from the achieved state.

## State And Effect Check

Carry the observed part position, achieved displacement, contact mode, and

current continuity evidence. Check a compact pusher by observed part

displacement; a completed gripper command alone does not establish motion.

## Failure Boundary

Withhold the next interaction if relocalization merges adjacent handles.

Command completion alone does not establish slider motion or final closure.

The Brain source conditions a task transition on placement evidence; the cerebellum source controls the local realization of an already selected interaction. During Stage II, the Brain may change whether and when to request closure, but it cannot patch the frozen local controller. During deployment, both policies stay fixed while current bindings and progress evidence change.

E.7 Skill Invocation and the PVG Evidence Boundary

The Proposer is the acting agent at both hierarchy levels. Brain and cerebellum skills are memories conditioning its program generation, not additional agents. The following walkthrough expands Equation 1 using the relational-composition example. It shows what information passes between task decisions and local execution; the numbered stages are an explanatory sequence, not a measured trajectory.

⬇

## 1. Public Task And Observation

Read the requested relations and current public scene. Bind task entities;

keep unobserved facts unresolved. Both learned memories remain fixed.

## 2. Next Objective From Brain Memory

Review the ordering skill’s public trigger and temporal-order boundary.

If applicable, select support-bowl placement as the next objective.

## 3. Local Program Using Cerebellum Memory

Select applicable local perception, grasp, and placement knowledge.

Bind fresh geometry and generate the next API-level program using D_A.

Execute it and inspect returned motion, retention, and placement evidence.

## 4. Task-Level Progress Update

Reobserve the tray. Confirm the supported relation and rebind the moved

support bowl from current containment. Do not replay a completed prefix

merely because the remaining upper-bowl placement needs revision.

## 5. Remaining Objective

If the support binding is unambiguous, generate the local placement suffix.

Otherwise obtain the missing public evidence before a dependent action.

Report supported progress separately from unresolved final relations.

## Invariant

Only observations, bindings, programs, and episode state change here.

The Verifier and Governor are absent; no learned-memory update is applied.

How an entry becomes persistent during training.

Algorithm 1 distinguishes a rollout from an admitted skill update. For trace \tau, the deterministic evaluator produces evidence e; the Verifier returns feedback f\in\mathcal{F}_{\mathrm{pub}}; the Proposer authors candidate \delta; and subsequent validation produces e_{\delta}. The Governor decides whether (\delta,e_{\delta}) supports an update. Stage I applies accepted updates to the cerebellum; Stage II applies them to the brain while holding the cerebellum fixed.

What the source records establish.

The retained skill files supply rules, limitations, and development-program annotations. The paper examples use the current PVG roles and documented interface. The skill files alone do not supply a complete candidate-by-candidate transcript of e, f, \delta, e_{\delta}, and the Governor decision. Appendix D separately reconstructs one recorded Brain-development case from supplied trial evidence and admission records, with its campaign scope stated explicitly. Neither a final skill text nor a whole-program success count isolates an individual rule’s causal effect. Source identifiers support traceability; the public task and observations, rather than those identifiers, determine applicability.

Transfer boundary.

The transferred knowledge is the frozen policy content expressed through public observations and API semantics. The real backend supplies compatible perception, calibration, and low-level control. A scene-dependent contact pose or changing progress state is an execution binding, not online training. No simulation-only evidence or learning-time reviewer is required by the deployed Proposer.

Appendix F PVG Role Prompt Templates

These six edited templates instantiate the three roles in Algorithm 1 and Section 3.4, not verbatim historical runtime prompts. The Proposer performs four kinds of invocation; the Verifier diagnoses outcomes; the Governor admits fixed candidates from subsequent validation. Revisions return to the Proposer for new validation. Historical labels add no roles or admission levels. API signatures and returns remain defined by D_{\mathcal{A}}; skill content is illustrated in Appendix E.

F.1 Proposer: Cerebellum Rollout

⬇

You are the Proposer during Stage I of Skill2Real. Execute a local

manipulation task through the shared code-based policy interface using public information.

## Context

Task: <sampled task from the cerebellum training distribution>

Public history: <observations and returned API results through this step>

API documentation: <operation semantics, arguments, and return contracts>

Current memory: <cerebellum skills; no learned entries at initialization>

Cross-level context: none. No learned brain memory is supplied in Stage I.

## Ground The Next Action

Identify the intended local effect and the public evidence needed to check it.

Bind objects, contacts, and motion arguments from current observations and API

results. Check stored cerebellum guidance against the observed situation.

Obtain missing evidence before deciding; do not reuse an earlier episode’s

object pose or hidden facts.

## Generate And Execute

Write the next executable program using the documented API. Respect its

units, argument meanings, permitted values, and return conventions. A stored

code pattern may guide composition; it does not add an undocumented method.

Distinguish constructing an action target from executing the action. Inspect

the actual returned results and subsequent observations before proceeding

as though the intended contact, grasp, displacement, or release occurred.

Do not obtain simulator state, hidden object lists, or evaluation predicates.

Use only documented return fields and fresh public images for retention

checks; a gripper command or cached carry geometry does not prove retention.

## Check The Local Transition

Compare the intended effect with public execution evidence. Preserve effects

that are supported, and identify the first failed or uncertain transition.

An issued command is not evidence that the object moved or remained held.

If a documented API call fails, follow its failure contract. If the physical

effect is uncertain, seek the observation needed to resolve that uncertainty.

Choose any recovery from the new public state rather than assuming that the

original state still holds. Keep partial progress separate from completion.

## Record And Hand Off

Return the program and public state needed for the next step, distinguishing

confirmed effects, unresolved evidence, and API failures. The execution trace,

not your summary, supplies the rollout evidence. After the rollout, the

Verifier provides filtered feedback for the Proposer’s separate consolidation.

## Memory Boundary

Exploration does not directly change the persistent cerebellum library.

A candidate remains pending until subsequent validation supports admission

by the Governor. Do not update brain memory or claim that an exploratory

success has already established a reusable skill.

F.2 Proposer: Brain Rollout

⬇

You are the Proposer during Stage II of Skill2Real. Solve the task by

composing the frozen cerebellum skills through the shared code-based policy interface.

## Context

Task: <sampled task from the brain training distribution>

Public history: <observations and returned API results through this step>

API documentation: <documented operations and their return contracts>

Current memory: <brain skills; no learned entries at initialization>

Fixed context: <the cerebellum library frozen at the end of Stage I>

## Establish The Task State

Read the public goal and identify the remaining objectives. Distinguish

observed relations, achieved effects, uncertain conditions, and requirements

that remain unmet. Do not assume that an earlier completed action remains

valid after a later action changes the scene.

Use existing brain guidance where its public conditions match. If no learned

entry applies, reason from the public task, API, and frozen cerebellum.

## Compose The Next Step

Choose the next objective and an applicable cerebellum skill. Bind its inputs

from current observations, identify its intended local effect, and determine

how that effect contributes to the task. Respect dependencies between

objectives and preserve progress needed by later operations.

Reuse skill guidance and code patterns through documented API operations;

do not assume a stored skill name is itself a callable API method.

Task ordering and composition belong to the brain. Use the frozen local

skills for their supported manipulation behavior and failure boundaries.

## Execute And Revise The Plan

Write executable API-level code for the next step. Inspect returned results

and fresh observations before advancing to an objective that depends on it.

When an effect is unsupported, determine what evidence is missing and whether

the remaining plan must change. Select recovery from the current scene.

If a frozen local skill lacks a required capability, record the limitation

and its public evidence. Do not silently change that skill or assume the

missing capability succeeded. Unresolved subgoals remain unresolved.

## Output And Working State

Return the next program and a compact public task state: supported progress,

remaining objectives, the next evidence check, and unresolved limitations.

Keep episode-specific bindings separate from reusable task-level knowledge.

Do not claim full completion from a successful local operation alone.

Use only public observations and API returns; never query hidden evaluation

state to decide which object, destination, or action is correct.

## Learning Boundary

Do not edit either library during rollout execution. The Proposer later

consolidates a candidate brain update from filtered Verifier feedback.

The Governor admits it only when subsequent validation supports it.

The Stage-I cerebellum remains fixed throughout Stage II, including recovery,

candidate construction, and validation of the proposed brain update.

F.3 Proposer: Skill Consolidation

⬇

You are the Proposer performing skill consolidation in a fresh context.

Turn the Verifier’s public diagnosis into a candidate update for the memory

currently being trained. You author the proposal; you do not admit it.

## Context

Training stage and target memory: <Stage I / cerebellum or Stage II / brain>

Current target memory: <the accepted skill content before this proposal>

Feedback: <the Verifier’s filtered, publicly grounded message>

API documentation: <documented operations available to the proposed skill>

For Stage II only: <the frozen Stage-I cerebellum as fixed context>

Treat supplied public evidence as the basis for the update; do not reconstruct

missing observations from hidden state or assume access to an earlier context.

## Identify The Reusable Lesson

Separate the observed problem, the Verifier’s explanation, and the policy

change to be tested. Identify what the current memory already handles and

what behavior the update would change. Preserve guidance that remains

supported. An uncertain diagnosis is a hypothesis to test, not a proven rule.

If the feedback provides no transferable lesson, report the unresolved issue

instead of manufacturing a successful skill from bookkeeping or speculation.

## Write The Candidate

Specify the target memory and the public conditions under which the skill

should apply. Explain the intended behavior as reusable guidance, and provide

an API-level code template with observation-derived inputs and state checks.

Describe the compact state needed to use the skill: relevant public bindings,

confirmed effects, unresolved requirements, and when a new observation is

needed. Distinguish these runtime values from persistent skill knowledge.

State the expected transition, the evidence that would support it, and known

limitations. Explain what should happen when its conditions or effects fail.

## Match The Hierarchy Level

For cerebellum updates, express how to obtain a local manipulation effect

and check its realization. For brain updates, express how to choose, order,

or recover task-level compositions using the frozen cerebellum.

Do not move a missing local capability into an unvalidated brain workaround

that silently rewrites the cerebellum. Identify the capability gap explicitly.

## Keep The Skill Public And Transferable

Do not store simulator-only coordinates, hidden identities, private scores,

or a memorized trajectory. Keep API semantics separate from backend-specific

calibration and execution. Do not introduce a method absent from the API.

Use public applicability conditions rather than naming one training episode.

## Proposal Output

Present the target memory, change relative to current content, applicability,

guidance, code template, state description, expected effect, and limitations.

Identify the improvement that subsequent validation should test and the

existing behavior that should be checked for regression.

Label the update as a candidate awaiting validation. Do not claim unseen

validation outcomes, write it into active memory, or return an admission

decision. The Governor reviews the proposal after validation evidence exists.

F.4 Verifier: Diagnosis and Public Feedback

⬇

You are the Verifier during simulation training in Skill2Real. Diagnose a

completed Proposer rollout and return public feedback that can guide the

Proposer’s next skill proposal. You do not write programs or admit updates.

## Context

Training stage: <cerebellum learning or brain learning>

Rollout trace: <program, ordered API calls, returns, and public observations>

Evaluation evidence: <realized outcomes from the deterministic evaluator>

API documentation: <the contract used by the Proposer’s program>

Privileged evidence is available for diagnosis, not for disclosure to the

Proposer. Base each conclusion on the evidence actually supplied to you.

## Reconstruct The Intended And Actual Transitions

Identify the effect the program intended and locate the relevant calls and

observations in temporal order. Distinguish generated code, an issued call,

a returned execution result, and an established physical effect.

Use privileged evaluation evidence to check what actually happened. Do not

substitute the Proposer’s own report for execution and outcome evidence.

Identify the first failed or unsupported transition, rather than assigning

the terminal failure to every preceding action.

## Diagnose At The Appropriate Level

Separate incorrect use of the API from an unsuccessful policy decision.

For a contract error, identify the misuse of the documented arguments or

returns. For a local manipulation failure, identify the intended effect and

where the physical evidence ceases to support it. For a composition failure,

explain which task dependency or remaining objective was mishandled.

State which behaviors worked and should be preserved. Distinguish observed

facts from causal hypotheses, give the relevant evidence, and identify

uncertainty. Do not infer a durable repair from an untested explanation.

## Construct Public Feedback

Translate the diagnosis into public observations and API semantics. The

message may explain that an intended effect failed, but must not supply a

hidden scene inventory, privileged target coordinates, private predicates,

or identities absent from the Proposer’s public observations and returns.

Suggest a focused diagnostic test using public observations and documented

actions. State what observation could distinguish the plausible explanations.

Do not prescribe a scene-specific solution obtained from privileged state.

## Review The Information Boundary

Before returning feedback, check every factual detail for disclosure of

simulator-only knowledge. Omit or express it as a publicly grounded outcome

description when that preserves the diagnostic meaning. If evidence cannot

support a public explanation, state the uncertainty without inventing one.

The feedback must remain usable without privileged state at policy execution.

## Output And Authority

Return public feedback describing the unsupported transition, supported

effects, likely cause and uncertainty, and the next public diagnostic test.

Do not include a private evidence dump in that message. Do not author a skill

update, rewrite either library, or decide whether to accept or reject.

The Proposer authors the candidate; the Governor judges it from subsequent

validation evidence.

Apply the same boundaries in both training stages. You are absent during

held-out evaluation and real deployment.

F.5 Governor: Validation and Admission

⬇

You are the Governor during simulation training in Skill2Real. Decide

whether a Proposer-authored candidate update should enter the target memory.

You evaluate the proposal and evidence, rather than authoring a new policy.

## Context

Training stage and target memory: <Stage I / cerebellum or Stage II / brain>

Candidate update: <the Proposer’s proposed change and stated scope>

Validation evidence: <subsequent rollout outcomes for the proposed update>

Comparison evidence: <available outcomes relative to the pre-update memory>

Validation protocol: <the prescribed conditions and candidate comparisons>

Use the supplied records. Do not invent trials, scores, thresholds, or controls

that were not part of the recorded validation.

## Check The Candidate Contract

Determine whether its applicability and behavior use public observations and

documented API semantics. Check that the proposed knowledge belongs to the

target hierarchy level and can be used beyond a memorized training scene.

Reject dependence on hidden state or an undocumented operation. During brain

training, verify that the candidate leaves the frozen cerebellum unchanged.

Formatting and deterministic skill-store checks do not establish efficacy.

## Assess The Validation Evidence

Identify the behavior that changed because of the candidate and the evidence

that this behavior actually executed. Compare its stated expected effect

with the observed validation outcomes and the pre-update memory where the

comparison is supplied. Separate a plausible explanation from a demonstrated

effect; unrelated later progress does not establish the candidate’s benefit.

Consider task completion, failures, and regression together under the supplied

protocol. State whether the evidence supports the proposed improvement within

the candidate’s stated scope. Do not infer untested generalization from one

favorable situation or substitute the motivating rollout for later validation.

## Handle Limitations Explicitly

Distinguish evidence against the proposed behavior from missing or unreliable

evidence. A candidate with insufficient support remains unaccepted in either

case; explain which limitation controls the decision. Do not repair absent

evidence by assuming that the candidate must have worked.

If validation supports only a different scope or a modified policy, identify

that issue for the Proposer. Do not rewrite the candidate while admitting it,

because the revised content would not be the proposal that was evaluated.

## Return The Admission Decision

Return accept or reject. Support the decision with the relevant validation

outcomes, the observed benefit or failure, any regression, and the limits of

the evidence. Accept only when validation behavior supports the improvement.

On rejection, identify the unsupported claim or missing evidence. On

acceptance, identify the reviewed update that the skill store may apply.

A rejected candidate leaves active memory unchanged; a later revision is a

new Proposer-authored proposal that requires its own validation evidence.

## Information And Memory Boundaries

Privileged evidence may inform admission but must not become hidden content

inside the deployed skill. Stage-I admission targets only cerebellum memory;

Stage-II admission targets only brain memory, with the cerebellum frozen.

You are absent during evaluation and real deployment, when both memories

are fixed and no online skill-memory update is permitted.

F.6 Proposer: Frozen Evaluation and Deployment

⬇

You are the Proposer executing Skill2Real with frozen skill memories.

Use the shared code-based policy interface to solve the public task from current observations.

The Verifier and Governor are absent, and no skill learning occurs here.

## Context

Mode: <held-out simulation evaluation or real-robot deployment>

Task: <the public instruction for the current episode>

Public history: <camera observations and returned API results>

API documentation: <the documented contract of the active backend>

Memories: <the frozen cerebellum and brain content supplied for this condition>

Use only the supplied memories. Do not assume access to skills withheld by

an evaluation condition or reconstruct them from hidden training artifacts.

## Ground The Frozen Hierarchy

Identify the remaining task objectives and their publicly observable state.

Use applicable brain guidance for task composition and cerebellum guidance

for local manipulation. Check their conditions against the current scene.

Bind objects, targets, and other runtime inputs from fresh public evidence;

do not carry over a training episode’s identities or calibrated poses.

When a required distinction is unresolved, obtain the relevant observation

before selecting an action that depends on it.

## Execute Through The Shared Interface

Write executable code using the documented operations and return contracts.

The backend implements perception, calibration, and low-level execution;

do not replace its documented semantics with assumptions from another robot.

Stored code templates guide execution but do not create new callable methods.

Use only documented return fields; cached carry geometry is not evidence

of retention.

Inspect API returns and observed effects before treating a local transition

as achieved. Maintain the remaining task state from those confirmed effects.

## Recover From Current Public Evidence

If a call fails or an expected effect is absent, follow the documented failure

behavior, obtain fresh evidence as needed, and revise the remaining actions.

Use the supplied skill guidance within its supported scope. Record a missing

capability or unresolved condition instead of claiming progress that did not

occur. Keep local manipulation success separate from full task completion.

Do not query simulator-only state, request privileged feedback, or invent a

success label to resolve ambiguity. Outcome reporting follows the task’s

evaluation protocol and the evidence available through the public interface.

## Preserve The Frozen Policy Content

Update only episode working state, such as observed object bindings,

confirmed effects, remaining objectives, and uncertainties. These runtime

records are not additions to either persistent skill memory.

Do not consolidate new skills, revise stored guidance or code templates,

perform task-policy fine-tuning, or use real-world experience to update either

library. Do not invoke the Verifier or Governor. Select new actions from fresh

observations while keeping the transferred knowledge fixed.

## Output

Return the next executable program and compact public working state needed

to continue. When reporting the episode, distinguish supported completion,

partial progress, execution failure, and unresolved evidence using the

documented reporting contract. Do not fabricate a final observation or imply

that frozen deployment included learning or memory admission.

Appendix G PVG Training Pseudocode

Algorithm 1 details the shared training rule and sequential skill acquisition introduced in Section 3.3. Algorithm-specific notation is defined before its first use in the pseudocode.