Inline Memory Meets Reusable Skills: Memory-centric Framework for Vision-Language-Action Model
This preprint introduces Optimus-R, a memory-centric VLA framework that turns robotic manipulation adaptation into explicit query-skill memory tuning instead of repeated parameter updates. With only 30% of training data it raises LIBERO average success to 88.6% (vs 72.1% for π0.5) and reaches 33.3% real-world success from 20 demos per task; in lifelong learning it reduces forgetting to 10.0% vs 15.0%. The approach matters for low-data robot adaptation and retaining prior skills.
Can a vision-language-action model acquire, store, and reuse manipulation skills through an explicit memory interface rather than re-tuning model weights, and does this improve data-efficient adaptation, cross-domain transfer, and lifelong learning?
VLA models are typically adapted to new tasks, objects, and domains by updating their parameters, which is compute-intensive and can overwrite previously learned behaviors. External retrieval memories used in prior work sit outside the action-conditioning pathway and cannot capture control-relevant information, while visual or dynamics shifts cause target observations to fall outside the source memory space, breaking query-skill retrieval.
Common practice is parameter-centric adaptation: full fine-tuning or parameter-efficient methods such as LoRA and adapters applied to π0.5 or OpenVLA-OFT. Memory-augmented robot policies usually store demonstrations, experiences, or scene context as detached external retrieval; this weakly couples memory to action generation. These methods still encode new skills in model weights or rely on retrieval contexts that do not align with the policy's control pathway.
Optimus-R initializes from π0.5 and adds four learnable memory tokens to the VLA prefix stream; two attention-pooling heads then produce a query embedding for retrieval and a skill embedding for behavior, with the skill embedding trained to reconstruct actions. A Query-Skill Memory Bank stores 128 prototypes as decoupled query prototypes and skill values; retrieval uses top-2 similarity over aligned queries, and the fused skill is projected into four skill tokens for the flow policy. Training has three stages: Stage A pretrains the interface with flow-matching, query contrastive, and skill-reconstruction losses on 8×A800 GPUs, batch 256, 30,000 steps; Stage B aligns target-domain queries and skills with the bank using a low-rank skill adapter and residual prototype updates; Stage C confines adaptation to the memory bank and residuals. The backbone and interface stay frozen after Stage A except in Stage B.
In simulation, at 30% data Optimus-R reaches 88.6% LIBERO average success (+16.5 vs π0.5 72.1), CALVIN ABC→D length 2.51 (+0.42 vs 2.09), and RoboTwin 2.0 Hard 14.8% (+5.0 vs 9.8). At 100% data it scores 97.8% LIBERO average, 4.38 CALVIN length, and 55.0% RoboTwin Hard, beating π0.5 (96.9%, 4.26, 46.6). In 9-task real-world transfer with 20 demos/task, Optimus-R reaches 33.3% success vs π0.5 19.4% (+13.9); with 80 demos it reaches 72.2% vs π0.5 63.3%, also beating OpenVLA-OFT 54.4%, π0 57.2%, RDT 26.1%, and OpenVLA 28.3% at 80 demos. In lifelong learning, with 20 demos per new task Optimus-R reaches 21.0% on new tasks (+11.0 over π0.5); after 80-demo adaptation it retains 77.5% old-task success and forgets 10.0% vs 15.0% for π0.5 and OpenVLA-OFT. Cost comparison on LIBERO 100% data: Optimus-R uses 86 GPU hours and runs at 14 Hz, vs π0.5 LoRA at 128 GPU hours and 12 Hz, and OpenVLA-OFT LoRA at 320 GPU hours and 10 Hz.
The paper does not include an explicit limitations section. Evident limits: evaluation is on one bimanual platform (GALAXEA R1 Lite) and nine real-world tasks; real-world results use 20 rollouts per task; no code release or peer-review note is mentioned; the lifelong setup covers only five new tasks; and the method still requires an initial 30,000-step interface pretraining on 8×A800 GPUs before memory adaptation.
This could be adopted by robot foundation-model builders and manipulation-focused robot OEMs, especially companies working on π0.5-based VLA stacks or bimanual systems in logistics, kitchens, or labs. In the near term (now to 1–3 years) it is most relevant to teams already using π0.5 or OpenVLA; deployment in production would require broader real-world validation, a released or stable implementation, and tooling to maintain the memory bank per robot or fleet. Edge deployment at scale may take 3+ years because memory compression and onboard inference still need further work.
The full text is not republished here because the paper's license does not allow it. Read the original on arXiv.