JitterVLA: A Replay Benchmark for Motion Quality in VLA-Driven Human-Robot Manipulation

CCS: Computer systems organization Robotics

Abstract.

Vision-language-action (VLA) policies issue sparse targets whose arrival at the robot controller is delayed and irregular, so a reasonable predicted action can still produce abrupt motion near people. Remedies are evaluated on different streams, delays and metrics. JitterVLA replays identical commands and arrival traces through alternative controllers, isolating the controller’s effect. We apply it to synthetic reaches, hand trajectories from 20 people, three open VLA pipelines (838 episodes) and a physical UR3. Interpolation lowered modeled p99 jerk for every source, by 83% (615 to 106 m/s3) on human motion with 200 ms delay, at a 2.0 cm tracking cost, and the effect replicated on 598 new episodes; jerk-limited generation lowered jerk further at a larger cost. Entropy–discontinuity associations had opposite, replicated signs in two pipelines, and entropy gating helped in one but not the other. On the UR3, interpolation lowered accelerometer-derived jerk by 3.0–14.0% in all six exploratory pairs, while a position-derived metric did not resolve the contrast. JitterVLA supports screening before human-centred studies; code and data will be released.

Keywords:

robot motion quality, vision-language-action models, command timing, jitter, paired replay, benchmark

1. Introduction

A person hands a manipulator an object, guides its target, or works beside it. The high-level policy produces a sensible action, but its targets arrive sparsely and at irregular times, so the low-level controller receives discontinuous goals. What the person experiences is not the predicted action but the motion the robot executes. VLA policies such as OpenVLA (Kim et al., 2025b) run at a few hertz, one to two orders of magnitude below the robot’s control loop (Jiang et al., 2026), and remote or shared inference adds network delay. Human-robot interaction (HRI) research links onset latency to teleoperation performance (Rakita et al., 2020; Khalil and Kwon, 2026) and robot motion to observers’ responses and perceived safety (Kulić and Croft, 2007; Rubagotti et al., 2022).

Timing is not only latency. A constant delay shifts every command, whereas timing jitter (variation in delay and arrival spacing, including bunching) changes which targets the controller sees and when. Remedies exist at several layers: action chunking and asynchronous execution (Zhao et al., 2023; Black et al., 2025b; Zhao et al., 2025; Guo and Guo, 2026; Kim et al., 2025a), interpolation and jerk-limited trajectory generation such as Ruckig (Berscheid and Kröger, 2021), geometry-guided jerk reduction in VLA execution (Yun et al., 2026), and stiffness adaptation (Ferraguti et al., 2013). Each paper evaluates its remedy with its own inputs, delays, plant and metrics, so the remedies cannot be compared fairly. JitterVLA addresses that comparison problem. It evaluates the middle of the chain (command stream, timing, controller, executed motion) before candidate behaviours are shown to people. It does not replace human studies.

We ask four research questions (RQ1–RQ4). RQ1 (timing): how do timing irregularities change motion-quality proxies for identical command streams? RQ2 (control): how do stepped, interpolated, entropy-gated, constant-gain and jerk-limited strategies move the jerk–tracking trade-off? RQ3 (cross-source): do these relationships hold across synthetic, human and three VLA action representations, and on new episode draws? RQ4 (physical): are the contrasts observable on a real UR3 under a bounded protocol?

Contributions. (1) A paired replay benchmark and protocol that holds the command source and delay realization fixed while varying the controller. (2) An evaluation that progresses from controlled synthetic reaches to 20-person human motion, three VLA pipelines with a planned replication on 598 new episodes, and measured network traces (Table 1). (3) Findings: a consistent jerk–tracking trade-off for interpolation, representation-dependent uncertainty associations, and a failure of entropy gating on some inputs. (4) A bounded UR3 check, including a null result for the planned position-derived metric. Code, the transformed VLA streams, the human windows and the traces will be released.

Terms. Latency is delay; jitter is its variation. Command discontinuity is the spatial jump between successive targets or chunks. Jerk is the benchmark plant’s jerk proxy, or a sensor-derived estimate on the UR3. Tracking error is the root-mean-square error (RMSE) against the undelayed reference. Jerk, tracking error, log dimensionless jerk (LDLJ) and spectral arc length (SPARC) (Balasubramanian et al., 2015) are motion-quality proxies; comfort, preference and safety were not measured.

2. Benchmark design

Paired replay. Within a comparison, every controller receives the same source (episode, person or synthetic reach) and the same delay realization (seed or trace offset). Differences are therefore attributable to the controller rather than to different stochastic inputs. Statistics are paired over the source unit: episodes, people or seeds. Intervals are 95% unit bootstraps (5000 resamples), not time-sample intervals.

Timing pipeline. For VLAs, measured inference service times from one RTX 5090 feed a single-worker first-in, first-out (FIFO) queue. A network delay is then added: zero, Gamma-distributed with mean 0–400 ms (shape 2), or sequential samples from 12 h of measured lab-path traces: local area network (LAN), internet round trip, and a cold HTTPS request. Delivery is in order, and the newest delivered target wins. Separating the mean delay from its spread lets RQ1 test jitter as well as latency.

Plant and controllers. Delivered targets drive a generic three-axis point mass (3 kg, critically damped impedance, 20 ms force lag, 3000 N/s force-rate cap, 1 ms steps). It is not a UR3 model; absolute jerk values are not hardware predictions. Controllers:

• B0 holds each delivered target (stepped).
• B2 interpolates linearly over one command period.
• A2 adds entropy-gated stiffness to interpolation.
• B3 (Ruckig, jerk limit 25 m/s3) is an explicit jerk-limited trajectory generator.

Interpolation is the simple smoothing baseline; B3 tests whether gains survive a stronger motion-generation method. A constant-gain grid (8 fixed stiffness values, stepped and interpolated) asks whether gating beats the best attainable constant gain at no worse tracking, rather than an obviously stiff baseline. We report 99th-percentile (p99) and root-mean-square (RMS) jerk and tracking RMSE, plus LDLJ and SPARC. LDLJ and SPARC are established dimensionless movement-smoothness measures, and they check that the conclusions do not depend on p99 jerk alone.

Evidence hierarchy. Protocols were frozen before data were seen: frozen analyses (v1.1, v1.3), a planned replication on new draws (v1.4), post-outcome robustness checks (v1.5: traces, LDLJ/SPARC, Ruckig, pi0.5 (Black et al., 2025a)), and an exploratory hardware follow-up (v1.2a). We label results accordingly and never pool across these labels.

3. Results

R1: interpolation trades jerk for tracking on every source (RQ1–3; frozen, replicated). Interpolation lowered modeled p99 jerk and raised tracking error for synthetic, human and all three VLA sources (Table 2). The effect held on the new episode draws. The reduction is not confined to synthetic or learned commands: it appears when replaying reaches from 20 people. Timing irregularity, not mean delay, drove stepped-goal jerk. In the frozen v1.0 sweep on synthetic reaches, pure fixed delays of 0, 200 and 400 ms left stepped p99 jerk unchanged (535, 534 and 533 m/s3), whereas a 200 ms delay spread raised it to 623. Under the three measured traces, the B2-B0 direction held for every source; for example, pi0-FAST gave -368, -350 and -331 m/s3 for LAN, internet and HTTPS. LDLJ and SPARC favoured B2 in every tested cell, and no interval crossed zero (post-outcome). The generic plant’s force-rate cap accentuates stepped-goal differences, so these magnitudes are not hardware predictions.

R2: lower jerk is not free (RQ2). Figure 2 shows the geometry on human input at 200 ms mean delay. Stepped B0 averages 615 m/s3 at 9.91 cm tracking error. Interpolation (B2) moves to 106 m/s3 at 11.87 cm. Jerk-limited B3 reaches 23.5 m/s3, but at 14.79 cm. Entropy-gated A2 (150 m/s3, 16.72 cm) is worse than B2 on both axes for this input. The benchmark thus identifies a useful intervention, a costly one, and a plausible heuristic that fails.

R3: uncertainty is representation-dependent (RQ3; frozen, replicated). In OpenVLA, mean translational-token entropy was weakly negatively associated with the successive goal step (mean within-episode Spearman \rho=-.065 [-.110, -.021]; new 200: -.089 [-.117, -.061]). In pi0-FAST, chunk-token entropy was positively associated with the chunk-boundary jump (\rho=.221 [.169, .271]; new 199: .217 [.181, .253]) (Fig. 3). Both directions replicated. Because the events and representations differ, the defensible conclusion is not that one model’s uncertainty is better. Instead, whether uncertainty tracks downstream discontinuity depends on the pipeline, so uncertainty-based control needs pipeline-specific validation. Controller outcomes follow the same split. On pi0-FAST, A2 lowered p99 jerk against B2 by 208 m/s3 (new draw: 199) for +0.18 cm (+0.16) tracking. It also beat the best attainable constant gain on feasible subsets (-12.4 m/s3 on 66/80; -5.6 on 162/199); feasibility was incomplete, and that audit was post hoc for the original set. On OpenVLA, A2 did not help: +2.3 [-0.7, 5.2]; new draw +3.4 [1.9, 4.8]. Table 3 summarises what replicated: all pre-registered pi0-FAST contrasts (28/28) and 22/28 for OpenVLA, whose six misses were mostly cells near zero.

R4: a bounded physical check (RQ4). The frozen v1.2 protocol replayed an author teleoperation trace in 30 small-amplitude runs on a UR3 CB3 at 125 Hz. Position-derived p99 filtered jerk was 77.7–81.9 m/s3 across conditions, and the planned contrast was not resolved at that logging resolution. That null result exposed a measurement limit, so an explicitly exploratory v1.2a session repeated the same motion plan while logging the robot’s tool-flange accelerometer (Fig. 4). In direct mode, B0 exceeded B2 in all three paired repeats at each delay, by 3.07 m/s3 on average without added delay and 5.79 with 200 ms mean delay. B2 reductions were 3.0–14.0%, and plant-mode differences had mixed signs. The sensor read p99 3.79 when stationary and 43.05–51.52 during motion; its gain, bandwidth and the source of the moving signal were not established. The same session’s position metric also showed positive direct contrasts, so the accelerometer is not the sole source of the direction. It does not validate the point-mass plant as a UR3 model, and it is not a calibrated tool-centre-point jerk or safety result.

4. Code structure, usage and maintenance

The Python code has four parts: core/ (command timing and FIFO delivery, the impedance plant, the paired stepped, interpolated and entropy-gated conditions and their metrics; NumPy and SciPy), benchmark/ (frozen protocols, experiment drivers, paired analysis, and the v1.5 add-ons: Ruckig jerk-limited controllers, LDLJ/SPARC and trace delays), jittervla/ (the command-line tool) and tests/ (34 unit tests); scripts/ holds the UR3 session and analysis tools. python -m jittervla replay --csv my.csv replays one’s own time-stamped targets and stream --npz a stored VLA stream, each printing every condition under the same arrivals. The code will be released under Apache-2.0 after the review period, in a public repository with versioned releases and an archived snapshot linked from the project page. Issues and pull requests are reviewed by the authors, and any change to controller semantics or results is released as a new protocol version rather than silently replacing a frozen result.

5. Availability and ethics

The code release will contain the code (Apache-2.0), tests, a CPU demo, frozen protocols, the transformed VLA streams (CC BY 4.0, BridgeData/DROID attribution), the 20 hand windows and the latency traces; the reported tables regenerate on a CPU without model weights or a robot. Hand trajectories are a secondary analysis of the public, de-identified 3D-ARM-Gaze dataset (Lento et al., 2024) (Apache-2.0; consent and ethics approval are reported by its authors), redistributed with attribution and a statement of changes. The UR3 teleoperation episodes were recorded by an author and are released at https://huggingface.co/datasets/armteam/JitterVLA-teleop-episodes; no human-subject study was performed.

6. Limitations and conclusion

• Model. The main results use one generic point-mass plant with open-loop VLA predictions; closed-loop behaviour, task success and robot-specific dynamics are outside scope.
• Comparability. The three pipelines differ jointly in model, dataset, action representation, entropy definition and discontinuity event, so they do not form a controlled model comparison. Service times and traces come from one GPU and one lab’s network paths.
• Hardware. The UR3 study is small (30 + 30 runs, one session each), the accelerometer is uncalibrated, and the v1.2a follow-up was designed after the v1.2 null result.
• HRI interpretation. All metrics are motion-quality proxies. Comfort, preference, trust and perceived safety were not measured; JitterVLA identifies which conditions warrant human-centred evaluation.

Conclusion. Across 69,988 paired replays from synthetic reaches, 20 people and three VLA pipelines, interpolation cut modeled p99 jerk for every source (by 83% on human motion with 200 ms mean delay) at a 0.3–2.4 cm tracking cost, and the effect replicated on 598 new episodes. Entropy gating helped on pi0-FAST, whose uncertainty tracked chunk jumps, but not on OpenVLA. On a real UR3, interpolation lowered accelerometer-derived jerk by 3.0–14.0% in all six exploratory pairs. The released code and data let HRI researchers screen controller choices before human-centred studies.

References

1. Balasubramanian et al. (2015) Sivakumar Balasubramanian, Alejandro Melendez-Calderon, Agnes Roby-Brami, and Etienne Burdet. 2015. On the analysis of movement smoothness. Journal of NeuroEngineering and Rehabilitation 12 (2015), 112. doi:10.1186/s12984-015-0090-9
2. Berscheid and Kröger (2021) Lars Berscheid and Torsten Kröger. 2021. Jerk-limited Real-time Trajectory Generation with Arbitrary Target States. In Robotics: Science and Systems XVII (RSS). Robotics: Science and Systems Foundation. doi:10.15607/RSS.2021.XVII.015
3. Black et al. (2025a) Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Robert Equi, Chelsea Finn, Niccolo Fusai, Manuel Y. Galliker, et al. 2025a. \pi_{0.5}: a Vision-Language-Action Model with Open-World Generalization. In Proceedings of The 9th Conference on Robot Learning (CoRL) (Proceedings of Machine Learning Research, Vol. 305). PMLR, 17–40.
4. Black et al. (2025b) Kevin Black, Manuel Y. Galliker, and Sergey Levine. 2025b. Real-Time Execution of Action Chunking Flow Policies. In Advances in Neural Information Processing Systems 38 (NeurIPS). Neural Information Processing Systems Foundation, 37596–37620. doi:10.52202/085713-1122
5. Ferraguti et al. (2013) Federica Ferraguti, Cristian Secchi, and Cesare Fantuzzi. 2013. A tank-based approach to impedance control with variable stiffness. In IEEE International Conference on Robotics and Automation (ICRA). IEEE, 4948–4953. doi:10.1109/ICRA.2013.6631284
6. Guo and Guo (2026) Tiecheng Guo and Meng Guo. 2026. Action ControlNet: A Lightweight Delay-Aware Adapter for Smooth Asynchronous Control in Vision-Language-Action Models. arXiv preprint arXiv:2606.25985 (2026).
7. Jiang et al. (2026) Wenqi Jiang, Jason Clemons, Karu Sankaralingam, and Christos Kozyrakis. 2026. How Fast Can I Run My VLA? Demystifying VLA Inference Performance with VLA-Perf. arXiv preprint arXiv:2602.18397 (2026).
8. Khalil and Kwon (2026) Aws Khalil and Jaerock Kwon. 2026. Nonlinear Performance Degradation of Vision-Based Teleoperation under Network Latency. arXiv preprint arXiv:2603.06850 (2026).
9. Khazatsky et al. (2024) Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, et al. 2024. DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset. In Robotics: Science and Systems XX (RSS). Robotics: Science and Systems Foundation. doi:10.15607/RSS.2024.XX.120
10. Kim et al. (2025a) Moo Jin Kim, Chelsea Finn, and Percy Liang. 2025a. Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success. In Robotics: Science and Systems XXI (RSS). Robotics: Science and Systems Foundation. doi:10.15607/RSS.2025.XXI.017
11. Kim et al. (2025b) Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan P. Foster, Pannag R. Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. 2025b. OpenVLA: An Open-Source Vision-Language-Action Model. In Proceedings of The 8th Conference on Robot Learning (CoRL) (Proceedings of Machine Learning Research, Vol. 270). PMLR, 2679–2713.
12. Kulić and Croft (2007) Dana Kulić and Elizabeth Croft. 2007. Physiological and subjective responses to articulated robot motion. Robotica 25, 1 (2007), 13–27. doi:10.1017/S0263574706002955
13. Lento et al. (2024) Bianca Lento, Effie Segas, Vincent Leconte, Emilie Doat, Frederic Danion, Renaud Péteri, Jenny Benois-Pineau, and Aymar de Rugy. 2024. 3D-ARM-Gaze: a public dataset of 3D Arm Reaching Movements with Gaze information in virtual reality. Scientific Data 11 (2024), 951. doi:10.1038/s41597-024-03765-4
14. Pertsch et al. (2025) Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. 2025. FAST: Efficient Action Tokenization for Vision-Language-Action Models. In Robotics: Science and Systems XXI (RSS). Robotics: Science and Systems Foundation. doi:10.15607/RSS.2025.XXI.012
15. Rakita et al. (2020) Daniel Rakita, Bilge Mutlu, and Michael Gleicher. 2020. Effects of Onset Latency and Robot Speed Delays on Mimicry-Control Teleoperation. In ACM/IEEE International Conference on Human-Robot Interaction (HRI). ACM, 519–527. doi:10.1145/3319502.3374838
16. Rubagotti et al. (2022) Matteo Rubagotti, Inara Tusseyeva, Sara Baltabayeva, Danna Summers, and Anara Sandygulova. 2022. Perceived safety in physical human–robot interaction—A survey. Robotics and Autonomous Systems 151 (2022), 104047. doi:10.1016/j.robot.2022.104047
17. Walke et al. (2023) Homer Rich Walke, Kevin Black, Tony Z. Zhao, Quan Vuong, Chongyi Zheng, Philippe Hansen-Estruch, Andre Wang He, Vivek Myers, Moo Jin Kim, Max Du, et al. 2023. BridgeData V2: A Dataset for Robot Learning at Scale. In Conference on Robot Learning (CoRL) (Proceedings of Machine Learning Research). PMLR Vol. 229, 1723–1736.
18. Yun et al. (2026) Sangho Yun, Minsoo Kim, Minwoo Cho, and Hwanjo Yu. 2026. GROOVE: Geometry-Guided Reduction of Operational-Space Jerk in VLA Execution. arXiv preprint arXiv:2609.13695 (2026).
19. Zhao et al. (2023) Tony Z. Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. 2023. Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware. In Robotics: Science and Systems XIX (RSS). Robotics: Science and Systems Foundation. doi:10.15607/RSS.2023.XIX.016
20. Zhao et al. (2025) Yongsheng Zhao, Lei Zhao, Baoping Cheng, Gongxin Yao, Xuanzhang Wen, and Han Gao. 2025. VLA-RAIL: A Real-Time Asynchronous Inference Linker for VLA Models and Robots. arXiv preprint arXiv:2512.24673 (2025).