Is It the Curriculum or Just the Easy Start? Benchmarking Unsupervised Environment Design on Quadrupeds
Unsupervised environment design (UED) methods such as PLR, ACCEL, SFL and PAIRED are compared mostly on 2D mazes, grid worlds and BipedalWalker, while terrain curricula for legged robots are hand-designed and evaluated against a few baselines. We present a benchmark that runs fourteen UED, domain-randomisation and robotics curricula under one protocol on five 3D quadrupeds in MuJoCo MJX: the closed-chain W01-TEK and the open-chain Unitree Go1 and Go2, Google Barkour vB and Boston Dynamics Spot.
From the abstract. Our summary is in progress.
Full paper
Is It the Curriculum or Just the Easy Start? Benchmarking Unsupervised Environment Design on Quadrupeds
Published under CC BY 4.0. Reproduced with attribution; original at arXiv:2610.40364 (PDF).
Abstract
Unsupervised environment design (UED) methods such as PLR, ACCEL, SFL and PAIRED are compared mostly on 2D mazes, grid worlds and BipedalWalker, while terrain curricula for legged robots are hand-designed and evaluated against a few baselines. We present a benchmark that runs fourteen UED, domain-randomisation and robotics curricula under one protocol on five 3D quadrupeds in MuJoCo MJX: the closed-chain W01-TEK and the open-chain Unitree Go1 and Go2, Google Barkour vB and Boston Dynamics Spot. Levels are vectors of eight interpretable terrain parameters, and all curricula drive the same PPO student at matched environment-step budgets. The evaluation reports success on random and held-out structured terrains, interquartile means with stratified bootstrap intervals, capability envelopes and a level-by-level domination comparison. We compare methods within each robot and report, per level set, the top group of methods not significantly below the best. No method is in the top group on every robot and level set. Methods with easy or low-difficulty training levels, curated or not, fill the top groups on in-distribution random levels of Go1 and Barkour, and of W01-TEK apart from PAIRED+BiBC. The success-driven terrain curriculum from legged-robot practice is in a top group most often, domain randomisation on uniform levels in none, and PAIRED in none outside Spot, where the fourteen methods lie within 3.8 points on random levels of difficulty 0.25. On three robots the leader on in-distribution random levels is not the leader on held-out terrain. We collect the results into a per-robot guide to choosing a curriculum.
Keywords:
Unsupervised Environment Design, Curriculum Learning, Reinforcement Learning, Legged Locomotion, Benchmark
1Wrocław University of Science and Technology
Wybrzeże Wyspiańskiego 27, 50-370 Wrocław, Poland
2Machinekind, Wrocław, Poland
3Koło Naukowe Solvro, Wrocław University of Science and Technology, Wrocław, Poland
{275409,279690,279903}@student.pwr.edu.pl
\theta\in[0,1]^{8} of eight terrain parameters, and the same PPO student trains on them for each of five quadrupeds. Adaptive curricula choose the next levels from the student’s rollouts (dashed).†
\icml@noticeprintedtrue
1 Introduction
Learned locomotion controllers are trained on distributions of terrain, and that distribution shapes what the controller can do. Practical pipelines use hand-designed curricula that promote a robot along rows of increasing difficulty (Rudin et al., 2021) or adaptive samplers over a parameterised terrain family (Lee et al., 2020). Recent work automates this with learning-progress (Li et al., 2026), learned-difficulty (Liu et al., 2026) and safety-critic (Ovinnikov et al., 2026) curricula, each with its own robot, baselines and protocol. Unsupervised environment design (UED) studies the same problem in general form: a teacher proposes environment parameters (levels) so that a student generalises. PAIRED (Dennis et al., 2020) learns a level generator from a regret signal. PLR and its robust variant PLR⟂ (Jiang et al., 2021b; Jiang et al., 2021a) curate random levels by a regret score, ACCEL (Parker-Holder et al., 2022) mutates curated levels, and SFL (Rutherford et al., 2024) scores levels by learnability. These methods are compared mostly on mazes, grid worlds, car racing and the 2D BipedalWalker, with fast reference implementations in minimax (Jiang et al., 2023) and JaxUED (Coward et al., 2024). To our knowledge no benchmark runs the UED family under one protocol on 3D legged locomotion, where the student is a contact-rich, high-dimensional controller and a level is a terrain.
We build one and ask which UED findings carry over. Contributions:
- A five-robot benchmark in MuJoCo MJX (Todorov et al., 2012): the closed-kinematic-chain quadruped W01-TEK and four open-chain robots from MuJoCo Playground (Zakka et al., 2025) and Menagerie (Zakka et al., 2022) (Go1, Go2, Barkour vB (Caluwaerts et al., 2023), Spot), on a shared terrain level space.
- Fourteen curricula on one student: DR and two easy-start DR controls, the game-inspired terrain curriculum of Rudin et al. (2021), PLR⟂, SFL, ACCEL with two mutation/score variants and a learning-progress score (Li et al., 2026), and PAIRED with behaviour-cloning and evolutionary variants, all driving one brax PPO (Schulman et al., 2017; Freeman et al., 2021) student at matched environment steps. Five seeds each, 350 runs.
- An evaluation protocol: success on random levels of graded difficulty and on held-out structured terrain, cross-fitted CVaR, aggregate interquartile means and probabilities of improvement (Agarwal et al., 2021), capability envelopes and a level-by-level domination comparison with a binomial noise reference (§4).
- Findings and guide (§5, §6), for one PPO student at matched environment steps, as answers to six questions asked per robot: which methods solve in-distribution random levels, which transfer to held-out terrain, which handle hard levels, which terrain features limit them, what they cost and how repeatable they are. No method is in the top group everywhere. Terrain-CL is in 13 of the 20 robot and level-set top groups, ACCEL in 11 and DR on uniform levels in none. On Go1, Go2 and Barkour the leader on random levels is not the leader on held-out terrain. On Spot the fourteen methods lie within 3.8 points on
d=0.25.
A method interacts with the benchmark only through the levels it proposes, so new curricula plug in without changing the level space or evaluation. The appendix holds the robot, level-space, method and protocol details and all additional tables. Code, level space and evaluation suite are available at https://github.com/Solvro/ml-wojtek-ued under the MIT licence. The W01-TEK model is available at https://github.com/machinekind/w01-tek under the Apache 2.0 licence.
2 Related Work
UED.
PAIRED (Dennis et al., 2020) trains a generator against the return gap of two students. Mediratta et al. (2023) stabilise it with behaviour-cloning distillation between them. PLR⟂ (Jiang et al., 2021a) trains only on replayed high-regret levels, ACCEL (Parker-Holder et al., 2022) adds level edits and reports gains on BipedalWalker, and SFL (Rutherford et al., 2024) shows that common regret estimates track success rate and replaces them with learnability. We keep these algorithms close to their published form and change the domain.
Locomotion curricula.
Terrain curricula for quadrupeds are usually proposed one at a time and compared with a few baselines (Lee et al., 2020; Rudin et al., 2021; Li et al., 2026; Liu et al., 2026; Ovinnikov et al., 2026). The game-inspired curriculum of Rudin et al. (2021), which moves each robot up or down a grid of terrain difficulties by the distance it walked, is the de facto standard in legged-robot training pipelines. LP-ACRL (Li et al., 2026) samples terrain tasks by learning progress, the change in episodic return between consecutive training stages. We include both, adapted to our level space (§3). LLM-written courses (Liang et al., 2024) and authored terrain datasets (Zhang and Yang, 2022) also come with their own baselines. None runs the UED family side by side under a shared level space and evaluation, and none compares robots of different morphology and size on one level space.
3 Benchmark
Robots and task.
W01-TEK is a quadruped built at Wrocław University of Science and Technology, with 12 actuated joints. Its legs close through a passive four-bar linkage (one connect equality constraint per leg). The four open-chain robots use MuJoCo Playground’s joystick reward and observations on the default models of their manufacturers or creators, with feet-only collisions and raised PD gains on Go2 (App. A). Go2, which Playground lacks, runs Go1’s environment on the Menagerie model. Every robot tracks a velocity command (v_{x},v_{y},\omega_{z}) drawn from the forward half of its native command box at a 0.02 s control step. The actor is blind. The critic also sees a 5\times 5 terrain height scan, so value-based level scores can tell terrains apart. A robot falls when its trunk drops below 0.6 of its standing height above the terrain beneath it or tilts past a gravity threshold, the same rule for all five (Barkour also keeps Playground’s joint-limit check). Table 1 lists the per-robot values. App. A gives the full specification.
Success.
Lengths are given in reference units and multiplied by each robot’s size factor s (App. A). An episode succeeds if the robot never falls and ends at least s metres from spawn. Relative to its size, Spot’s top commanded speed is lower than the other robots’, so its episodes are longer (780 steps), which keeps the success distance a similar share of the distance at top speed for all robots (App. A.3).
- W01-TEK
- closed chain (four-bar)
- 300 steps
- MJX
- Go1
- open chain
- 300 steps
- MJX
- Go2
- open chain
- 300 steps
- MJX
- Barkour vB
- open chain
- 300 steps
- MJX
- Spot
- open chain
- 780 steps
- MJX
Level space.
A level is \theta\in[0,1]^{8} plus a noise seed. The eight coordinates map linearly to a radial slope (-20^{\circ} to +18^{\circ}), radial stairs (-8 to +5 cm high, 15-60 cm wide), smooth noise (0-10 cm amplitude, 0.2-1 m wavelength), gap trenches (0-6 cm wide) and piecewise-constant blocks (0-6 cm high, 15-60 cm cells), in reference units (Table 6). Features start at the edge of a flat spawn disc and are radial or seeded fields, so a level does not depend on heading. Each parameter has a flat value: zero for slopes, heights and gaps, mid-range for widths, cell sizes and wavelength, which have no effect at zero height. The level at difficulty d is \theta_{d}=\theta_{\text{flat}}+d\,(\theta-\theta_{\text{flat}}). The ranges are set on W01-TEK so that uniform levels are neither trivial nor mostly unsolvable. Levels are rendered into one shared heightfield: each of an iteration’s 256 levels occupies its own tile, so every lane of 32 environments trains on its own level without recompiling the model (App. B.2).
Student and training loop.
All methods train the same student: brax PPO networks and loss with MuJoCo Playground’s locomotion hyperparameters (MLPs 512\text{-}256\text{-}128, ELU, observation normalisation, 20-step unrolls, Table 7). One iteration resets 32 environments onto each of 256 levels, with a fresh physics-randomisation draw (Tobin et al., 2017) where the robot defines one, and steps them for one episode, with a PPO update after every unroll when the iteration trains. Runs last 300M environment steps for W01-TEK and 1B for the other robots, five seeds per method and robot.
Methods.
Table 2 lists the fourteen curricula. They differ in where new levels come from (uniform, an easy generator that shrinks a uniform level to difficulty d=0.1, a per-level random difficulty, a difficulty row, or a learned generator), whether a buffer of the 4000 highest-scoring levels is replayed by rank and staleness, whether replayed levels are edited, and how levels are scored: positive value loss (mean positive GAE advantage (Jiang et al., 2021b)), learnability p(1-p) with p the lane’s success fraction (Rutherford et al., 2024), learning progress, or PAIRED regret (best antagonist return minus mean protagonist return). DR-easy and DR-mixed are controls: they isolate “train on easy levels” from curation. Terrain-CL adapts Rudin et al. (2021) to the tiled arena: each of the 256 lanes keeps one uniform level (its column) for the whole run and a row r\in\{0,\dots,10\}, starting from r\sim\mathcal{U}\{0..5\}, and trains on the column shrunk to d=r/10. After every iteration a lane moves up a row when at least half of its environments succeed, down when its mean travel is below half the commanded displacement, and to a random row when it passes the top. ACCEL-LP scores ACCEL’s levels by learning progress (Li et al., 2026): a visit’s score is the lane’s mean first-episode return minus the return of the level’s previous visit, signed, and 0 on a level’s first visit. All methods get the same number of environment steps. Non-training scoring iterations count toward it, and PAIRED’s antagonist does not, so PAIRED simulates twice as much (App. C).
- DR (Tobin et al., 2017)
- uniform
- -
- -
- -
- DR-easy
- easy
- -
- -
- -
- DR-mixed
- mixed
- -
- -
- -
- Terrain-CL (Rudin et al., 2021)
- rows
- -
- -
- -
- PLR⟂ (Jiang et al., 2021a)
- uniform
- buffer
- -
- PVL
- SFL† (Rutherford et al., 2024)
- uniform
- buffer
- -
- learnability
- ACCEL (Parker-Holder et al., 2022)
- easy
- buffer
- edit
- PVL
- ACCEL-grow
- easy
- buffer
- edit + grow
- PVL
- ACCEL-SFL
- easy
- buffer
- edit + grow
- learnability
- ACCEL-LP
- easy
- buffer
- edit
- learning progress
- PAIRED (Dennis et al., 2020)
- generator
- -
- -
- PAIRED regret
- PAIRED+UniBC (Mediratta et al., 2023)
- generator
- -
- -
- PAIRED regret
- PAIRED+BiBC (Mediratta et al., 2023)
- generator
- -
- -
- PAIRED regret
- PAIRED+Evo
- easy
- buffer
- edit
- PAIRED regret
4 Evaluation Protocol
Each final policy runs deterministically, without physics randomisation, with commands drawn as in training except that the forward speed v_{x} is high enough to reach the success distance. Every level is rolled out in two independent passes of four episodes.
Level sets.
(i) Random: 2048 uniform genomes at difficulty d\in\{0.25,0.5\}, the same genomes at both, and a second, independent draw of 2048 uniform genomes at full difficulty. (ii) Held-out suite: seven structured terrain families no method trains on (stairs up and down, slopes up and down, trenches, waves, random blocks), each at four strengths with 64 instances. (iii) Flat: no terrain, a check that terrain skills were not bought with basic walking. (iv) Capability envelopes: one feature ramped to the end of its range with the others flat. The envelope is the largest magnitude passed with 50% success (App. D).
Headline difficulty.
For the open-chain robots full-difficulty levels are rarely solved: the best method solves 19% of them on Go1 and at most 5.0% on Go2, Barkour and Spot, against 43% on W01-TEK. We therefore report d=0.25 as the headline set and full-difficulty results in App. E.1.
Cross-fitted CVaR.
Robustness is the mean success over the worst 25% of levels. Ranking and scoring levels on the same episodes biases this low through selection noise (with 2000 levels of true success 0.5 and 4 episodes each, same-pass CVaR25 reads 0.19), so levels are ranked on one pass and scored on the other, in both directions.
Domination comparison.
Mean success hides whether one method is better everywhere or the two win on different levels. For methods A and B, each level’s solve rate is pooled over all seeds and both passes (5\times 8=40 episodes). We report the fraction of levels on which A’s rate is higher, lower or tied, the mean per-level difference with a 95% bootstrap interval, and the mean absolute difference against a binomial noise reference: the mean |A-B| that two identical policies would show from episode sampling alone.
Statistics.
Tables report the mean \pm standard deviation over five seeds. Methods are compared within each robot with two-sided Welch t-tests over the per-seed means. A robot’s top group on a level set is the method with the best mean and every method not significantly below it (p\geq 0.05). Aggregates over robots (App. E) follow Agarwal et al. (2021): each robot’s per-seed scores are min-max normalised over the method means (0 = worst method, 1 = best), and we report the interquartile mean (IQM) over the 25 robot-seed scores with a 95% stratified bootstrap interval (robots as strata), and the probability of improvement P(X>Y), the Mann-Whitney probability that a seed of X beats a seed of Y on the same robot, averaged over robots. We call a difference significant at p<0.05, without multiple-comparison correction, and give p-values for the pairwise comparisons cited in the text.
5 Results
\pm std over 5 seeds. Bold: the robot’s top group on that level set, the best mean and every method not significantly below it (Welch t-test, p\geq 0.05). d{=}0.25: 2048 random levels at difficulty 0.25. OOD suite: held-out structured terrains. CVaR, d=0.5, flat ground and full-difficulty levels are in App. E.d=0.25 (top) and d=0.5 (bottom) during training. Mean over 5 seeds, shaded min-max. PAIRED and its variants dashed.We answer six questions, each per robot. The same levels are not equally hard for every robot (DR solves 84.5% of d=0.25 levels on W01-TEK and 14.9-34.9% on the others), so we compare methods within a robot and do not pool rankings across robots. A method is in a robot’s top group on a level set when its mean is the best or not significantly below the best (Welch t-test, p\geq 0.05, 5 seeds). A method outside the top group is significantly worse than the best at uncorrected p<0.05, while a method inside it is not shown to be equal. Membership also depends on the seed spread: on W01-TEK’s d=0.25 levels PAIRED+BiBC (70.3\pm 26.4\%) is in the top group and DR-easy (90.2\pm 0.7\%) is not. Table 4 (§6) collects the top groups of all level sets. Aggregates over robots are in App. E.
Q1: Which methods solve in-distribution random levels?
On W01-TEK all methods except PAIRED+BiBC solve 80.5-91.2% of d=0.25 levels, and the top group is ACCEL-grow (91.2%), ACCEL, DR-mixed and PAIRED+BiBC. On Go1 it is ACCEL-LP (64.0%), Terrain-CL, DR-easy and DR-mixed, with DR at 34.9% and the PAIRED family at 24.6-33.3%. On Go2 it is Terrain-CL (34.1%), DR-easy, ACCEL-LP, ACCEL and SFL. On Barkour it is Terrain-CL (60.1%), ACCEL-SFL, DR-easy, ACCEL-LP and ACCEL. On Spot the fourteen methods lie within 14.1-17.9%, a spread of 3.8 points, and the top group is PAIRED+BiBC, PAIRED, ACCEL, ACCEL-SFL and PAIRED+Evo. On Go1 and Barkour every member of the top group uses easy or low-difficulty training levels (the easy generator, mixed difficulty or Terrain-CL’s low rows), and on W01-TEK every member except PAIRED+BiBC does. The methods that train on the full uniform range (DR, PLR⟂, SFL) are outside it on all three robots, and so are the learned-generator methods apart from PAIRED+BiBC on W01-TEK. At d=0.5 the top group is a subset of the d=0.25 group on W01-TEK, Go1, Barkour and Spot. On Go2 it widens to eight methods, because seeds that never learn to walk inflate the variance (Q6).
Q2: Which methods transfer to held-out terrain?
On the held-out suite the top group of W01-TEK is ACCEL-grow (59.0%), Terrain-CL, DR-mixed, PAIRED+Evo and ACCEL. On Go1 it is Terrain-CL (45.8%), ACCEL and SFL, on Go2 ACCEL (33.4%), SFL, ACCEL-LP and ACCEL-SFL, on Barkour SFL (45.2%) and Terrain-CL, and on Spot PAIRED+BiBC (23.2%) and PAIRED+Evo. On Go1, Go2 and Barkour the leader on d=0.25 is not the leader on the suite. Methods that tie on random levels can differ widely on held-out terrain: DR-easy and Terrain-CL solve the same share of Go1’s d=0.25 levels (61.6 vs. 62.3%) but differ by 7.2 points on the suite (38.6 vs. 45.8%), and ACCEL-LP and Terrain-CL tie on Barkour’s d=0.25 levels (59.0 vs. 60.1%) and differ by 12.7 points on its suite (30.1 vs. 42.8%). SFL shows the reverse: it is outside the d=0.25 top group on Go1 and Barkour and in the suite’s top group on both.
Q3: Which methods handle hard levels?
Uniform levels at full difficulty separate the methods on W01-TEK and Go1, where the best method, Terrain-CL in both cases, solves 43.5% and 19.1%. On Go2, Barkour and Spot no method exceeds 5.0% (Table 9). Terrain-CL is in the uniform-level top group on four robots: with DR-mixed, ACCEL-grow, PAIRED+BiBC and PAIRED+Evo on W01-TEK, with ACCEL-LP and ACCEL on Go1, with ACCEL on Go2 and alone on Barkour. DR, which trains on exactly these levels, is in the top group on no robot. On the hardest quarter of the d=0.25 levels (CVaR25) the top group is the mean’s top group or a subset of it on every robot except Spot, so on these robots the methods that solve more levels do not do so by giving up their hardest ones (Table 8).
Q4: Which terrain features limit each method?
Capability envelopes (Table 11, reference units) show where each method stops. On W01-TEK all methods except PAIRED+BiBC reach the end of the range on rough ground, downhill slopes and downward stairs, and stair-up height separates them (5.0 cm for DR-mixed, Terrain-CL, ACCEL, ACCEL-grow, ACCEL-SFL and PAIRED+Evo, 4.0 cm for DR). On Go1 stair-up height (Terrain-CL 4.1, ACCEL 3.9, SFL 3.6 cm) and gaps (at most 2.7 cm, ACCEL-LP) bind. On Barkour SFL alone reaches the best block height (4.0 cm) and shares the stair-up top group with PAIRED+Evo. On Go2 and Spot no method passes any gap. Spot’s uphill-slope envelope is at most 5.2^{\circ}.
Q5: How fast do methods learn, and at what cost?
On W01-TEK every method but PAIRED+BiBC plateaus within about 100M steps (Figure 2). Go2 and Spot plateau early at low success. On Go1 and Barkour the curves are still rising at 1B steps, so their rankings hold at this budget rather than at convergence. At matched environment steps every method outside the PAIRED family takes about the same wall clock on a given robot (0.5-0.9 GPU-hours per run, Table 12). Buffer methods spend a fifth to a quarter of their steps on scoring iterations that do not update the student (786M student steps of 1B for ACCEL on Go1). PAIRED and its variants simulate twice as many steps, for the antagonist, and take 1.8-1.9 times the wall clock.
Q6: How repeatable are the results across seeds?
Runs that never learn to walk concentrate on Go2, where standard deviations reach 12.7 points (ACCEL-LP, d=0.25) and the top groups widen. Outside Go2 the largest standard deviations on d=0.25 and the suite, apart from PAIRED+BiBC on W01-TEK, are SFL’s on Go1 (8.2 and 4.4), ACCEL’s on Barkour’s d=0.25 levels (7.2) and PAIRED+Evo’s on Barkour’s suite (7.8). Terrain-CL stays at or below 3.1 points outside Go2, and DR-mixed at or below 2.8 on every robot.
6 Which Curriculum Should I Use?
\bullet: the best mean or not significantly below it, Welch t-test, p\geq 0.05, 5 seeds). .25, .5: random levels at d=0.25, 0.5. OOD: held-out suite. U: uniform levels at full difficulty. On Spot all methods lie within 3.8 points on d=0.25.Table 4 marks, per robot and level set, the methods in the top group. We summarise it per family of methods. Every statement holds for one PPO student at this budget.
Randomisation (DR, DR-easy, DR-mixed).
DR on uniform levels is in no top group. Changing only the distribution it samples from changes this. DR-easy is in the d=0.25 top group on Go1, Go2 and Barkour and in no held-out top group. DR-mixed is in every top group on W01-TEK and in both random-level groups on Go1. Both cost the same as DR and need no buffer or score.
Terrain curriculum (Terrain-CL).
Terrain-CL is in more top groups than any other method (13 of 20): all four on Go1 and Barkour, three on Go2, the suite and uniform levels on W01-TEK, and none on Spot. It needs the task’s success test and legged_gym’s travel threshold, and no buffer or score. Its rows need no robot-specific setting.
Replay of uniform levels (PLR⟂, SFL).
PLR⟂ is in no top group, but on Go1 and Barkour it improves on DR, which samples the same levels (45.9 vs. 34.9%, p=0.002, and 33.5 vs. 20.9%, p=0.003, at d=0.25). On W01-TEK, Go2 and Spot it does not. SFL, the same buffer scored by learnability, is in the held-out top group on Go1, Go2 and Barkour, leads it on Barkour, and has a wide seed spread on Go1 and Go2 (std up to 12.2 points on d=0.25 and the suite).
ACCEL and its variants.
ACCEL is in the held-out top group on W01-TEK, Go1 and Go2, leads it on Go2, and in the d=0.25 group on every robot except Go1, eleven groups in total. ACCEL-grow, which also enlarges edited levels, leads W01-TEK’s d=0.25, d=0.5 and held-out sets and enters no other robot’s group except Go2’s at d=0.5. ACCEL-LP leads Go1’s random levels and is in Go2’s held-out group, but in no other held-out group. ACCEL-SFL is in Barkour’s random-level groups, Go2’s held-out group and Spot’s d=0.25 group, and is below ACCEL on W01-TEK (83.8 vs. 90.5%, p=0.002).
PAIRED and its variants.
PAIRED is in no top group outside Spot’s d=0.25 set, and below PLR⟂ on W01-TEK, Go1, Go2 and Barkour (on Go1 28.9 vs. 45.9%, p<0.001). Distilling the antagonist into the protagonist (UniBC) changes PAIRED’s d=0.25 success by at most 1.3 points on any robot. Distilling both ways (BiBC) collapses on Go2 (0.2% success) and leads every Spot set. PAIRED+Evo, which replaces the learned generator with ACCEL’s buffer and edits under the same regret, is in the held-out top group on W01-TEK and Spot. All four cost about twice the wall clock of the other methods.
Short answers.
For in-distribution random levels, a method with easy or low-difficulty training levels is in the d=0.25 top group on every robot, whether it curates (ACCEL family, Terrain-CL) or not (DR-easy, DR-mixed). For held-out terrain the choice depends on the robot: Terrain-CL or ACCEL on Go1, ACCEL on Go2, SFL or Terrain-CL on Barkour, PAIRED+BiBC or PAIRED+Evo on Spot, and ACCEL or ACCEL-grow, Terrain-CL, DR-mixed or PAIRED+Evo on W01-TEK. Without a budget to compare methods on a new robot, Terrain-CL is the method found in a top group most often here, and ACCEL is second. Evaluating only on in-distribution random levels would have picked a different leader on Go1, Go2 and Barkour.
7 Discussion and Limitations
Relation to 2D UED results.
On BipedalWalker ACCEL outperforms PLR⟂, PAIRED and DR (Parker-Holder et al., 2022). Here ACCEL is in eleven of twenty top groups, and Terrain-CL, a curriculum from legged-robot practice, is in thirteen. JaxUED and SFL report DR as a strong baseline in 2D domains (Coward et al., 2024; Rutherford et al., 2024). Here DR on uniform levels is in no top group, while DR on easy or mixed-difficulty levels is in the random-level top groups of four robots. PAIRED trails replay-based methods in 2D (Parker-Holder et al., 2022; Jiang et al., 2021a) and here, and the behaviour-cloning fixes proposed for it (Mediratta et al., 2023) do not change that outside Spot.
Benchmark statement.
The benchmark fixes one training budget per robot, shared by all methods. Training can be extended, and longer runs may improve the policies. On three robots the leader on in-distribution random levels is not the leader on held-out terrain, so a new curriculum should be reported on both.
Limitations.
- A robot’s structure and build affect the share of levels it solves, whatever the method. We therefore compare methods within each robot, and the aggregates over robots (App. E) average scores normalised per robot rather than raw success.
- Terrain-CL and ACCEL-LP are adaptations: Terrain-CL’s columns are uniform levels of the level space rather than legged_gym’s terrain types, and promotion is decided per lane of 32 environments. ACCEL-LP ranks a replay buffer by learning progress measured over one visit, instead of LP-ACRL’s softmax over a fixed task set.
Conclusion.
Across fourteen curricula and five quadrupeds no method is in the top group on every robot and level set. Methods with easy or low-difficulty training levels, curated or not, fill the random-level top groups of Go1 and Barkour, and of W01-TEK apart from PAIRED+BiBC. The success-driven terrain curriculum is in a top group most often and ACCEL second, while DR on uniform levels is in none and PAIRED in none outside Spot. The leader on held-out terrain differs from the leader on in-distribution random levels on three robots. The benchmark, level space and protocol are built so that a new curriculum is added by changing only how levels are proposed.
Acknowledgments and Disclosure of Funding
Calculations have been carried out in Wrocław Centre for Networking and Supercomputing (http://www.wcss.pl), grant No. hpc-danbor2008-1756464546. W01-TEK was built in the Laboratory of Mechatronics and Robotics, Wrocław University of Science and Technology, under the scientific supervision of Jarosław Szrek. We thank Machinekind for providing its robot model.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here. The authors hope that this benchmark and its per-robot guide will support the development of Physical AI and of learning-based robotics, by making curricula for legged robots easier to compare, reproduce and choose.
References
Appendix A Robots and Task
A.1 W01-TEK
W01-TEK is a quadruped built in the Laboratory of Mechatronics and Robotics, Wrocław University of Science and Technology. Machinekind provided the robot model we simulate (https://github.com/machinekind/w01-tek). It has 12 actuated joints (three per leg). Each leg is a closed kinematic chain: the actuated joints drive the foot through a passive four-bar linkage, closed in MuJoCo by one connect equality constraint per leg. The closure adds constraint dynamics that serial legs do not have (the linkage can reach a toggle configuration, which the reward penalises). Training randomises terrain friction, base and link masses, and actuator gain and damping scale per environment. W01-TEK runs on the MuJoCo Warp backend of MJX. The four Playground robots run on the JAX backend.
A.2 Open-chain robots
Go1, Barkour vB and Spot use MuJoCo Playground’s joystick environments (Zakka et al., 2025): Playground’s reward and observations, feet-only collisions and, where Playground defines one, its physics randomisation. Go2 has no Playground environment. It runs Go1’s environment on the Menagerie model (Zakka et al., 2022), rewritten to feet-only collisions and the sensor names Go1’s environment reads. Every robot keeps its own reward. We change the scene (the tiled heightfield arena), the commands (forward-walking slice of the native command box), the termination (the shared fall rule below) and the critic input (a height scan). Table 5 lists the per-robot values.
- Model
- Machinekind
- Playground
- Menagerie
- Menagerie
- Playground
- Episode (steps)
- 300
- 300
- 300
- 300
- 780
- Physics DR
- own
- Playground
- Go1’s
- none
- none
Models.
Every robot keeps the default actuator and joint settings of the model released by its manufacturer or creators (the MJCF/URDF files of MuJoCo Menagerie and MuJoCo Playground, and Machinekind’s model of W01-TEK). Only Go2’s PD gains are raised (position gain doubled, velocity gain and joint damping scaled by \sqrt{2}, keeping Menagerie’s damping ratio).
Size.
s is set per robot from its settled standing pose over flat ground, relative to a fixed reference, and rounded to 0.1. It scales the arena, every level length, the eval suite, the critic’s height-scan grid and the success distance. Slopes and the fall rule’s ratio do not scale.
Fall rule.
An episode ends in a fall when the trunk is below 0.6 of the standing height above the terrain under it (read from the heightfield) or the gravity vector’s z-component in the trunk frame exceeds -0.4. This replaces Playground’s own terminations, which use world heights (wrong on a pit or a raised tile). Barkour keeps Playground’s joint-limit check.
Critic.
The critic additionally receives a clean 5\times 5 terrain height scan ahead of the robot (a fixed grid scaled by s). The actor is blind.
A.3 Success distance and episode length
Success needs s metres of travel within the episode. With 300-step (6 s) episodes, this is 28-30% of the distance at top commanded speed for W01-TEK, Go1, Go2 and Barkour. Relative to its size, Spot’s top commanded speed is lower than the other robots’, so in 6 s its success distance would need 72% of its top speed from rest. With 6 s episodes even a perfect flat walker would succeed on about half of the levels. We lengthen Spot’s episodes to 780 steps (15.6 s), which restores the 28% ratio and keeps the success distance a size-relative quantity. Evaluation draws v_{x} from 1.2\times the success distance over the episode duration up to the maximum, so every evaluation command can reach the distance. The distance is then 83% of the travel at the slowest evaluation command for all five robots. Spot redraws its command every 200 steps, Go1 and Go2 every 5 s on average, and Barkour keeps one command per episode. With turning, perfect tracking of evaluation commands reaches the distance on 62% of Spot’s levels, 70% of Go1’s and Go2’s and 95% of Barkour’s, which approximately caps flat-ground success (Table 8).
Appendix B Level Space and Arena
B.1 Level space
- Noise amplitude
- 0-0.10 m
- Noise wavelength
- 0.2-1.0 m
- Radial slope
- -20^{\circ} to +18^{\circ}
- Stair height
- -0.08 to +0.05 m
- Stair width
- 0.15-0.60 m
- Gap width
- 0-0.06 m
- Block height
- 0-0.06 m
- Block size
- 0.15-0.60 m
Each parameter has a flat value (zero slope, step, amplitude, gap and block height, and mid-range stair width, noise wavelength and block size, which have no effect at zero height). Because the slope and stair ranges are asymmetric, flat is not the centre of the cube. Difficulty d moves a genome toward flat, \theta_{d}=\theta_{\text{flat}}+d(\theta-\theta_{\text{flat}}). The easy generator of DR-easy and ACCEL is d=0.1.
B.2 Rendering and the tiled arena
A level is rendered into a square patch of half-width 3.0 m at 0.05 m resolution with heights clipped to \pm 0.3 m (all \times s). A flat spawn disc of radius r_{0}=0.55 m blends into the terrain over w=0.2 m. Let r be the distance from the patch centre, \delta=\max(r-r_{0}-w,0) and \tau(r)=\mathrm{clip}((r-r_{0})/w,0,1). Then
h(x,y)=\tau(r)\Big[\tan(\phi)\,\delta+h_{s}\Big\lceil\frac{\delta}{w_{s}}\Big\rceil+A\,n_{\lambda}(x,y)+h_{b}\,b_{c}(x,y)-D\,\mathbf{1}\big[\delta>0,\ (\delta\bmod P)<g\big]\Big],where \phi is the slope, h_{s},w_{s} stair height and width, A,\lambda noise amplitude and wavelength, n_{\lambda} a seeded bilinear noise field in [-1,1] with control points every \lambda, b_{c} a seeded piecewise-constant field in [-1,1] with cells of size c and height h_{b}, and g the gap width with depth D=0.15 m and period P=0.6 m.
All levels of an iteration share one MuJoCo heightfield: the arena is a grid of 16\times 16 tiles separated by flat aprons, each level is written into its own tile and its environments spawn at the tile’s centre. The heightfield data is an array leaf of the MJX model, so writing 256 new levels needs no recompilation, and all environments share one model. Figure 3 shows each feature at the end of its range and Figure 4 the five robots on one level.
d=0.5 and d=1. Features start at the edge of the flat spawn disc.d=0.6) rendered for each robot in its own arena. Terrain lengths and camera distance scale with the robot’s size s, so robots of different proportions fill their panels differently.Appendix C Methods and Hyperparameters
Training loop.
Brax’s own training loop fixes each environment’s model for a whole run, so we drive brax’s PPO networks, loss and running observation normaliser from a custom loop. One iteration runs 256 lanes \times 32 environments. All environments of a lane are reset onto its level, with a fresh physics-randomisation draw where the robot defines one, and stepped for one episode in 20-step unrolls, with a PPO update after each unroll when the iteration trains. An iteration is 256\cdot 32\cdot 300\approx 2.46M environment steps (Spot: 6.4M).
Scores.
Positive value loss (PVL): the mean positive GAE advantage of a lane over the iteration (Jiang et al., 2021b). Learnability: p(1-p) with p the fraction of the lane’s environments that never fell and reached the success distance (Rutherford et al., 2024). Learning progress (after LP-ACRL (Li et al., 2026)): R is the lane’s mean over its 32 environments of the undiscounted return of each environment’s first episode on the level (up to 300 steps, Spot 780, steps after a fall and reset still train PPO but do not count toward R). A visit’s score is R minus the R of the level’s previous visit, signed, and 0 on a level’s first visit, including a mutated child’s. PAIRED regret: the best antagonist return minus the mean protagonist return on the lane, clipped at 0.
DR, DR-easy, DR-mixed.
Every iteration trains on 256 fresh levels: uniform (DR), uniform shrunk to d=0.1 (DR-easy, the same easy generator as ACCEL), or shrunk by a per-level d\sim\mathcal{U}(0,1) (DR-mixed).
Terrain-CL.
The game-inspired curriculum of Rudin et al. (2021), adapted to lanes: all 32 environments of a lane share one level, so promotion is decided per lane. At the start of the run each lane draws a uniform genome and noise seed (its column), kept for the whole run, and a row r\sim\mathcal{U}\{0,\dots,5\} (legged_gym’s initial-level cap). Every iteration trains on all lanes, each at its column shrunk toward flat by d=r/10, the same shrink as the random evaluation sets. After the iteration a lane moves to r+1 when at least 50% of its environments succeeded. Otherwise to r-1 (clipped at 0) when its mean distance from spawn is below half its mean commanded planar displacement, integrated from the commanded velocities and heading rate over the episode (legged_gym: distance below half the commanded speed times the episode length). A lane promoted past row 10 moves to a uniformly random row.
PLR⟂ (Jiang et al., 2021a) and SFL.
A buffer of the 4000 highest-scoring levels. Once it holds 256 levels, an iteration replays with probability 0.8: it samples 256 levels by rank prioritisation (temperature 0.3) mixed with staleness (weight 0.3), trains on them and refreshes their scores. Otherwise it scores 256 fresh uniform levels without training and adds them to the buffer. PLR⟂ scores by PVL. Our SFL scores by learnability in the same buffer. This is an approximation of SFL (Rutherford et al., 2024), which periodically rebuilds a top-K buffer from a large random pool and trains on a mix of buffer and random levels.
ACCEL (Parker-Holder et al., 2022) and variants.
PLR⟂ with two changes: new levels come from the easy generator, and every replayed level is mutated by perturbing 2 randomly chosen parameters by \mathcal{U}(-0.1,0.1) (clipped to [0,1], seed kept). The next non-training iteration scores the children of the most recent replay batch. A non-training iteration with no queued children scores fresh easy levels. ACCEL-grow adds a “grow” step that scales the mutated genome’s distance from flat by \mathcal{U}(1,1.3). ACCEL-SFL is ACCEL-grow scored by learnability. ACCEL-LP is ACCEL scored by learning progress. With signed scores, levels whose return fell since their last visit rank last and are the first evicted from a full buffer. LP-ACRL samples a fixed task set by a softmax over learning progress. ACCEL-LP keeps ACCEL’s rank and staleness sampling over an evolving buffer.
PAIRED (Dennis et al., 2020) and variants.
A learned generator maps a 16-dimensional Gaussian latent through an MLP to a sigmoid-squashed Gaussian over the 8 parameters. A random seed is appended. Each iteration it proposes 256 levels, on which a protagonist and an antagonist (two independent students, each with its own normaliser) both train in separate rollouts. The generator is trained on PAIRED regret by single-step PPO (entropy coefficient 0.1, learning rate 10^{-4}). UniBC and BiBC (Mediratta et al., 2023) add a behaviour-cloning loss (weight 0.5, every iteration) that distils the antagonist into the protagonist, or both ways. PAIRED+Evo replaces the generator with ACCEL’s buffer and edits, scored by PAIRED regret.
Budget accounting.
All methods get the same number of environment steps. Non-training iterations of the buffer methods count toward the budget, so they take fewer gradient steps than DR. PAIRED’s budget counts protagonist steps only, so PAIRED simulates twice as much (Table 12).
- Steps per level (episode)
- 300 (Spot 780)
- Environment steps
- 300M (W01-TEK), 1B (others)
- PPO
- brax, 20-step unroll, 4 epochs \times 32 minibatches
- \gamma, \lambda, learning rate, entropy, clip, grad norm
- 0.97, 0.95, 3\times 10^{-4}, 0.01, 0.2, 1.0
- Networks
- MLP 512-256-128, ELU, observation normalisation
- Buffer
- 4000 levels, replay probability 0.8
- Replay sampling
- rank (temperature 0.3) + staleness (0.3)
- Easy generator
- shrink to d=0.1
- Terrain-CL
- rows d=r/10, r\in\{0..10\}, r_{0}\sim\mathcal{U}\{0..5\}, up at \geq50% success,
- —
- down at travel <0.5\times commanded displacement
- Mutation
- 2 parameters, \pm 0.1, grow \times\,\mathcal{U}(1,1.3)
- PAIRED generator
- latent 16, lr 10^{-4}, entropy 0.1
- Behaviour cloning
- weight 0.5, every iteration
Appendix D Evaluation Protocol
Episodes.
Each final policy is evaluated deterministically (tanh of the policy mean), without physics randomisation and without early episode cuts. Commands are drawn as in training, with v_{x} restricted to the evaluation range (App. A.3). Every level is rolled out in two independent passes of 4 episodes each.
Level sets.
Random: 2048 uniform genomes, shrunk to d\in\{0.25,0.5\}. The genomes are the same at both. Uniform: a second, independent draw of 2048 uniform genomes, the training distribution of DR. Flat: 64 flat levels. Held-out suite: seven structured terrain families, each at strength k=1..4 with 64 instances, radially symmetric like the training levels (reference units, \times s): stairs up and down (2k cm steps every 0.3 m), slopes up and down (5k^{\circ}), trenches (10 cm deep, 5k cm wide, every 0.5 m), waves (\pm 2k cm, 0.6 m wavelength) and random blocks (\pm 2k cm, 0.3 m cells). We report mean success over the 28 cells.
Cross-fitted CVaR.
CVaR25 is the mean success over the worst 25% of levels, after the adversarial evaluation of Rutherford et al. (2024). Levels are ranked on one pass and scored on the other, in both directions, and averaged. In a simulation with 2000 levels of true success 0.5 and 4 episodes each, same-pass CVaR25 reads 0.19 and the cross-fitted estimate 0.49.
Solvable subset.
Uniform success restricted to levels that some policy passes: a level is solvable if at least one run of any method on that robot succeeds in at least one episode of pass A. Every run is scored on pass B over those levels and vice versa, and the two are averaged. The subset depends on the set of evaluated runs, so the metric is comparable only within one robot.
Capability envelopes.
For each of seven features (rough, slope up, slope down, stairs up, stairs down, gaps, blocks), the feature takes six evenly spaced magnitudes up to the end of its range with every other feature flat, 64 seeded levels per magnitude. The envelope is the largest magnitude still passed with 50% success (linearly interpolated, 0 if the smallest magnitude already fails). An envelope at the end of the range is censored.
Domination comparison.
For methods A and B, a level’s solve rate is the fraction of its 40 episodes (5 seeds \times 2 passes \times 4) that succeed. The heatmap counts levels in a 10\times 10 grid of (A, B) solve rates. We report the fractions of levels where A’s rate is higher, lower or equal, the mean difference A-B with a 95% bootstrap interval over levels, and the mean |A-B| against a noise reference: per level, both rates are drawn as \mathrm{Binomial}(n,p)/n at p=(A+B)/2, and the reference is the exact expectation of their absolute difference, averaged over levels. A mean |A-B| at the reference means the two methods are indistinguishable level by level.
Statistics.
Per-seed metrics are averaged over five seeds. Tables give the mean \pm sample standard deviation, and per-robot method comparisons use two-sided Welch t-tests over the per-seed means, uncorrected for multiple comparisons. Aggregates over robots follow Agarwal et al. (2021): per robot, scores are min-max normalised over the method means (so the bounds depend on the set of methods). The IQM is the 25%-trimmed mean of the 25 robot-seed scores. Intervals are 95% percentile stratified bootstrap intervals with robots as strata (10 000 resamples). The probability of improvement P(X>Y) is, per robot, the Mann-Whitney statistic U/(nm) of X’s against Y’s seeds (ties count one half), averaged over robots, with a stratified bootstrap interval (2000 resamples).
Appendix E Additional Results
\pm std over 5 seeds. Bold: top group per robot and column (best mean or not significantly below it, Welch t-test, p\geq 0.05). CVaR25 is over the d=0.25 levels, cross-fitted.E.1 Full-difficulty (uniform) levels
\pm std over 5 seeds. Bold: top group per robot and column (Welch t-test, p\geq 0.05).E.2 All method pairs
The domination comparison (§4) shows whether one method is better on most levels or the two win on different levels. Figure 6 gives it for ACCEL and DR on d=0.25 as an example (Figure 5 on uniform levels), and Figure 7 for every pair.
d=0.25 levels. Each level’s solve rate is pooled over 5 seeds \times 8 episodes. The colour is the fraction of levels (log scale) per cell. Mass below the diagonal: levels ACCEL solves more often. Panel titles give the fractions of levels ACCEL wins, loses and ties, the mean difference with its 95% bootstrap interval, and the mean absolute difference against the binomial noise reference.d=0.25 levels on which the row method’s solve rate exceeds the column method’s, per robot. Darker cells: the row method wins on more levels. The diagonal is blank, and 1-M_{ab}-M_{ba} is the fraction of ties.E.3 Aggregates over robots
The main text compares methods within each robot. For completeness, Figure 8, Table 10 and Figure 9 pool the five robots after min-max normalisation (§4). By mean rank over robots, ACCEL and Terrain-CL are first on d=0.25 (both 4.0) and on the held-out suite (3.0 and 3.4), and DR is last on the suite (12.2). By IQM (Table 10), ACCEL-LP (0.94), DR-easy and Terrain-CL (both 0.92) are first on d=0.25, and Terrain-CL (0.89), ACCEL (0.87) and PAIRED+Evo (0.86) on the suite, with overlapping intervals. The method rankings on d=0.25 agree across W01-TEK, Go1, Go2 and Barkour (Spearman \rho=0.53-0.86, p<0.05 for all pairs but W01-TEK-Barkour, p=0.05). Spot’s ranking is unrelated to or inverted against the others (\rho=-0.61 to -0.28).
\times five seeds: success min-max normalised per robot over the method means, interquartile mean and mean with 95% stratified bootstrap intervals.\times 5 seeds with 95% stratified bootstrap intervals (App. D).P(X>Y) for the named method pairs, with 95% stratified bootstrap intervals, on d=0.25 and the held-out suite.E.4 Capability envelopes
\pm std over 5 seeds. Bold: top group per robot and column (Welch t-test, p\geq 0.05). In reference units (lengths are multiplied by s, slopes are not). An envelope at the end of the range (rough 10 cm, slope -20^{\circ} to +18^{\circ}, stairs -8 to +5 cm, gaps 6 cm, blocks 6 cm) is censored.On W01-TEK every method except PAIRED+BiBC reaches the end of the range for rough ground, downhill slope and downward stairs. Stair-up height is the binding feature (DR 4.0, ACCEL 5.0 of 5 cm). The open-chain robots’ gap envelopes are at most 2.7 of 6 cm, and 0 for 42 of the 56 robot-method pairs. Spot’s uphill-slope envelope is at most 5.2^{\circ}.
E.5 Curriculum composition
Figure 10 shows the terrain each curriculum trains on over the run.
Appendix F Compute
\pm std over seeds. GPU-h: wall clock on one GPU. Sim steps count every simulated environment step (PAIRED’s antagonist, the non-training iterations of the buffer methods). Student steps are those that fed a student PPO update.Every run used one NVIDIA H100. The 350 training runs took 306 GPU-hours (W01-TEK 40, Go1 63, Go2 77, Barkour 66, Spot 60). A W01-TEK run (300M steps) takes 0.6 hours on average. A run of the other robots (1B steps, JAX backend) takes 0.9-1.1 hours. PAIRED and its variants take 1.8-1.9 times the wall clock of the other methods.