A GO2 policy carried from Isaac Lab to the real robot, through gates

Most open-source quadruped RL stops at "trained in sim, deployed once." The deployment either works on video or it does not, and either way the failures that happen on hardware never make it back into the training distribution. The gap between a policy that scores well in simulation and one that is trustworthy on a robot is exactly where the interesting engineering lives.

Domain randomization is applied up front, blind, and generously, in the hope of covering whatever reality does. That is expensive and it is not targeted: the sweep is not informed by any failure the robot actually had. Meanwhile the export path from PyTorch to the runtime on the robot is usually taken on faith, so a numerical discrepancy introduced at export is indistinguishable from a policy that simply did not transfer.

Gate every stage, and make the hardware failure a training input rather than a postmortem. Phoenix is built to record real failures as trajectories, replay them in simulation under a randomized physics sweep seeded from that specific failure, fine-tune on the resulting distribution and send the policy back to the robot. That loop is wired and unit-tested; it has not yet run on real captured data.

Training

PPO through rsl_rl on a layered-YAML Isaac Lab environment, driven by roughly ten shell entry points.

Export gate

ONNX export with a torch / onnxruntime parity check that fails the build rather than warning.

Runtime

A ROS 2 policy node on the Jetson behind three bridges and one shared slew cap, all fail-closed.

Capture

Rule-based attitude, collapse, and slip detection writing Parquet trajectory logs.

Replay

Failure-seeded curriculum fine-tune, wired and unit-tested, awaiting real hardware parquets.

The whole loop: the Isaac Lab environment layering, the training entry points, the parity-gated export, the ROS 2 deploy stack and its safety layer, the failure detector and logging format, and the replay and fine-tune path. The evidence ledger that classifies each claim as verified, inferred, or not-validated is also mine, and it is what keeps this page honest.

◆Verified

Simulation training and the layered environment: complete, with unit tests across the stages.

◆Verified

stand-v3-h25 sim evaluation: 32 of 32 successful episodes, nominal and under full actuator randomization, on the standing task.

◆Measured

ONNX / torch parity gate: maximum absolute difference 4.77e-06 on the stand-v3-h25 export (2026-06-20), against a 1e-4 tolerance. Re-verification is owed: a 2026-05-21 audit found earlier exports dropped observation normalization, and a parity check cannot catch a wrapper error that both sides share.

◆Verified

The deploy stack ran end to end on the physical GO2 from the Jetson (April 2026); no stand passed. 236 unit tests pass in CI.

◆Verified

Live stand on the physical GO2 (2026-10-06). A newer walking policy, commanded to stand still, passed every stage gate: after a scripted stand-up ramp it held the robot for 2 s, 5 s and three 10 s windows, each ended by its timer, with body tilt under 0.09 rad and none of its targets altered by the safety layer. Every live start was from the folded pose on the floor, and no velocity command was sent.

◆Verified

First walking commands on the physical GO2, on a free floor (October 2026). The policy walks but under-tracks: 0.71 m measured for a 0.3 m/s, 4 s command (about 60 %) and 17-19 deg for a 0.5 rad/s, 3 s turn (about 20 %), against 89 % and 76 % for the same profiles replayed in MuJoCo.

◆Measured

Joint static friction on the hung robot with an active breakaway probe, validated in MuJoCo: hips 0.24-0.35, thighs 0.17-0.34, calves 0.42-0.96 N m. The training simulator modelled none; replaying the walk with these values gives 66 % forward and 34-47 % turning, so friction explains most of the gap.

◆Verified

On a free floor (October 2026): the learned policy maintained standing, executed forward walking, and demonstrated limited turning capability. A recorded turn achieved 17° of 86° commanded. Getting up from lying flat and sitting down are scripted, not learned; the policy was trained from near the standing pose and does not stand the robot up.

◇Open

The stand-v3-h25 policy itself has not been re-run on hardware. In its last live run (2026-04-21) about 33% of steps hit the slew limit at zero command; which of three train/deploy mismatches caused it is pending an ablation.

◆Verified in sim

Friction-aware retrain (October 2026): two walking policies retrained in Isaac Lab with a joint-friction actuator model, one randomized around the measured friction values and one over a broad 0 to 1.6 N m range. Both finished the command curriculum and pass the sim-to-sim export gate and parity check.

◇Open

Walking that tracks the command on the robot. A first preregistered hardware comparison of the retrained policies against the current one ran in October 2026 on one forward command; results will be reported after turns, faster commands and a second floor surface are tested.

The slew saturation is the point of the whole project. A policy with a clean simulation record hit a per-step rate limit as soon as it ran on real actuators, and the cause is still being isolated.

The current evidence points away from the robot and to three mismatches between training and deployment: a per-step rate limiter that exists only at deploy time, hip joints set to 0.0 in the deploy config where training used plus or minus 0.1, and a velocity input the policy node fed as zeros. An earlier single-cause explanation (an actuator-gain randomization term) is superseded, and which of the three dominates is pending an ablation. Separately, a custom gain startup term and a per-motor slew hinge penalty brought the stand policy to 32 of 32 simulation episodes under full actuator randomization.

A related audit finding: the parity gate compares ONNX against TorchScript exports of the same wrapper, so it verifies numeric parity but cannot catch a wrong wrapper. It did not: the exporter was searching for normalizer keys that rsl_rl 3.x never writes, so every pre-2026-05-21 export silently dropped observation normalization. A green gate was not the same as a correct export, and finding that is exactly what the ledger exists for.

October 2026, the first steps: the walking policy passed every live stand gate, then walked short of its command, about 60 % of the distance and 20 % of the turn. Rather than guess, I replayed the exact hardware commands in MuJoCo and swept candidate causes: floor friction and latency did not reproduce the gap, joint friction did. I then built a breakaway probe that ramps one joint at a time on the hung robot and reads friction from the torque at which it starts to move, validated it in simulation, and measured 0.2-1.0 N m per joint. The training simulator had modelled none.

The pipeline is YAML-configured with shell entry points per stage, and the parity gate is part of the deploy verification rather than an afterthought. The parity check (phoenix.sim2real.verify_deploy) checks each stage in order:

Export

torch → onnx: ok

Parity

max abs drift, tolerance 1e-4: 4.77e-06

Sim

stand-v3-h25 evaluation: 32 / 32 success

Hardware

stand-v3-h25 live stand, awaiting re-verified export (a newer walking policy held the robot standing on 2026-10-06): OPEN

Walk the friction-aware policies on the robot. They are retrained in simulation; a first preregistered A/B/C comparison against the current policy, on the same free floor with every walk taped, ran in October 2026 on one forward command. Turns, faster commands and a second floor surface come before its results are reported. In parallel, rank the three April train/deploy mismatches with the ablation, and feed real captured parquets into the replay curriculum so the loop closes on real data for the first time.