The reliability layer

A classical system fails legibly. A joint faults, a solver diverges, a check fires, and something in the stack knows. A learned policy does not have that structure. It maps observations to actions, and when the observations leave the region it was trained on it keeps mapping: the outputs stay numerically plausible, the rates stay nominal, and the confidence it exports does not move.

I measured this on a driver-assistance model used on public roads. Under simulated input shift its perception collapsed almost completely, and none of its exported uncertainty channels moved out of the nominal range, so a monitor reading them would have seen nothing. The information needed to catch it was present the whole time, one layer inside the network, and simply never surfaced.

So the interesting problem is not making policies better. It is building the layer around them that knows when to stop trusting them, and that can prove afterwards what it knew and when.

A learned policy is competent until the world stops matching its training set. Nothing announces that moment: the model stays confident and every consumer keeps trusting it. Three versions of it, each from something I measured rather than from a survey.

1. 01

   It fails silently

   openpilot's shipped driving model loses 8 of 10 output readouts under simulated input, and its own exported uncertainty never leaves the nominal range. Nothing downstream could tell.

2. 02

   It fails off-camera

   A GO2 locomotion policy that passed its simulation evaluation hit a per-step slew saturation when it ran on real actuators. The current evidence points to mismatches between how the policy was trained and how it was deployed; which one dominates is still being isolated.

3. 03

   It fails without a record

   When a run does collapse, the interesting moment is usually seconds before anything visible happened, and by then nobody kept the state needed to find it.

Each of my projects sits at one stage of the same chain. Read end to end, they are an attempt to build it; read individually, they are the stage each one happens to occupy.

1. Policy

   Train something competent, and know exactly how competent.

   A locomotion policy trained in Isaac Lab and carried to a physical GO2 through a parity-gated export. The point is not the policy; it is that every stage between the training run and the robot is gated, so a failure on hardware can be attributed to the policy rather than to the pipeline.

   go2-phoenix

2. Distribution shift

   Establish what actually happens when the input leaves the training distribution.

   Not a benchmark: a shipped checkpoint, reproduced frame-exactly first, then driven with a continuous real-to-simulated blend. On one route the collapse is a sharp discontinuity, on another a gradual slide, which is why averaged metrics miss it.

   supercombo-blindspot

3. Detection

   Notice it from inside the model, because the outside says nothing.

   This is the thread with the strongest evidence and the one I would build a dissertation on. A model's exported uncertainty stayed flat through total perceptual collapse; the same failure was legible in its recurrent state. Detection has to read internals, and it has to report its own coverage bound.

   supercombo-blindspot · policy-health-monitor

4. Arbitration

   Decide whether the policy is still allowed to drive.

   Detection is worth nothing without a decision attached. HELIX turns raw symptoms into structured FaultEvents, diagnoses a likely cause, and puts a cooldown-aware auditor between the diagnosis and any command that reaches the robot.

   HELIX

5. Recovery

   Physically do something about it. This is where my work currently stops.

   HELIX publishes audited zero-velocity commands, and its evidence records exactly what it published and when. In the audited session that stream had no downstream subscriber, so physical recovery on hardware is the next gate.

   HELIX

6. Evidence

   Be able to prove afterwards what was true.

   The main repositories carry a ledger that classifies each claim as verified, inferred, or not validated, and the ledger is what decides the wording on these pages. It is also what let a preregistered study falsify a tool I had already built, and archive it, instead of quietly dropping it.

   BlackBoxRS · IPFD (archived)

Three of these are refuted, null, or falsified, and one of them killed a tool I had already built.

◆Verified in sim

A shipped L2 driving model can lose perception almost entirely on simulator-rendered input while every exported uncertainty channel stays nominal. Parity-controlled, 1,159-frame control; real night and glare do not trigger it; shown on v0.9.7, not reproduced on v0.9.6.

◆Verified in sim

The same failure is detectable from the model's internal recurrent state. 2.41% mean held-out FPR over four real corpora, 95% CI [0%, 5.17%], worst fold 6.90%; deployment-grade FPR unproven.

○Refuted

That internal detector generalizes to other corruption families. AUROC 0.52-0.74 across ImageNet-C, which bounds the detector.

◆Verified on hardware

A monitor can read a real locomotion policy's internals onboard the robot and stop on a frozen internal state. Initial fault-detection experiments used shadow mode on a stationary GO2. Subsequent experiments monitored Phoenix's walking policy during active standing and walking, with zero false alarms across 49 seconds of policy control. PHM remains advisory..

◇Not shown

That monitor warns before a policy's behavior fails. The faults on the robot were injected in software, on a standing robot and during walking under Unitree's own controller; warning ahead of a real behavioral failure has not been tested.

◆Verified on hardware

A ROS 2 reliability layer can run inside a real robot graph and turn its signals into structured, audited fault events. Eight bounded GO2 sessions; an injected log fault was detected end to end; live rate alerts were unlabeled threshold crossings.

◇Not shown

That layer detects physical sensor faults on the robot. LiDAR cover, mic disconnect and a 44 s sensor dropout raised no alert in the builds tested; the dropout path is fixed in software, not re-tested.

◇Not proven

That layer physically holds the robot when it fires. The recovery stream had no downstream subscriber.

◇Open

A sim-trained locomotion policy transfers to the physical GO2. A walking policy holds a stand and takes its first steps on the robot after a scripted stand-up, but under-tracks the command; joint friction is measured as the cause, friction-aware policies are retrained in simulation and a first preregistered hardware comparison has run on one forward command; turns, faster commands and a second floor surface come before results are reported.

○Null

Failures carry uniquely useful information for policy repair or simulator diagnosis. Five preregistered CPU toy studies (cart-pole), all NO-GO; no quadruped claim made.

○Falsified

A restored simulator branch substitutes for the uninterrupted episode. Preregistered 5-seed study: disagreement 4.05% to 2.48%, a 38.9% reduction against a 50% bar. IPFD archived.

The strongest result I have is that a deployed model's failure was invisible at its interface and legible in its internals. The obvious next question is whether that generalizes into a runtime shield: a monitor that sits beside a learned locomotion policy, decides in real time whether it is still inside its competent region, and hands control to a fallback when it is not.

That is what policy-health-monitor and the Phoenix reliability work are for, and it is the direction I would pursue in a PhD. The monitor now runs onboard the GO2 while a learned walking policy controls the robot, reading that policy's own internals. It does not yet take control away. The hard part is not the detector. It is proving the arbitration is safe on a robot that can hurt someone, which is why the hardware ledger matters more to me than another simulation number.

Read the teardown → Technical reports →