Research

The reliability layer

My question is narrow and it does not move: when a learned robot policy is about to fail, can the system tell, in time to do something about it?

Threads

Four decompositions of one question.

Runtime failure detection Watch a shipped policy's own internals for the moment it leaves the region where it is competent, and catch it before it acts. This is the thread with the strongest evidence behind it, and the one I would build a dissertation on. supercombo-blindspot · policy-health-monitor · HELIX
Failure localization Given a trajectory that ended badly, find the step where recovery stopped being possible. I built this on counterfactual replay, then preregistered a study that falsified the substitution it depends on, and archived it. The method is open; the assumption is not safe. IPFD (archived) · BlackBoxRS
Honest sim-to-real Parity-verified transfer from simulation to real edge hardware, and precision about what actually runs on the robot versus what is still in simulation. Most of the value is in refusing to blur that line. go2-phoenix · physx-newton-bench
Assistive autonomy My M.S. thesis: a quadruped guiding blind and low-vision users, where following the wrong person is a worse failure than following nobody. A domain where the reliability argument is not academic. Thesis · private until defense

Claim ledger

What I actually claim, and how far it holds.

Three of these are refuted, null, or falsified, and one of those killed a tool I had already built. They stay on the page because a ledger that only records wins is not a ledger.

A shipped L2 driving model can lose perception entirely while every exported signal stays nominal Parity-controlled, 1,159-frame control; LOCO-evaluated detector Verified in sim
The same failure is recoverable from the model's internal recurrent state 2.41% mean held-out FPR, 95% CI [0%, 5.17%] Verified in sim
That internal detector generalizes to other corruption families AUROC 0.52-0.74 across ImageNet-C; reported as a bound, not hidden Refuted
A ROS 2 reliability layer can detect and structure live faults inside a real robot graph Eight GO2 and Jetson sessions; 1.16 ms callback path Verified on hardware
That layer physically holds the robot when it fires The recovery stream had no downstream subscriber Not proven
A sim-trained locomotion policy transfers to the physical GO2 Deploy chain verified end to end; Gate 7 stand has not cleared Open
Failure-seeded curriculum improves quadruped RL outcomes Effect did not survive an honest seed count; result published as null Null
A restored simulator branch substitutes for the uninterrupted episode Preregistered 5-seed study: disagreement 4.05% to 2.48%, a 38.9% reduction against a 50% bar. IPFD archived Falsified

Direction

Where this goes next.

The strongest result I have is that a deployed model's failure was invisible at its interface and legible in its internals. The obvious next question is whether that generalizes into a runtime shield: a monitor that sits beside a learned locomotion policy, decides in real time whether it is still inside its competent region, and hands control to a fallback when it is not.

That is what policy-health-monitor and the Phoenix reliability work are for, and it is the direction I would pursue in a PhD. The hard part is not the detector. It is proving the arbitration is safe on a robot that can hurt someone, which is why the hardware ledger matters more to me than another simulation number.

Read the teardown Papers & evidence