Projects

Things that run

Grouped by what they contribute rather than by what they are built with. Every entry states how far it has actually been validated, and where footage exists the demo plays inline.

Flagship case studies

Read these first
HELIX Runtime fault detection, diagnosis, and audited recovery on ROS 2 Case study supercombo-blindspot Silent-failure teardown of a shipped L2 driving model Case study go2-phoenix Closed-loop sim-to-real on the physical Unitree GO2 Case study

Localize failures

Once something has gone wrong, find the moment it became unrecoverable and keep enough state to prove it.

01 ISAAC LAB · FALSIFICATION STUDY

IPFD

The Isaac Policy Failure Debugger localized the step where a failed Franka lift stopped being recoverable. Everything it reported rested on one assumption: that a restored branch stands in for what the uninterrupted episode would have done. I preregistered a five-seed study to test that assumption, and it failed. Restored branches whose state, observation and replayed actions all matched the reference still reached different outcomes. A better restoration protocol cut disagreement 4.05% to 2.48%, a 38.9% relative reduction against a preregistered 50% bar, so the work is archived rather than shipped.

ARCHIVED HONEST NEGATIVE · ASSUMPTION FALSIFIED Isaac Lab Counterfactual replay Preregistration Negative result
Incident report generated by replaying a real GO2 rosbag
02 ROS 2 · FLIGHT RECORDER

BlackBoxRS

A flight recorder for ROS 2 robots. Incident capture and post-failure forensics so that after a crash you can actually answer what happened, rather than guessing from whatever happened to be in the terminal scrollback.

REAL-HARDWARE BAG REPLAY SHIPPED ROS2 Forensics Bag replay

Detect failures at runtime

Watch the policy's own internals, and the timing around it, for the signature of a controller operating outside its competence.

Policy Health Monitor: the OK, DEGRADED, INTERVENE and STOP arbitration states
03 LEARNED-POLICY SAFETY · C++

policy-health-monitor

A runtime safety net for learned policies: detect when a neural controller's internals drift out of distribution and intervene before it acts on the result. This is the reliability shield generalized beyond any one robot.

SIM ONLY · HARDWARE TRACK PENDING OOD detection Runtime safety ROS2 C++
04 ROUTE-RISK MEMORY

riskgraph-go2

Persistent route-risk memory: the robot remembers where things went wrong and scores safer paths the next time through, with the reasoning exposed rather than hidden in a cost map.

MVP SHIPPED · HARDWARE-UNVERIFIED Graph Risk scoring ROS2

Validate before you trust

Benchmarks and harnesses that decide whether a result is real. Including the one where the answer was no.

05 ISAAC LAB · BACKEND BENCHMARK

physx-newton-bench

PhysX against Newton (MuJoCo-Warp) in Isaac Lab: throughput scaling, per-process VRAM, ten-seed learning curves, and an open-loop dynamics-equivalence probe on the GO2 task. Measured 1.93× step time and 5.3× VRAM between backends. Filed upstream to NVIDIA's Newton repository.

PUBLISHED · UPSTREAMED Isaac Lab PhysX Newton Benchmark
Success-rate comparison across failure-driven curriculum variants, showing no significant effect
06 FAILURE-DRIVEN RL · NEGATIVE RESULT

ashfall

Failure-driven reinforcement learning for the GO2: seed training with the failures the robot actually had. The headline effect did not survive an honest seed count, so the null result stands as the finding and the repository says so on its front page.

NULL RESULT HOLDS · REPORTED AS-IS RL Curriculum Null result

Deploy to real robots

Everything that has to work before any of the above matters: a physical GO2, a Jetson, and a ROS 2 graph that stays up.

07 REAL GO2 · VOICE CONTROL

go2-simple-workspace

Spoken commands driving a physical Unitree GO2. Speech is transcribed by Google Speech Recognition with an offline Whisper fallback, mapped to Unitree motion API IDs, and sent to the robot over ROS 2. The clip is the real robot in the lab taking a spoken command and standing up on its hind legs.

RUNS ON THE PHYSICAL ROBOT ROS2 Whisper ASR Unitree API Real hardware
GO2 walking toward a target in the Isaac Sim digital twin
08 AUDIO-VISUAL · GO2

come-here

Someone says "come here" from outside the camera's field of view, and the dog has to solve all of it: hear the command, work out where it came from, turn, find the caller, walk over, stop, sit. The bearing comes from the microphone array's firmware DOA register, because software SRP-PHAT did not survive the mount, and a bounded whole-token Whisper matcher gates the wake so ordinary lab talk does not trigger it. YOLOv8 takes over for acquisition, alignment, approach and a bounding-box stop. Two live end-to-end successes on the robot, one of them a blind call with the caller's position undisclosed: bearing -59°, turn -56° in 1.16 s, 0.66 m of approach, about 9.6 s from wake word to seated. A systemd unit brings the whole stack up at boot, listening 45 s after power-on with no laptop attached.

LIVE END-TO-END ON THE GO2 · DEMO VIDEO PENDING ReSpeaker DOA Whisper YOLOv8 ROS2
09 GAZEBO · NAV2 · YOLOv8

ros2-go2-nav2-yolo

Full Gazebo autonomy stack (Nav2, SLAM Toolbox, CHAMP, YOLOv8). YOLOv8n at 53 ms / 18.8 fps on CPU, with ten non-obvious DDS, TF, and SLAM bugs documented and fixed rather than worked around.

DOCUMENTED · REPRODUCIBLE Nav2 SLAM Toolbox YOLOv8 CHAMP

Understand the scene

Perception that a planner can be asked questions in plain language.

Open-vocabulary point-cloud query for a sofa across three views
10 GPU · OPEN-VOCAB MAPPING

openvocab-tsdf

GPU-accelerated open-vocabulary 3D mapping: build a TSDF and query it in natural language. Ask for a sofa and the corresponding geometry lights up.

MERGED TSDF CLIP CUDA
Semantic scene graph overlaid on a navigation map
11 GO2 · LANGUAGE-GROUNDED NAV

go2-semantic-nav

An open-vocabulary 3D semantic scene graph feeding a language-grounded Nav2 overlay, targeted at Jetson Orin NX.

SIM ONLY Scene graph Nav2 Jetson

Earlier work

Where I started, in hardware and embedded. Kept for the record.