C++ perception runtime · C++20, CUDA, TensorRT
A Python perception pipeline, rebuilt in C++ for the robot's own computer: about 4x lower p95 latency.
A real-time person-perception runtime for the GO2's front camera on its Jetson Orin NX: hardware H.264 decode, pose estimation and person re-identification in TensorRT, a target tracker and a ROS 2 node. Replaying 1,841 recorded front-camera frames at 30 fps, p95 latency is under 34 ms against 135 ms for the original Python pipeline, and it keeps up with the camera on 12% of one CPU core.
At a glance
- 135 → 33.5 ms
- p95 arrival-to-result latency, Python vs C++ (slowest of six C++ runs)
- ~0.1% vs 24%
- frames dropped at the 30 fps camera rate
- 12% vs 108%
- of one CPU core
Why it matters
A robot that follows a person has to act on what its camera sees now. Every millisecond and CPU percent the perception path gives back is headroom for the rest of the autonomy stack on the same embedded computer.
Measured on the Orin
Same Jetson, same 1,841 recorded front-camera frames replayed as H.264 at 30 fps, same arrival-to-result boundary. Python: median of three runs, 134.8 ms. C++: slowest of six, 33.5 ms (range 30.2 to 33.5). 134.8 / 33.5 = 4.0x; median to median, 4.1x.
It keeps up with the camera, dropping about 0.1% of frames where the original drops 24%, on 12% of one CPU core instead of 108%. Two offline parity runs on the Orin produced bit-identical output to each other.
Design decisions
TensorRT inference replayed as CUDA graphs; a GPU candidate filter and a small CPU NMS in place of library NMS; every frame buffer allocated once and passed as a move-only lease; a one-slot mailbox so the pipeline always works on the newest frame. Each decode call waits for the picture of the frame it pushed instead of returning the previous one, which removed a one-frame lag.
CI runs ASan, UBSan and TSan on every commit, and fails unless the desktop GPU suite, including compute-sanitizer memcheck and racecheck, passed on the same sources.
Limitations
- Input
- Latency measured on recorded camera frames replayed on the robot's computer. A first 60-second run on the live camera, through the ROS 2 node on the Orin, received frames at 14.3 fps, below the 29 fps the live gate assumed; sustained live, thermal and integration tests are still open.
- Parity
- The Orin build passes 4 of 7 pre-registered output-match gates against the x86 Python reference: detection, boxes, keypoints and person selection match; confidence, embedding similarity and re-identification decisions do not yet. The outputs are close, not equivalent. The gap traces to the Orin's colour-conversion table; the same build given the desktop table passes all 7.
- Conditions
- The C++ runs shared the Jetson with other work while the Python runs had it nearly to themselves, and the Python p95 counts only the 76% of frames it processed. Both favour the Python figure.