Vision · perception & localization

The vision system

Production-grade perception and localization on a single Jetson AGX Thor. Dual ZED 2i stereo cameras, Ouster OS0-128 LiDAR, IMU, and u-blox GNSS stream full-rate data into the limit-robot ROS 2 container — one documented interface for perception and localization.

W/S/A/D to drive · drag to orbit · scroll to zoom
LOADING LIMIT

What's inside

Full-rate perception and localization from a single platform. Every sensor — dual ZED 2i stereo (front + rear), Ouster OS0-128 LiDAR, IMU, and u-blox GNSS — publishes into one ROS 2 graph inside limit-robot, so downstream systems consume a single, stable data plane instead of five separate integrations.

cameras

Dual ZED 2i stereo

Front + rear stereo pairs — per-eye images, depth, point clouds, and the ZED IMU on a shared GPU-accelerated SDK path.

lidar

Ouster OS0-128

128-channel solid-state LiDAR over dedicated 1 GbE — dense point clouds for mapping, obstacle detection, and localization.

localization

IMU + GNSS

ZED IMU for short-term orientation and a u-blox GNSS receiver for absolute position, fused with LiDAR into one state estimate.

Jetson AGX Thor aarch64 ROS 2 Jazzy in limit-robot GPU-accelerated perception stack limit_msgs custom message types 100 topics · 29 nodes · 17 actions

How to interact

drive

W / S · A / D

W drives forward, S reverses, A/D yaw left and right. While a key is held, Limit animates its Run 01 gait; release all keys and it stands completely still.

camera

Drag · scroll

Drag to orbit around the model, scroll to zoom in and out. The camera stays free while the robot drives — you can watch it run from any angle.

console

Command log

Every drive input is logged as the real ROS 2 command it maps to (CTRL_VELOCITY), so the demo doubles as a reference for the actual drive interface.

Under the hood

sensors in

Raw streams

/zed_pub (per-eye images, depth, point cloud, IMU) · /ouster/os_driver (points, ~10 Hz) · /gnss_pub (fix).

bridge out

/limitx_*

A canonical bridge composites the two ZED units, multiplexes IMU, forwards LiDAR + GNSS, and publishes source health on limit_msgs types.

sim

Gazebo + Nav2

Simulation and the Nav2 stack share the same graph, so the interface you integrate against in sim is the interface you get on the real robot.

bash · inspect the graph
ros2 topic list -t                     # 100 topics with types
ros2 node info /zed_pub                # custom limit_msgs types in the graph
ros2 topic hz /ouster/os_driver/points # ~10 Hz point cloud stream
ros2 topic echo /gnss_pub/fix --once

What's next

STEP 1 · LIDAR
Power-cycle the Ouster
Confirm INITIALIZING clears and points stream ~10 Hz
→
STEP 2 · ZED
Restore the GPU path
Recreate limit-robot with --gpus all; SDK backend on both cameras
→
STEP 3 · NET
rmem_max → 4194304
Simultaneous 5-source streaming without the socket-buffer wall
→
STEP 4 · SENSORS
IMU + GNSS
Bring back the ZED IMU and connect the u-blox receiver
Definition of done: both ZED cameras on the SDK backend, IMU publishing, ~10 Hz LiDAR, GNSS fixed, all 7 services stable.