Virtual RGBD Sensor

In Rust!

Table of Contents

1. Features
2. Recent Updates
3. Instructions
4. Debugging Notes

Features:

1. Converted 32-channel RoboSense LiDAR outputs in the form of .pcap or MSOP/DIFOP packets to PointCloud2: (x, y, z, intensity, cluster_id). Implementation explanation is here.
2. Implemented Two-Layer-Graph Clustering for 32-channel point cloud segmentation, per frame and without cross-frame matching and consistency. Implementation explanation is here.

3. Ported cpp nav2_costmap_2d point cloud to costmap conversion to Rust. Implementation details are contained here.

Recent Updates:

Date	Changelog / Update Notes
9/26/26	- Rewrote ros2-planning costmap_2d implementation in Rust, also integrated costmap construction directly after RangeGraph initialization
9/14/26	- Integrated Two-Layer-Graph Clustering with the decoding of MSOP/DIFOP packets for optimized segmentation while point cloud are being processed. Specifically during the construction of the range and set graphs.
9/12/26	- Tested rslidar_sdk_node.rs on online LiDAR and offline .pcap files. Also integrated simple Bevy visualizer for point clouds.
9/8/26	- Rewrote cpp rslidar_sdk with RSHeliosDecoder for RoboSense 32-channel LiDAR specifically. This is for converting offline .pcap or online MSOP/DIFOP packets to point clouds

Instructions:

Run Zenoh publisher:

RUSTFLAGS="-C link-arg=-fuse-ld=gold" cargo run --bin rslidar_viz --release

Run Zenoh subscriber:

RUSTFLAGS="-C link-arg=-fuse-ld=gold" cargo run --bin zenoh_viz --release

For Offline Demo:

To run the point cloud conversion and segmentation on the provided offline LiDAR packets, make sure that pcap_path in config.yaml is set to whatever file path points to test_cloud.pcap.

For Online Demo:

If you want to run rslidar_sdk_node on online LiDAR via MSOP/DIFOP ports, make sure the pcap_path in config.yaml is empty.

Zenoh Mappings

A brief overview on the Zenoh pub/sub elements in virtual-rgbd-sensor:

Publishers:

1. rslidar/points/raw: raw (x, y, z, intensity) point cloud from RoboSense LiDAR MSOP/DIFOP packets
2. rslidar/points/segmented: segmented (x, y, z, cluster_id) point cloud from segmentation.rs's two-layer-graph clustering
3. rslidar/costmap: occupancy grid containing free space, obstacle regions, and inflation layers from Rust implementation of ros-planning costmap_2d in /costmap

Subscribers:

In zenoh_test.rs, there are example point cloud and costmap subscribers in _points_subscriber and _costmap_subscriber, respectively.

Debugging Notes:

Our LiDAR is pinged via 192.168.1.102:

sudo ip link set enP8p1s0 up
sudo ip addr ad 192.168.1.102/24 dev enP8p1s0

If working in a Docker container, it needs to be able to access IP addresses.

x11 host on Docker so I can GUI on local:

ssh -X mini-dos@cev_jetson0.coecis.cornell.edu

in cev_jetson0, to test if GUI working

xclock

sudo docker run -it \
    --network host \
    -e DISPLAY=$DISPLAY \
    -e XAUTHORITY=/root/.Xauthority \
    -v ~/.Xauthority:/root/.Xauthority:ro \
    --name dbimage-container \
    dbimage:lidar-dev \
    /bin/bash

in docker

rviz2

should actually display.