Feature: RTAB-Map SuperPoint feature extraction with OpenVINO (CPU/NPU) by naidjeldias · Pull Request #3 · Synkar/rtabmap

naidjeldias · 2026-06-17T20:13:20Z

Summary

Adds SuperPoint feature extraction via Intel OpenVINO, enabling SuperPoint inference on Intel CPU and Meteor Lake NPU as an alternative to the existing Torch backend.

This is a focused Synkar fork change (~8 files): SuperPointOpenVINO backend, CMake/Docker wiring, and model conversion script.

What changed

New SuperPointOpenVINO / SPDetectorOpenVINO backend (mirrors Torch integration pattern)
CMake option WITH_OPENVINO and RTABMAP_OPENVINO guards in Features2d.cpp
Parameters: SuperPoint/OpenVINOModelPath, SuperPoint/OpenVINODevice (CPU, NPU, GPU, AUTO)
Noble Dockerfile: OpenVINO 2025 + Intel NPU driver stack
convert_superpoint.py: TorchScript → OpenVINO IR export (fixed 848×480 input for NPU)

Design notes

Feature type ID `99` (intentional)

kFeatureSuperPointOpenVINO=99 is deliberate on this fork to avoid enum collisions with future upstream extractors (17, 18, …). Synkar DBs and configs use:

Kp/DetectorStrategy=99
Vis/FeatureType=99

Required runtime parameter: `Kp/BadSignRatio=0`

Validation on dabi_floor0 (reprocessed from Torch reference DB) showed that the default Kp/BadSignRatio=0.5 interacts badly with OpenVINO SuperPoint:

Threshold = 0.5 × Kp/MaxFeatures → 250 keypoints with default max features
OpenVINO’s CPU greedy NMS yields slightly fewer keypoints than Torch on some frames
RTAB-Map then keeps keypoints but strips all descriptors for nodes below the threshold (~153 nodes, ~2.9% of features)
Loop closures drop from ~700 to ~666

This is not an OpenVINO inference failure. With Kp/BadSignRatio=0, descriptor completeness is restored and loop counts match Torch.

Recommended Synkar production config:

Kp/DetectorStrategy=99
Vis/FeatureType=99
SuperPoint/OpenVINOModelPath=/etc/synkar/models/superpoint.xml
SuperPoint/OpenVINODevice=NPU
Kp/BadSignRatio=0
SuperPoint/Threshold=0.010
SuperPoint/NMS=true
SuperPoint/NMSRadius=4

Validation results (Intel Meteor Lake, dabi_floor0)

Reprocessed from Torch reference DB (dabi_floor0-5.5.0-2026-04-29.db) with Kp/BadSignRatio=0:

Metric	NPU (`OpenVINODevice=NPU`)	CPU (`OpenVINODevice=CPU`)	Torch reference
Loop closures	700	702	705
Zero-descriptor features	0%	0%	0%
SLAM avg time / frame	182 ms	250 ms	345 ms
Trajectory length	1017.0 m	1017.0 m	1017.0 m
RMSE	0.000 m	0.000 m	0.000 m
Words (LTM)	310,606	311,691	294,603

Conclusions:

NPU and CPU OpenVINO produce equivalent SLAM quality (Δ2 loop closures)
NPU is ~1.4× faster than CPU OpenVINO on the same chip (~1.9× vs Torch)
Feature/descriptor pipeline is correct when BadSignRatio is disabled

Build

docker build -f docker/noble/Dockerfile -t rtabmap:openvino .

cmake -DWITH_OPENVINO=ON -DOpenVINO_DIR=<path> ..

Model conversion

python3 corelib/src/superpoint_openvino/convert_superpoint.py \
  --input superpoint.pt \
  --output superpoint.xml

Input images must match the exported shape (848×480 grayscale) or the IR model must be re-exported for your camera resolution.

Test plan

Build with -DWITH_OPENVINO=ON (Noble Docker image)
Reprocess Torch DB with Kp/DetectorStrategy=99 + valid IR model
Compare loop closures vs Torch on dabi_floor0
Verify SuperPoint/OpenVINODevice=NPU on Intel Meteor Lake NPU
Confirm zero-descriptor rate is 0% with Kp/BadSignRatio=0
Preferences UI entry for OpenVINO detector (CLI/params work today)
Graceful UERROR logging on model load / inference failure

Known limitations

ROI-based keypoint extraction not supported (same constraint as Torch backend)
CPU greedy NMS differs from Torch max-pool NMS (minor keypoint count differences; negligible SLAM impact with BadSignRatio=0)
OpenVINO GUI integration pending

Implemented Superpoint feature extraction module from RTAB-Map to run on Intel NPU hardware.

naidjeldias

Summary

The actual OpenVINO work is a focused, well-structured addition (commit 98cd9a89, 8 files / ~473 lines): a SuperPointOpenVINO backend that mirrors the existing Torch integration, caches descriptor tensors between generateKeypointsImpl / generateDescriptorsImpl, and adds Noble Docker + CMake wiring for Intel GPU/NPU.

However, the PR as opened is not merge-ready in its current form.

Blocking issues

1. PR scope (217 commits / 358 files)

The branch contains a large upstream sync plus the OpenVINO feature. GitHub reports mergeable_state: dirty. For review and maintenance, please split into:

PR A: rebase/sync noetic-devel (or merge upstream separately)
PR B: OpenVINO SuperPoint only (~8 files)

2. Build breakage risk

WITH_OPENVINO=ON by default + FIND_PACKAGE(OpenVINO REQUIRED) will fail cmake on systems without OpenVINO. Recommend opt-in default (OFF).

3. Enum value `kFeatureSuperPointOpenVINO=99`

Looks temporary. Should be the next sequential enum value (17) to stay consistent and avoid config collisions.

4. Missing GUI integration

No entry in preferencesDialog.ui / PreferencesDialog.cpp for model path, device, or detector selection. Torch and Rpautrat are fully wired — OpenVINO is CLI/params-only today.

5. Error visibility

Model load / inference failures are silent (catch(...) {}, no UERROR). Users will see empty keypoints with no explanation.

6. ROI handling

roi parameter is ignored; Torch explicitly errors on non-zero ROI. Please align behavior.

Non-blocking but recommended

Area	Note
Naming	Typo "OpenVINNO" in PR title + commit message
Comments	Portuguese comments in `.cpp` — use English for upstream consistency
Formatting	Leading spaces on every line in new `.h`/`.cpp` files
Constructor	Initialize members from `Parameters::default*` like `SuperPointTorch`
NMS	CPU greedy NMS differs from Torch max-pool NMS — may change keypoint sets
Default device	Parameter default is `CPU`; PR goal is NPU — consider `NPU` or `AUTO` default + docs
Docs	No README on exporting/converting SuperPoint to OpenVINO IR (`.xml`/`.bin`)
Tests	No unit/smoke test for model load + keypoint/descriptor output
Thread safety	`mutable` descriptor cache + `const` descriptor method — verify against RTAB-Map threading model
Performance	`image.clone()` + `memcmp` on every descriptor call — consider pointer/size check only

What looks good

Clean adapter split (SPDetectorOpenVINO + SuperPointOpenVINO)
Reuses existing SuperPoint params (threshold, NMS, radius)
Handles both flat heatmap and 65-channel softmax output layouts
Descriptor bilinear sampling + L2 norm matches Torch path
#ifdef RTABMAP_OPENVINO guards in Features2d.cpp
Noble Dockerfile documents Intel OpenVINO 2025 + NPU driver stack

Suggested test plan

Build with -DWITH_OPENVINO=ON on Noble Docker image
Run with Kp/DetectorStrategy set to the OpenVINO enum + valid IR model path
Compare keypoint count / descriptor norms vs Torch backend on a fixed frame
Verify SuperPoint/OpenVINODevice=NPU on Intel NPU hardware
Confirm graceful fallback + log message when model path is invalid
Confirm Preferences UI can select and persist OpenVINO settings (once added)

Verdict: Promising feature implementation, but needs work before merge — split scope, resolve conflicts, and address enum/GUI/error-handling issues; make OpenVINO opt-in by default.

naidjeldias · 2026-06-24T20:45:10Z

 option(WITH_ORB_OCTREE    "Include ORB Octree feature support"   ON)
-option(WITH_TORCH         "Include Torch support (SuperPoint)"   OFF)
+option(WITH_TORCH         "Include Torch support (SuperPoint)"   ON)
+option(WITH_OPENVINO      "Include OpenVINO support (SuperPoint)" ON)


WITH_OPENVINO defaults to ON and FIND_PACKAGE(OpenVINO REQUIRED ...) will fail configuration on machines without OpenVINO installed. Consider defaulting to OFF (like WITH_TORCH historically) so the feature is opt-in and does not break existing build pipelines.

naidjeldias · 2026-06-24T20:45:11Z

+     try {
+         compiledModel_ = core_.compile_model(path, device_);
+         inferRequest_ = compiledModel_.create_infer_request();
+     } catch(...) {}


Model load failures are swallowed silently (catch(...) {} and early return without logging). The Torch backend logs UERROR when the path is empty/missing — please mirror that so users know why SuperPoint returns zero keypoints.

naidjeldias · 2026-06-24T20:45:13Z

+     } catch(...) {}
+ }
+
+ SuperPointOpenVINO::SuperPointOpenVINO(const ParametersMap & parameters)


Unlike SuperPointTorch, this constructor does not initialize path_, threshold_, nms_, minDistance_, or device_ before parseParameters(). If parsing misses a key, members may be uninitialized. Please follow the SuperPointTorch initializer-list pattern.

naidjeldias · 2026-06-24T20:45:13Z

+     }
+ }
+
+ std::vector<cv::KeyPoint> SuperPointOpenVINO::generateKeypointsImpl(const cv::Mat & image, const cv::Rect & roi, const cv::Mat & mask)


roi is ignored. SuperPointTorch::generateKeypointsImpl explicitly rejects non-zero ROI with UERROR. Either add the same guard or document that ROI-based keypoint extraction is unsupported for this backend.

naidjeldias · 2026-06-24T20:45:15Z

+#endif
+#ifdef RTABMAP_OPENVINO
+case Feature2D::kFeatureSuperPointOpenVINO:
+	feature2D = new SuperPointOpenVINO(parameters);


No Preferences UI wiring for OpenVINO model path / device (Torch and Rpautrat both have combo entries and parameter fields in preferencesDialog.ui + PreferencesDialog.cpp). Without this, users cannot select SUPERPOINT_OPENVINO from the GUI.

…ion to the repository

RTAB-Map feature extraction with OpenVINNO.

98cd9a8

Implemented Superpoint feature extraction module from RTAB-Map to run on Intel NPU hardware.

naidjeldias commented Jun 24, 2026

View reviewed changes

naidjeldias changed the base branch from noetic-devel to master June 25, 2026 14:24

naidjeldias changed the title ~~Feature: rtab map feature extraction with openvinno~~ Feature: RTAB-Map SuperPoint feature extraction with OpenVINO (CPU/NPU) Jun 26, 2026

feat(superpoint_openvino/convert_superpoint.py): adding model convers…

fc9a484

…ion to the repository

naidjeldias mentioned this pull request Jul 2, 2026

Feature: SuperPoint OpenVINO feature extraction (noetic-devel) #4

Open

5 tasks

Navigation Menu

Uh oh!

Feature: RTAB-Map SuperPoint feature extraction with OpenVINO (CPU/NPU) - #3

Feature: RTAB-Map SuperPoint feature extraction with OpenVINO (CPU/NPU)#3
naidjeldias wants to merge 2 commits into
masterfrom
feature/rtab-map-feature-extraction-with-openvinno

naidjeldias commented Jun 17, 2026 •

edited

Loading

Uh oh!

naidjeldias left a comment

Uh oh!

Uh oh!

naidjeldias Jun 24, 2026

Uh oh!

naidjeldias Jun 24, 2026

Uh oh!

naidjeldias Jun 24, 2026

Uh oh!

naidjeldias Jun 24, 2026

Uh oh!

naidjeldias Jun 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

naidjeldias commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What changed

Design notes

Feature type ID 99 (intentional)

Required runtime parameter: Kp/BadSignRatio=0

Validation results (Intel Meteor Lake, dabi_floor0)

Build

Model conversion

Test plan

Known limitations

Uh oh!

naidjeldias left a comment

Choose a reason for hiding this comment

Summary

Blocking issues

1. PR scope (217 commits / 358 files)

2. Build breakage risk

3. Enum value kFeatureSuperPointOpenVINO=99

4. Missing GUI integration

5. Error visibility

6. ROI handling

Non-blocking but recommended

What looks good

Suggested test plan

Uh oh!

Uh oh!

naidjeldias Jun 24, 2026

Choose a reason for hiding this comment

Uh oh!

naidjeldias Jun 24, 2026

Choose a reason for hiding this comment

Uh oh!

naidjeldias Jun 24, 2026

Choose a reason for hiding this comment

Uh oh!

naidjeldias Jun 24, 2026

Choose a reason for hiding this comment

Uh oh!

naidjeldias Jun 24, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

naidjeldias commented Jun 17, 2026 •

edited

Loading

Feature type ID `99` (intentional)

Required runtime parameter: `Kp/BadSignRatio=0`

3. Enum value `kFeatureSuperPointOpenVINO=99`