ui: Show camera viewing ray when hovering a pixel in the image viewer#4584
Open
behnamasadi wants to merge 2 commits into
Open
ui: Show camera viewing ray when hovering a pixel in the image viewer#4584behnamasadi wants to merge 2 commits into
behnamasadi wants to merge 2 commits into
Conversation
Hovering the mouse over a pixel in the model viewer's image window now draws the back-projected viewing ray from that camera's center through the pixel into the 3D scene (rerun-style). If the hovered pixel is near an observed keypoint, the ray snaps to and highlights the corresponding 3D point; otherwise it extends across the scene along the pixel's direction. The pixel is unprojected with Camera::CamFromImg and transformed by the image pose; the ray is drawn with a dedicated LinePainter/PointPainter in the model viewer. The image viewer installs an event filter on its graphics view to track hover, and clears the ray when the cursor leaves the image.
Adds a "Reprojection error" option to the image colormap dropdown in the model
viewer, coloring each camera frustum by its mean reprojection error (accumulated
from the 3D point tracks) through the existing JetColormap.
Uses an *absolute* color scale (0 px = blue, "Reproj. error for red [px]" = red)
rather than a per-model min/max normalization, so colors stay comparable across
models: a well-registered model stays blue/cyan and only genuinely high-error
cameras turn red, instead of stretching a tiny error spread over the whole
spectrum. The upper bound is user-adjustable via a spinbox (with an explanatory
tooltip), and applying the colormap logs the model's actual per-image error range
so the scale maps to concrete numbers.
Default colormap is unchanged ("Uniform color").
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a rerun-style interaction to the model viewer: hovering a pixel in a
camera's image draws that pixel's back-projected viewing ray in the 3D view.
pixel into the 3D scene, updating live as the cursor moves and clearing when
it leaves the image.
highlights the corresponding 3D point (yellow); otherwise it extends across
the scene along the pixel's direction.
This makes the 2D image and the 3D reconstruction directly cross-referenceable:
point at anything in a photo and see where it lies in 3D.
Demo
How it works
its
QGraphicsViewviewport (mouse tracking on) and maps the cursor to imagepixel coordinates.
Camera::CamFromImg(pixel)gives the normalizedcamera-frame direction
(x, y, 1), transformed by the image pose(
Inverse(image.CamFromWorld())) into a world-space ray fromimage.ProjectionCenter().Point2Dwith apoint3D_idwithin ~1% of the image size is found; if present, the ray ends exactly at
that 3D point and highlights it.
LinePainter(ray) andPointPainter(highlight)render in
ModelViewerWidget::paintGL, using the samemodel_scale_ * (world + model_origin_)transform as the rest of the scene.Notes
model_viewer_widget.{h,cc}(SetHoverRay/ClearHoverRay + painters),image_viewer_widget.{h,cc}(event filter + unprojection).image_viewer_widget.hmoves
graphics_scene_/graphics_view_from private to protected so thedatabase image viewer can attach the hover filter.
-DGUI_ENABLED=ONand verified interactively; notcovered by the headless unit tests. The ray geometry was checked numerically
(computed pixel direction matches the camera→3D-point direction).
Note on the second commit (cherry-picked)
For convenience this branch also carries a second commit,
"ui: Add reprojection-error image colormap for camera frustums", cherry-picked
from #4579, so both GUI viewer features can be built and demoed from a single
branch. That colormap change is reviewed separately in #4579 — happy to drop
the commit here and keep this PR scoped to the hover-ray change alone if you
prefer.