Skip to content

ui: Show camera viewing ray when hovering a pixel in the image viewer#4584

Open
behnamasadi wants to merge 2 commits into
colmap:mainfrom
behnamasadi:feat/hover-pixel-ray
Open

ui: Show camera viewing ray when hovering a pixel in the image viewer#4584
behnamasadi wants to merge 2 commits into
colmap:mainfrom
behnamasadi:feat/hover-pixel-ray

Conversation

@behnamasadi

Copy link
Copy Markdown
Contributor

Summary

Adds a rerun-style interaction to the model viewer: hovering a pixel in a
camera's image draws that pixel's back-projected viewing ray in the 3D view.

  • Double-click a camera frustum → its photo opens in the image window (existing).
  • Hover a pixel in that photo → a ray shoots from the camera center through the
    pixel into the 3D scene
    , updating live as the cursor moves and clearing when
    it leaves the image.
  • If the hovered pixel is near an observed keypoint, the ray snaps to and
    highlights the corresponding 3D point
    (yellow); otherwise it extends across
    the scene along the pixel's direction.

This makes the 2D image and the 3D reconstruction directly cross-referenceable:
point at anything in a photo and see where it lies in 3D.

Demo

hover-pixel-ray demo

How it works

  1. Capture the hovered pixel — the image window installs an event filter on
    its QGraphicsView viewport (mouse tracking on) and maps the cursor to image
    pixel coordinates.
  2. Unproject to a world rayCamera::CamFromImg(pixel) gives the normalized
    camera-frame direction (x, y, 1), transformed by the image pose
    (Inverse(image.CamFromWorld())) into a world-space ray from
    image.ProjectionCenter().
  3. Snap to a 3D point — the nearest observed Point2D with a point3D_id
    within ~1% of the image size is found; if present, the ray ends exactly at
    that 3D point and highlights it.
  4. Draw in 3D — a dedicated LinePainter (ray) and PointPainter (highlight)
    render in ModelViewerWidget::paintGL, using the same
    model_scale_ * (world + model_origin_) transform as the rest of the scene.

Notes

  • Files: model_viewer_widget.{h,cc} (SetHoverRay/ClearHoverRay + painters),
    image_viewer_widget.{h,cc} (event filter + unprojection). image_viewer_widget.h
    moves graphics_scene_/graphics_view_ from private to protected so the
    database image viewer can attach the hover filter.
  • GUI feature — built with -DGUI_ENABLED=ON and verified interactively; not
    covered by the headless unit tests. The ray geometry was checked numerically
    (computed pixel direction matches the camera→3D-point direction).

Note on the second commit (cherry-picked)

For convenience this branch also carries a second commit,
"ui: Add reprojection-error image colormap for camera frustums", cherry-picked
from #4579, so both GUI viewer features can be built and demoed from a single
branch. That colormap change is reviewed separately in #4579 — happy to drop
the commit here and keep this PR scoped to the hover-ray change alone if you
prefer.

Hovering the mouse over a pixel in the model viewer's image window now draws the
back-projected viewing ray from that camera's center through the pixel into the
3D scene (rerun-style). If the hovered pixel is near an observed keypoint, the
ray snaps to and highlights the corresponding 3D point; otherwise it extends
across the scene along the pixel's direction.

The pixel is unprojected with Camera::CamFromImg and transformed by the image
pose; the ray is drawn with a dedicated LinePainter/PointPainter in the model
viewer. The image viewer installs an event filter on its graphics view to track
hover, and clears the ray when the cursor leaves the image.
Adds a "Reprojection error" option to the image colormap dropdown in the model
viewer, coloring each camera frustum by its mean reprojection error (accumulated
from the 3D point tracks) through the existing JetColormap.

Uses an *absolute* color scale (0 px = blue, "Reproj. error for red [px]" = red)
rather than a per-model min/max normalization, so colors stay comparable across
models: a well-registered model stays blue/cyan and only genuinely high-error
cameras turn red, instead of stretching a tiny error spread over the whole
spectrum. The upper bound is user-adjustable via a spinbox (with an explanatory
tooltip), and applying the colormap logs the model's actual per-image error range
so the scale maps to concrete numbers.

Default colormap is unchanged ("Uniform color").
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant