Skip to content

map: point clouds, and Zarr dimensions set from the style panel - #1370

Merged
amanbagrecha merged 7 commits into
mainfrom
feat/map-pointclouds-zarr-dims
Oct 5, 2026
Merged

amanbagrecha merged 7 commits into
mainfrom
feat/map-pointclouds-zarr-dims

Conversation

@amanbagrecha

@amanbagrecha amanbagrecha commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

What

Point clouds: a new pointcloud layer kind for LAS / LAZ / COPC / EPT.

  • Rendering: maplibre-gl-lidar (deck.gl PointCloudLayer), one hidden control per layer so each cloud styles on its own.
  • COPC and EPT stream by viewport. LAS/LAZ ≤ 1.3 are read whole in the browser.
  • LAS 1.4 and CRS problems: LAS 1.4, a CRS stored only in GeoTIFF keys, or a CRS the user names go through prepare.py's new pointcloud action. laspy rewrites the file as LAS 1.2 in lon/lat, thinned to 5M points and cached. The user names a CRS with options.crs, entered in the style panel when the file has none.
  • Style panel: colour by elevation, intensity, classification or RGB; colormap and range; ASPRS class toggles; point size; elevation filter; height offset; tilt to 3D.
  • Bundle: moves to deck.gl 9.4, which the lidar library needs. Rasters are unchanged and covered by the existing e2e tests. laz-perf.wasm now ships beside the bundle instead of loading from unpkg.
  • MCP and file browser: both know the new kind and the new extensions.

Zarr dimensions: every non-map dimension (time, band, depth…) gets a value list, previous/play/next buttons and a slider. The style panel now opens when a layer is added or its card clicked. Testing against real stores turned up bugs, fixed here:

  • Pyramids: multiscale stores (carbonplan/ndpyramid) failed with "Not found: v3 array or group".
  • Unconsolidated v3: stores without consolidated metadata failed. The page now lists the bucket (ListObjectsV2) and falls back to Python inspect.
  • String coordinates: band names showed as bare indexes. Picking a named dimension now recomputes the automatic colour range from that slice only.
  • Placement: read from the coordinate arrays and passed to the layer. This covers descending latitude, 0..360 longitude, CF-attribute or WRF-style dimension names, and projected grids (a new crs action returns proj4 and lon/lat bounds).
  • Extension-less URLs: a remote URL without an extension is checked for Zarr metadata before Python is asked.

Tested against

  • Point clouds: giswqs autzen-classified.copc.laz (10.6M points) and madison.copc.laz, the USGS IA_FullState EPT, PDAL autzen_trim.las and autzen.laz, plus local LAS 1.2 and 1.4 files.
  • Zarr: giswqs noaa-oisst-v2-monthly.zarr and stac-zarr/example.zarr (v3, unconsolidated), the carbonplan 4-D tavg-prec-month pyramid, the NEMO sos_abs pyramid, and local v3, UTM and WRF-style stores.

Tests

  • test_map_e2e.py: three new cases cover point clouds in the browser and through Python, the CRS box, and Zarr placement plus dimension controls. 11 pass.
  • test_prepare.py: five new cases cover the converter, thinning, the CRS refusal and override, inspect/plan, and the crs action. 32 pass.
  • tests/test_map_doc.py and tests/test_map_runtime_contract.py: pass, with a new assertion that the vendored LAZ decoder is used.
  • Known flake: test_an_open_map_file_follows_agent_edits_and_writes_its_own is flaky on main too: it failed 2 of 6 runs without this change.

Known limits

  • Point clouds draw on deck.gl's canvas, above all map layers, whatever their order in the layer list.
  • Classification codes above 31 read as unclassified in files Python converts, because LAS 1.2 can't hold them.

Note

Medium Risk
Large viewer and prepare.py changes (new conversion pipeline, Zarr placement) affect layer loading for multiple formats; mitigated by extensive new e2e and unit tests.

Overview
Adds point cloud layers (LAS/LAZ/COPC/EPT) end-to-end: new pointcloud map kind and style/options (color_scheme, class filters, options.crs), browser rendering via maplibre-gl-lidar, and a prepare.py pointcloud action that rewrites unsupported LAS (e.g. 1.4, missing CRS) to thinned LAS 1.2 in lon/lat with caching. MCP, discovery, and the style panel gain point-cloud controls (including CRS entry on failure).

Zarr loading and the style panel are expanded: richer lat/lon axis detection, projected-grid placement via a new crs action, multiscale pyramids, remote listing for unconsolidated stores, per-dimension slice UI (slider, pick list, play), and auto-clim that follows named dimensions. Extension-less remote URLs can be probed as Zarr before calling Python.

Azure blob SAS handling is normalized: unsigned() strips SAS query params for cache keys, signing replaces stale tokens, and download resume matches sources across token rotation.

Reviewed by Cursor Bugbot for commit 6f57182. Bugbot is set up for automated code reviews on this repo. Configure here.

Point clouds. LAS/LAZ/COPC/EPT load as a new `pointcloud` layer kind through
maplibre-gl-lidar (deck.gl PointCloudLayer), one hidden control per layer so
each cloud styles independently. COPC and EPT stream by viewport; LAS/LAZ
<= 1.3 are read whole in the browser. LAS 1.4, a CRS only in GeoTIFF keys,
or a CRS the user names (`options.crs`, typed into the style panel when the
file has none) go through prepare.py's new `pointcloud` action: laspy
rewrites the file as LAS 1.2 in lon/lat, thinned to 5M points and cached.
The style panel colours by elevation/intensity/classification/RGB, toggles
ASPRS classes, and sets point size, elevation filter, height offset and tilt.
The bundle moves to deck.gl 9.4 (the lidar library needs it; rasters are
unchanged) and ships laz-perf.wasm beside it instead of loading it from unpkg.

Zarr. Every non-map dimension (time, band, depth...) gets a value list,
previous/play/next and a slider in the style panel, which now opens when a
layer is added or its card clicked. Fixed along the way:
- multiscale pyramids (carbonplan/ndpyramid) failed to load
- Zarr v3 stores without consolidated metadata failed; the page now lists
  the bucket (ListObjectsV2) and falls back to Python
- string coordinates (band names) were unlabelled; picking a named
  dimension recomputes the automatic colour range for that slice only
- placement is read from the coordinate arrays (descending latitude,
  0..360 longitude, CF-attribute or WRF-style dimension names, projected
  grids via a new `crs` action that returns proj4 + lon/lat bounds)
- an extension-less remote URL is probed for Zarr metadata before Python

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread fused_render/templates/map/template.html
Comment thread fused_render/templates/map/template.html
Comment thread fused_render/templates/map/template.html Outdated
Comment thread fused_render/templates/map/template.html
…quieter point-cloud errors

- zarrPlacement asks pyproj whether the store's declared CRS is geographic
  before trusting a lon/lat-sized coordinate range, so projected grids in
  km or cell indices are no longer pinned near 0,0.
- inspect reports each variable's grid_mapping so unlistable stores can
  still open spatial_ref.
- looksLikeZarr keeps the URL query (SAS/token) on every probed key.
- A point-cloud loaderror after the load settled no longer rejects unhandled.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit bf2e999. Configure here.

Comment thread fused_render/templates/map/prepare.py Outdated
aman bagrecha and others added 2 commits October 2, 2026 18:42
@github-actions

github-actions Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  fused_render/templates/map
  discover.py 86, 88, 90
  map_doc.py 146
  map_tools.py 214
  prepare.py 168-175, 180-185, 256-269, 282-350, 354-355, 360-366, 561-568, 613, 615, 617, 619-620, 639-652, 672, 674, 676, 678
Project Total  

This report was generated by python-coverage-comment-action

@amanbagrecha
amanbagrecha requested a review from iamsdas October 4, 2026 17:57
@amanbagrecha
amanbagrecha merged commit fafb80a into main Oct 5, 2026
13 of 16 checks passed
@amanbagrecha
amanbagrecha deleted the feat/map-pointclouds-zarr-dims branch October 5, 2026 07:19
@vasu2001 vasu2001 mentioned this pull request Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants