Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ jobs:
toolchain: '1.88.0'
- uses: Swatinem/rust-cache@v2
- run: cargo test --workspace --exclude superstac-python --locked
- run: cargo test -p superstac-cli -p superstac-engine --features geoparquet --locked
- run: cargo check -p superstac-python --no-default-features --locked

wheels:
name: Wheel (${{ matrix.target }})
Expand Down Expand Up @@ -61,6 +63,11 @@ jobs:
python -m pip install --no-deps --force-reinstall dist/*.whl
- name: Test the installed wheel
run: python -m unittest discover -s crates/python/tests -v
- name: Execute the GeoParquet notebook against local fixtures
if: matrix.target == 'x86_64-unknown-linux-gnu'
run: |
python -m pip install pyarrow nbformat nbclient ipykernel
python scripts/test-geoparquet-notebook.py
- uses: actions/upload-artifact@v4
with:
name: wheels-${{ matrix.target }}
Expand Down
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,23 @@ releases.

## [Unreleased]

### Added — v0.3.0
- Optional Rust/CLI GeoParquet backend and GeoParquet-enabled Python wheels.
- Scoped, named, resumable STAC metadata ingestion with progress and storage caps.
- Snapshot and automatic API fallback modes, freshness checks, and gap-aware scope unions.
- Per-catalog sorting, local collection summaries, row-group pruning, and a reproducible benchmark.
- Explicit acquisition-window incremental overlays, atomic compaction, and cleanup with reader protection.
- Sync/async Python ingestion, backend configuration, maintenance, and type hints.
- Automatically versioned Parquet provenance and search diagnostics.
- A GeoParquet Jupyter/Colab notebook, release highlights page, and documented local benchmark.

### Changed — v0.3.0
- Dataset manifests use version 2; legacy version-1 inventories remain readable.
- Release publishing includes the GeoParquet crate before the engine; CI checks both feature modes.

### Fixed — v0.3.0
- Forward sort fields to live STAC searches instead of silently discarding them.

## [0.2.0] - 2026-09-27

### Added
Expand Down
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

15 changes: 14 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,12 @@ normalized to canonical names — regardless of which catalog they came from.
> **Status: alpha.** APIs and YAML schema are not yet stable. Pre-1.0; expect
> breaking changes.

**[What’s new in v0.3](https://spatialnode.com/superstac/docs/releases)** · [Try GeoParquet in Colab](https://colab.research.google.com/drive/1JIx8j0Vi7jypKe5Q6klgCzYQMFHMaKGc?usp=sharing) · [Local search benchmark](https://spatialnode.com/superstac/docs/guides/benchmarks)

## Features

- **Federated search:** Query multiple STAC catalogs concurrently with spatial, temporal, collection, and item filters.
- **Reusable satellite-data inventories:** Save metadata for your area of interest and explore it repeatedly without querying every provider again. Fall back to live catalogs when saved coverage is missing or stale.
- **Collection discovery:** Find available collections and the catalogs that serve them.
- **Consistent names:** Map provider-specific collection IDs and asset keys to canonical names through configurable aliases.
- **Deduplicated results:** Merge items across catalogs by item ID, with source provenance available in Rust and CLI results.
Expand Down Expand Up @@ -69,6 +72,7 @@ cargo add superstac-search
cargo add superstac-engine
cargo add superstac-cli
cargo add superstac-config
cargo add superstac-geoparquet
```

### From source
Expand Down Expand Up @@ -106,7 +110,9 @@ superstac --json search -c landsat-c2-l2 -l 10 | jq '.metadata'
superstac collections microsoft sentinel-2-l2a
```

Run `superstac --help` for the full surface.
Run `superstac --help` for the full surface. For saved metadata inventories, build
with `--features geoparquet` and follow the [GeoParquet guide](crates/geoparquet/README.md).
Published Python wheels include this backend.

## Configuration

Expand Down Expand Up @@ -178,11 +184,18 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
| [`superstac-core`](https://crates.io/crates/superstac-core) | domain models, errors, storage trait |
| [`superstac-config`](https://crates.io/crates/superstac-config) | YAML config loading |
| [`superstac-search`](https://crates.io/crates/superstac-search) | federated search logic |
| [`superstac-geoparquet`](https://crates.io/crates/superstac-geoparquet) | Reusable satellite-data inventories for repeated searches |
| [`superstac-engine`](https://crates.io/crates/superstac-engine) | runtime (health, introspection, search orchestration) |
| [`superstac-cli`](https://crates.io/crates/superstac-cli) | the `superstac` binary |

## Logs and debugging

Include `superstac --version` or Python’s `superstac.__version__` in bug reports.
Search diagnostics also include `metadata.superstac_version`. Provider requests
identify the library as `superstac/<version>` through the User-Agent header.
New ingested and compacted Parquet files record the writer version in their footer.


Logs flow through `tracing`. The default level comes from `settings.log_level`
in your config; override at runtime:

Expand Down
31 changes: 19 additions & 12 deletions ROADMAP.MD
Original file line number Diff line number Diff line change
Expand Up @@ -3,29 +3,36 @@
Our priority is reliable federated STAC search. These are areas of focus,
subject to feedback and capacity, with no fixed delivery dates.

Last Updated: September 28th, 2026
Last updated: October 2, 2026

## Implemented for v0.3

- [x] [Reusable satellite-data inventories with GeoParquet](https://github.com/Spatialnode/superstac/issues/31): scoped ingestion with progress and resume, local search and API fallback, Python integration, scope unions, sorting, incremental refresh, compaction and cleanup.



## Short term

- Fix search and iterator edge cases, backed by focused tests.
- Improve configuration errors, logging, and documentation.
- Support private catalogs through custom headers and environment-based credentials.
- [ ] [Support private catalogs](https://github.com/Spatialnode/superstac/issues/32) through per-catalog headers and environment-based credentials, consistently across search, discovery, health checks and ingestion.
- [ ] [Add AI/agent integration](https://github.com/Spatialnode/superstac/issues/22): define a supported tool interface for catalog discovery and bounded searches, with a working agent example. Existing Python APIs can be called by agents; a dedicated integration is still pending.
- [ ] [Explore WASM bindings](https://github.com/Spatialnode/superstac/issues/21): validate a browser-compatible API search client first, including CORS and runtime constraints. Native filesystem and GeoParquet support should not be assumed to work in browsers.

## Near term

- Add catalog preferences and improve health-based fallback.
- Introduce SQLite persistence.
- Improve pagination and result handling across Python, Rust, and the CLI.
- [ ] Add explicit preferred-catalog ordering and failover between providers. Improve recovery of catalogs unavailable at startup. Health checks, retries, concurrent search and snapshot-to-API fallback already exist.
- [ ] Implement SQLite persistence for catalog registration, provider configuration and settings.
- [ ] Expose resumable federated pagination and bounded/lazy result iteration across Python, Rust and the CLI. Provider pagination is already followed internally, but callers currently receive materialized, capped results rather than a federated cursor or lazy stream.

## Long term

Explore broader authentication, additional storage backends, and ecosystem
integrations as real use cases emerge.
Explore OAuth/token refresh and asset signing, additional configuration storage
backends, and ecosystem integrations as real use cases emerge. These extend the
basic credential support above rather than duplicate it.

## v1.0 release

- Stable public APIs and configuration format.
- Well-tested core search, with predictable results and failure handling.
- Clear documentation, supported-platform coverage, and migration guidance.
- [ ] Stable public APIs and configuration format, with a compatibility policy.
- [ ] Release confidence in core search, with predictable results and failure handling. Existing tests are a foundation; scale, interoperability and failure-recovery coverage remain ongoing work.
- [ ] Maintain documentation, supported-platform coverage and migration guidance for every stable release. Documentation and packaging already exist; this is a continuing release requirement.

The v1.0 release depends on core stability; longer-term ideas are optional.
Loading
Loading