Skip to content

POC: Add weather station collections - #1202

Draft
rchlfryn wants to merge 10 commits into
mainfrom
feat/weather-station-collections
Draft

POC: Add weather station collections#1202
rchlfryn wants to merge 10 commits into
mainfrom
feat/weather-station-collections

Conversation

@rchlfryn

@rchlfryn rchlfryn commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Description

Models the station registry currently living in src/constants/weatherStations.ts as Payload collections, so the pages can become center-dynamic. Nothing reads these collections yet — this is inert on its own, and the page cutover follows separately.

The registry describes pages, not stations: 17 of the 32 groups cover more than one logger, and the layout is a property of the combination rather than of any one station (Alpental lists Summit, Mid and Base temperatures together before moving on to humidity). So groups stay editorial and carry the table layout, while stations sync from SnowObs — which has no concept of a page, a slug or a column order.

Related Issues

First of three for #1169. Then the page cutover, then alerting.

Key Changes

  • stations — synced from SnowObs station/metadata, fully read-only. Includes weatherStationPartner (who owns the site), which Django carried and we'd otherwise have dropped.
  • stationGroups — the pages: slug, legacy slug, region, the stations the page covers, and the table layout as one row per reading naming the loggers that report it. That is where the 275 hand-ordered [stid, variable] pairs go, transposed into 242 rows — air_temp → Summit, Mid, Base instead of three separate entries. No station is on two pages, so nothing is configured twice.
  • stationRegions — index headings, ranked north to south. A collection rather than a select because that order matters; Django models the same thing as Area.order.
  • A SnowObs tab on Settings — source, token, a sync button and the pages switch. weatherPagesEnabled will replace the hardcoded STATIONS_TENANT_SLUG gate. Settings is already the per-tenant singleton, so this avoids a one-row collection that reads oddly in the admin. The token is encrypted with payload.encrypt, the same mechanism holding the MCP plugin's API keys, and field access is gated to global roles.
  • /api/cron/sync-stations — hourly, per tenant, upserting on (tenant, source, stid). Never deletes: groups point at these rows, and a station going quiet is a fault to report rather than a reason to forget it.
  • POST /api/settings/:id/sync-stations — the same sync on demand, behind the button, so a new source or token can be checked the moment it's saved instead of an hour later.
  • pnpm seed:weather — local-only. See below.

How to test

pnpm seed                     # tenants + content, schema pushed
pnpm seed:weather             # regions, stations, groups, SnowObs config
pnpm dev                      # Weather → Station Groups → Alpental Ski Area

Verified locally against live SnowObs:

  • 30 of 32 groups reproduce their legacy column layout exactly, checked by seeding every group through payload.create and reading it back at depth: 2. The two that differ hold the identical columns in a different order: Alpental and Crystal Green Valley interleaved a pair of snow readings per station, which one row per reading cannot express.
  • 242 column rows across the 32 pages, down from 275 flat [stid, variable] pairs.
  • Sync on a seeded database: created 0, updated 0, unchanged 60 in 410ms — no-op writes are skipped.
  • Corrupting a station's name and elevation, then re-running: updated 1, unchanged 59, values restored.
  • The sync endpoint returns 401 unauthenticated, 403 for another center's settings, and 400 with a readable message when the source or token is missing.
  • Token confirmed stored as ciphertext, not plaintext.

Migration Explanation

One migration creates the station tables, adds three snowobs_* columns to settings, and seeds 14 regions, 60 stations and 32 groups. Additive throughout — down() drops the tables and the columns.

It holds its own copy of the seed rather than importing the constants, matching 20260505_045200_backfill_nav_builtin_pages, so it keeps working once they're deleted. The station rows are a point-in-time snapshot; the sync corrects anything stale on its first run. If the nwac tenant is absent it logs and skips rather than failing the deploy.

That last point is why pnpm seed:weather exists. Locally the schema is pushed rather than migrated, and migrations that do run execute before /next/seed creates any tenant — so the backfill finds no nwac and skips. The script does the same work against a pushed database, building regions and groups from the constants and fetching stations through the real syncStations(). It's idempotent. Deployed environments are unaffected; the tenant already exists there.

Future enhancements / Questions

  • Next: page cutover — generateStaticParams and the index read from Payload, NWAC_WEATHER_STATION_GROUPS deleted. Then alerting.
  • 60 stations, not 54. SnowObs holds seven that no page shows: stid 15 (retired 2019) and six "5 minute" variants of Mission Ridge and Mt Hood Meadows. They report no battery voltage, so they're inert — but they're real and now visible.
  • The SnowObs tab is not seeded on deploy — it holds a credential, so NWAC's is filled in by hand in the admin. The sync skips any center without a source and token.
  • Two pages reorder a column or two (above). Worth a look from Dennis before the page cutover, since these are read daily.
  • seed:weather reads the legacy constants, which the page cutover deletes. It'll need repointing then — most likely at a dump of the real data.
  • Sync failures return 500 so a broken run surfaces rather than sitting in a log. Wiring that to Sentry is worth doing when alerting lands.
  • The 27k-line migration .json is Payload's generated schema snapshot, the same as every other migration in the repo.

@github-actions

github-actions Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Migration Safety Check

Found 13 potential issues:

20260822_185959_weather_station_collections.ts

Warning (line 1113): DELETE keyword detected - review for data loss

FOREIGN KEY (\`_parent_id\`) REFERENCES \`station_groups\`(\`id\`) ON UPDATE no action ON DELETE cascade

Warning (line 1133): DELETE keyword detected - review for data loss

FOREIGN KEY (\`tenant_id\`) REFERENCES \`tenants\`(\`id\`) ON UPDATE no action ON DELETE set null,

Warning (line 1134): DELETE keyword detected - review for data loss

FOREIGN KEY (\`region_id\`) REFERENCES \`station_regions\`(\`id\`) ON UPDATE no action ON DELETE set null

Warning (line 1156): DELETE keyword detected - review for data loss

FOREIGN KEY (\`parent_id\`) REFERENCES \`station_groups\`(\`id\`) ON UPDATE no action ON DELETE cascade,

Warning (line 1157): DELETE keyword detected - review for data loss

FOREIGN KEY (\`stations_id\`) REFERENCES \`stations\`(\`id\`) ON UPDATE no action ON DELETE cascade

Warning (line 1181): DELETE keyword detected - review for data loss

FOREIGN KEY (\`tenant_id\`) REFERENCES \`tenants\`(\`id\`) ON UPDATE no action ON DELETE set null

Warning (line 1209): DELETE keyword detected - review for data loss

FOREIGN KEY (\`tenant_id\`) REFERENCES \`tenants\`(\`id\`) ON UPDATE no action ON DELETE set null

Warning (line 1217): ALTER keyword detected - review for data loss

await db.run(sql`ALTER TABLE \`settings\` ADD \`snowobs_source\` text;`)

Warning (line 1218): ALTER keyword detected - review for data loss

await db.run(sql`ALTER TABLE \`settings\` ADD \`snowobs_token\` text;`)

Warning (line 1220): ALTER keyword detected - review for data loss

sql`ALTER TABLE \`settings\` ADD \`snowobs_weather_pages_enabled\` integer DEFAULT false;`,

Warning (line 1223): ALTER keyword detected - review for data loss

sql`ALTER TABLE \`payload_locked_documents_rels\` ADD \`station_groups_id\` integer REFERENCES station_groups(id);`,

Warning (line 1226): ALTER keyword detected - review for data loss

sql`ALTER TABLE \`payload_locked_documents_rels\` ADD \`station_regions_id\` integer REFERENCES station_regions(id);`,

Warning (line 1229): ALTER keyword detected - review for data loss

sql`ALTER TABLE \`payload_locked_documents_rels\` ADD \`stations_id\` integer REFERENCES stations(id);`,

Review these patterns and add backup/restore logic if needed. See docs/migration-safety.md for guidance.

@rchlfryn rchlfryn changed the title Add weather station collections WIP: Add weather station collections Aug 21, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Models the registry from src/constants/weatherStations.ts as data, ahead of
#1169. Nothing reads these yet.

Each station carries the readings it contributes to the NOW table; a group is
an ordered list of stations. Columns derive from the two, so the 275
hand-ordered pairs in the constants become zero rows of data. The selection
has to be per-station: 47 of 53 loggers report more than their table shows,
and availability can't stand in for it -- a station that goes offline would
silently lose its columns.

Graphs keep their own presets, hence tableVariables rather than variables.
Regions get a collection rather than a select because the index is ranked
north to south. Per-center SnowObs source, token and switches live on a
Settings tab.

The migration seeds 14 regions, 60 stations and 32 groups from the constants,
holding its own copy so it survives their deletion.
Upserts on (tenant, source, stid) for every center with weather pages
enabled. Never deletes: groups point at these rows, and a station going quiet
is a fault to report rather than a reason to forget it existed.

Skips no-op writes, so a run with nothing to do costs one query.
@rchlfryn rchlfryn changed the title WIP: Add weather station collections POC: Add weather station collections Aug 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant