POC: Add weather station collections - #1202
Conversation
Migration Safety CheckFound 13 potential issues: 20260822_185959_weather_station_collections.ts Warning (line 1113): DELETE keyword detected - review for data loss FOREIGN KEY (\`_parent_id\`) REFERENCES \`station_groups\`(\`id\`) ON UPDATE no action ON DELETE cascadeWarning (line 1133): DELETE keyword detected - review for data loss FOREIGN KEY (\`tenant_id\`) REFERENCES \`tenants\`(\`id\`) ON UPDATE no action ON DELETE set null,Warning (line 1134): DELETE keyword detected - review for data loss FOREIGN KEY (\`region_id\`) REFERENCES \`station_regions\`(\`id\`) ON UPDATE no action ON DELETE set nullWarning (line 1156): DELETE keyword detected - review for data loss FOREIGN KEY (\`parent_id\`) REFERENCES \`station_groups\`(\`id\`) ON UPDATE no action ON DELETE cascade,Warning (line 1157): DELETE keyword detected - review for data loss FOREIGN KEY (\`stations_id\`) REFERENCES \`stations\`(\`id\`) ON UPDATE no action ON DELETE cascadeWarning (line 1181): DELETE keyword detected - review for data loss FOREIGN KEY (\`tenant_id\`) REFERENCES \`tenants\`(\`id\`) ON UPDATE no action ON DELETE set nullWarning (line 1209): DELETE keyword detected - review for data loss FOREIGN KEY (\`tenant_id\`) REFERENCES \`tenants\`(\`id\`) ON UPDATE no action ON DELETE set nullWarning (line 1217): ALTER keyword detected - review for data loss await db.run(sql`ALTER TABLE \`settings\` ADD \`snowobs_source\` text;`)Warning (line 1218): ALTER keyword detected - review for data loss await db.run(sql`ALTER TABLE \`settings\` ADD \`snowobs_token\` text;`)Warning (line 1220): ALTER keyword detected - review for data loss sql`ALTER TABLE \`settings\` ADD \`snowobs_weather_pages_enabled\` integer DEFAULT false;`,Warning (line 1223): ALTER keyword detected - review for data loss sql`ALTER TABLE \`payload_locked_documents_rels\` ADD \`station_groups_id\` integer REFERENCES station_groups(id);`,Warning (line 1226): ALTER keyword detected - review for data loss sql`ALTER TABLE \`payload_locked_documents_rels\` ADD \`station_regions_id\` integer REFERENCES station_regions(id);`,Warning (line 1229): ALTER keyword detected - review for data loss sql`ALTER TABLE \`payload_locked_documents_rels\` ADD \`stations_id\` integer REFERENCES stations(id);`,Review these patterns and add backup/restore logic if needed. See |
|
Preview deployment: https://featxweather-station-collections.preview.avy-fx.org |
852a177 to
74e40e3
Compare
74e40e3 to
a06c249
Compare
a06c249 to
283d706
Compare
Models the registry from src/constants/weatherStations.ts as data, ahead of #1169. Nothing reads these yet. Each station carries the readings it contributes to the NOW table; a group is an ordered list of stations. Columns derive from the two, so the 275 hand-ordered pairs in the constants become zero rows of data. The selection has to be per-station: 47 of 53 loggers report more than their table shows, and availability can't stand in for it -- a station that goes offline would silently lose its columns. Graphs keep their own presets, hence tableVariables rather than variables. Regions get a collection rather than a select because the index is ranked north to south. Per-center SnowObs source, token and switches live on a Settings tab. The migration seeds 14 regions, 60 stations and 32 groups from the constants, holding its own copy so it survives their deletion.
Upserts on (tenant, source, stid) for every center with weather pages enabled. Never deletes: groups point at these rows, and a station going quiet is a fault to report rather than a reason to forget it existed. Skips no-op writes, so a run with nothing to do costs one query.
283d706 to
b4d0517
Compare
Description
Models the station registry currently living in
src/constants/weatherStations.tsas Payload collections, so the pages can become center-dynamic. Nothing reads these collections yet — this is inert on its own, and the page cutover follows separately.The registry describes pages, not stations: 17 of the 32 groups cover more than one logger, and the layout is a property of the combination rather than of any one station (Alpental lists Summit, Mid and Base temperatures together before moving on to humidity). So groups stay editorial and carry the table layout, while stations sync from SnowObs — which has no concept of a page, a slug or a column order.
Related Issues
First of three for #1169. Then the page cutover, then alerting.
Key Changes
stations— synced from SnowObsstation/metadata, fully read-only. IncludesweatherStationPartner(who owns the site), which Django carried and we'd otherwise have dropped.stationGroups— the pages: slug, legacy slug, region, the stations the page covers, and the table layout as one row per reading naming the loggers that report it. That is where the 275 hand-ordered[stid, variable]pairs go, transposed into 242 rows —air_temp → Summit, Mid, Baseinstead of three separate entries. No station is on two pages, so nothing is configured twice.stationRegions— index headings, ranked north to south. A collection rather than a select because that order matters; Django models the same thing asArea.order.Settings— source, token, a sync button and the pages switch.weatherPagesEnabledwill replace the hardcodedSTATIONS_TENANT_SLUGgate. Settings is already the per-tenant singleton, so this avoids a one-row collection that reads oddly in the admin. The token is encrypted withpayload.encrypt, the same mechanism holding the MCP plugin's API keys, and field access is gated to global roles./api/cron/sync-stations— hourly, per tenant, upserting on(tenant, source, stid). Never deletes: groups point at these rows, and a station going quiet is a fault to report rather than a reason to forget it.POST /api/settings/:id/sync-stations— the same sync on demand, behind the button, so a new source or token can be checked the moment it's saved instead of an hour later.pnpm seed:weather— local-only. See below.How to test
Verified locally against live SnowObs:
payload.createand reading it back atdepth: 2. The two that differ hold the identical columns in a different order: Alpental and Crystal Green Valley interleaved a pair of snow readings per station, which one row per reading cannot express.[stid, variable]pairs.created 0, updated 0, unchanged 60in 410ms — no-op writes are skipped.updated 1, unchanged 59, values restored.Migration Explanation
One migration creates the station tables, adds three
snowobs_*columns tosettings, and seeds 14 regions, 60 stations and 32 groups. Additive throughout —down()drops the tables and the columns.It holds its own copy of the seed rather than importing the constants, matching
20260505_045200_backfill_nav_builtin_pages, so it keeps working once they're deleted. The station rows are a point-in-time snapshot; the sync corrects anything stale on its first run. If thenwactenant is absent it logs and skips rather than failing the deploy.That last point is why
pnpm seed:weatherexists. Locally the schema is pushed rather than migrated, and migrations that do run execute before/next/seedcreates any tenant — so the backfill finds nonwacand skips. The script does the same work against a pushed database, building regions and groups from the constants and fetching stations through the realsyncStations(). It's idempotent. Deployed environments are unaffected; the tenant already exists there.Future enhancements / Questions
generateStaticParamsand the index read from Payload,NWAC_WEATHER_STATION_GROUPSdeleted. Then alerting.seed:weatherreads the legacy constants, which the page cutover deletes. It'll need repointing then — most likely at a dump of the real data..jsonis Payload's generated schema snapshot, the same as every other migration in the repo.