diff --git a/AGENTS.md b/AGENTS.md index a1c641a..852f127 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -15,15 +15,15 @@ accepted ISAM 2026 demo paper. > [!IMPORTANT] > **This repository is in maintenance mode.** v5 is the live Cornell Tech -> deployment and stays on Notion. Active development continues in a separate -> repository — see below. Changes here should be fixes and operational -> improvements, not new architecture. +> deployment. Active development continues in a separate repository — see +> below. Changes here should be fixes and operational improvements, not new +> architecture. ## Repository layout — two apps | Path | Version | Status | Data layer | |---|---|---|---| -| `v5/` | v5 | **Live — work here** | Notion (`NOTION_DB_*`) | +| `v5/` | v5 | **Live — work here** | Postgres (`DATABASE_URL`); Notion read only by the one-time import | | `src/` (root) | v4 | Legacy, frozen | AirTable (`AIRTABLE_TABLE_*`) | **Default to `v5/` unless a task explicitly names v4.** The root app is retained @@ -62,7 +62,7 @@ Run from `v5/`: ```bash npm run dev # dev server (:3000) -npm run build # production build +npm run build # runs db:migrate, then production build npm run lint # eslint npm run typecheck # tsc --noEmit npm test # vitest (unit + integration + component) @@ -70,8 +70,9 @@ npm run test:e2e # playwright (needs: npx playwright install chromium) npm run test:all # lint + typecheck + vitest + playwright ``` -`npm run test:all` must pass before any merge. It needs no credentials — the -mock catalog and MSW cover every external service. +`npm run test:all` must pass before any merge. It needs no credentials — an +in-process PGlite database (seeded with demo data) covers Postgres and MSW +covers every other external service. ## Conventions @@ -90,8 +91,11 @@ mock catalog and MSW cover every external service. the default Node runtime. - The in-memory rate limiter is a per-process singleton and resets on cold start. Upstash backs it only when **both** `UPSTASH_REDIS_REST_*` vars are set. -- The mock-catalog fallback triggers when **any** Notion env var is missing *or* a - fetch throws. Great for tests; means a misconfigured deploy fails soft and - silently serves mock data. Check the logs before concluding the catalog is empty. +- `DATABASE_URL` unset serves an in-process PGlite database seeded with demo + data — great for tests and a fresh clone, but check that it's actually set + in a real deploy before concluding the catalog is empty. Unlike the old + Notion fallback, a **configured but unreachable** Postgres never falls back + to demo data — it fails toward stale cache or an explicit error state + (constitution Article 4). - Scripts under `v5/scripts/` run via `node --experimental-strip-types`. They are migration and maintenance tools, not part of the app build. diff --git a/CLAUDE.md b/CLAUDE.md index 7bdc9be..7f3751b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -7,9 +7,9 @@ one place stops the two documents from drifting apart. Two things to know before anything else: -1. **The active app is `v5/`** (Notion-backed), not the root `src/` tree (v4, - AirTable, frozen). When working in `v5/`, also read - [`v5/AGENTS.md`](./v5/AGENTS.md). +1. **The active app is `v5/`** (Postgres-backed, with Notion read only by the + one-time import), not the root `src/` tree (v4, AirTable, frozen). When + working in `v5/`, also read [`v5/AGENTS.md`](./v5/AGENTS.md). 2. **This repository is in maintenance mode.** v5 is the live Cornell Tech deployment. Next-generation development happens in a separate repository — see `AGENTS.md`. diff --git a/docs/handover.md b/docs/handover.md index 676826c..a96b648 100644 --- a/docs/handover.md +++ b/docs/handover.md @@ -13,19 +13,24 @@ ## 1. What this system is, in operational terms A website that lists the lab's machines and answers questions about them. The data lives in -**Notion** — seven databases that staff edit directly. The website reads from Notion and -adds an AI assistant on top. +**Postgres** (Neon, provisioned through Vercel) and the website reads from there, with an AI +assistant on top. **The website no longer reads Notion.** Notion was the source of truth +before this phase; its seven databases were read once, by a developer, to move everything +into Postgres, and are not read again. A one-way mirror back into an admin's own Notion +workspace is planned for a later phase. -**Editing the catalogue means editing Notion.** There is no admin login and no dashboard, -by design. If a machine's description is wrong, you fix it in Notion, and the website -catches up within minutes. +**There is no admin login or dashboard yet, so there is currently no way to edit the +catalogue from the website.** Editing the old Notion databases no longer has any effect — +the website stopped reading them. Until inventory editing ships on the website, a wrong +record needs a developer **[dev]** to fix directly in Postgres. Three moving parts, and it is worth knowing which is which when something is wrong: | Part | What it is | Who provides it | |---|---|---| | **The website** | The app itself | Hosted on Vercel | -| **The data** | Seven databases | Notion workspace | +| **The data** | Tools, units, categories, locations, resources, tickets, projects | Neon Postgres (provisioned through Vercel) | +| **Files** | Tool images, manuals, project photos | Vercel Blob | | **The assistant** | The AI | Anthropic API (Claude) | --- @@ -38,14 +43,15 @@ owner. | Thing | Where | Who owns it | Notes | |---|---|---|---| | Vercel project | vercel.com | ⬜ **TBD** | Hosting, deploys, env vars, logs | -| Notion workspace | notion.so | ⬜ **TBD** | The seven databases | -| Notion integration token | Notion settings | ⬜ **TBD** | `NOTION_API_KEY` | +| Neon Postgres | Vercel → Storage → Marketplace | ⬜ **TBD** | **The catalogue's source of truth.** Provisioned through the Vercel Marketplace; sets `DATABASE_URL` automatically | +| Notion workspace | notion.so | ⬜ **TBD** | The seven original databases. No longer read by the website — kept for reference until a one-way mirror is set up | +| Notion integration token | Notion settings | ⬜ **TBD** | `NOTION_API_KEY`. Needed only to re-run the one-time import; not required for the website to run | | `AI_GATEWAY_API_KEY` | Vercel → AI Gateway | ⬜ **TBD** | **The intended production path.** Model spend on the hosting invoice, with a platform spend limit | | Anthropic API key | console.anthropic.com | ⬜ **TBD** | **Fallback.** Costs money per use. Keep it — it is the lever back if the gateway fails | | GitHub repository | github.com | ⬜ **TBD** | The code | | Domain / DNS | ⬜ | ⬜ **TBD** | | | `ADMIN_REVALIDATE_SECRET` | Vercel env vars | ⬜ **TBD** | Forces the site to refresh | -| Vercel Blob store | Vercel → Storage | ⬜ **TBD** | Holds the nightly Notion backup. Sets `BLOB_READ_WRITE_TOKEN`. **Contains student PII — keep private** | +| Vercel Blob store | Vercel → Storage | ⬜ **TBD** | Holds tool images, manuals and project photos (public) alongside maintenance photos and the nightly backup (private). Sets `BLOB_READ_WRITE_TOKEN`. **The private files contain student PII — keep those private** | | `CRON_SECRET` | Vercel env vars | ⬜ **TBD** | Lets the nightly backup cron prove it is Vercel (§3) | > [!WARNING] @@ -96,20 +102,29 @@ for what remains to verify before production traffic goes through it. ### Add a machine to the catalogue -1. In Notion, add a row to the **Tools** database. -2. Fill in at least: name, description, category, location. -3. Add units in the **Units** database, linked to the tool — one row per physical machine, - with serial and status. -4. Add manuals or SOPs to **Resources**, linked to the tool. -5. Tick **published**. Until you do, it will not appear on the site. -6. Wait a few minutes, or force a refresh (§4). - -**The assistant picks it up automatically.** No deploy, no developer. +> [!WARNING] +> **This no longer works.** The steps below edited Notion, which the website read. The +> website now reads Postgres and does not read Notion at all, and the on-website editor that +> replaces this is not built yet. Adding a machine today needs a developer **[dev]** to +> insert it directly in Postgres. Once inventory editing ships on the website, this section +> will be rewritten to match it. +> +> ~~1. In Notion, add a row to the **Tools** database.~~ +> ~~2. Fill in at least: name, description, category, location.~~ +> ~~3. Add units in the **Units** database, linked to the tool — one row per physical machine, +> with serial and status.~~ +> ~~4. Add manuals or SOPs to **Resources**, linked to the tool.~~ +> ~~5. Tick **published**. Until you do, it will not appear on the site.~~ +> ~~6. Wait a few minutes, or force a refresh (§4).~~ ### Mark a machine out of service -Change the unit's **status** in the Units database to `Under Maintenance` or `Out of -Service`. The site shows it as offline, and the assistant stops recommending it. +> [!WARNING] +> **This no longer works** — editing Notion has no effect on the website (see above). +> ~~Change the unit's **status** in the Units database to `Under Maintenance` or `Out of +> Service`. The site shows it as offline, and the assistant stops recommending it.~~ Until +> inventory editing ships on the website, this needs a developer **[dev]** to change directly +> in Postgres. ### Handle a maintenance ticket @@ -125,10 +140,12 @@ it and how often, and write that down here: ### Fix something the assistant got wrong -Almost always a data problem, not an AI problem. The assistant answers from Notion, so a -wrong answer usually means a wrong or empty field. Fix the record; the answer changes. +Almost always a data problem, not an AI problem. The assistant answers from Postgres, so a +wrong answer usually means a wrong or empty field there — which, until inventory editing +ships on the website, needs a developer **[dev]** to fix directly. Editing the old Notion +record does nothing; the website does not read it. -If it is wrong *and* the Notion record is right, that is a real bug — see §6. +If it is wrong *and* the underlying record is right, that is a real bug — see §6. ### Change branding, colours, or the assistant's name @@ -185,7 +202,8 @@ There is no automated restore, on purpose — it is far more work than the failu ## 4. Forcing the site to refresh -The site caches Notion data for a few minutes. To make a change appear immediately: +The site caches catalogue data from Postgres for a few minutes. To make a change appear +immediately: ``` POST https:///api/admin/revalidate @@ -204,7 +222,8 @@ effect. |---|---|---| | Anthropic console | **Spend.** Set a limit and an alert. | Weekly, at minimum | | Vercel dashboard | Failed deploys, function errors | When something looks wrong | -| Vercel logs | `Falling back to mock catalog` | Whenever the catalogue looks odd | +| Vercel logs | `DbUnavailableError` (Postgres unreachable) | Whenever the catalogue looks odd | +| Vercel → Storage | The Neon database is reachable | Whenever the catalogue looks odd | | Vercel → Cron Jobs | The nightly backup ran green | Monthly — see §3 | | Notion: Maintenance_Logs | Open tickets | Per §3 | @@ -217,25 +236,33 @@ recoverable; an unbounded bill is not. ### The site shows machines the lab doesn't own -**Most likely cause, and it is not obvious.** The app falls back to a built-in demo -catalogue whenever it cannot reach Notion — a missing or misspelled environment variable, an -expired integration token, or a database that stopped being shared with the integration. +**Now much less likely, and it no longer fails silently.** Before this phase, the app fell +back to a built-in demo catalogue whenever it could not reach Notion, with no visible error. +That is no longer how a database problem shows up: -The site does **not** show an error. It looks perfectly healthy while serving fictional -equipment. +- If `DATABASE_URL` is missing from the Vercel project — a genuine misconfiguration — the + site serves the same small built-in demo catalogue used in development and testing, and + shows a banner saying so. +- If `DATABASE_URL` is set but Postgres cannot be reached, the site keeps serving whatever it + last cached rather than switching to demo data, and any page that isn't cached shows an + error instead of inventing machines. -1. Check the Vercel logs for `Falling back to mock catalog`. -2. Check every `NOTION_DB_*` variable and `NOTION_API_KEY` in Vercel. -3. In Notion, confirm each database is still shared with the integration. +1. Check the Vercel logs for `DbUnavailableError`. +2. Check `DATABASE_URL` is set in the Vercel project's environment variables. +3. Check the Neon dashboard (Vercel → Storage) — a suspended or deleted branch is the usual + cause. **[dev]** ### A machine is missing from the site -Check `published` is ticked in Notion. Then force a refresh (§4). +Check `published` is set on the tool in Postgres. Then force a refresh (§4). Until inventory +editing ships on the website, changing `published` needs a developer **[dev]**. -### A field is empty on the site but filled in Notion +### A field is empty on the site but filled in the old Notion databases -Someone probably renamed the property in Notion. The app matches property names and cannot -follow a rename. Compare against `v5/.env.example` and the other rows. **[dev]** if unclear. +The website reads Postgres, not Notion, so this means the one-time import either mapped that +field differently than expected or the field has since been edited in Postgres. **[dev]**: +compare the row in Postgres against the Notion page it was imported from +(`notion_page_id` on the row) to see where the two diverge. ### The assistant is down or erroring @@ -279,10 +306,14 @@ The live app is `v5/`. The root `src/` directory is the old v4 app and is not us Nothing about Cornell is hardcoded. -1. Duplicate the seven Notion databases into the new workspace. -2. Create a Notion integration and share all seven databases with it. -3. Deploy the repo to Vercel. -4. Set `NOTION_API_KEY`, the seven `NOTION_DB_*` IDs, and `ANTHROPIC_API_KEY`. +1. Deploy the repo to Vercel. +2. Add Neon Postgres and a Blob store to the project through the Vercel Marketplace / + Storage tab — this sets `DATABASE_URL` and `BLOB_READ_WRITE_TOKEN` automatically. Run + `npm run db:migrate` once to apply the schema. +3. Set `ANTHROPIC_API_KEY` (or `AI_GATEWAY_API_KEY`, §2). +4. **Only if migrating an existing Notion-based catalogue:** set `NOTION_API_KEY` and the + seven `NOTION_DB_*` IDs, and run `npm run import:notion` once against the new database. A + brand-new lab with no existing data skips this step entirely. 5. Override the `NEXT_PUBLIC_*` branding variables. 6. Replace the logo in `v5/public/`. @@ -293,9 +324,13 @@ Full variable list with explanations: `v5/.env.example`. ## 9. Known limitations — say these out loud at handover - **No sign-in yet.** The assistant is open to anyone with the URL. Specced, not built. -- **The fallback is silent** (§6). Fix specced in the operational-hardening spec. -- **No analytics.** Notion cannot aggregate, so there is no way to see which machines get - asked about most. This is the main reason a successor project exists. +- **No catalogue editing on the website yet.** Postgres is the source of truth, but the + admin inventory pages that let staff edit it there are a later phase (§6, §8). Until then, a + wrong or missing record needs a developer. +- **The old silent fallback is fixed.** A configured-but-unreachable database now fails + toward stale cached data or an explicit error, never toward invented equipment (§6). +- **No analytics.** There is no way to see which machines get asked about most. This is the + main reason a successor project exists. - **No backup beyond Notion's own version history.** Notion keeps page history; there is no separate export. Consider a periodic manual export of the databases. - **One person built this.** That is the risk this document exists to reduce. If something diff --git a/docs/specs/2026-09-14-v5-data-platform-design.md b/docs/specs/2026-09-14-v5-data-platform-design.md index 65d5c54..c9cfa90 100644 --- a/docs/specs/2026-09-14-v5-data-platform-design.md +++ b/docs/specs/2026-09-14-v5-data-platform-design.md @@ -1012,3 +1012,20 @@ Appended per [`DRIFT.md`](DRIFT.md). Original text above is never edited — the - **`db:migrate` is not yet part of the Vercel build.** Phase 2 adds it to the build command when Neon exists; until then a build needs no database. **Status.** Accepted. Each is a detail the spec left to implementation. + +### 2026-09-15 — Phase 2 built, with as-built details + +**What changed.** Phase 2 (§9) is implemented: every read path is on Postgres through `src/lib/data/` (catalogue, projects, resources, maintenance history), `src/lib/catalog.ts` and `src/lib/projects.ts` keep their exports and their `"use cache"` tags over the new modules, the health route probes Postgres, `/tools/` redirects to the slug, the QR-label script reads Postgres, and the repo docs say Postgres. `npm run build` now runs `db:migrate` first. Details the spec did not name: + +- **A bridge from Postgres ids to Notion page ids, for the writes that are still on Notion.** Every id the app hands around is now a Postgres uuid, but a correction, a maintenance ticket and a project submission still create Notion pages whose relations address Notion *page* ids, and Notion refuses a relation it cannot find. `src/lib/data/notion-ids.ts` reads the page id the import recorded on `tools` and `units`; a row that has none (created after the import, or the demo seed) is written *without* the relation and logged, never with an id Notion would reject. The module leaves with those writes in Phase 3. +- **Tool and project pages are no longer prerendered at build.** `generateStaticParams` needed a reachable database during `next build`, which §3.2 says a build must not need, and Cache Components rejects an empty list. The pages render on first request and are then served from the `catalog` / `projects` cache tags. Two visible consequences: an unknown slug and a legacy-id redirect are resolved inside the dynamic part of a partially prerendered page, so the HTTP status is 200 and the browser performs the not-found render or the redirect; printed QR labels still land on the right page. A real 404 status and a real 308 are a follow-up, not a Phase 2 goal. +- **E2E runs against a production build, with one retry.** Under `next dev`, Playwright's fourteen parallel workers triggered first compiles of several routes at once and Turbopack rewrote the root layout's client chunk mid-response (`ERR_CONTENT_LENGTH_MISMATCH`), so pages intermittently never hydrated and four tests failed on every run. The Playwright web server is now `npm run build && next start` with `DATABASE_URL` and the Notion env blanked, which also exercises the bundled server with PGlite — the same configuration a Vercel preview without a database runs. That leaves about one test execution in a hundred failing, always the same shape: a `Link` click or the locale switch whose request is sent and answered within milliseconds (the trace shows the RSC fetch completing) but whose result the App Router never commits — the URL or the rendered locale stays put for the full assertion timeout. The server is not the bottleneck (56 concurrent requests answer in ~150 ms), no console error is logged, and a hydration-marker wait made no difference, so it is filed as a client-router race under load rather than an app defect. `retries: 1` keeps the gate green and Playwright reports such a test as *flaky*, so the count stays visible instead of hidden. Worth revisiting on the next Next.js upgrade. +- **`/api/health` reports `database: "ok" | "unreachable" | "demo"`** and `catalog: "live" | "demo"` in place of the Notion fields; a demo substrate is `status: "ok"`, an unreachable database `degraded`, and no error text ever reaches the body. +- **Cache tags.** Every catalogue read is tagged `catalog` only; §3.9's `tool:` tag on detail reads waits for the write paths that would invalidate it, which is also where the choice of key (slug or uuid) belongs. +- **`src/lib/data/uuid.ts`** guards every id lookup so free text from a tool call never reaches Postgres as a bad uuid cast. `listResources()` has no production caller yet; the chat route uses `listResourcesForTool()`. +- **Retired but not deleted:** `src/components/mock-catalog.ts` (only the offline eval fixtures still pin its data), the Notion read helpers in `src/lib/notion.ts`, `test/fixtures/catalog.ts`, and the four Notion maintenance scripts §3.10 lists. Each deletion is proposed separately (constitution working agreements). +- **English fallback for messages is in** (§6, Article 6 as amended): `src/i18n/request.ts` lays each locale file over `en.json`, so a string added only to English renders in English elsewhere. The first such strings are the sign-in notices in the header, which now say why a click on *Sign in* went nowhere instead of doing nothing (this deployment has no sign-in configured, or the request failed). +- **Project URLs use the slug.** `MakerLabProject` gains `slug`, the gallery and a tool's "built with this" link to `/projects/`, and the project page accepts either the slug or the uuid. +- **The demo seed has one published sample project** (a laser-cut lamp built with both demo tools, with two bundled photos under `public/sample-projects/`, materials and an outside link) and gives the Trotec a photo attachment, since its bundled image file is not named after the tool. The gallery, the project page and "built with this" are therefore exercised end to end without a database. + +**Status.** Accepted. The bridge and the prerender change are the two a reader should know about before Phase 3. diff --git a/v5/.gitignore b/v5/.gitignore index ed3d3e8..a354760 100644 --- a/v5/.gitignore +++ b/v5/.gitignore @@ -10,3 +10,5 @@ tsconfig.tsbuildinfo /playwright-report/ /blob-report/ /playwright/.cache/ +.vercel +.env* diff --git a/v5/AGENTS.md b/v5/AGENTS.md index e2b8f56..fe0b927 100644 --- a/v5/AGENTS.md +++ b/v5/AGENTS.md @@ -6,8 +6,8 @@ active application. Everything below is scoped to the `v5/` directory. > [!IMPORTANT] > The **root `CLAUDE.md` describes v4** — an AirTable-backed app with > `AIRTABLE_TABLE_*` env vars. **That does not apply to v5.** v5's data layer is -> **Notion** (`NOTION_DB_*`). When working in `v5/`, follow this file, not the -> root v4 doc. +> **Postgres** (`DATABASE_URL`), not Notion and not AirTable. When working in +> `v5/`, follow this file, not the root v4 doc. ## What this is @@ -24,23 +24,54 @@ the catalog to external agents. White-labelled via env vars. - **MCP:** `@modelcontextprotocol/sdk` (HTTP JSON-RPC server at `/api/mcp`). - **Validation:** `zod`. **Search:** `match-sorter` (fuzzy, ranked). -## Data layer — Notion (not AirTable) - -Seven Notion databases, IDs supplied via env (`NOTION_DB_TOOLS`, `_CATEGORIES`, -`_LOCATIONS`, `_UNITS`, `_RESOURCES`, `_MAINTENANCE_LOGS`, `_FLAGS`) plus -`NOTION_API_KEY`. See `.env.example` for the full list and defaults. - -- `src/lib/notion.ts` — server-only Notion REST client: page→record parsers, pagination, 429 retry, `createMaintenanceLog`. Tolerant of snake_case **and** Title-case property names. -- `src/lib/catalog.ts` — orchestration: fetch + join + derive `MakerLabTool`s, cached with `cacheTag("catalog")` / `cacheLife("minutes")`. -- **Mock-catalog fallback (important):** `getCatalogTools()` / `getCatalogTool(id)` return the built-in `src/components/mock-catalog.ts` data whenever `hasNotionCatalogEnv()` is false (**any** Notion env var missing) **or** a fetch throws. So the app — and the whole E2E/test suite — runs with no Notion creds. +## Data layer — Postgres (not Notion, not AirTable) + +**Postgres is the source of truth (constitution Article 7).** Neon Postgres, +provisioned through the Vercel Marketplace, reached through Drizzle ORM. +`DATABASE_URL` set → Neon. `DATABASE_URL` unset → an in-process PGlite +database, migrated and seeded with demo data on first use (two tools: "Form +4", "Trotec Speedy 400"), so a fresh clone, local dev, and the whole test +suite run with no credentials and no network. See `.env.example` for the full +variable list. + +- `src/lib/db/client.ts` — the one entry point: `getDb()` (a Drizzle handle), + `dataSubstrate()` (`"neon" | "pglite-demo"`), `pingDb()` (throws + `DbUnavailableError`), `resetDbForTests()`. +- `src/lib/db/schema/index.ts` — tables and vocabulary constants (stored + snake_case, e.g. `in_use`; display text is derived, never stored). +- `src/lib/catalog.ts` — orchestration: fetch + join + derive `MakerLabTool`s + from Postgres, cached with `cacheTag("catalog")` / `cacheLife("minutes")`. +- `src/lib/data/*.ts` — the query modules underneath: `catalog.ts`, + `projects.ts`, `maintenance.ts`, `resources.ts`, plus `uuid.ts` (the shape + guard every untrusted id passes before it reaches a uuid column) and + `notion-ids.ts`. Relative imports with `.ts` extensions, no `@/` alias, no + `"server-only"` — `scripts/` loads them under plain Node. +- **Notion is read only by the one-time import** (`npm run import:notion`). + No request path reads Notion. A one-way mirror (app → an admin's own Notion + workspace) is a later phase, not built yet. +- **The three writes still on Notion address pages, not rows.** A correction, + a maintenance ticket and a project submission create Notion pages whose + `relation` properties need *Notion page ids*, while everything the app hands + around is now a Postgres uuid. `src/lib/data/notion-ids.ts` translates + through `notion_page_id`, and a row that has none is written without the + relation rather than with an id Notion would reject — a ticket a human has + to link by hand beats a ticket that never arrived (Article 4). It goes away + with those writes in Phase 3. +- **Files** (tool images, manuals, project photos) live in **Vercel Blob**, + not Notion attachments — see `next.config.ts`'s `images.remotePatterns`. +- **Failing toward stale, not wrong (Article 4).** `DATABASE_URL` unset serves + the PGlite demo seed with `DemoDataBanner` shown. `DATABASE_URL` set but + unreachable never falls back to demo or invented data — cached pages keep + serving and an uncached read renders the error state. ## Key files | Path | Purpose | |---|---| | `src/lib/site-config.ts` | White-label branding (env-driven, all have defaults) | -| `src/lib/notion.ts` | Notion API client (server-only) | -| `src/lib/catalog.ts` | Catalog orchestration + cache + mock fallback | +| `src/lib/db/client.ts` | `getDb()`, `dataSubstrate()`, `pingDb()` — the one entry point to Postgres/PGlite | +| `src/lib/notion.ts` | Notion API client — write paths still on Notion until Phase 3 (tickets, corrections, project submission); no request path reads it | +| `src/lib/catalog.ts` | Catalog orchestration + cache, reading Postgres | | `src/lib/rate-limit.ts` | In-memory (or Upstash) sliding-window limiter | | `src/lib/types.ts` / `src/components/catalog-types.ts` | Notion record types / resolved view types | | `src/app/api/chat/route.ts` | Claude chat: streaming, tools (`get_unit_details`, `report_issue`, `web_fetch`), PDF manual attach | @@ -62,8 +93,8 @@ Seven Notion databases, IDs supplied via env (`NOTION_DB_TOOLS`, `_CATEGORIES`, ## Testing -Comprehensive, **fully-mocked** suite (no live Notion/Anthropic/Redis). Run -everything with one command: +Comprehensive, **fully-mocked** suite (no live Notion/Anthropic/Redis/Postgres — +Article 3). Run everything with one command: ```bash npm run test:all # lint + typecheck + vitest + playwright @@ -74,7 +105,10 @@ Or individually: `npm test` (Vitest: unit + integration + component), first), `npm run test:coverage`. - **Vitest** (jsdom) + React Testing Library + MSW; **Playwright** for E2E. -- E2E boots its own server on **port 3100** with `NOTION_*` unset (mock catalog) and intercepts `/api/chat` — it never touches your `:3000` dev server or real services. +- Catalogue reads run against an in-process PGlite database seeded with demo + data (`getCatalogTools()` needs no Notion env at all); write paths still on + Notion (§ above) are covered with `vi.stubEnv` + MSW as before. +- E2E boots its own server on **port 3100** with `DATABASE_URL` unset (PGlite demo catalog) and intercepts `/api/chat` — it never touches your `:3000` dev server or real services. - Tests are colocated (`*.test.ts(x)` next to source); shared harness in `test/`. - **Read these before writing tests:** `TESTING.md` (runbook), `test/README.md` (harness internals + the `streamText`-capture and env-stubbing patterns), and `docs/specs/2026-05-29-v5-test-suite-design.md` (design + coverage matrix). The harness deps/scripts are already wired — don't hand-edit `package.json` for them. @@ -82,7 +116,7 @@ first), `npm run test:coverage`. ```bash npm run dev # dev server (:3000) -npm run build # production build +npm run build # runs db:migrate, then production build npm run lint # eslint npm run typecheck # tsc --noEmit npm run test:all # full test suite diff --git a/v5/TESTING.md b/v5/TESTING.md index 43bced1..2babbef 100644 --- a/v5/TESTING.md +++ b/v5/TESTING.md @@ -9,15 +9,15 @@ import paths), see [`test/README.md`](./test/README.md). ## Overview The suite has four layers, all **fully mocked — no live services**. Tests never -hit Notion, Anthropic, or Upstash; there are no API keys and no network cost. -Everything is deterministic and offline. +hit Postgres, Notion, Anthropic, or Upstash; there are no API keys and no +network cost. Everything is deterministic and offline. | Layer | What | Where it lives | Runner | |---|---|---|---| | Unit | `src/lib`, `src/i18n` pure logic | colocated `*.test.ts` next to source | Vitest | | Integration | API routes (`/api/*`) with HTTP + module mocks | colocated `route.test.ts` next to the route | Vitest | | Component | React UI via React Testing Library | colocated `*.test.tsx` next to the component | Vitest | -| E2E | Full app against the mock catalog | `e2e/*.spec.ts` | Playwright | +| E2E | Full app against the PGlite demo catalog | `e2e/*.spec.ts` | Playwright | Shared infra lives in `test/` (MSW server + handlers, fixtures, mocks, the RTL `render` helper). Tests import from there; **don't edit `package.json` or run @@ -58,16 +58,21 @@ Playwright boots its own dev server (see E2E notes below), so no separate `test/mocks/next-cache.ts`. - **`server-only`** is aliased to an empty stub (`test/mocks/server-only.ts`) in `vitest.config.ts`, so `rate-limit.ts` and its importers load under Vitest. -- **Mock-catalog fallback rule.** `getCatalogTools()` / `getCatalogTool(id)` - return the built-in `src/components/mock-catalog.ts` data when - `hasNotionCatalogEnv()` is false — i.e. **any** of the 8 Notion env vars - (`NOTION_API_KEY` + the 7 `NOTION_DB_*`) is missing — and also on any thrown - error during fetch. So: - - **Mock path** (default in tests): leave `NOTION_*` unset → mock data, no MSW - needed. - - **Real Notion path**: `vi.stubEnv` all 8 vars (set `NOTION_DB_*` to the - `DB_IDS` sentinels from `test/msw/handlers.ts`) → MSW serves - `api.notion.com`. See the `stubNotionEnv()` helper in `test/README.md`. +- **The PGlite rule.** `getCatalogTools()` / `getCatalogTool(id)` read from an + in-process **PGlite** database — real Postgres, compiled to WebAssembly, + migrated and seeded with demo data (two tools: `form-4`, + `trotec-speedy-400`) — whenever `DATABASE_URL` is unset, which is the + default in every test. So: + - **Demo-seed path** (default in tests): leave `DATABASE_URL` unset → the + seeded PGlite database, no MSW needed, no Notion env at all. Put + `// @vitest-environment node` at the top of any file that touches PGlite. + - **An isolated database**: `createPgliteDb()` from `src/lib/db/pglite.ts` + returns a fresh instance for a test that seeds its own rows. + - **Write paths still on Notion this phase** (tickets, corrections, project + submission, uploads, intake): `vi.stubEnv` all 8 Notion vars (set + `NOTION_DB_*` to the `DB_IDS` sentinels from `test/msw/handlers.ts`) → MSW + serves `api.notion.com`. See the `stubNotionEnv()` helper in + `test/README.md`. ## How to add a test @@ -147,18 +152,17 @@ directly. Also mock `@ai-sdk/anthropic`. The full verified snippet is in ## E2E notes -- Playwright's `webServer` boots `npx next dev -p 3100` with all `NOTION_*` vars - set to `""`, so `hasNotionCatalogEnv()` is false and the app serves the - built-in mock catalog (`src/components/mock-catalog.ts`) regardless of your +- Playwright's `webServer` boots `npx next dev -p 3100` with `DATABASE_URL` + unset, so the app serves the seeded PGlite demo database regardless of your dev shell's environment. `testDir` is `./e2e`; `baseURL` is `http://localhost:3100`. - `/api/chat` is intercepted **inside each spec** at the network layer via `page.route()` returning a UI-message stream chunk — no real Anthropic call. - `reuseExistingServer: false` — Playwright **always** boots its own fresh - mock-backed server on the dedicated port 3100. This means E2E never collides - with (or accidentally reuses) a `next dev` you have running on the default - port 3000 against real Notion, so results are deterministic no matter what you - have running locally. Port 3000 is left untouched. + PGlite-backed server on the dedicated port 3100. This means E2E never + collides with (or accidentally reuses) a `next dev` you have running on the + default port 3000 against a real database, so results are deterministic no + matter what you have running locally. Port 3000 is left untouched. ## Coverage diff --git a/v5/e2e/corrections.spec.ts b/v5/e2e/corrections.spec.ts index 31d9b44..dadd94c 100644 --- a/v5/e2e/corrections.spec.ts +++ b/v5/e2e/corrections.spec.ts @@ -10,11 +10,12 @@ import { test, expect } from "@playwright/test"; // intercepts that request with page.route() instead — the same technique // chat.spec.ts uses — so nothing leaves the machine. // -// Mock catalog: "Form 4" is slug `form-4`, Notion id `tool-form-4` -// (src/components/mock-catalog.ts). Strings are `flag.*` in messages/en.json. +// Demo catalogue: "Form 4" is slug `form-4`; its id is the Postgres uuid the +// seed assigned (src/lib/db/demo-seed.ts). Strings are `flag.*` in +// messages/en.json. const FLAG_ENDPOINT = "**/api/flags"; -const TOOL_ID = "tool-form-4"; +const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/; test.describe("Report a correction", () => { test("the control is reachable from a tool detail page", async ({ page }) => { @@ -102,9 +103,9 @@ test.describe("Report a correction", () => { ).toHaveCount(0); // The report identifies the specific field, not just the tool (spec §2), and - // carries the tool's Notion id rather than its slug. + // carries the tool's database id rather than its slug. expect(submitted).toMatchObject({ - tool_id: TOOL_ID, + tool_id: expect.stringMatching(UUID), field_flagged: "description", issue_description: "The build volume says 220 mm, but it is 256 mm.", }); diff --git a/v5/e2e/gallery.spec.ts b/v5/e2e/gallery.spec.ts index cc71e35..ed833f7 100644 --- a/v5/e2e/gallery.spec.ts +++ b/v5/e2e/gallery.spec.ts @@ -1,11 +1,11 @@ import { test, expect } from "@playwright/test"; -// The app boots with NOTION_* unset (see playwright.config.ts webServer.env), -// so getCatalogTools() serves the built-in mock catalog -// (src/components/mock-catalog.ts): "Form 4" and "Trotec Speedy 400". +// The app boots with DATABASE_URL and NOTION_* unset (see playwright.config.ts +// webServer.env), so getCatalogTools() serves the PGlite demo seed +// (src/lib/db/demo-seed.ts): "Form 4" and "Trotec Speedy 400". test.describe("Gallery", () => { - test("loads at / and shows the mock-catalog tools", async ({ page }) => { + test("loads at / and shows the demo-catalogue tools", async ({ page }) => { await page.goto("/"); // Gallery heading from messages/en.json gallery.title => "TOOLS // MACHINES". @@ -34,7 +34,7 @@ test.describe("Gallery", () => { test("shows the catalog status strip count", async ({ page }) => { await page.goto("/"); // GlobalChrome status strip renders status.toolsInInventory: - // "{count} TOOLS IN INVENTORY" with count=2 (mock catalog has 2 tools). + // "{count} TOOLS IN INVENTORY" with count=2 (the demo seed has 2 tools). await expect(page.getByText("2 TOOLS IN INVENTORY")).toBeVisible(); }); }); diff --git a/v5/e2e/projects.spec.ts b/v5/e2e/projects.spec.ts index beb43a3..c5a09ba 100644 --- a/v5/e2e/projects.spec.ts +++ b/v5/e2e/projects.spec.ts @@ -1,11 +1,10 @@ import { test, expect } from "@playwright/test"; -// The app boots with NOTION_* unset (see playwright.config.ts webServer.env), -// so `hasProjectsEnv()` is false: `getPublishedProjects()` returns [] without -// ever calling Notion, and `POST /api/projects` answers 503 "not configured". -// That is the correct production behaviour for a lab that has not created the -// Projects database yet — so the gallery here is asserted as an *intentional* -// empty state, not a broken one. +// The app boots with no DATABASE_URL and NOTION_* unset (see +// playwright.config.ts webServer.env), so the gallery reads the PGlite demo +// seed — one published sample project, "Laser-cut plywood lamp" +// (src/lib/db/demo-seed.ts) — and `POST /api/projects`, still a Notion write +// until Phase 3, would answer 503 "not configured". // // The submit path is therefore exercised with `page.route("**/api/projects")` // standing in for the route handler, exactly as chat.spec.ts stands in for @@ -31,9 +30,7 @@ async function fillRequiredFields(page: import("@playwright/test").Page) { } test.describe("Projects gallery", () => { - test("renders an intentional empty state with no Projects database", async ({ - page, - }) => { + test("renders the sample project and opens its page", async ({ page }) => { await page.goto("/projects"); // projects.title => "STUDENT PROJECTS". @@ -41,23 +38,31 @@ test.describe("Projects gallery", () => { page.getByRole("heading", { name: "STUDENT PROJECTS", level: 1 }) ).toBeVisible(); - // The lede describes the gallery rather than apologising for it, and its - // {institution} placeholder resolved. + // The lede describes the gallery, and its {institution} placeholder resolved. await expect( page.getByText(/Builds, experiments, and course outcomes from/i) ).toBeVisible(); - // projects.empty — reads as "nothing published yet", not as an error. No - // failure language, and the invitation to submit is still offered. - await expect( - page.getByText("No projects published yet. Be the first to share your build.") - ).toBeVisible(); + // The demo seed's one published project is listed, with no failure language. + const card = page.getByRole("link").filter({ hasText: "Laser-cut plywood lamp" }).first(); + await expect(card).toBeVisible(); await expect( page.getByText(/error|failed|unavailable|something went wrong/i) ).toHaveCount(0); // Whole-page check: no locale placeholder rendered literally anywhere. await expect(page.locator("body")).not.toContainText(PLACEHOLDER_LEAK); + + // Its page shows the write-up, the outside link and the tools it was built with. + await card.click(); + await expect(page).toHaveURL(/\/projects\/laser-cut-plywood-lamp$/); + await expect( + page.getByRole("heading", { name: "Laser-cut plywood lamp", level: 1 }) + ).toBeVisible(); + await expect( + page.getByRole("link", { name: /wikipedia/i }).first() + ).toHaveAttribute("href", /en\.wikipedia\.org\/wiki\/Laser_cutting/); + await expect(page.getByRole("link", { name: "Trotec Speedy 400" })).toBeVisible(); }); test("the empty state's call to action reaches the submit form", async ({ @@ -105,7 +110,7 @@ test.describe("Project submission form", () => { page.getByRole("textbox", { name: "Link (optional)" }) ).toBeVisible(); - // Tools-used chips come from the mock catalog via getCatalogTools(). + // Tools-used chips come from the demo catalogue via getCatalogTools(). await expect( page.getByRole("button", { name: "Form 4", exact: true }) ).toBeVisible(); @@ -223,7 +228,10 @@ test.describe("Project submission form", () => { const payload = bodies[0] as Record; expect(payload.title).toBe("Parametric stool"); expect(payload.author).toBe("Ada Lovelace"); - expect(payload.tools).toEqual(["tool-trotec-speedy-400"]); + // The chosen tool travels as its database id (a uuid), not its slug. + expect(payload.tools).toEqual([ + expect.stringMatching(/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/), + ]); expect(payload).not.toHaveProperty("published"); }); diff --git a/v5/e2e/tool-detail.spec.ts b/v5/e2e/tool-detail.spec.ts index e00dda9..b8485e3 100644 --- a/v5/e2e/tool-detail.spec.ts +++ b/v5/e2e/tool-detail.spec.ts @@ -1,7 +1,9 @@ import { test, expect } from "@playwright/test"; -// Mock catalog: Form 4 (slug "form-4") has one unit "Form 4 // A" (serial -// ML-F4-001) and two resource links (Form 4 SOP / Resin handling safety). +// Demo seed: Form 4 (slug "form-4") has one unit "Form 4 // A" (serial +// ML-F4-001), two resource links (Form 4 SOP / Resin handling safety), and a +// `notionPageId` of 1f2e3d4c-5b6a-4789-8abc-def012345678 (src/lib/db/demo-seed.ts) +// used below to exercise the legacy `/tools/` redirect (spec Goal 2). test.describe("Tool detail", () => { test("deep-link /tools/form-4 shows name, units and resources", async ({ @@ -47,4 +49,24 @@ test.describe("Tool detail", () => { // Trotec unit serial. await expect(page.getByText("ML-LSR-400").first()).toBeVisible(); }); + + test("a legacy Notion-id link redirects permanently to the slug, preserving ?src=qr", async ({ + page, + }) => { + // Undashed, as it appears in a printed QR code — lands on the current slug. + await page.goto("/tools/1f2e3d4c5b6a47898abcdef012345678"); + await expect(page).toHaveURL(/\/tools\/form-4$/); + await expect( + page.getByRole("heading", { name: "Form 4", level: 1 }) + ).toBeVisible(); + + // Dashed, with the QR marker — the marker survives the redirect. + await page.goto( + "/tools/1f2e3d4c-5b6a-4789-8abc-def012345678?src=qr" + ); + await expect(page).toHaveURL(/\/tools\/form-4\?src=qr$/); + await expect( + page.getByRole("heading", { name: "Form 4", level: 1 }) + ).toBeVisible(); + }); }); diff --git a/v5/evals/fixtures.ts b/v5/evals/fixtures.ts index 8b2d6a8..e23b84b 100644 --- a/v5/evals/fixtures.ts +++ b/v5/evals/fixtures.ts @@ -2,13 +2,16 @@ import { mockTools } from "@/components/mock-catalog"; import type { MakerLabTool } from "@/components/catalog-types"; /** - * Eval fixtures (design spec §3). The eval suite runs with every `NOTION_*` - * variable unset, so `getCatalogTools()` serves `src/components/mock-catalog.ts` - * — a fixed, two-machine catalog. This module pins that same data into the - * shape the assertions need, so an assertion can name exact values instead of - * guessing what the live catalog happens to contain today. + * Eval fixtures (design spec §3). The eval suite runs with `DATABASE_URL` + * unset, so `getCatalogTools()` serves the PGlite demo seed — a fixed, + * two-machine catalog — and `run.eval.ts` builds the fixture from that same + * catalogue, so an assertion can name exact values and can never police a + * machine the model was not given. * - * Everything here is pure data derived from the mock catalog plus two small + * {@link evalFixture} is the offline default the assertion unit tests use: the + * same two machines, pinned as literal data so those tests need no database. + * + * Everything here is pure data derived from a catalog plus two small * hand-maintained lists: * * - {@link EXTRA_ALIASES} — other names a catalog machine legitimately goes by @@ -198,7 +201,7 @@ function toFixtureTool(tool: MakerLabTool): EvalFixtureTool { }; } -/** Build the fixture from a catalog (defaults to the mock catalog). */ +/** Build the fixture from a catalog (defaults to the pinned two machines). */ export function buildFixture(tools: MakerLabTool[] = mockTools): EvalFixture { const fixtureTools = tools.map(toFixtureTool); @@ -213,5 +216,5 @@ export function buildFixture(tools: MakerLabTool[] = mockTools): EvalFixture { }; } -/** The fixture the runner uses: the mock catalog, pinned. */ +/** The offline fixture the assertion tests use: the two machines, pinned. */ export const evalFixture: EvalFixture = buildFixture(); diff --git a/v5/evals/harness.test.ts b/v5/evals/harness.test.ts index 35c60a5..0209dba 100644 --- a/v5/evals/harness.test.ts +++ b/v5/evals/harness.test.ts @@ -6,9 +6,11 @@ import { composeCase, stubWrites } from "./harness"; vi.mock("next/cache", () => nextCacheMock()); -// Blank the Notion contract so the catalog is the mock catalog no matter what -// the developer has in their shell — the same rail `npm run eval` applies. +// Blank `DATABASE_URL` so the catalog is the demo seed, and the Notion contract +// so no write path can reach Notion, no matter what the developer has in their +// shell — the same rails `npm run eval` applies. beforeEach(() => { + vi.stubEnv("DATABASE_URL", ""); for (const key of getNotionEnvContract()) vi.stubEnv(key, ""); }); diff --git a/v5/evals/run.eval.ts b/v5/evals/run.eval.ts index f732d01..d3bdaa9 100644 --- a/v5/evals/run.eval.ts +++ b/v5/evals/run.eval.ts @@ -3,7 +3,7 @@ import { fileURLToPath } from "node:url"; import path from "node:path"; import { anthropic } from "@ai-sdk/anthropic"; import { generateText, stepCountIs, type LanguageModel } from "ai"; -import { hasNotionCatalogEnv } from "@/lib/catalog"; +import { getCatalogTools, isDemoCatalog } from "@/lib/catalog"; import { CHAT_MODEL_ID, GATEWAY_CHAT_MODEL_ID, @@ -12,7 +12,7 @@ import { } from "@/lib/model"; import { getNotionEnvContract } from "@/lib/notion"; import { loadCases, type EvalCase } from "./cases"; -import { evalFixture } from "./fixtures"; +import { buildFixture } from "./fixtures"; import { composeCase } from "./harness"; import { formatReport, runSuite, type CaseExecution } from "./runner"; @@ -30,8 +30,9 @@ import { formatReport, runSuite, type CaseExecution } from "./runner"; * eval that tested a reimplementation of the prompt would test nothing. * * Safety rails, in order: - * - every `NOTION_*` variable is blanked, so the catalog is the fixed mock - * catalog and no Notion request is possible; + * - `DATABASE_URL` and every `NOTION_*` variable are blanked, so the catalog + * is the fixed demo seed in an in-process PGlite database (spec §3.2) and + * neither a real database nor Notion can be reached; * - every `write` capability tool is stubbed, so an eval can never create a * Notion record even if the model decides to call one; * - the provider-native `web_search` / `web_fetch` tools the chat route adds @@ -79,11 +80,13 @@ async function executeCase(evalCase: EvalCase): Promise { describe("agent evals", () => { beforeAll(() => { - // Blank the Notion contract so `getCatalogTools()` serves the mock catalog - // regardless of what the developer has in their shell. + // Blank `DATABASE_URL` so `getCatalogTools()` serves the demo seed, and the + // Notion contract so the write paths cannot reach Notion either, regardless + // of what the developer has in their shell. + vi.stubEnv("DATABASE_URL", ""); for (const key of getNotionEnvContract()) vi.stubEnv(key, ""); - if (hasNotionCatalogEnv()) { - throw new Error("refusing to run: the Notion environment is still configured"); + if (!isDemoCatalog()) { + throw new Error("refusing to run: the catalog is a real database, not the demo seed"); } if (!process.env.ANTHROPIC_API_KEY && !process.env.AI_GATEWAY_API_KEY) { throw new Error( @@ -96,7 +99,11 @@ describe("agent evals", () => { const cases = loadCases(); console.info(`Running ${cases.length} eval cases against ${MODEL_LABEL}…`); - const report = await runSuite(cases, executeCase, evalFixture, { + // Built from the catalogue the harness actually composes with, so an + // assertion can never police a machine or a manual the model was not given. + const fixture = buildFixture(await getCatalogTools()); + + const report = await runSuite(cases, executeCase, fixture, { onCase: (result) => console.info(` ${result.status.toUpperCase()} ${result.id}`), }); diff --git a/v5/messages/en.json b/v5/messages/en.json index 57d88c6..8429f3c 100644 --- a/v5/messages/en.json +++ b/v5/messages/en.json @@ -18,6 +18,8 @@ "addSeed": "I'd like to add new equipment to the inventory.", "signIn": "SIGN IN", "signInAria": "Sign in with your {institution} account", + "signInUnconfigured": "Sign-in isn't set up on this deployment yet.", + "signInFailed": "Couldn't start sign-in. Try again.", "signOut": "SIGN OUT", "signedInAria": "Signed in as {name}" }, diff --git a/v5/next.config.ts b/v5/next.config.ts index 7c730ce..0ba2390 100644 --- a/v5/next.config.ts +++ b/v5/next.config.ts @@ -5,11 +5,22 @@ const withNextIntl = createNextIntlPlugin("./src/i18n/request.ts"); const nextConfig: NextConfig = { cacheComponents: true, + // PGlite ships its WASM build and its extension tarballs as files it locates + // with `import.meta.url`. Bundled into the server output those become + // `/_next/static/media/...` URLs that nothing can read from disk, so a build + // or a request without `DATABASE_URL` dies on "Extension bundle not found". + // Kept external, it loads from node_modules and the demo database works in a + // built app exactly as it does under `next dev` (spec §3.2). + serverExternalPackages: ["@electric-sql/pglite"], images: { localPatterns: [ { pathname: "/tool-images/**", }, + { + // Photos for the demo seed's sample project (src/lib/db/demo-seed.ts). + pathname: "/sample-projects/**", + }, { pathname: "/makerlab-logo-transparent.png", }, @@ -18,6 +29,10 @@ const nextConfig: NextConfig = { }, ], remotePatterns: [ + { + protocol: "https", + hostname: "*.public.blob.vercel-storage.com", + }, { protocol: "https", hostname: "prod-files-secure.s3.us-west-2.amazonaws.com", @@ -34,6 +49,9 @@ const nextConfig: NextConfig = { protocol: "https", hostname: "v5.airtableusercontent.com", }, + // The Notion/S3/Airtable patterns above can go once every image has been + // re-imported to Vercel Blob; until then, rows imported before the switch + // may still reference them. ], minimumCacheTTL: 3600, }, diff --git a/v5/package.json b/v5/package.json index 2fde6ec..8e7bd83 100644 --- a/v5/package.json +++ b/v5/package.json @@ -4,7 +4,7 @@ "private": true, "scripts": { "dev": "next dev", - "build": "next build", + "build": "npm run db:migrate && next build", "start": "next start", "lint": "eslint", "typecheck": "tsc --noEmit", diff --git a/v5/playwright.config.ts b/v5/playwright.config.ts index a2d2d11..d31f250 100644 --- a/v5/playwright.config.ts +++ b/v5/playwright.config.ts @@ -1,22 +1,35 @@ import { defineConfig, devices } from "@playwright/test"; -// E2E runs against the app booted with NO Notion env vars, so getCatalogTools() -// serves the built-in mock catalog (src/components/mock-catalog.ts). The -// /api/chat call is intercepted at the network layer inside each spec via +// E2E runs against a PRODUCTION build (`next build && next start`) booted with +// no `DATABASE_URL` and no Notion env, so the catalogue is the in-process +// PGlite database seeded with the two demo tools (src/lib/db/demo-seed.ts). +// The /api/chat call is intercepted at the network layer inside each spec via // page.route(). No real external services are touched. // +// Why not `next dev`: with parallel workers, the first request to each route +// compiled it on demand and Turbopack rewrote the root layout's client chunk +// while another worker was downloading it (ERR_CONTENT_LENGTH_MISMATCH), so +// pages intermittently never hydrated (data platform spec 2026-09-14, Phase 2 +// amendment). A production server has no compiler in the loop, and it is the +// same bundle a Vercel preview without a database runs. +// // NOTE FOR THE E2E AGENT: run `npx playwright install chromium` once before the // first `npm run test:e2e` — the foundation harness does not install browsers. export default defineConfig({ testDir: "./e2e", fullyParallel: true, forbidOnly: !!process.env.CI, - retries: process.env.CI ? 1 : 0, + // One retry everywhere, not only in CI. With fourteen workers on a large + // machine a handful of interactions a few hundred milliseconds after a + // navigation still occasionally miss (about one test execution in a hundred + // as of Phase 2); a retry keeps the gate green and Playwright reports the + // test as "flaky" rather than hiding it, so the count stays visible. + retries: 1, reporter: process.env.CI ? "github" : "list", use: { // Dedicated test port (not 3000) so the suite never collides with — or - // accidentally reuses — a `next dev` you have running locally against real - // Notion. The webServer below always boots its own mock-backed instance. + // accidentally reuses — a `next dev` you have running locally against a + // real database. The webServer below always boots its own demo instance. baseURL: "http://localhost:3100", trace: "on-first-retry", }, @@ -27,17 +40,22 @@ export default defineConfig({ }, ], webServer: { - command: "npx next dev -p 3100", + // `npm run build` runs `db:migrate` first, which is a no-op with the + // blanked DATABASE_URL below, then `next build`. + command: "npm run build && npx next start -p 3100", url: "http://localhost:3100", - // Always boot a fresh mock-backed server; never reuse whatever is on the + // Always boot a fresh demo-backed server; never reuse whatever is on the // port. Keeps E2E deterministic regardless of the local dev environment. reuseExistingServer: false, - timeout: 120_000, - // Unset Notion env so the app serves the mock catalog. Spreading - // process.env first, then overwriting with "" makes hasNotionCatalogEnv() - // return false regardless of the dev shell's environment. + // The build is most of this budget; `next start` itself is seconds. + timeout: 300_000, + // Blank `DATABASE_URL` so the catalogue is the demo seed, and the Notion + // env so no write path can reach Notion. Spreading process.env first, then + // overwriting with "" wins over both the dev shell and `.env.local`, which + // Next will not override for a key that is already present. env: { ...process.env, + DATABASE_URL: "", NOTION_API_KEY: "", NOTION_DB_TOOLS: "", NOTION_DB_CATEGORIES: "", diff --git a/v5/public/sample-projects/laser-cut-lamp-lit.png b/v5/public/sample-projects/laser-cut-lamp-lit.png new file mode 100644 index 0000000..9896bfb Binary files /dev/null and b/v5/public/sample-projects/laser-cut-lamp-lit.png differ diff --git a/v5/public/sample-projects/laser-cut-lamp-parts.png b/v5/public/sample-projects/laser-cut-lamp-parts.png new file mode 100644 index 0000000..b70a263 Binary files /dev/null and b/v5/public/sample-projects/laser-cut-lamp-parts.png differ diff --git a/v5/scripts/generate-qr-labels.test.ts b/v5/scripts/generate-qr-labels.test.ts index f056c6d..dc32e93 100644 --- a/v5/scripts/generate-qr-labels.test.ts +++ b/v5/scripts/generate-qr-labels.test.ts @@ -1,3 +1,4 @@ +// @vitest-environment node import { resolve } from "node:path"; import { buildLabelSheet, @@ -5,6 +6,7 @@ import { deriveLabels, formatLabelLocation, loadSheetStrings, + loadSources, MIN_QR_MM, parseArgs, pickSheetStrings, @@ -13,7 +15,6 @@ import { type LabelSheetStrings, type QrLabelSource, } from "./generate-qr-labels.ts"; -import { mockTools } from "@/components/mock-catalog"; import enMessages from "../messages/en.json"; const MESSAGES_DIR = resolve(process.cwd(), "messages"); @@ -173,27 +174,31 @@ describe("renderLabelSheet", () => { }); }); -describe("buildLabelSheet (integration, mock catalog)", () => { - it("produces one label per tool in the mock catalog", async () => { - const html = await buildLabelSheet(mockTools, { +// The whole script, end to end, over the catalogue it reads in production — +// the demo seed standing in for a real database (spec §3.2). +describe("buildLabelSheet (integration, demo catalogue)", () => { + it("produces one label per tool in the catalogue", async () => { + const sources = await loadSources(); + const html = await buildLabelSheet(sources, { baseUrl: BASE, strings: STRINGS, encodeQr: fakeEncoder, }); const labels = html.match(/class="qr-label"/g) || []; - expect(labels).toHaveLength(mockTools.length); - expect(mockTools.length).toBeGreaterThan(0); + expect(labels).toHaveLength(sources.length); + expect(sources.length).toBeGreaterThan(0); }); it("encodes each tool's own ?src=qr URL and prints its name", async () => { - const html = await buildLabelSheet(mockTools, { + const sources = await loadSources(); + const html = await buildLabelSheet(sources, { baseUrl: BASE, strings: STRINGS, encodeQr: fakeEncoder, }); - for (const tool of mockTools) { + for (const tool of sources) { expect(html).toContain(`${BASE}/tools/${tool.slug}?src=qr`); expect(html).toContain(tool.name); } @@ -201,8 +206,9 @@ describe("buildLabelSheet (integration, mock catalog)", () => { }); it("drops unpublished tools from the sheet", async () => { + const sources = await loadSources(); const html = await buildLabelSheet( - [...mockTools.map((tool) => ({ ...tool, published: false })), source()], + [...sources.map((tool) => ({ ...tool, published: false })), source()], { baseUrl: BASE, strings: STRINGS, encodeQr: fakeEncoder } ); @@ -211,6 +217,26 @@ describe("buildLabelSheet (integration, mock catalog)", () => { }); }); +describe("loadSources (Postgres, PGlite demo seed)", () => { + it("reads published, non-archived tools ordered by name", async () => { + const sources = await loadSources(); + + expect(sources.map((source) => source.slug)).toEqual(["form-4", "trotec-speedy-400"]); + expect(sources.every((source) => source.published)).toBe(true); + + const form4 = sources.find((source) => source.slug === "form-4"); + expect(form4).toMatchObject({ name: "Form 4", location: "MakerLab", zone: "Resin Bench" }); + }); + + it("produces working label URLs end to end from Postgres", async () => { + const sources = await loadSources(); + expect(deriveLabels(sources, BASE).map((label) => label.url)).toEqual([ + "https://tools.example.edu/tools/form-4?src=qr", + "https://tools.example.edu/tools/trotec-speedy-400?src=qr", + ]); + }); +}); + describe("sheet strings", () => { it("picks the qr namespace out of a locale catalog", () => { const strings = pickSheetStrings(enMessages); diff --git a/v5/scripts/generate-qr-labels.ts b/v5/scripts/generate-qr-labels.ts index bcbdcb5..5d31c21 100644 --- a/v5/scripts/generate-qr-labels.ts +++ b/v5/scripts/generate-qr-labels.ts @@ -15,11 +15,15 @@ import { readFile, writeFile } from "node:fs/promises"; import { join } from "node:path"; import { fileURLToPath } from "node:url"; +import { and, asc, eq, isNull } from "drizzle-orm"; import { getDirection, isSupportedLocale, DEFAULT_LOCALE } from "../src/i18n/config.ts"; +import { getDb } from "../src/lib/db/client.ts"; +import { locations, tools } from "../src/lib/db/schema/index.ts"; /** * The minimum a label needs: something to route to, something to print. Both - * the resolved catalog (`MakerLabTool`) and raw Notion metadata satisfy it. + * the resolved catalog (`MakerLabTool`) and a raw Postgres `tools` row (joined + * to its location) satisfy it. */ export interface QrLabelSource { id: string; @@ -338,34 +342,37 @@ export function parseArgs(argv: string[]): CliOptions { } /** - * Catalog source. With Notion configured we read the published Tools DB - * directly — `src/lib/catalog.ts` caches through `next/cache`, which only - * works inside a Next runtime. Without credentials we fall back to the mock - * catalog, so staff can dry-run the layout before wiring anything up. + * Catalog source: published, non-archived tools straight from Postgres + * (spec §3.10 — old labels still resolve because `notion_page_id` isn't + * touched here, and new ones are printed from `slug`). `getDb()` resolves to + * the PGlite demo seed when `DATABASE_URL` is unset, so staff can dry-run the + * layout before a real database is wired up. */ -async function loadSources(): Promise { - const { getNotionEnvContract, fetchAllTools, fetchAllLocations, resolveTools } = - await import("../src/lib/notion.ts"); - - if (!getNotionEnvContract().every((key) => Boolean(process.env[key]))) { - console.warn("Notion env not configured — generating labels from the mock catalog."); - const { mockTools } = await import("../src/components/mock-catalog.ts"); - return mockTools; - } - - const [tools, locations] = await Promise.all([fetchAllTools(), fetchAllLocations()]); - // `fetchAllTools` already filters to published records; keep the flag so - // `deriveLabels` stays the single place that decides. - const publishedById = new Map(tools.map((tool) => [tool.id, tool.fields.published])); - - return resolveTools(tools, [], locations).map((tool) => ({ - id: tool.id, - // Notion-backed tool pages route by page id (catalog.ts sets slug = id). - slug: tool.id, - name: tool.name, - location: tool.location_room, - zone: tool.location_zone, - published: publishedById.get(tool.id), +export async function loadSources(): Promise { + const db = await getDb(); + + const rows = await db + .select({ + id: tools.id, + slug: tools.slug, + name: tools.name, + room: locations.room, + zone: locations.zone, + }) + .from(tools) + .leftJoin(locations, eq(tools.locationId, locations.id)) + .where(and(eq(tools.published, true), isNull(tools.archivedAt))) + .orderBy(asc(tools.name)); + + return rows.map((row) => ({ + id: row.id, + slug: row.slug, + name: row.name, + location: row.room, + zone: row.zone, + // The query already filters to published tools; the flag stays true so + // `deriveLabels` remains the single place that decides. + published: true, })); } diff --git a/v5/src/app/api/chat/rate-limit.route.test.ts b/v5/src/app/api/chat/rate-limit.route.test.ts index 3f1bc50..5266036 100644 --- a/v5/src/app/api/chat/rate-limit.route.test.ts +++ b/v5/src/app/api/chat/rate-limit.route.test.ts @@ -1,3 +1,4 @@ +// @vitest-environment node /** * `/api/chat` at and over the tiered ceiling (auth design spec §8, §10). * @@ -6,13 +7,15 @@ * an anonymous visitor and a signed-in student hitting the same endpoint from * the same IP get different allowances, and the refusal offers a way forward. * - * The model is still stubbed at the `streamText` boundary — no network (Art. 3). + * The model is still stubbed at the `streamText` boundary, and the catalogue + * comes from the demo-seeded PGlite database — no network (Art. 3). */ import { SESSION_COOKIE_NAME, createSessionPayload, signSession, } from "@/lib/auth/session-cookie"; +import { resetDbForTests } from "@/lib/db/client"; const AUTH_SECRET = "chat-ceiling-test-secret"; @@ -42,19 +45,9 @@ vi.mock("ai", async (importOriginal) => { }; }); -// Notion is never reached (NOTION_* unset → mock catalog), but catalog.ts -// imports it, so stub the module surface it uses. -vi.mock("@/lib/notion", () => ({ - fetchMaintenanceLogsByUnit: vi.fn(async () => []), - createMaintenanceLog: vi.fn(), - fetchAllResources: vi.fn(async () => []), - getNotionEnvContract: () => ["NOTION_API_KEY"], - fetchAllTools: vi.fn(async () => []), - fetchAllCategories: vi.fn(async () => []), - fetchAllLocations: vi.fn(async () => []), - fetchAllUnits: vi.fn(async () => []), - resolveTools: vi.fn(() => []), -})); +// Notion is not stubbed here: no request path in this suite reaches it. The +// catalogue reads Postgres, and the one Notion write left in the capability +// layer (`report_issue`, spec Phase 3) is never called from a ceiling test. vi.mock("next/cache", () => ({ cacheTag: vi.fn(), @@ -97,6 +90,11 @@ function chatRequest({ ip, cookie }: { ip: string; cookie?: string }): Request { beforeEach(() => { vi.stubEnv("AUTH_SECRET", AUTH_SECRET); + vi.stubEnv("DATABASE_URL", ""); +}); + +afterAll(() => { + resetDbForTests(); }); describe("POST /api/chat — anonymous ceiling", () => { diff --git a/v5/src/app/api/chat/route.test.ts b/v5/src/app/api/chat/route.test.ts index d56e5c1..1c0a0c3 100644 --- a/v5/src/app/api/chat/route.test.ts +++ b/v5/src/app/api/chat/route.test.ts @@ -1,7 +1,7 @@ +// @vitest-environment node /* eslint-disable @typescript-eslint/no-explicit-any -- this suite inspects the loosely-typed { system, messages, tools } object captured from the mocked streamText call; precise typing here would add noise without value. */ -import type { MaintenanceLogRecord, ResourceRecord } from "@/lib/types"; // ── Captured streamText args ───────────────────────────────────────── // streamText is mocked so we can inspect the { system, messages, tools } @@ -46,16 +46,9 @@ vi.mock("ai", async (importOriginal) => { const mocks = vi.hoisted(() => ({ rateLimitAsync: vi.fn(), checkRateLimit: vi.fn(), - fetchMaintenanceLogsByUnit: vi.fn(), createMaintenanceLog: vi.fn(), - fetchAllResources: vi.fn(), })); -const { - checkRateLimit, - fetchMaintenanceLogsByUnit, - createMaintenanceLog, - fetchAllResources, -} = mocks; +const { checkRateLimit, createMaintenanceLog } = mocks; // ── Mock the rate limiter (default: allowed, set in beforeEach) ────── // The route calls the identity-keyed `checkRateLimit`; the rest of the module @@ -71,29 +64,14 @@ vi.mock("@/lib/rate-limit", async (importOriginal) => { }; }); -// ── Mock Notion calls used by the route's tools / manual collection ── -vi.mock("@/lib/notion", () => ({ - fetchMaintenanceLogsByUnit: mocks.fetchMaintenanceLogsByUnit, - createMaintenanceLog: mocks.createMaintenanceLog, - fetchAllResources: mocks.fetchAllResources, - // catalog.ts also imports from notion.ts, but with NOTION_* env unset the - // catalog uses the mock fallback and never touches these, so stub the rest. - getNotionEnvContract: () => [ - "NOTION_API_KEY", - "NOTION_DB_TOOLS", - "NOTION_DB_CATEGORIES", - "NOTION_DB_LOCATIONS", - "NOTION_DB_UNITS", - "NOTION_DB_RESOURCES", - "NOTION_DB_MAINTENANCE_LOGS", - "NOTION_DB_FLAGS", - ], - fetchAllTools: vi.fn(async () => []), - fetchAllCategories: vi.fn(async () => []), - fetchAllLocations: vi.fn(async () => []), - fetchAllUnits: vi.fn(async () => []), - resolveTools: vi.fn(() => []), -})); +// ── Mock the one Notion call left in the route's tools ─────────────── +// `report_issue` still files its ticket in Notion (spec §9, Phase 3). Every +// other export stays real: nothing else in this request path reads Notion, and +// a full replacement would have to track every import in the capability layer. +vi.mock("@/lib/notion", async (importOriginal) => { + const actual = await importOriginal(); + return { ...actual, createMaintenanceLog: mocks.createMaintenanceLog }; +}); // next/cache is imported by catalog.ts ("use cache" / cacheTag / cacheLife). vi.mock("next/cache", () => ({ @@ -102,29 +80,75 @@ vi.mock("next/cache", () => ({ revalidateTag: vi.fn(), })); +import { eq, inArray } from "drizzle-orm"; import { POST } from "@/app/api/chat/route"; +import { getDb, resetDbForTests } from "@/lib/db/client"; +import { DEMO_FORM_4_UNIT_NOTION_PAGE_ID } from "@/lib/db/demo-seed"; +import { + attachments, + maintenanceLogs, + resources, + tools as toolsTable, + units, +} from "@/lib/db/schema/index"; import { SESSION_COOKIE_NAME, createSessionPayload, signSession, } from "@/lib/auth/session-cookie"; -// A mock-catalog resource fixture builder (mirrors ResourceRecord shape). -function resource( - partial: Partial & { id?: string } -): ResourceRecord { - const { id = "res-1", ...fields } = partial; - return { - id, - createdTime: "2024-01-01T00:00:00.000Z", - lastEditedTime: "2024-01-01T00:00:00.000Z", - fields: { - title: "Manual", - tool: ["tool-form-4"], - published: true, - ...fields, - }, - } as ResourceRecord; +/** + * The catalogue, the units and the resources all come from the demo-seeded + * PGlite database `getDb()` hands out with `DATABASE_URL` unset — the two + * tools `Form 4` (unit `Form 4 // A`) and `Trotec Speedy 400`. The seed carries + * no maintenance logs and no PDF resources; the tests that need them write + * them and clean up after themselves. + */ + +/** Resource rows this file inserted, removed after each test. */ +const insertedResourceIds: string[] = []; + +async function toolId(slug: string): Promise { + const db = await getDb(); + const [row] = await db + .select({ id: toolsTable.id }) + .from(toolsTable) + .where(eq(toolsTable.slug, slug)); + return row.id; +} + +async function unitId(label: string): Promise { + const db = await getDb(); + const [row] = await db + .select({ id: units.id }) + .from(units) + .where(eq(units.unitLabel, label)); + return row.id; +} + +/** Add resources to a seeded tool (default: the Form 4). */ +async function addResources( + rows: Array<{ title: string; url?: string | null; published?: boolean; slug?: string }> +): Promise { + const db = await getDb(); + const values = await Promise.all( + rows.map(async ({ slug = "form-4", ...row }) => ({ + toolId: await toolId(slug), + type: "Manual", + ...row, + })) + ); + const created = await db.insert(resources).values(values).returning({ id: resources.id }); + insertedResourceIds.push(...created.map((row) => row.id)); +} + +async function addMaintenanceLogs( + label: string, + rows: Array> +): Promise { + const db = await getDb(); + const id = await unitId(label); + await db.insert(maintenanceLogs).values(rows.map((row) => ({ title: "Issue", unitId: id, ...row }))); } function chatRequest( @@ -150,6 +174,7 @@ const userMessage = (text: string) => ({ beforeEach(() => { captured.args = undefined; + vi.stubEnv("DATABASE_URL", ""); // Undo any `vi.stubGlobal("fetch", …)` from a prior PDF test (the shared // setup file does not call vi.unstubAllGlobals). vi.unstubAllGlobals(); @@ -161,13 +186,25 @@ beforeEach(() => { retryAfterSeconds: 3600, role: "student", }); - fetchMaintenanceLogsByUnit.mockReset(); - fetchMaintenanceLogsByUnit.mockResolvedValue([]); - fetchAllResources.mockReset(); - fetchAllResources.mockResolvedValue([]); createMaintenanceLog.mockReset(); }); +afterEach(async () => { + const db = await getDb(); + await db.delete(maintenanceLogs); + // The seed carries no attachments, so clearing the table only drops what a + // test wrote. + await db.delete(attachments); + if (insertedResourceIds.length > 0) { + await db.delete(resources).where(inArray(resources.id, insertedResourceIds)); + insertedResourceIds.length = 0; + } +}); + +afterAll(() => { + resetDbForTests(); +}); + describe("POST /api/chat — rate limiting", () => { it("returns 429 with Retry-After and never calls streamText when denied", async () => { checkRateLimit.mockResolvedValueOnce({ @@ -248,7 +285,7 @@ describe("get_unit_details.execute", () => { unit_label: "no-such-unit", }); expect(result.found).toBe(false); - // Sample list drawn from the mock catalog's units. + // Sample list drawn from the seeded catalogue's units. expect(result.message).toMatch(/Form 4 \/\/ A|Trotec Speedy 400/); }); @@ -265,21 +302,18 @@ describe("get_unit_details.execute", () => { expect(result.maintenance_logs).toEqual([]); }); - it("surfaces maintenance logs, sliced to 10", async () => { - const logs: MaintenanceLogRecord[] = Array.from({ length: 12 }, (_, i) => ({ - id: `log-${i}`, - createdTime: "2024-01-01T00:00:00.000Z", - lastEditedTime: "2024-01-01T00:00:00.000Z", - fields: { + it("surfaces maintenance logs, capped at 10 and newest first", async () => { + await addMaintenanceLogs( + "Form 4 // A", + Array.from({ length: 12 }, (_, i) => ({ title: `Issue ${i}`, - type: "Issue Report", - priority: "Medium", - status: "Open", - date_reported: "2024-09-01", + type: "issue_report", + priority: "medium", + status: "open", description: `desc ${i}`, - }, - })) as MaintenanceLogRecord[]; - fetchMaintenanceLogsByUnit.mockResolvedValueOnce(logs); + dateReported: `2024-09-${String(i + 1).padStart(2, "0")}`, + })) + ); const tools = await getTools(); const result = await tools.get_unit_details.execute({ @@ -288,8 +322,9 @@ describe("get_unit_details.execute", () => { expect(result.found).toBe(true); expect(result.maintenance_logs).toHaveLength(10); + // Stored snake_case, shown in the words the assistant has always seen. expect(result.maintenance_logs[0]).toMatchObject({ - title: "Issue 0", + title: "Issue 11", type: "Issue Report", priority: "Medium", status: "Open", @@ -346,13 +381,14 @@ describe("report_issue.execute", () => { priority: "High", }); + // The caller is told the Postgres uuid the catalogue resolved; the ticket, + // still filed in Notion, carries the unit's imported Notion page id — the + // only id that database's `unit` relation can address. + const form4A = await unitId("Form 4 // A"); expect(result.success).toBe(true); - expect(result.unit_resolved).toEqual({ - id: "unit-form-4-a", - label: "Form 4 // A", - }); + expect(result.unit_resolved).toEqual({ id: form4A, label: "Form 4 // A" }); expect(createMaintenanceLog).toHaveBeenCalledWith( - expect.objectContaining({ unit: ["unit-form-4-a"] }) + expect.objectContaining({ unit: [DEMO_FORM_4_UNIT_NOTION_PAGE_ID] }) ); }); @@ -451,11 +487,11 @@ describe("PDF manual collection (focused tool)", () => { } it("attaches small PDFs to the first user message and caps at 3", async () => { - fetchAllResources.mockResolvedValueOnce([ - resource({ id: "r1", title: "Manual 1", url: "https://x.test/m1.pdf" }), - resource({ id: "r2", title: "Manual 2", url: "https://x.test/m2.pdf" }), - resource({ id: "r3", title: "Manual 3", url: "https://x.test/m3.pdf" }), - resource({ id: "r4", title: "Manual 4", url: "https://x.test/m4.pdf" }), + await addResources([ + { title: "Manual 1", url: "https://x.test/m1.pdf" }, + { title: "Manual 2", url: "https://x.test/m2.pdf" }, + { title: "Manual 3", url: "https://x.test/m3.pdf" }, + { title: "Manual 4", url: "https://x.test/m4.pdf" }, ]); const fetchMock = vi.fn(async () => okPdf(PDF_SMALL)); @@ -482,12 +518,12 @@ describe("PDF manual collection (focused tool)", () => { }); it("skips PDFs that are too large or return non-ok, and skips non-PDF resources", async () => { - fetchAllResources.mockResolvedValueOnce([ - resource({ id: "r1", title: "Good", url: "https://x.test/good.pdf" }), - resource({ id: "r2", title: "Huge", url: "https://x.test/huge.pdf" }), - resource({ id: "r3", title: "Dead", url: "https://x.test/dead.pdf" }), + await addResources([ + { title: "Good", url: "https://x.test/good.pdf" }, + { title: "Huge", url: "https://x.test/huge.pdf" }, + { title: "Dead", url: "https://x.test/dead.pdf" }, // Non-PDF url → skipped entirely (never fetched). - resource({ id: "r4", title: "Webpage", url: "https://x.test/page.html" }), + { title: "Webpage", url: "https://x.test/page.html" }, ]); const fetchMock = vi.fn(async (url: string) => { @@ -525,19 +561,9 @@ describe("PDF manual collection (focused tool)", () => { }); it("ignores unpublished resources and resources for other tools", async () => { - fetchAllResources.mockResolvedValueOnce([ - resource({ - id: "r1", - title: "Unpublished", - url: "https://x.test/u.pdf", - published: false, - }), - resource({ - id: "r2", - title: "OtherTool", - url: "https://x.test/o.pdf", - tool: ["tool-trotec-speedy-400"], - }), + await addResources([ + { title: "Unpublished", url: "https://x.test/u.pdf", published: false }, + { title: "OtherTool", url: "https://x.test/o.pdf", slug: "trotec-speedy-400" }, ]); const fetchMock = vi.fn(async () => okPdf(PDF_SMALL)); vi.stubGlobal("fetch", fetchMock); @@ -550,16 +576,43 @@ describe("PDF manual collection (focused tool)", () => { expect(captured.args.system).not.toContain("Available manuals"); }); + it("attaches a resource's uploaded file when the resource itself has no url", async () => { + await addResources([{ title: "Scanned manual", url: null }]); + const db = await getDb(); + const [scanned] = await db + .select({ id: resources.id }) + .from(resources) + .where(eq(resources.title, "Scanned manual")); + await db.insert(attachments).values({ + ownerType: "resource", + ownerId: scanned.id, + blobPathname: "resources/scanned-abc123.pdf", + access: "public", + publicUrl: "https://blob.test/resources/scanned-abc123.pdf", + }); + + const fetchMock = vi.fn(async () => okPdf(PDF_SMALL)); + vi.stubGlobal("fetch", fetchMock); + + await POST( + chatRequest({ messages: [userMessage("help")], toolId: "form-4" }) + ); + + expect(fetchMock).toHaveBeenCalledWith( + "https://blob.test/resources/scanned-abc123.pdf", + expect.anything() + ); + expect(captured.args.system).toContain("Scanned manual"); + }); + it("does not run manual collection when no toolId is provided", async () => { - fetchAllResources.mockResolvedValueOnce([ - resource({ id: "r1", title: "Manual 1", url: "https://x.test/m1.pdf" }), - ]); + await addResources([{ title: "Manual 1", url: "https://x.test/m1.pdf" }]); const fetchMock = vi.fn(async () => okPdf(PDF_SMALL)); vi.stubGlobal("fetch", fetchMock); await POST(chatRequest({ messages: [userMessage("hi")] })); - expect(fetchAllResources).not.toHaveBeenCalled(); expect(fetchMock).not.toHaveBeenCalled(); + expect(captured.args.system).not.toContain("Available manuals"); }); }); diff --git a/v5/src/app/api/chat/route.ts b/v5/src/app/api/chat/route.ts index 67f741d..0c983c6 100644 --- a/v5/src/app/api/chat/route.ts +++ b/v5/src/app/api/chat/route.ts @@ -13,9 +13,11 @@ import { type UserModelMessage, } from "ai"; import { getCatalogTool, getCatalogTools } from "../../../lib/catalog"; -import { fetchAllResources } from "../../../lib/notion"; +import { + listResourcesForTool, + type ToolResource, +} from "../../../lib/data/resources"; import type { MakerLabTool } from "../../../components/catalog-types"; -import type { ResourceRecord } from "../../../lib/types"; import { checkRateLimit, type RateLimitDecision } from "../../../lib/rate-limit"; import { resolveIdentity } from "../../../lib/auth/identity"; import { siteConfig } from "../../../lib/site-config"; @@ -50,7 +52,7 @@ interface ChatRequest { export async function POST(req: Request) { // Who is asking, then how much they are allowed — both before any expensive - // work (Notion fetch / model call). Anonymous visitors get a small allowance + // work (catalogue read / model call). Anonymous visitors get a small allowance // keyed by hashed IP; signed-in callers get a generous one keyed by user id // (auth design spec §8). const identity = await resolveIdentity(req); @@ -372,18 +374,15 @@ function uniqueHosts(urls: string[]): string[] { return [...set]; } -function isPdfUrl(url: string | undefined): boolean { +function isPdfUrl(url: string | null | undefined): boolean { if (!url) return false; const cleaned = url.split("?")[0].toLowerCase(); return cleaned.endsWith(".pdf"); } -function pickPdfUrl(resource: ResourceRecord): string | null { - if (isPdfUrl(resource.fields.url)) return resource.fields.url ?? null; - const file = (resource.fields.files || []).find( - (f) => isPdfUrl(f.filename) || isPdfUrl(f.url) - ); - return file?.url || null; +function pickPdfUrl(resource: ToolResource): string | null { + if (isPdfUrl(resource.url)) return resource.url; + return resource.fileUrls.find(isPdfUrl) ?? null; } /** @@ -429,38 +428,35 @@ async function fetchPdfAsBase64( async function collectToolManuals( toolId: string ): Promise<{ manuals: AttachedManual[]; skipped: number }> { - let resources: ResourceRecord[]; + // Only the focused tool's resources are read (spec §3.10, Article 4's "load + // context lazily") — the whole resource table used to come back from Notion + // just to be filtered down to one tool's rows here. + let forTool: ToolResource[]; try { - resources = await fetchAllResources(); + forTool = await listResourcesForTool(toolId); } catch (err) { console.warn("[chat] failed to load resources for manuals", err); return { manuals: [], skipped: 0 }; } - const forTool = resources.filter( - (r) => r.fields.published !== false && (r.fields.tool || []).includes(toolId) - ); - const manuals: AttachedManual[] = []; let skipped = 0; try { for (const r of forTool) { const url = pickPdfUrl(r); if (!url) { - if (r.fields.url) { - console.info( - `[chat] skipping non-PDF resource: ${r.fields.title} (${r.fields.url})` - ); + if (r.url) { + console.info(`[chat] skipping non-PDF resource: ${r.title} (${r.url})`); } continue; } if (manuals.length >= MAX_PDFS_PER_CHAT) { console.info( - `[chat] PDF cap reached (${MAX_PDFS_PER_CHAT}); skipping: ${r.fields.title}` + `[chat] PDF cap reached (${MAX_PDFS_PER_CHAT}); skipping: ${r.title}` ); continue; } - const title = r.fields.title || "Manual"; + const title = r.title || "Manual"; const data = await fetchPdfAsBase64(title, url); if (!data) { skipped += 1; diff --git a/v5/src/app/api/flags/route.test.ts b/v5/src/app/api/flags/route.test.ts index fddc1b0..4ce6754 100644 --- a/v5/src/app/api/flags/route.test.ts +++ b/v5/src/app/api/flags/route.test.ts @@ -1,3 +1,4 @@ +// @vitest-environment node import { http, HttpResponse } from "msw"; import { server } from "../../../../test/msw/server"; import { DB_IDS } from "../../../../test/msw/handlers"; @@ -6,10 +7,27 @@ import { nextCacheMock } from "../../../../test/mocks/next-cache"; // The route reaches the catalog (for tool resolution) which imports next/cache. vi.mock("next/cache", () => nextCacheMock()); +import { getCatalogTools } from "@/lib/catalog"; +import { resetDbForTests } from "@/lib/db/client"; +import { DEMO_FORM_4_NOTION_PAGE_ID } from "@/lib/db/demo-seed"; import { POST } from "./route"; const NOTION = "https://api.notion.com/v1"; -const FORM_4_ID = "tool-form-4"; + +// The route resolves the flagged tool against the demo-seeded PGlite database +// (`DATABASE_URL` unset). Its id is a Postgres uuid minted at seed time, so it +// is looked up rather than hard-coded. +let FORM_4_ID = ""; + +beforeEach(async () => { + vi.stubEnv("DATABASE_URL", ""); + const tools = await getCatalogTools(); + FORM_4_ID = tools.find((tool) => tool.slug === "form-4")?.id ?? ""; +}); + +afterAll(() => { + resetDbForTests(); +}); // The in-memory limiter is a per-process singleton keyed by IP, so each test // gets its own IP to avoid bleeding a spent window into the next one. @@ -79,6 +97,8 @@ describe("POST /api/flags", () => { expect(creates[0].properties).toMatchObject({ status: { select: { name: "New" } }, field_flagged: { select: { name: "location" } }, + // The relation addresses the tool's Notion page, not its Postgres id. + tool: { relation: [{ id: DEMO_FORM_4_NOTION_PAGE_ID }] }, }); }); diff --git a/v5/src/app/api/health/route.test.ts b/v5/src/app/api/health/route.test.ts index 1d367d0..ef6bb9b 100644 --- a/v5/src/app/api/health/route.test.ts +++ b/v5/src/app/api/health/route.test.ts @@ -1,40 +1,18 @@ -import { http, HttpResponse } from "msw"; - -import { server } from "../../../../test/msw/server"; -import { DB_IDS } from "../../../../test/msw/handlers"; +// @vitest-environment node +// `nextCacheMock` is imported first on purpose: `vi.mock` is hoisted above +// every import, and its factory can only reach a module imported before the +// one it replaces. import { nextCacheMock } from "../../../../test/mocks/next-cache"; vi.mock("next/cache", () => nextCacheMock()); import { cacheLife } from "next/cache"; -import { mockTools } from "../../../components/mock-catalog"; import { HEALTH_CACHE } from "../../../lib/cache"; +import { resetDbForTests } from "../../../lib/db/client"; import { GET } from "./route"; // ── Helpers ───────────────────────────────────────────────────────── -function stubNotionEnv() { - vi.stubEnv("NOTION_API_KEY", "secret_test"); - vi.stubEnv("NOTION_DB_TOOLS", DB_IDS.tools); - vi.stubEnv("NOTION_DB_CATEGORIES", DB_IDS.categories); - vi.stubEnv("NOTION_DB_LOCATIONS", DB_IDS.locations); - vi.stubEnv("NOTION_DB_UNITS", DB_IDS.units); - vi.stubEnv("NOTION_DB_RESOURCES", DB_IDS.resources); - vi.stubEnv("NOTION_DB_MAINTENANCE_LOGS", DB_IDS.maintenance_logs); - vi.stubEnv("NOTION_DB_FLAGS", DB_IDS.flags); -} - -function unsetNotionEnv() { - vi.stubEnv("NOTION_API_KEY", ""); - vi.stubEnv("NOTION_DB_TOOLS", ""); - vi.stubEnv("NOTION_DB_CATEGORIES", ""); - vi.stubEnv("NOTION_DB_LOCATIONS", ""); - vi.stubEnv("NOTION_DB_UNITS", ""); - vi.stubEnv("NOTION_DB_RESOURCES", ""); - vi.stubEnv("NOTION_DB_MAINTENANCE_LOGS", ""); - vi.stubEnv("NOTION_DB_FLAGS", ""); -} - // The in-memory limiter is a per-process singleton keyed by IP, so every test // gets its own address rather than eating a shared window. let ipCounter = 0; @@ -45,196 +23,130 @@ function makeRequest(ip = `10.0.0.${++ipCounter}`) { } beforeEach(() => { - // The degraded paths log the real reason server-side; keep it out of the + // Force the demo substrate regardless of the host shell's own env, matching + // the rest of the catalogue test suite (see src/lib/catalog.test.ts). + vi.stubEnv("DATABASE_URL", ""); + // The degraded path logs the real reason server-side; keep it out of the // test output (and prove nothing about it reaches the response body below). vi.spyOn(console, "warn").mockImplementation(() => {}); }); -// ── Healthy ───────────────────────────────────────────────────────── +afterEach(() => { + resetDbForTests(); +}); -describe("GET /api/health — healthy", () => { - it("returns 200 with status ok, notion ok, and a live catalog", async () => { - stubNotionEnv(); +// ── Healthy (demo substrate) ──────────────────────────────────────── +// +// `DATABASE_URL` unset is the default in every test, and it is a normal, +// expected state — not a failure — so it reports `status: "ok"` with +// `database: "demo"`, distinct from a real Postgres that is actually down. +describe("GET /api/health — healthy (demo substrate)", () => { + it("returns 200 with status ok, database demo, and the seeded tool count", async () => { const res = await GET(makeRequest()); expect(res.status).toBe(200); const body = await res.json(); expect(body.status).toBe("ok"); - expect(body.notion).toBe("ok"); - expect(body.catalog).toBe("live"); - expect(body.toolCount).toBeGreaterThan(0); + expect(body.database).toBe("demo"); + expect(body.catalog).toBe("demo"); + expect(body.toolCount).toBe(2); // the seeded demo catalogue: form-4, trotec-speedy-400 expect(Number.isNaN(Date.parse(body.checkedAt))).toBe(false); }); - it("probes Notion with a single page_size:1 query against the Tools database", async () => { - stubNotionEnv(); - - const probes: Array<{ db: string; body: unknown }> = []; - server.use( - http.post( - `https://api.notion.com/v1/databases/${DB_IDS.tools}/query`, - async ({ request }) => { - probes.push({ db: DB_IDS.tools, body: await request.json() }); - return HttpResponse.json({ results: [], has_more: false, next_cursor: null }); - } - ) - ); - - await GET(makeRequest()); - - // First hit is the health probe itself; anything after is the catalog read. - expect(probes.length).toBeGreaterThan(0); - expect(probes[0].body).toEqual({ page_size: 1 }); - }); - it("caches the probe on the 30s health profile", async () => { - stubNotionEnv(); - await GET(makeRequest()); expect(vi.mocked(cacheLife)).toHaveBeenCalledWith(HEALTH_CACHE); }); it("does not let a client or CDN store the verdict", async () => { - stubNotionEnv(); - const res = await GET(makeRequest()); expect(res.headers.get("cache-control")).toBe("no-store"); }); }); -// ── Unconfigured ──────────────────────────────────────────────────── - -describe("GET /api/health — unconfigured", () => { - it("returns 503 with notion unconfigured when the Notion env is empty", async () => { - unsetNotionEnv(); - - const res = await GET(makeRequest()); - - // The status code is the point: an uptime monitor alerts on 503, and would - // never notice a 200 carrying "degraded". - expect(res.status).toBe(503); - const body = await res.json(); - expect(body.status).toBe("degraded"); - expect(body.notion).toBe("unconfigured"); - expect(body.catalog).toBe("mock"); - expect(body.toolCount).toBe(mockTools.length); - }); - - it("returns 503 when a single NOTION_DB_* variable is dropped", async () => { - stubNotionEnv(); - vi.stubEnv("NOTION_DB_UNITS", ""); - - const res = await GET(makeRequest()); +// ── Unreachable (a configured database that pingDb cannot reach) ──── +// +// `dataSubstrate()` and `pingDb()` are mocked directly rather than pointed at +// a real broken connection string, matching the `src/lib/catalog.test.ts` +// "a database failure" pattern: `vi.doMock` + a fresh dynamic `import()`, so +// only this one route instance sees the failure and every other test keeps +// the real module and its memoised PGlite handle. - expect(res.status).toBe(503); - const body = await res.json(); - expect(body.notion).toBe("unconfigured"); +describe("GET /api/health — unreachable", () => { + afterEach(() => { + vi.doUnmock("../../../lib/db/client"); + vi.resetModules(); }); -}); -// ── Unreachable ───────────────────────────────────────────────────── + it("returns 503 with database unreachable, degraded status, no live catalog, and no invented count", async () => { + vi.resetModules(); + vi.doMock("../../../lib/db/client", () => ({ + dataSubstrate: () => "neon", + pingDb: () => Promise.reject(new Error("connection terminated unexpectedly")), + })); + const { GET: GETWithBrokenDb } = await import("./route"); -describe("GET /api/health — unreachable", () => { - it("returns 503 with notion unreachable when the Notion query throws", async () => { - stubNotionEnv(); - server.use( - http.post("https://api.notion.com/v1/databases/:id/query", () => HttpResponse.error()) - ); - - const res = await GET(makeRequest()); + const res = await GETWithBrokenDb(makeRequest()); expect(res.status).toBe(503); const body = await res.json(); expect(body.status).toBe("degraded"); - expect(body.notion).toBe("unreachable"); - expect(body.catalog).toBe("mock"); - expect(body.toolCount).toBe(mockTools.length); - }); - - it("returns 503 with notion unreachable when Notion answers non-2xx", async () => { - stubNotionEnv(); - server.use( - http.post("https://api.notion.com/v1/databases/:id/query", () => - HttpResponse.json({ object: "error", code: "unauthorized" }, { status: 401 }) - ) - ); - - const res = await GET(makeRequest()); - - expect(res.status).toBe(503); - expect((await res.json()).notion).toBe("unreachable"); + expect(body.database).toBe("unreachable"); + expect(body.catalog).toBe("demo"); + // Article 4: never invented data — a Postgres this probe just proved down + // is never asked for a tool count. + expect(body.toolCount).toBe(0); }); }); // ── Disclosure ────────────────────────────────────────────────────── // -// A health endpoint that names the missing variable or echoes the Notion error -// hands an attacker the configuration. The body is three enum words, a count, -// and a timestamp — nothing else, on any path. +// A health endpoint that echoes a driver error or connection string hands an +// attacker the configuration. The body is three enum words, a count, and a +// timestamp — nothing else, on any path. describe("GET /api/health — leaks nothing", () => { - const FORBIDDEN = [ - "NOTION_", - "notion.com", - "secret_test", - "Bearer", - DB_IDS.tools, - DB_IDS.units, - ]; - - it("names no env var, Notion id, or error text when unconfigured", async () => { - unsetNotionEnv(); - - const raw = await (await GET(makeRequest())).text(); - - for (const needle of FORBIDDEN) expect(raw).not.toContain(needle); - }); - - it("does not echo the Notion error body when unreachable", async () => { - stubNotionEnv(); - server.use( - http.post("https://api.notion.com/v1/databases/:id/query", () => - HttpResponse.json( - { - object: "error", - code: "unauthorized", - message: "API token is invalid: secret_test against db-tools", - }, - { status: 401 } - ) - ) - ); - - const raw = await (await GET(makeRequest())).text(); - - for (const needle of FORBIDDEN) expect(raw).not.toContain(needle); - expect(raw).not.toContain("API token is invalid"); - expect(raw).not.toContain("unauthorized"); + afterEach(() => { + vi.doUnmock("../../../lib/db/client"); + vi.resetModules(); }); it("returns exactly the documented fields and nothing more", async () => { - unsetNotionEnv(); - const body = await (await GET(makeRequest())).json(); expect(Object.keys(body).sort()).toEqual([ "catalog", "checkedAt", - "notion", + "database", "status", "toolCount", ]); }); + + it("does not echo the driver error or connection string when unreachable", async () => { + const SENSITIVE = "postgres://admin:hunter2@db.internal.example.com/prod"; + vi.resetModules(); + vi.doMock("../../../lib/db/client", () => ({ + dataSubstrate: () => "neon", + pingDb: () => Promise.reject(new Error(`could not connect to ${SENSITIVE}: ECONNREFUSED`)), + })); + const { GET: GETWithBrokenDb } = await import("./route"); + + const raw = await (await GETWithBrokenDb(makeRequest())).text(); + + for (const needle of [SENSITIVE, "hunter2", "ECONNREFUSED", "postgres://"]) { + expect(raw).not.toContain(needle); + } + }); }); // ── Rate limiting ─────────────────────────────────────────────────── describe("GET /api/health — rate limiting", () => { - it("429s a caller that hammers it, before touching Notion", async () => { - unsetNotionEnv(); + it("429s a caller that hammers it, before touching the database", async () => { const ip = "10.9.9.9"; let last = await GET(makeRequest(ip)); diff --git a/v5/src/app/api/health/route.ts b/v5/src/app/api/health/route.ts index a0c28b7..602accc 100644 --- a/v5/src/app/api/health/route.ts +++ b/v5/src/app/api/health/route.ts @@ -1,90 +1,84 @@ import { cacheLife } from "next/cache"; import { HEALTH_CACHE } from "../../../lib/cache"; import { getCatalogTools } from "../../../lib/catalog"; -import { getNotionEnvContract } from "../../../lib/notion"; +import { dataSubstrate, pingDb } from "../../../lib/db/client"; import { rateLimitAsync } from "../../../lib/rate-limit"; import { resolveIdentity } from "../../../lib/auth/identity"; -import { mockTools } from "../../../components/mock-catalog"; // `runtime` cannot be set when nextConfig.cacheComponents is enabled. // Default Node.js runtime is used. -const NOTION_API_URL = "https://api.notion.com/v1"; -const NOTION_VERSION = "2022-06-28"; const PROBE_TIMEOUT_MS = 5_000; -type NotionHealth = "ok" | "unreachable" | "unconfigured"; +type DatabaseHealth = "ok" | "unreachable" | "demo"; interface HealthReport { status: "ok" | "degraded"; - notion: NotionHealth; - catalog: "live" | "mock"; + database: DatabaseHealth; + catalog: "live" | "demo"; toolCount: number; checkedAt: string; } /** - * One minimal, uncached Notion call — `page_size: 1` against the Tools database. - * It deliberately does *not* go through `catalog.ts`, whose day-long cache would - * happily keep answering through an outage (which is the point of that cache and - * the reason this probe exists). + * `pingDb()` carries no signal of its own (spec §3.2), and a hung connection + * must not hang this endpoint — race it against a plain timer instead. + */ +function withTimeout(promise: Promise, ms: number): Promise { + let timer: ReturnType; + const timeout = new Promise((_, reject) => { + timer = setTimeout(() => reject(new Error("Health probe timed out")), ms); + }); + return Promise.race([promise, timeout]).finally(() => clearTimeout(timer)); +} + +/** + * One `select 1` against Postgres — or `"demo"` with no round trip at all when + * `DATABASE_URL` is unset, since that runs the app on the in-process PGlite + * demo seed (spec §3.10) rather than on nothing. Demo is a known, expected + * substrate, not a failure, so it gets its own word instead of collapsing into + * `"ok"` or `"unreachable"`. * - * Nothing about the failure escapes this function: the caller gets one of three - * words. A health endpoint that reports `NOTION_DB_TOOLS is missing` or echoes a - * Notion error body hands an attacker the configuration, so the detail is logged - * server-side and nowhere else. + * Nothing about a real failure escapes this function beyond one of three + * words — a health endpoint that echoes a connection string or driver error + * hands an attacker the configuration, so the detail is logged server-side + * and nowhere else. */ -async function probeNotion(): Promise { - const configured = getNotionEnvContract().every((key) => Boolean(process.env[key])); - if (!configured) return "unconfigured"; +async function probeDatabase(): Promise { + if (dataSubstrate() === "pglite-demo") return "demo"; try { - const res = await fetch( - `${NOTION_API_URL}/databases/${process.env.NOTION_DB_TOOLS}/query`, - { - method: "POST", - headers: { - Authorization: `Bearer ${process.env.NOTION_API_KEY}`, - "Content-Type": "application/json", - "Notion-Version": NOTION_VERSION, - }, - body: JSON.stringify({ page_size: 1 }), - signal: AbortSignal.timeout(PROBE_TIMEOUT_MS), - } - ); - - if (!res.ok) { - console.warn("Health probe: Notion query failed with status", res.status); - return "unreachable"; - } + await withTimeout(pingDb(), PROBE_TIMEOUT_MS); return "ok"; } catch (error) { - console.warn("Health probe: Notion unreachable:", error); + console.warn("Health probe: database unreachable:", error); return "unreachable"; } } /** - * Cached for ~30s so a monitor polling every minute costs at most one Notion - * call per 30s and the endpoint cannot be used to hammer Notion. `checkedAt` is + * Cached for ~30s so a monitor polling every minute costs at most one probe + * per 30s and the endpoint cannot be used to hammer Postgres. `checkedAt` is * captured with the cached value, so it reports when the probe actually ran. * - * The tool count comes from the catalog only when Notion answered; otherwise the - * catalog is on its mock fallback and counting it would mean five more failing - * Notion calls to learn something already known. + * The tool count comes from the catalogue whenever there is one to safely + * read: on the demo seed `getCatalogTools()` reads the local PGlite instance, + * which costs nothing extra to ask. A configured-but-unreachable Postgres is + * the one case that skips it — counting from a database this same probe just + * proved was down would mean either inventing a number or making the failing + * call twice, and Article 4 rules out the former. */ async function checkHealth(): Promise { "use cache"; cacheLife(HEALTH_CACHE); - const notion = await probeNotion(); - const live = notion === "ok"; - const toolCount = live ? (await getCatalogTools()).length : mockTools.length; + const database = await probeDatabase(); + const toolCount = database === "unreachable" ? 0 : (await getCatalogTools()).length; return { - status: live ? "ok" : "degraded", - notion, - catalog: live ? "live" : "mock", + status: database === "unreachable" ? "degraded" : "ok", + database, + catalog: database === "ok" ? "live" : "demo", toolCount, checkedAt: new Date().toISOString(), }; @@ -102,7 +96,7 @@ async function checkHealth(): Promise { * around. */ export async function GET(req: Request) { - // Rate limit before the probe — cheap as it is, it still touches Notion. + // Rate limit before the probe — cheap as it is, it still touches Postgres. const identity = await resolveIdentity(req); const { allowed } = await rateLimitAsync(`health:${identity.rateLimitKey}`, { limit: 30, diff --git a/v5/src/app/api/mcp/route.test.ts b/v5/src/app/api/mcp/route.test.ts index 5cb397b..fa72b54 100644 --- a/v5/src/app/api/mcp/route.test.ts +++ b/v5/src/app/api/mcp/route.test.ts @@ -1,3 +1,4 @@ +// @vitest-environment node /** * Integration tests for the MCP API route (`src/app/api/mcp/route.ts`). * @@ -7,10 +8,10 @@ * the route's `POST` handler and assert the JSON responses. * - `enableJsonResponse: true` in the route means the transport returns a single * JSON object (not an SSE stream), so `await res.json()` works directly. - * - Catalog comes from the mock-catalog fallback (no NOTION_* env set), so tool - * results are deterministic and offline. - * - `@/lib/notion`'s `fetchMaintenanceLogsByUnit` is mocked so the maintenance - * tools are deterministic without any network. + * - Catalog and maintenance history come from the demo-seeded PGlite database + * `getDb()` hands out with `DATABASE_URL` unset, so results are deterministic + * and offline. The seed has no maintenance logs; the tests that need them + * write them. * * SDK quirk discovered: the WebStandardStreamableHTTPServerTransport is created * with `sessionIdGenerator: undefined` (stateless mode). In that mode the @@ -23,27 +24,30 @@ * each method statelessly. */ +// `nextCacheMock` is imported first on purpose: `vi.mock` is hoisted above every +// import, and its factory can only reach a module imported before the one it +// replaces. `catalog.ts` imports `cacheTag`/`cacheLife`, which only work inside +// the Next build. +import { nextCacheMock } from "../../../../test/mocks/next-cache"; +import { eq } from "drizzle-orm"; import { POST, GET } from "@/app/api/mcp/route"; +import { getDb, resetDbForTests } from "@/lib/db/client"; +import { maintenanceLogs, units } from "@/lib/db/schema/index"; -// Partial-mock @/lib/notion: override ONLY `fetchMaintenanceLogsByUnit` and keep -// every other export real. (catalog.ts also imports from this module — e.g. -// getNotionEnvContract, fetchAllTools — so a full replacement would break the -// catalog's mock-fallback path.) The catalog itself still comes from the -// mock-catalog because NOTION_* env stays unset. -vi.mock("@/lib/notion", async (importOriginal) => { - const actual = await importOriginal(); - return { - ...actual, - fetchMaintenanceLogsByUnit: vi.fn(async () => []), - }; -}); - -import { fetchMaintenanceLogsByUnit } from "@/lib/notion"; - -const fetchLogsMock = vi.mocked(fetchMaintenanceLogsByUnit); +vi.mock("next/cache", () => nextCacheMock()); const MCP_URL = "http://localhost/api/mcp"; +/** The uuid of a seeded unit — what the capability resolves a label to. */ +async function unitId(label: string): Promise { + const db = await getDb(); + const [row] = await db + .select({ id: units.id }) + .from(units) + .where(eq(units.unitLabel, label)); + return row.id; +} + /** Build a JSON-RPC POST Request with the Accept header the transport requires. */ function rpcRequest( body: unknown, @@ -89,8 +93,18 @@ async function callTool(name: string, args: Record = {}) { } beforeEach(() => { - fetchLogsMock.mockReset(); - fetchLogsMock.mockResolvedValue([]); + vi.stubEnv("DATABASE_URL", ""); +}); + +afterEach(async () => { + // The seeded catalogue rows are read, never written; only the logs a test + // wrote need clearing. + const db = await getDb(); + await db.delete(maintenanceLogs); +}); + +afterAll(() => { + resetDbForTests(); }); // ── GET handler ────────────────────────────────────────────────────── @@ -223,7 +237,7 @@ describe("tools/call: list_tools", () => { it("returns a structured summary of the whole catalog", async () => { const { json } = await callTool("list_tools"); const parsed = JSON.parse(resultText(json)); - // mock-catalog has 2 tools (Form 4, Trotec Speedy 400). + // The demo seed is 2 tools (Form 4, Trotec Speedy 400). expect(parsed.count).toBe(2); const names = parsed.tools.map((t: { name: string }) => t.name); expect(names).toContain("Form 4"); @@ -301,7 +315,7 @@ describe("tools/call: get_tool_details", () => { // ── tools/call: get_unit_details ───────────────────────────────────── describe("tools/call: get_unit_details", () => { - it("returns JSON for a real mock unit label", async () => { + it("returns JSON for a seeded unit label", async () => { const { json } = await callTool("get_unit_details", { unit_label: "Form 4 // A" }); const text = resultText(json); const parsed = JSON.parse(text); @@ -324,37 +338,44 @@ describe("tools/call: get_unit_details", () => { describe("tools/call: get_maintenance_history", () => { it("returns a JSON list of logs when the unit has maintenance history", async () => { - fetchLogsMock.mockResolvedValue([ + const db = await getDb(); + const form4A = await unitId("Form 4 // A"); + await db.insert(maintenanceLogs).values([ + { + title: "Resin tank cloudy", + unitId: form4A, + type: "repair", + priority: "high", + status: "open", + dateReported: "2024-09-01", + description: "The resin tank film is clouded and needs replacement.", + }, + // A log on another unit, to prove the tool reads by unit rather than + // returning everything. { - id: "log-1", - createdTime: "2024-09-01T10:00:00.000Z", - lastEditedTime: "2024-09-01T10:00:00.000Z", - fields: { - title: "Resin tank cloudy", - type: "Repair", - priority: "High", - status: "Open", - date_reported: "2024-09-01", - description: "The resin tank film is clouded and needs replacement.", - }, + title: "Lens dirty", + unitId: await unitId("Trotec Speedy 400"), + dateReported: "2024-09-02", }, - ] as Awaited>); + ]); const { json } = await callTool("get_maintenance_history", { unit_label: "Form 4 // A", }); const text = resultText(json); const parsed = JSON.parse(text); + expect(parsed.unit_id).toBe(form4A); expect(parsed.unit_label).toBe("Form 4 // A"); expect(parsed.maintenance_logs).toHaveLength(1); expect(parsed.maintenance_logs[0].title).toBe("Resin tank cloudy"); + // Stored `repair` / `high` / `open`, shown in the words the assistant has + // always been given. + expect(parsed.maintenance_logs[0].type).toBe("Repair"); expect(parsed.maintenance_logs[0].priority).toBe("High"); - // Confirm the fetch was driven with the matched unit's id. - expect(fetchLogsMock).toHaveBeenCalledWith("unit-form-4-a"); + expect(parsed.maintenance_logs[0].status).toBe("Open"); }); it("returns an empty log list when the unit has no logs", async () => { - fetchLogsMock.mockResolvedValue([]); const { json } = await callTool("get_maintenance_history", { unit_label: "Form 4 // A", }); diff --git a/v5/src/app/api/projects/route.test.ts b/v5/src/app/api/projects/route.test.ts index 2f9c367..c9a79a6 100644 --- a/v5/src/app/api/projects/route.test.ts +++ b/v5/src/app/api/projects/route.test.ts @@ -1,3 +1,4 @@ +// @vitest-environment node import { http, HttpResponse } from "msw"; import { server } from "../../../../test/msw/server"; @@ -6,11 +7,38 @@ import { createSessionPayload, signSession, } from "@/lib/auth/session-cookie"; +import { nextCacheMock } from "../../../../test/mocks/next-cache"; + +// The route translates catalogue ids to Notion page ids through Postgres, and +// that module's neighbours import cacheTag/cacheLife. +vi.mock("next/cache", () => nextCacheMock()); + +import { getCatalogTools } from "@/lib/catalog"; +import { resetDbForTests } from "@/lib/db/client"; +import { DEMO_FORM_4_NOTION_PAGE_ID } from "@/lib/db/demo-seed"; const NOTION = "https://api.notion.com/v1"; const PROJECTS_DB = "db-projects"; const AUTH_SECRET = "projects-route-test-secret"; +// `tools_used` is a Notion relation, but the form now submits catalogue ids — +// Postgres uuids minted at seed time. The demo seed is the substrate here +// (`DATABASE_URL` unset): the Form 4 came from Notion and has a page id, the +// Trotec did not and has none. +let FORM_4_ID = ""; +let TROTEC_ID = ""; + +beforeEach(async () => { + vi.stubEnv("DATABASE_URL", ""); + const tools = await getCatalogTools(); + FORM_4_ID = tools.find((tool) => tool.slug === "form-4")?.id ?? ""; + TROTEC_ID = tools.find((tool) => tool.slug === "trotec-speedy-400")?.id ?? ""; +}); + +afterAll(() => { + resetDbForTests(); +}); + // ── Helpers ───────────────────────────────────────────────────────── // The in-memory limiter is a per-process singleton keyed by `projects:` — @@ -68,7 +96,7 @@ function validPayload(overrides: Record = {}) { title: "Lamp from scrap plywood", author: "Ada Lovelace", body: "Cut on the laser, glued, sanded.", - tools: ["tool-form-4"], + tools: [FORM_4_ID], materials: ["Plywood"], photos: [{ id: "file-upload-1", name: "cover.png" }], ...overrides, @@ -167,7 +195,10 @@ describe("POST /api/projects (drafts by default)", () => { await post( submitRequest( - validPayload({ link: "https://example.com/lamp", tools: ["tool-form-4", "tool-2"] }) + validPayload({ + link: "https://example.com/lamp", + tools: [FORM_4_ID, TROTEC_ID], + }) ) ); @@ -179,8 +210,10 @@ describe("POST /api/projects (drafts by default)", () => { rich_text: [{ text: { content: "Ada Lovelace" } }], }); expect(properties.link).toEqual({ url: "https://example.com/lamp" }); + // Each catalogue id is translated to the tool's Notion page; the Trotec has + // no imported page, so it drops out rather than failing the whole write. expect(properties.tools_used).toEqual({ - relation: [{ id: "tool-form-4" }, { id: "tool-2" }], + relation: [{ id: DEMO_FORM_4_NOTION_PAGE_ID }], }); expect(properties.materials).toEqual({ multi_select: [{ name: "Plywood" }] }); // Photos go in as file_upload references from /api/upload-notion. @@ -439,14 +472,16 @@ describe("POST /api/projects (validation)", () => { it("accepts exactly 20 tools", async () => { stubProjectsEnv(); const calls = captureNotionCreate(); - const tools = Array.from({ length: 20 }, (_, i) => `tool-${i}`); + const tools = Array.from({ length: 20 }, () => crypto.randomUUID()); const res = await post(submitRequest(validPayload({ tools }))); + // The cap is what is under test: twenty is not one too many. None of these + // ids is a real tool, so nothing survives the Notion-page translation and + // the submission is filed without a relation rather than refused. expect(res.status).toBe(201); - expect( - (calls[0].properties?.tools_used as { relation: unknown[] }).relation - ).toHaveLength(20); + expect(calls).toHaveLength(1); + expect(calls[0].properties?.tools_used).toBeUndefined(); }); it("ignores non-string entries in tools and materials", async () => { @@ -455,13 +490,13 @@ describe("POST /api/projects (validation)", () => { const res = await post( submitRequest( - validPayload({ tools: ["tool-form-4", 7, null], materials: [{}, "Plywood"] }) + validPayload({ tools: [FORM_4_ID, 7, null], materials: [{}, "Plywood"] }) ) ); expect(res.status).toBe(201); expect(calls[0].properties?.tools_used).toEqual({ - relation: [{ id: "tool-form-4" }], + relation: [{ id: DEMO_FORM_4_NOTION_PAGE_ID }], }); expect(calls[0].properties?.materials).toEqual({ multi_select: [{ name: "Plywood" }], diff --git a/v5/src/app/api/projects/route.ts b/v5/src/app/api/projects/route.ts index 1222f98..1aefd9a 100644 --- a/v5/src/app/api/projects/route.ts +++ b/v5/src/app/api/projects/route.ts @@ -4,6 +4,7 @@ import { hasProjectsEnv, type ProjectWriteFields, } from "../../../lib/notion"; +import { notionPageIdsForTools } from "../../../lib/data/notion-ids"; import type { ProjectRecord } from "../../../lib/types"; import { rateLimitAsync } from "../../../lib/rate-limit"; import { resolveIdentity } from "../../../lib/auth/identity"; @@ -144,6 +145,13 @@ export async function POST(req: NextRequest) { ); } + // `tools` arrives as catalogue ids, which are Postgres uuids since the read + // path moved (spec §3.10), while `tools_used` is a Notion relation. Translate + // through `tools.notion_page_id` and drop what does not resolve: Notion + // rejects the whole page for one unknown relation id, and losing a student's + // whole write-up over a tool link is the wrong way to fail (Article 4). + const toolPageIds = await notionPageIdsForTools(tools); + try { const record = await submitProject({ title, @@ -154,7 +162,7 @@ export async function POST(req: NextRequest) { // no session, no email, submission still succeeds. author_email: identity.email || undefined, link: link || undefined, - tools_used: tools, + tools_used: toolPageIds, materials, photo_uploads: photoUploads, }); diff --git a/v5/src/app/projects/page.tsx b/v5/src/app/projects/page.tsx index 4d0b583..ad5ca51 100644 --- a/v5/src/app/projects/page.tsx +++ b/v5/src/app/projects/page.tsx @@ -36,7 +36,7 @@ export default async function ProjectsPage() { projects.map((project) => (