Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions .claude/skills/frontend-htmx/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,9 +62,11 @@ Server can control behavior via htmx response headers (`HX-Reswap`, `HX-Redirect

## Checking a frontend change

The browser suite (`e2e/browser`, from W6 on; `docs/testing.md` has the "Browser tests" section) runs the
real app with freshly built assets in Chromium. It fails on console errors, uncaught exceptions and CSP
violations on every page, so an inline script or a style without the nonce shows up without a dedicated test.
The browser suite (`e2e/browser`; `docs/testing.md` has the "Browser tests" section with a worked example)
runs the real app with freshly built assets in Chromium. It fails on console errors, uncaught exceptions,
CSP violations and failed or 404/5xx requests on every page, so an inline script or a style without the
nonce shows up without a dedicated test. A test that provokes an error on purpose exempts it with
`browser.Allow`.

- `make test-ui RUN='<TestName>'` while iterating; `HEADED=1 SLOWMO=250` to watch it run. The target runs
`yarn build` first. Run the whole suite once before committing.
Expand Down
7 changes: 4 additions & 3 deletions .claude/skills/test-failure/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,10 @@ Goal: find the cause while reading as little as possible. Stop at the first step
3. **One test, verbose:** `go test ./pkg/x/ -run '^TestName$' -count=1 -v 2>&1 | tail -n 40`.
Add `/subtest_name` to the `-run` pattern for table tests.
4. **Search the log, don't print it:** `grep -n -A5 'TestName' <log path>`. Never `cat` a log.
**Browser tests** (`e2e/browser`, `make test-ui`): the report line names a screenshot and a trace. Look
at the screenshot first (the Read tool shows images), then the guard message: console error, CSP
violation or failed request. Open the trace only if both leave it unclear. Re-run one test with
**Browser tests** (`e2e/browser`, `make test-ui`; see "Browser tests" in `docs/testing.md`): the
report line names a screenshot and a trace in `.ui-artifacts/`. Look at the screenshot first (the Read
tool shows images), then the guard message (`browser: console.error: …`, a CSP violation, a failed
request or a 404/5xx response). Open the trace only if both leave it unclear. Re-run one test with
`make test-ui RUN='^TestName$'`. Stale assets are a common cause, and `make test-ui` rebuilds them.
A locator timeout usually means the markup changed, not that the app is slow; don't raise timeouts.
An `architecture` test failure (`pkg/arch`, from RS on) names the package and the forbidden import:
Expand Down
9 changes: 7 additions & 2 deletions .claude/skills/wave-run/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,10 @@ subagents, whose context is thrown away.
the plan says so (contract-defining work), or after a task has failed twice.
- Tasks that own disjoint files go out in **one message** with several `Agent` calls,
`subagent_type: "general-purpose"`. Never `fork`: a fork drags the coordinator's context along.
- Parallel tasks that write files in **one package** (W3's `e2e`, W6's `e2e/browser`) each iterate under
their own build tag (`//go:build browser && b3`, run with `-tags browser,b3`) and switch to the shared
tag before reporting, so one agent's half-written file doesn't break the others' compile. Shared build
steps such as `yarn build` run once in the coordinator before dispatch, never in each agent.
- A broad question ("where is X used across the handlers?") goes to an `Explore` subagent, which returns the
answer rather than the files. The coordinator's own lookups use the LSP tool.
- **Subagents run no git commands** and don't touch `go.mod` (only W0 and W4.S2 do, each as a single task).
Expand Down Expand Up @@ -88,14 +92,15 @@ Variations by wave:

- **W6 (browser):** step 1 is `make test-ui RUN=<the task's tests>`, then again with `COUNT=3` (a flaky
test is sent back, not accepted). The mutation check breaks a Stimulus
controller or an htmx attribute the task covers.
controller or an htmx attribute the task covers. At the end of the wave, empty each controller's
`connect()` in turn and run the suite: a controller that survives is untested, whatever the reports say.
- **R-waves and RS (refactors):** replace step 1 with `make test-q PKG=<task packages>` plus
`git diff --stat -- e2e/`, which must be empty. Also check that the task shrank the `pkg/arch` allowlist
and didn't grow it. The mutation check becomes: break the service method the task extracted and watch an
E2E or service test fail. Run `make test-ui` once per commit, next to `make check-q`.

Not part of the budget: reading every test file. Read a test only when the mutation check fails to fail.
Run `make check-q` once before each commit; commit per task.
Run `make check-q` and `make lint` (CI lints; `check-q` does not) once before each commit; commit per task.

For each bug a subagent reports: check `gh issue list --label bug` and the "Known bugs" list in
`docs/plan/wb.md`, file an issue if it is new (security bugs go to the owner, not a public issue), and put
Expand Down
50 changes: 50 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,56 @@ jobs:
commit_message: "chore: apply go fix"
file_pattern: "*.go"

browser:
name: Browser
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7

- name: Set up Go
uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: true

- name: Set up Node.js
uses: actions/setup-node@v7
with:
node-version: "24"
cache: "yarn"
cache-dependency-path: cmd/web/yarn.lock

- name: Install system dependencies
run: sudo apt-get update && sudo apt-get install -y libvips-dev

- name: Install frontend dependencies
run: yarn install --frozen-lockfile
working-directory: cmd/web

- name: Cache Playwright driver and Chromium
uses: actions/cache@v6
with:
path: |
~/.cache/ms-playwright
~/.cache/ms-playwright-go
key: playwright-${{ runner.os }}-${{ hashFiles('go.sum') }}

- name: Install Chromium
run: make ui-deps UI_DEPS_FLAGS=--with-deps

- name: Browser tests
run: make test-ui
env:
QRUN_MAX: "400"

- name: Upload traces and screenshots
if: failure()
uses: actions/upload-artifact@v7
with:
name: ui-artifacts
path: .ui-artifacts/
if-no-files-found: ignore

frontend:
name: Frontend
runs-on: ubuntu-latest
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
.cover/
coverage.out
.ui-artifacts/
2 changes: 2 additions & 0 deletions .golangci.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
version: "2"
run:
tests: true
build-tags:
- browser
linters:
enable:
- misspell
Expand Down
34 changes: 29 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
.PHONY: shell tunnel lint test test-short cover build check fix check-q test-q vet-q cover-q model
.PHONY: shell tunnel lint test test-short cover build check fix check-q test-q vet-q cover-q model ui-deps test-ui ui-trace

PKG ?= ./...
TAGS ?=
tags_flag = $(if $(TAGS),-tags $(TAGS))

shell:
flyctl postgres connect -a pcomdb
Expand Down Expand Up @@ -46,20 +48,42 @@ fix:
# Quiet variants for agents: one line on success, a trimmed report on failure
# (full output goes to a log file). They run the same steps as `make check`,
# which stays the verbose CI form.
# Narrow with PKG, for example `make test-q PKG=./pkg/links/...`.
# Narrow with PKG, for example `make test-q PKG=./pkg/links/...`, and pass
# build tags with TAGS, for example `make vet-q PKG=./e2e/browser/... TAGS=browser`.
check-q:
@tools/qrun.sh build go build -o /dev/null ./...
@tools/qrun.sh vet go vet ./...
@tools/qrun.sh test go test ./...

test-q:
@tools/qrun.sh test go test $(PKG)
@tools/qrun.sh test go test $(tags_flag) $(PKG)

vet-q:
@tools/qrun.sh vet go vet $(PKG)
@tools/qrun.sh vet go vet $(tags_flag) $(PKG)

cover-q:
@QRUN_SHOW_OK=1 tools/qrun.sh cover go test -cover $(PKG)
@QRUN_SHOW_OK=1 tools/qrun.sh cover go test $(tags_flag) -cover $(PKG)

# Browser tests (e2e/browser, build tag `browser`). `make ui-deps` installs the
# Playwright driver and Chromium once (UI_DEPS_FLAGS=--with-deps also installs
# the system libraries, on Linux). `make test-ui` builds the frontend and runs
# the suite quietly: RUN=<regex> narrows it, COUNT=<n> repeats it, and
# HEADED=1 SLOWMO=250 shows the browser (headless otherwise). A failed test logs the path of its
# trace; open it with `make ui-trace F=<path>`.
PLAYWRIGHT = go run github.com/mxschmitt/playwright-go/cmd/playwright
RUN ?=
COUNT ?= 1
UI_DEPS_FLAGS ?=

ui-deps:
$(PLAYWRIGHT) install $(UI_DEPS_FLAGS) chromium

test-ui:
@tools/qrun.sh ui-build yarn --cwd cmd/web build
@HEADED=$(HEADED) SLOWMO=$(SLOWMO) tools/qrun.sh test-ui go test -tags browser -count=$(COUNT) $(if $(RUN),-run '$(RUN)') ./e2e/browser/...

ui-trace:
$(PLAYWRIGHT) show-trace $(F)

# Shape of a generated model without reading pkg/model/core:
# `make model` lists the models, `make model T=User` prints one.
Expand Down
22 changes: 22 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,28 @@ make lint # golangci-lint
Docker must be running: tests that touch the database start a Postgres
container via `pkg/testutil/testdb`.

### Browser tests

User flows (forms, htmx swaps, Stimulus controllers) are tested in a real
Chromium driven by [playwright-go](https://github.com/mxschmitt/playwright-go).
They live in `e2e/browser` behind the build tag `browser`, so `make test` and
`make check` don't run them.

```
make ui-deps # once: installs the Playwright driver and Chromium
make test-ui # builds the frontend, then runs the suite
make test-ui RUN=TestSmoke COUNT=3 # narrow it, repeat it
make test-ui HEADED=1 SLOWMO=250 RUN=TestSmoke_LoginAndBoostedNavigation
make ui-trace F=.ui-artifacts/<Test>.trace.zip # inspect a failure
```

Chromium runs **headless** by default, so no window opens and a passing run
prints only `ok: test-ui`. Add `HEADED=1` to see the browser and `SLOWMO=<ms>`
to slow each step down. Narrow the run with `RUN` when watching, because the
tests run in parallel and each one opens its own window. A failed test saves a
screenshot and a trace under `.ui-artifacts/` and prints their paths.
`docs/testing.md` explains how the suite works and how to write a test.

### psql access

```
Expand Down
53 changes: 53 additions & 0 deletions docs/archive/history.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,3 +49,56 @@ results, and the subagents at 43k–167k (the factories were the most expensive:
each. model-shape was used through `make model` (5 calls). LSP was loaded once
in total and test-failure never, because nothing failed. This is the first
recorded wave, so there is no earlier cost line to compare with.

## W6 — Browser tests (2026-09-26, branch `test/w6-browser`)

**Built.**

- B0 (coordinator): `e2e/browser` behind the `browser` build tag, playwright-go,
`e2e.WithRealAssets()`, logged-in pages from the session cookie, guards on page
errors, `console.error`, CSP violations and failed same-origin requests, a trace
and screenshot on failure, `make ui-deps`/`test-ui`/`ui-trace`, and a `Browser`
CI job. Smoke tests for login, boosted navigation, an action button and the
error toast.
- B1–B7: one file per area (navigation, writing, comments, actions, settings,
layout sweep, accounts). The suite runs in about 35 seconds, and passes
`-count=3`.
- Every Stimulus controller except `selfsubmit` (used by no template) and
`collapse` (only under the mobile-menu test skipped on #140) fails a test when
its `connect()` is emptied. Removing `json-enc`, `head-support`, the
`htmx:responseError` handler or the `htmx:sendError` handler each fails a test.
- Every mutating browser route is used successfully, except `signup` (skipped on
#139) and `signup_waiting_list`, which is switched off in code (Q6). The API
routes are W3's.
- Bugs filed: #139 (pages rendered from a bare map have no CSP nonce: `/signup`,
`/confirm_signup`, `/articles`, `/confirm_waiting_list`), #140 (opening the
mobile menu violates `style-src-attr`), #141 (a submit right after the post
form re-renders itself can go out natively and get a 403).

**Wrong.**

- The plan said comments update "without a full reload". The comment form
reloads the page on purpose (it keeps the scroll position), so a subagent
pinned the intended behavior as a bug. The test now asserts the reload
behavior, and no issue was filed.
- The share link has no copy button, so the planned "copy the share link"
step had nothing to click. The clipboard controller is tested through the
API key instead.
- The haiku layout sweep was vacuous (`window.scrollWidth` is undefined, so
the overflow check could never fail) and allowed CSP violations. It was
redone at sonnet.

**Left out.** `/confirm_signup` in the layout sweep (it needs a user with a
known confirmation seed, and #139 blocks the page anyway). The `Browser`
check becomes required on `master` right after this PR merges (not before, or
open PRs without the job would wait forever); that is the owner's step.

**Cost.** 2 coordinator sessions (B0, then B1–B7) and 8 subagents (7 sonnet,
1 haiku), 945 turns in total. The coordinators peaked at 133k and 140k
context with 92k and 108k of tool results; subagents at 79k–262k, with the
settings and writing tasks the most expensive (150–167 turns, 234k–271k of
results). 91 wasteful calls, almost all `cat` of whole files. Skills:
frontend-htmx by 4 agents, wave-run twice, wave-close once; model-shape and
test-failure never, and LSP barely (2k of results). Compared with W0, the
wave cost about three times the turns for twice the subagents: browser
tasks iterate far more than unit tests do.
2 changes: 1 addition & 1 deletion docs/implementation-plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Related documents:
| W3 | End-to-end HTTP tests: server rules | W0 | not started | `test/w3-e2e` |
| W4 | Local stack (Postgres, tommy for mail and S3), dev tooling container, app in compose, seed | W0 | not started | `test/w4-local-stack` |
| W5 | Coverage ratchet | W1–W4 | not started | `test/w5-ratchet` |
| W6 | Browser tests (playwright-go): user flows | W0 | not started | `test/w6-browser` |
| W6 | Browser tests (playwright-go): user flows | W0 | done | `test/w6-browser` |
| WB | Bug-fix wave (#108–#117, #119–#122) | W1–W3 | not started | `fix/wb-survey-bugs` |
| R1 | Router decomposition (move handlers) | W3, W6, WB | planned | `refactor/r1-router` |
| RS | Repositories and services, thin handlers | R1 | planned | `refactor/rs-layers` |
Expand Down
17 changes: 17 additions & 0 deletions docs/lessons.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,20 @@ Generalizable lessons from running the waves. Wave-specific notes go in
- **Subagents cat files.** Even with AGENTS.md's reading rules, W0's
subagents printed whole files 33 times and loaded LSP once. The task
preamble now says it explicitly.
- **Parallel subagents in one Go package.** One half-written file breaks the
compile for every agent in the package. Give each agent its own build tag
while it iterates (`//go:build browser && b3`, run with `-tags browser,b3`)
and have it switch to the shared tag before reporting. Shared build steps
(`yarn build`) run once in the coordinator, not in every agent, or they race
on the output directory.
- **A cheap tier needs a self-checking task.** A layout sweep asserts things
that pass whether or not the check works (`undefined > n` is false). Such a
task can't catch its own mistakes, so it is not cheap. Ask for a proof that
each check can fail.
- **Verify bug reports against intent.** A subagent told the wrong expected
behavior reports the real one as a bug. Before filing, check the code and
its history (`git log -S`) for a deliberate choice.
- **Mutation-sweep the controllers.** Emptying each Stimulus controller's
`connect()` in turn and running the suite found the one controller whose
tests only ever took the happy path (confirm: every test accepted the
dialog). The agents' own "covers" lists overstated coverage.
12 changes: 10 additions & 2 deletions docs/plan/r2.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,16 @@ Replace the switch with explicit settings, such as `--secure-cookies`,
- The E2E harness starts a tommy test container (the same one that serves S3,
below) and adds
`app.Mails(t, filter)`, which polls `GET /api/v1/events?plugin=mail` until
the expected mail arrives. Tests that asserted on the `outgoing_emails`
queue gain an assertion on delivered mail.
the expected mail arrives. **E2E and browser tests switch from the
`outgoing_emails` queue to delivered mail**: they assert on what tommy
received (recipient, subject, body), and flows that follow a link from an
email (signup confirmation in `e2e/browser/accounts_test.go`) take it from
the delivered message. The queue is an implementation detail that R4 may
replace with a job queue, so after R2 no test under `e2e/` reads it
(`git grep ListOutgoingEmails e2e/` is empty). Today that is
`e2e/browser/accounts_test.go`, `comments_test.go`, `settings_test.go`,
plus any W3 test added by then. `factory.ListOutgoingEmails` stays for
package tests of `dbsender` itself.
- Unit and package tests keep using `fakesender`.

**Object storage everywhere; local file storage is dropped.**
Expand Down
Loading
Loading