test(cli): flag registry, ratchet contract test, CLI snapshot (#94 F1a)

.github/workflows/ci.yml

@@ -255,6 +255,17 @@ jobs:
       - run: pip install --require-hashes -r requirements.lock.txt -r requirements-ci.lock.txt
       - name: mypy
         run: mypy
+      # The CLI snapshot, on the pinned Python: its content check (tests/test_cli_flags.py)
+      # runs only on 3.13, and a PR runs no 3.13 test leg. Regenerate with `make cli-snapshot`.
+      - name: CLI snapshot is current
+        # Still runs when mypy fails, so a stale snapshot shows on the same push.
+        if: ${{ !cancelled() }}
+        run: |
+          PYTHONPATH=. python scripts/cli_surface.py dump "$RUNNER_TEMP/cli-interface.json"
+          diff -u docs/reference/cli-interface.json "$RUNNER_TEMP/cli-interface.json" || {
+            echo "docs/reference/cli-interface.json is stale: run 'make cli-snapshot' in dokimi/ and commit the result"
+            exit 1
+          }
       # Dependency hygiene, config in [tool.deptry] of pyproject.toml: an import
       # not declared (DEP001), a runtime dependency never imported (DEP002), an
       # import that only resolves through another package's dependencies (DEP003),

dokimi/CONTRIBUTING.md

@@ -214,6 +214,35 @@ Guidelines:
 
 Versioning, the changelog and releases: [docs/releasing.md](docs/releasing.md).
 
+### Adding or renaming a CLI flag
+
+Every concept a `dokimi` flag can name is one row of `CONCEPTS` in
+`dokimi/cli_flags.py`: its canonical spellings, its type and the synonyms banned for it.
+`tests/test_cli_flags.py` walks every parser of `dokimi.cli.build_parser()` and fails when
+a flag is neither a canonical spelling nor a row of `tests/data/cli_flags_allowlist.txt`.
+A banned synonym fails too, with the canonical spelling in the message. Until the flags
+move onto the registry, `cli_flags.py` is data only: no command reads it.
+
+- **New flag.** Use the canonical spelling of its concept. A new concept needs a row in
+  `CONCEPTS`, which reviewers look at.
+- **Snapshot.** From `dokimi/`, after `make dev`, run `make cli-snapshot` and commit
+  `docs/reference/cli-interface.json` with the change. CI's `typecheck` job and
+  `tests/test_cli_flags.py` check it against the code, so a flag change without a
+  regenerated snapshot fails the PR. The ratchet compares spellings only: a reviewer checks
+  that the option's type matches the `kind` of its concept.
+- **Non-canonical spelling.** Only if the flag must keep it, add one row to
+  `tests/data/cli_flags_allowlist.txt` with the reason after `#`, for example
+  `--sample-size  # the spelling the external runner already passes`. An allowlist row
+  exempts its spelling on every command, so a new command may use an allowlisted banned
+  spelling without failing.
+- **Removing or renaming.** An allowlist row that no parser uses fails the test, and so
+  does a row for a spelling that is now canonical. Delete the row when the flag goes.
+- **Why the snapshot test runs on one Python.** The dumper hashes validators with
+  `ast.dump`, whose output differs between Python versions. The snapshot is written by the
+  pinned interpreter (`make dev`, Python 3.13), and its content check runs there only: the
+  `typecheck` job on every PR, and the Python 3.13 test leg on `main`. The other test legs
+  run the ratchet and the structural checks.
+
 ## Docs-drift checklist
 
 When your change touches user-facing behavior, confirm:

dokimi/Makefile

@@ -13,7 +13,7 @@
 #   make benchmark-compare                      # or: make benchmark-compare BASE=<path>
 BASE ?= baselines/baseline.json
 
-.PHONY: validate-benchmark-spec benchmark-run benchmark-compare dev ci-local hooks
+.PHONY: validate-benchmark-spec benchmark-run benchmark-compare cli-snapshot dev ci-local hooks
 
 # `make dev` venv: at the repo root, outside this directory, so `bandit -r .`
 # and the docs benchmark (both run from here) do not scan it. CI's Python:
@@ -31,6 +31,14 @@ benchmark-compare:
 	@test -f "$(BASE)" || { echo "ERROR: base snapshot $(BASE) not found. Take one with 'make benchmark-run', copy baselines/current.json to it, or pass BASE=<path>."; exit 1; }
 	python cli.py compare --base $(BASE) --candidate baselines/current.json --out-json reports/compare.json --out-md reports/compare.md
 
+# Regenerate the CLI snapshot after a command or flag changes. tests/test_cli_flags.py fails
+# while docs/reference/cli-interface.json is stale. It uses the venv's Python (the one
+# `make dev` pins): the dump hashes validator source with ast.dump, whose output differs
+# between Python versions.
+cli-snapshot:
+	@test -x $(VPY) || { echo "ERROR: $(VENV_ABS) not found -- run 'make dev' first"; exit 1; }
+	PYTHONPATH=. $(VPY) scripts/cli_surface.py dump docs/reference/cli-interface.json
+
 validate-benchmark-spec:
 	@command -v yq >/dev/null 2>&1 || { echo "ERROR: yq is not installed"; exit 1; }
 	@echo "Checking YAML parse for benchmarks/spec.v1.yaml"

dokimi/changelog.d/20261026_224106_napetrov_test-cli-flag-registry.md

@@ -0,0 +1,3 @@
+### Documentation
+
+- **CLI flag registry and how to add a flag** — `dokimi/cli_flags.py` lists each flag concept with its canonical spelling, type and banned synonyms, and `tests/test_cli_flags.py` fails a flag that is neither canonical nor on `tests/data/cli_flags_allowlist.txt`, or a banned synonym, with the canonical spelling in the message. A contributor adding or renaming a flag now finds the steps in `CONTRIBUTING.md` (“Adding or renaming a CLI flag”): run `make cli-snapshot` and commit `docs/reference/cli-interface.json`. No command reads the registry yet; no flag, default or output changes.