diff --git a/docs/governance/workstreams.md b/docs/governance/workstreams.md index da3e076..8768c8e 100644 --- a/docs/governance/workstreams.md +++ b/docs/governance/workstreams.md @@ -232,6 +232,7 @@ state to its index: [`docs/adr/`](../adr/README.md), [`docs/pdr/`](../pdr/README | D216 | **Out-of-cycle work had no hold** (#385). On a multi-agent project, 2026-11-13/14, about 6 of 14 pull requests were out of cycle — a gap found in review, a flaky check — assigned in a chat message only: two sessions could take one issue unseen. `nstack claim` held only a deliverable of the accepted cycle | **Done**: `nstack claim '#'` holds an issue on `nstack/hold/issues/issue-`, the same events, the same refusal of a second taker, take-over and staleness; its work branches `issue--…`, named from the issue's title when the forge answers; in a framed project or not. On GitHub a comment on the issue says who holds it, as a person writes « I take it »: the assignee stays the person's to set. `nstack unclaim '#'` gives it back once its work is merged — an issue's hold does not end with a cycle; `nstack plan` lists the issues held. `14` and `issue-14` name it too, `#` starting a shell comment. The issue's comment names the person, never an email address. Market: a forge's assignee (GitHub, GitLab, Jira) refuses nobody; the hold, git's first push wins, does on any forge. Not built: K5 for out-of-cycle pull requests — a hold says, it does not yet gate. Five tests seen red first; an independent verifier's findings written in | | D217 | **The platform tests hung while another project ran its end-to-end tests.** On a workstation where a project ran a long end-to-end run, 2026-11-14, the platform suite stopped at the same test, again and again: `test_evidence.py` starts a real run, which waits on the machine's end-to-end lock (`$XDG_RUNTIME_DIR/nstack-e2e.lock`, D84's one run at a time) — held, rightly, by the other project's run | **Done**: every test has a runtime folder of its own (`conftest.py`), so no test ever takes or waits on the machine's lock; the tests of the lock itself already set theirs. Seen red first: the file hung under a held lock, and passes in seconds under the same lock | | D218 | **A review's failed run blocked the merge for good** (#394). On an attested project, 2026-11-14, a code owner approved before the record was complete — the usual moment —: « Test sheet and cycle », triggered by the review, failed (A2). The record completed by a notes push, which is no event; a later run of the same head passed, and the ruleset still refused the merge on the review's run, kept beside it. An agent cannot re-run a workflow: the person had to | **Done**: the job runs on the pull request's own events only. A review or a comment stays a verifier's hand (T2), read at the next run, which any edit of the description starts — an agent's included; T2's message and the verification playbook say so. Not chosen: skipping the job on a review, which leaves a required check « skipped », counted as passed; a separate job, since this one reads no approval. A pull request already blocked by a review's failed run is unblocked by a new push, whose head starts afresh — or, before taking this version, by a new review; never by an edit, nor by a re-run, which replays the old event's payload and fails again (seen on that project). The workflow's test seen red first | +| D219 | **A workflow red on the default branch went unseen** (#387). On a desktop project, 2026-11-07 to 13, the Windows packaging workflow failed on every run of the default branch — six runs — and was noticed by chance: it is no required check, so every pull request merged green beside it | **Done**: `nstack doctor` reads, workflow by workflow — a quiet one, filtered by `paths:` or scheduled, never drops out of a window busier ones fill — the last completed runs of the default branch's own events (push, schedule, dispatch: never a pull request's, a fork's included) with the token it already has (Actions: read). It says, as information (N2) that never changes its exit, each active workflow whose last three or more runs all failed — timed out and failed to start count — with the first failure's link; cancelled and skipped runs say nothing either way. Unreadable runs are said, with the permission to add. Market: GitHub shows a red run and notifies nobody once the check is not required. Not built: a fitness check on a `paths:` filter that leaves out a module its jobs build, until it pays. Three tests seen red first; an independent verifier's findings written in (the per-workflow read, forks' runs, timeouts) | | v0.6.0 | **What is announced refuses, and the documents say what is true**: D51 to D54, and the documentary lot of the internal audit | **Done (2026-09-20)**: five workstreams merged (#83, #84, #85, #86, #87), **v0.6.0 published**, tag on `8ae544c`, verified on published artefacts — including an update of `tool-library` from v0.5.0, 9 files and no conflict, still compliant. Found doing it: D55. Next: the pilot, and M11 beside it | | Pilot | **`NapkinStack/flet`, the first real project**, run by a session that knew nothing of this repository, with the maintainer as decider and this repository's agent as observer — public on the Free plan, where the whole checklist is available, rather than private on Team where G5 is paid and G6 does not exist | **Set aside by the maintainer, 2026-10-15**, at its cycle boundary; the real product followed since is `summit`. **Ran from 2026-09-20**: discovery decided *go* after three rounds and two independent challenges, charter accepted; cycle 01 closed `stopped/reframed`, **cycle 02 closed** with two deliverables accepted and one deferred (its #27), **cycle 03 framed and approved** (#28, #30), which starts its first `critical` module; the framework taken to v0.6.1 by itself at the boundary (#29), v0.6.2 proposed (#31). It has produced D56 to D63, D68, D70 to D77, replicated D26 and D55, and measured D49 — the first figures on what an independent verifier finds (PDR-0003) | | M11 | **Ceremony follows consequence** (PDR-0011): the framework charges what a failure costs, per module and per project, and states each price before it is paid; with PDR-0004's routing, and PDR-0007, PDR-0008 and PDR-0010 — through to v0.7.0 | **Published as v0.7.0 (2026-10-12).** Planned 2026-09-23 (#98). Merged: **M11f** (#101), **M11a** (#102), **M11b** (#103), **M11d** (#104), **M11e** (#105), **M11c** (#106), **M11h** (#107), **M11g** (#108) closed by its probe — not clean, returned to M13; **M11m** (D68, D74, D75); **M11l** (PDR-0010, D59, D60); **M11n** (D79); **M11o** (PDR-0010 extended: every document has its moment); **M11p** (PDR-0002's three levels given a place: `docs/project/rules/`); **M11j** (PDR-0007: the unframed stage); **M11q** (no folder created empty; PDR-0012 proposed: a project's tree is the project's). **M11k** (PDR-0008's print mode: `nstack settings`; D77, D78). **M11s** (PDR-0012 accepted; the framework's files under `.nstack/`; an update carries what a project adapted); **M11u** (an update rewrites the old addresses a project wrote in its own files, D85). **M11t** (`nstack new-contract`: the contracts module is created by the first contract; PDR-0012 built in full); **M11v** (what the third project found: D81, D82, D83, D86, D87). **M11i**: v0.7.0 published 2026-10-12 (#125, D88). To do: **M11r** PDR-0008's apply mode and D57, after a probe on the real forge; the two earlier projects take v0.7.0 themselves, observed | diff --git a/platform/tests/test_default_branch_runs.py b/platform/tests/test_default_branch_runs.py new file mode 100644 index 0000000..f33ea2d --- /dev/null +++ b/platform/tests/test_default_branch_runs.py @@ -0,0 +1,59 @@ +"""D219 (#387): a workflow that fails on every run of the default branch goes unseen when it is not a +required check — every pull request merges green beside it. `nstack doctor` says so (N2), as +information: the exit code does not change. Read workflow by workflow: a quiet one never drops out +of a window busier ones fill. +""" + +from __future__ import annotations + +from napkinstack import doctor + + +def _run(conclusion: str, day: int, event: str = "push", workflow: str = "w") -> dict: + return {"conclusion": conclusion, "status": "completed", "event": event, + "created_at": f"2026-11-{day:02d}T10:00:00Z", "html_url": f"https://forge.example/runs/{workflow}-{day}"} + + +class Forge: + def __init__(self, workflows: dict[str, list[dict]] | None, default: str = "trunk"): + self.workflows, self.default, self.read = workflows, default, [] + + def get(self, path, missing=False): + self.read.append(path) + if path == "": + return {"default_branch": self.default} + if self.workflows is None: + raise doctor.NotVerified("read refused (HTTP 403): repository out of reach, or the token lacks the " + "Administration: read permission") + if path.startswith("/actions/workflows?"): + return {"workflows": [{"id": index, "name": name, "state": "active"} + for index, name in enumerate(self.workflows)]} + index = int(path.split("/")[3]) + return {"workflow_runs": list(self.workflows.values())[index]} + + +def test_a_workflow_red_on_its_last_runs_of_the_default_branch_is_named_with_its_first_failure(capsys): + forge = Forge({"Package Windows": [_run("failure", day, workflow="win") for day in (13, 12, 11, 10)] + + [_run("timed_out", 9, workflow="win"), _run("success", 6, workflow="win")], + "CI": [_run("success", 13), _run("failure", 12)]}) + doctor._default_branch_runs(forge) + out = capsys.readouterr().out + assert any("branch=trunk" in path for path in forge.read), "the default branch, whatever its name" + assert "RED Package Windows: failed its last 5 runs on trunk, since 2026-11-09" in out, out + assert "https://forge.example/runs/win-9" in out, "a timed-out run is a failure" + assert "CI:" not in out, "a workflow whose last run passed is not said" + + +def test_two_failures_are_not_a_streak_and_a_fork_s_pull_request_is_not_the_branch_s(capsys): + forge = Forge({"Nightly": [_run("failure", 13, "schedule"), _run("failure", 12, "schedule"), + _run("success", 11, "schedule")], + "CI": [_run("failure", day, "pull_request") for day in (13, 12, 11)] + [_run("success", 10)]}) + doctor._default_branch_runs(forge) + out = capsys.readouterr().out + assert "RED" not in out and "no workflow failing" in out, out + + +def test_runs_that_cannot_be_read_are_said_never_judged_nor_blamed_on_administration(capsys): + doctor._default_branch_runs(Forge(None)) + out = capsys.readouterr().out + assert "NOT READ" in out and "Actions: read" in out and "Administration" not in out, out diff --git a/skeleton/.nstack/handbook/INSTALL.md.jinja b/skeleton/.nstack/handbook/INSTALL.md.jinja index 9b7e907..d1604ae 100644 --- a/skeleton/.nstack/handbook/INSTALL.md.jinja +++ b/skeleton/.nstack/handbook/INSTALL.md.jinja @@ -297,8 +297,9 @@ registry is asked one other question: `nstack update` says when the version it l on PyPI yet, since the project's CI installs it from there and stays red until it is. `nstack doctor` reads the token from `GH_TOKEN`, otherwise `GITHUB_TOKEN`: a fine-grained -token, limited to the repository, with the **Administration: read** permission. Without a -token, the GitHub part stays "not verified", never compliant. +token, limited to the repository, with the **Administration: read** permission — and +**Actions: read**, to be told of a workflow failing run after run on the default branch, which +no required check shows. Without a token, the GitHub part stays "not verified", never compliant. {% elif forge == "none" %}## No forge — nothing refuses a change Without a forge, nothing stands between a commit and the main branch but the team's habits: diff --git a/src/napkinstack/doctor.py b/src/napkinstack/doctor.py index 517ab0f..dc08d28 100644 --- a/src/napkinstack/doctor.py +++ b/src/napkinstack/doctor.py @@ -12,7 +12,7 @@ https://api.github.com). No answer leaves L1 as it is. GitHub is read with a token supplied by a human (GH_TOKEN, otherwise GITHUB_TOKEN): -fine-grained, limited to the repository, Administration: read permission. Without a token, +fine-grained, limited to the repository, Administration: read permission (Actions: read for N2). Without a token, or when the API refuses a read, the setting is "not verified", never compliant. No write. Rules: @@ -38,6 +38,9 @@ N1 information, never a gap (D91): a published version newer than the project's, read from the registry — the one place a project learns of it. Elsewhere, only `nstack update` asks it, whether the version it lays is published yet (D201) + N2 information, never a gap (D219): a workflow whose last runs on the default branch all + failed — RED_STREAK or more in a row — named with its first failing run. A workflow outside + the required checks can stay red for weeks while every pull request merges green G1-G21 the GitHub settings of CHECKLIST; G6 is not applicable outside a public repository, G21 outside `approval: agent`, and on a private one G1-G5, G12, G13, G17, G18, G20 and G21 name the GitHub plan or option required @@ -607,6 +610,56 @@ def _newer(answers: dict) -> None: "--refresh-package napkinstack --with-executables-from pre-commit, then nstack update") +RED_STREAK = 3 # failures in a row on the default branch before a workflow is said red: one is a flake +RUNS = 10 # the last runs read per workflow +FAILED = {"failure", "timed_out", "startup_failure"} +PASSED = {"success", "neutral"} # cancelled, skipped, stale and action_required say nothing either way +OWN = {"push", "schedule", "workflow_dispatch"} # the branch's own runs: never a pull request's, a fork's included + + +def _default_branch_runs(client: GitHub) -> None: + """Says the workflows whose last runs on the default branch all failed (N2), read workflow by + workflow — a quiet one, a `paths:` filter or a schedule, never drops out of a window busier + workflows fill. Information: a workflow is the project's to keep or drop, never a gap.""" + print("\nDefault branch") + try: + default = str(client.get("").get("default_branch") or "main") + workflows = [one for one in client.get("/actions/workflows?per_page=100").get("workflows") or [] + if isinstance(one, dict) and one.get("state") == "active"] + branch = urllib.parse.quote(default, safe="") + found = {one["id"]: (str(one.get("name") or one["id"]), + client.get(f"/actions/workflows/{one['id']}/runs?branch={branch}&status=completed" + f"&per_page={RUNS}").get("workflow_runs") or []) + for one in workflows if "id" in one} + except (NotVerified, AttributeError) as reason: + said = str(reason).split(":")[0] + print(f" NOT READ its workflow runs ({said})\n" + " Action: a token that may also read Actions (Actions: read), to be told of a workflow " + "that keeps failing there") + return + red = [] + for name, runs in found.values(): + own = sorted((run for run in runs if isinstance(run, dict) and run.get("event") in OWN), + key=lambda run: str(run.get("created_at", "")), reverse=True) + failed, ended = [], False + for run in own: + if run.get("conclusion") in FAILED: + failed.append(run) + elif run.get("conclusion") in PASSED: + ended = True + break + if len(failed) >= RED_STREAK: + red.append((name, failed, ended)) + for name, failed, ended in sorted(red, key=lambda one: one[0]): + first = failed[-1] + print(f" RED {name}: failed its last {len(failed)}{'' if ended else ' or more'} runs on {default}, " + f"since {str(first.get('created_at', ''))[:10]} — information, not a gap\n" + f" First failure: {first.get('html_url')}\n" + " It is no required check: pull requests merge green beside it.") + if not red: + print(f" ok no workflow failing on its last {RED_STREAK} runs on {default}") + + def _private(client: GitHub) -> bool: """Repository not public (private or internal); visibility unreadable: treated as public.""" try: @@ -734,6 +787,8 @@ def run(root: Path) -> int: status, detail = OUT_OF_REACH, f"Reason: {PRIVATE_PLAN[rule]}" results.append((rule, setting, status, detail)) _display(rule, setting, status, detail) + if client is not None: + _default_branch_runs(client) gaps = sum(status == GAP for _, _, status, _ in results) unknown = sum(status == UNKNOWN for _, _, status, _ in results)