check_docs.py

#!/usr/bin/env python3
"""Assert the documents still agree with the configs they describe.

Documentation drift is this repository's most frequently-recurring defect, and
unlike the others it has never had a check. `oracle` was recorded as an
i5-1235U with 32 GB when it is a dual-core A6-9200 with 4 GB, and ADR-0008 notes
that wrong entry was load-bearing in planning. `shiva` was described as the
hypervisor for several revisions when it is the iLO. `roadmap.md` says of itself
that it "has already been wrong about the switch answering SNMP and about the
history purge". Alert-rule and panel counts went stale in eight places (#72),
one of them a verification step inside a deploy runbook.

Every other class of defect here was answered by moving truth somewhere CI can
check it. This does that for `docs/`, following the pattern
`scripts/snmp-targets.sh --check` already established for the SNMP inventory:

    The device list must live in exactly one place. It is currently spread
    across five ... and --check asserts the other copies still agree.

Twelve assertions, each comparing prose against something machine-readable:

  1. Counted claims        rules, unit-test coverage, dashboards, panels,
                           Alloy agents, ADRs, runbooks, and the size of this
                           list — README.md says how many assertions run, and
                           that number is read from the registry in main()
  2. SNMP targets          snmp.yaml <-> docs/network.md
  3. Host and stack table  docs/architecture.md <-> docs/network.md, stacks/
  4. Ports table           docs/architecture.md <-> compose.yaml
  5. Compute table         docs/hardware.md <-> docs/network.md, and
                           neither pinning a point release
  6. Image versions        no version pins in prose; compose.yaml owns them
  7. ADR numbering         one ADR per number, and each file's H1 agrees with
                           the number in its filename
  8. Firewall posture      docs/security.md <-> docs/firewall-claims.yaml. The
                           CI half of #363; the claims file is checked against
                           the running firewall by check_firewall_claims.py,
                           which cannot run here because the ruleset is not in
                           this repository and deliberately never will be.
  9. Guest claims          a host that says it runs no guests, while another
                           row says it is a guest on it. Checkable only against
                           its siblings, which is why it is its own assertion.
 10. Outstanding buys     README.md <-> the buy table in docs/roadmap.md. The
                           roadmap is a source here, not a target: that table
                           is the one place a purchase may enter or leave, so
                           it is what README's count answers to.
 11. Runbook URLs         every critical alert's runbook_url names a file in
                           docs/runbooks/ and, if it has one, an anchor that is
                           a heading in that file (#842).
 12. Relative links       every relative Markdown link, in every tracked file,
                           resolves to a file in the tree, and its anchor to a
                           heading. The one check that runs over records too: a
                           dead link is a structural fault, not a claim.

Only present-tense documents are checked for their claims — the relative-link
assertion (12) is the exception, and scans every tracked file including the
records, because a dead link is a structural fault, not a claim. `docs/changelog.md`,
`docs/roadmap.md` and `docs/adr/` record what was true when the work landed —
`changelog.md` still says "(34 rules)" in a 2026-08 entry, and that is correct
as written, and its *claims* are not checked here.
Failing on those would need an
ignore list, and this repository has already learned where that leads: the
`.gitleaksignore` deleted in the history purge "was an acknowledgement, not a
fix, and it existed because a CI job that is permanently red for a known reason
gets ignored". So the scope is a fixed list of files rather than a suppression
mechanism that grows.

A count is matched whether it is written in digits or spelled out. The first
version of this check only looked for the phrasings someone happened to think
of, which left "13 LogQL rules" in README.md unguarded next to a checked "13
log-based", and left "five dashboards" in docs/images/README.md unguarded in a
file that was not in scope at all. Both were correct, and both would have gone
stale silently — the exact failure #72 is about, surviving inside its own fix.

Usage: scripts/check_docs.py
"""

from __future__ import annotations

import functools
import json
import pathlib
import re
import subprocess
import sys

# PyYAML from the one pinned bootstrap, scripts/_deps.py (#848).
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
from _deps import require_yaml

yaml = require_yaml()

REPO = pathlib.Path(__file__).resolve().parent.parent
STACK = REPO / "stacks/observability"
COMPOSE = STACK / "compose.yaml"
ALERTMANAGER = STACK / "alertmanager/alertmanager.yaml"
NETWORK_MD = REPO / "docs/network.md"
ARCH_MD = REPO / "docs/architecture.md"
HARDWARE_MD = REPO / "docs/hardware.md"
ROADMAP_MD = REPO / "docs/roadmap.md"

# Present-tense documents. See the module docstring for why roadmap.md,
# changelog.md and adr/ are deliberately absent — they are records, not claims
# about now.
PROSE = (
    "README.md",
    "SECURITY.md",
    "docs/architecture.md",
    "docs/hardware.md",
    "docs/images/README.md",
    "docs/network.md",
    "docs/observability.md",
    "docs/security.md",
    # Every stack's README, globbed rather than listed. This was the single
    # literal "stacks/observability/README.md", so stacks/lab's README was
    # prose nothing checked — its image table could have carried a version pin
    # and gone stale silently, which is the #73 defect the whole PROSE list
    # exists to prevent (#263).
    *sorted(str(p.relative_to(REPO)) for p in REPO.glob("stacks/*/README.md")),
    *sorted(str(p.relative_to(REPO)) for p in (REPO / "docs/runbooks").glob("*.md")),
)

# Prose spells small numbers out, and a spelled count goes stale exactly as
# readily as a digit: "five dashboards" and "across six files" were both
# unguarded while the digits beside them were checked. Longest-first so "seven"
# cannot match inside "seventeen" and leave the rest of the pattern to fail.
#
# The word branch is case-insensitive because prose capitalises a number that
# starts a sentence, and a capital is not a different claim. It was matching
# lowercase only, so "There are five dashboards" was guarded while "Five
# dashboards are provisioned" two files away was not — and both were stale
# together (#81). `number()` already lowercased, so only the pattern was wrong.
#
# THE TABLE USED TO STOP AT TWENTY, described here as "comfortably above any
# count here — the cap keeps the alternation short, it is not a claim about the
# ceiling". That was true when it was written and stopped being true without
# anything noticing. On 2026-09-07 the unit-test coverage sentence in
# observability.md read "Coverage is forty-five rules of 67" and TWO rules were
# added that day: the digit was caught both times and the word was not, because
# "forty-five" was not in the table (#367). The check exists to prevent exactly
# that, and the defect survived inside it.
#
# So the words are GENERATED through ninety-nine rather than typed, and the
# ceiling is now asserted rather than assumed — see check_unparsed_counts().
# Beyond a hundred, prose here uses digits, and the assertion is what makes that
# a rule rather than a hope.
#
# The cost of widening it: a heading like "Two dashboards are not captured" is a
# claim about a subset, and this reads it as a claim about the total and fails.
# That is the right way round — a false positive is a reword, a false negative is
# a document that lies. Phrase a subset so it does not put a bare count in front
# of the noun.
_ONES = ["one", "two", "three", "four", "five", "six", "seven", "eight", "nine"]
_TEENS = ["ten", "eleven", "twelve", "thirteen", "fourteen", "fifteen", "sixteen", "seventeen", "eighteen", "nineteen"]
_TENS = ["twenty", "thirty", "forty", "fifty", "sixty", "seventy", "eighty", "ninety"]


def _number_words() -> dict[str, int]:
    words = {w: i + 1 for i, w in enumerate(_ONES)}
    words.update({w: 10 + i for i, w in enumerate(_TEENS)})
    for t_i, tens in enumerate(_TENS):
        base = 20 + t_i * 10
        words[tens] = base
        # Hyphenated is the only form used here and the only one generated.
        # "forty five" as two words would need the pattern to span whitespace,
        # which would also match "forty" ending one claim and "five" starting
        # the next.
        for o_i, one in enumerate(_ONES):
            words[f"{tens}-{one}"] = base + o_i + 1
    return words


NUMBER_WORDS = _number_words()


# Built as STRUCTURE rather than as a flat list of all ninety-nine words. A
# 99-branch alternation is correct and slow: it is applied for ~20 claim
# patterns across ~25 documents, and flattening it took check_docs from 1.6s to
# 5.5s — measured. Expressing the compounds as "tens, optionally hyphen ones"
# gives 27 branches for the same language and puts the cost back to noise.
#
# Longest-first within each group, so "seven" cannot match inside "seventeen"
# and leave the rest of the pattern to fail.
def _alt(words: list[str]) -> str:
    return "|".join(sorted(words, key=len, reverse=True))


_WORD_NUMBER = (
    rf"(?:{_alt(_TENS)})(?:-(?:{_alt(_ONES)}))?"  # twenty, forty-five
    rf"|{_alt(_TEENS)}"  # ten .. nineteen
    rf"|{_alt(_ONES)}"  # one .. nine
)
COUNT = r"\b(\d+|(?i:" + _WORD_NUMBER + r"))"

# Number words this table cannot turn into an integer. A claim built on one of
# these is invisible to every assertion below, so it is reported rather than
# skipped — the ceiling stays visible instead of becoming the next silent gap.
# This is the lesson of #367 rather than a guess about what comes next.
# The optional leading word absorbs the determiner or multiplier these always
# carry in prose — "a hundred panels", "two hundred panels" — because without it
# the article sits between the verb and the number and the claim pattern's
# whitespace cannot span it. That was the first version's bug, caught by testing
# the assertion rather than assuming it worked.
UNPARSEABLE_NUMBER = re.compile(r"(?:\w+[ \t]+)?\b(?i:hundred|thousand|million|billion|dozen|score)\b")


# Prose wraps, and a counted claim wraps with it. "It routes all seven\nVLANs"
# in restore-the-firewall.md was invisible to a line-by-line scan for the whole
# life of #209 — the claim was there, the grep that would have found it was not.
# So claims are matched against the whole file and the line is derived from the
# offset. One newline is allowed inside a claim and a blank line is not, so a
# count ending one paragraph cannot bind to a noun starting the next.
WS = r"(?:[ \t]+|[ \t]*\n[ \t]*)"


def number(token: str) -> int:
    """A counted claim, written either as digits or as a word."""
    return int(token) if token.isdigit() else NUMBER_WORDS[token.lower()]


# ---------------------------------------------------------------------------
# Markdown helpers
# ---------------------------------------------------------------------------
def cells(line: str) -> list[str]:
    """Split one markdown table row into stripped cells."""
    return [c.strip() for c in line.strip().strip("|").split("|")]


def is_separator(line: str) -> bool:
    return bool(re.fullmatch(r"\|[\s:|-]+\|", line.strip()))


def tables_under(text: str, heading: re.Pattern[str]) -> list[list[list[str]]]:
    """Every markdown table in the section introduced by `heading`.

    A section runs to the next heading of the same or higher level, so a
    table belonging to a later section is never attributed to this one.
    """
    out: list[list[list[str]]] = []
    lines = text.splitlines()
    start = None
    for i, line in enumerate(lines):
        if heading.match(line):
            start = i
            break
    if start is None:
        return out

    level = len(lines[start]) - len(lines[start].lstrip("#"))
    rows: list[list[str]] = []
    for line in lines[start + 1 :]:
        if line.startswith("#"):
            depth = len(line) - len(line.lstrip("#"))
            if depth <= level:
                break
        if line.lstrip().startswith("|"):
            if not is_separator(line):
                rows.append(cells(line))
        elif rows:
            out.append(rows)
            rows = []
    if rows:
        out.append(rows)
    return out


def strip_md(cell: str) -> str:
    """Reduce a table cell to its plain text: no backticks, links or emphasis."""
    cell = re.sub(r"\[([^\]]*)\]\([^)]*\)", r"\1", cell)  # links
    cell = re.sub(r"\[\^[^\]]*\]", "", cell)  # footnote refs
    cell = cell.replace("`", "").replace("*", "").replace("**", "")
    return cell.strip()


# A host-and-stack row for something that does not exist yet. See the block
# above check_host_stack_table() for what it does there; count_alloy_agents()
# below reads it too, because a row describing an undeployed host describes an
# undeployed agent.
#
# Matched against the RAW cell, never strip_md()'s output: that helper removes
# every `*`, which takes the emphasis with it and leaves the marker unfindable.
NOT_BUILT = re.compile(r"\*\*not built yet\*\*", re.IGNORECASE)


# ---------------------------------------------------------------------------
# Facts, computed from the configs
# ---------------------------------------------------------------------------
def count_alerts(paths) -> int:
    return sum(len(re.findall(r"^\s*-\s*alert:", p.read_text(encoding="utf-8"), re.MULTILINE)) for p in paths)


def count_recording_rules(paths) -> int:
    return sum(len(re.findall(r"^\s*-\s*record:", p.read_text(encoding="utf-8"), re.MULTILINE)) for p in paths)


def tested_alertnames(paths) -> set[str]:
    """Every alert named by a promtool unit test.

    `alertname:` appears twice per case — once selecting the rule, once inside
    exp_labels — so this is a set, not a count. A case asserting
    `exp_alerts: []` still counts the rule as covered: it was fed an input and
    its silence was asserted, which is the half of the pairing #63 was missing.

    This counts what the tests NAME, not what they prove. A test file naming a
    rule that no longer exists would inflate it — but `promtool test rules`
    fails on that first, so the two checks bracket each other.
    """
    names: set[str] = set()
    for path in paths:
        names.update(re.findall(r"^\s*alertname:\s*(\S+)", path.read_text(encoding="utf-8"), re.MULTILINE))
    return names


def count_panels(paths) -> int:
    """Every panel object, rows included.

    Rows are counted because the numbers already in the documents count them:
    84 with rows, 73 without. Pinning the definition here is what makes the
    figure reproducible rather than a number somebody once arrived at.
    """
    total = 0

    def walk(panels) -> None:
        nonlocal total
        for panel in panels or []:
            total += 1
            walk(panel.get("panels"))

    for path in paths:
        walk(json.loads(path.read_text(encoding="utf-8")).get("panels"))
    return total


def compose_services() -> dict:
    """Services that `make up` actually starts.

    Profile-gated services are excluded: `renderer` sits behind the `capture`
    profile precisely so the running stack stays six services, and counting it
    would make this check disagree with a document that is correct.
    """
    doc = yaml.safe_load(COMPOSE.read_text(encoding="utf-8"))
    return {name: svc or {} for name, svc in (doc.get("services") or {}).items() if not (svc or {}).get("profiles")}


def count_notifying_receivers() -> int:
    """Receivers in alertmanager.yaml that actually send somewhere.

    `null` is excluded. It exists to swallow `info` and has no *_configs at
    all, so counting it would make the prose say five and be wrong in the
    other direction. The test is "declares at least one delivery config"
    rather than a name blocklist, so a second discard receiver would be
    excluded on the same grounds without anyone remembering to add it here.
    """
    doc = yaml.safe_load(ALERTMANAGER.read_text(encoding="utf-8")) or {}
    return sum(1 for r in (doc.get("receivers") or []) if any(k.endswith("_configs") and r[k] for k in r))


# A Contents cell saying a host runs NO Alloy agent. Stripped before the
# substring test below, so that saying so does not read as saying the opposite.
#
# "no Alloy" covers both shapes the table uses: "runs no Alloy" on the six lab
# machines ADR-0029 scrapes rather than instruments, and "No Alloy agent" on a
# host that is scraped for a different reason. A negation this does NOT know
# would overcount — the failure is loud rather than silent, because the count
# is asserted against hardware.md's sentence, but it is worth extending here
# rather than reaching for a different phrasing in the table.
NO_ALLOY = re.compile(r"\bno\s+alloy\b", re.IGNORECASE)


def count_alloy_agents() -> int:
    """Hosts the architecture table says run an Alloy agent.

    The agents are not in this repository — deploy-agent.sh puts them on
    hosts, and nothing here lists the hosts — so the Host and stack mapping
    is the machine-readable side. A row whose Contents cell names Alloy is an
    agent; "two Alloy agents" in hardware.md was unguarded and stale for as
    long as it took to deploy a third (#88).

    A row marked NOT_BUILT is not counted, because an agent on a host that does
    not exist is not an agent. `stacks/lab` declares one for `alexander`, and it
    collects nothing until that guest is racked (#262). The exclusion is not a
    convenience: dropping the marker on the commit that builds the host pushes
    this count to four and fails hardware.md's "three Alloy agents" in the same
    run, which is exactly when that sentence should be forced to change.

    SAYING A HOST RUNS NO ALLOY USED TO COUNT IT AS RUNNING ONE. The test was a
    bare `"alloy" in cell`, which cannot tell "Alloy agent (Docker)" from "runs
    no Alloy". It was survivable only by accident: every row that says so is
    also marked NOT_BUILT, so the negation was masked by the exclusion above —
    and would have surfaced on the commit that dropped the marker, which is the
    commit already busy changing this count for a real reason. Found 2026-09-16
    when `smaug`'s row said "No Alloy agent" and pushed the count to five; six
    further rows (`bahamut`, `leviathan`, `titan`, `ramuh`, `carbuncle`,
    `siren`) carry "runs no Alloy" and are waiting to do the same.

    Negations are STRIPPED rather than the affirmative being matched, and that
    is the part worth writing down. Matching `alloy agent` instead looks
    tidier and is wrong: `prometheus` names Alloy in a service list — "…
    docker-socket-proxy, Alloy" — and never says "Alloy agent" at all, so that
    test would drop a host which genuinely runs one and quietly report three.
    An over-count fails loudly against hardware.md; an under-count would have
    been a checker agreeing with a stale sentence.
    """
    tables = tables_under(
        ARCH_MD.read_text(encoding="utf-8"),
        re.compile(r"^##\s+Host and stack mapping"),
    )
    if not tables:
        return 0
    return sum(
        1
        for row in tables[0][1:]
        if len(row) > 3 and "alloy" in NO_ALLOY.sub("", strip_md(row[3])).lower() and not NOT_BUILT.search(row[3])
    )


def count_vlans() -> int:
    """VLANs in docs/network.md's segment table.

    The table is the enumeration; the prose above it was the claim, and they
    disagreed five times over (#209). `WAN` and `LAN` carry a dash in the VLAN
    column precisely because they are not VLANs — the untagged switch-management
    LAN is a real network and a real seventh thing to count, which is why the
    wrong number was so durable. Counting the tag column rather than the rows
    keeps that distinction.
    """
    text = NETWORK_MD.read_text(encoding="utf-8")
    rows = tables_under(text, re.compile(r"^#\s+Network$", re.MULTILINE))
    if not rows:
        return 0
    return sum(1 for row in rows[0] if len(row) > 1 and strip_md(row[1]).isdigit())


def count_adrs() -> int:
    """ADR files in docs/adr/, superseded ones included.

    README.md is the only place this number is asserted and it was wrong by four
    when this was written — twenty-three against twenty-seven. It had been right
    once. Nothing failed as 0024 through 0027 landed, because "N ADRs" was a
    phrasing nobody had thought of, which is the #72 shape this whole file
    exists to prevent surviving inside the fix for it — exactly as "forty-five
    rules" did (#367).

    Superseded ADRs count. ADR-0002 and ADR-0013 are marked and not deleted, and
    ADR-0001 says why: "the history of what was believed and when is the point."
    A count that skipped them would be counting decisions still in force, which
    is a different claim from the one the README makes.

    This counts the directory rather than reading the files, and that is the
    module docstring's scoping rule showing through: docs/adr/ is deliberately
    outside PROSE, so an ADR stating a count in its own text stays untouched.
    """
    return len(list((REPO / "docs/adr").glob("[0-9][0-9][0-9][0-9]-*.md")))


def count_runbooks() -> int:
    """Runbooks in docs/runbooks/.

    THIS COUNT IS CORRECT TODAY, and it is guarded anyway, which needs saying
    because guarding a true number can look like work for its own sake.

    It is correct the way the ADR count above was correct once. #265 is in
    review as this lands and adds build-the-lab-domain.md, which makes it wrong
    — so this is the rare case where the drift can be watched arriving instead
    of found four ADRs later. Nothing about the change that falsifies it touches
    README.md, or any file a reviewer of that branch would think to reread: a
    directory gains a file, and a sentence somewhere else quietly stops being
    true. An unguarded number that happens to be right is not a number anybody
    is keeping right.

    Every .md here is a runbook and there is no index file, so the glob is the
    count. One added later would not be a runbook and would need excluding.
    """
    return len(list((REPO / "docs/runbooks").glob("*.md")))


def facts() -> dict:
    prom_rules = sorted((STACK / "prometheus/rules").glob("*.rules.yaml"))
    loki_rules = sorted((STACK / "loki/rules").glob("*.rules.yaml"))
    dashboards = sorted((STACK / "grafana/dashboards").glob("*.json"))
    prom = count_alerts(prom_rules)
    loki = count_alerts(loki_rules)
    recording = count_recording_rules(prom_rules)
    tested = tested_alertnames(sorted((STACK / "prometheus/tests").glob("*.test.yaml")))
    return {
        "prometheus_rules": prom,
        "loki_rules": loki,
        "total_rules": prom + loki,
        "recording_rules": recording,
        "rules_page": prom + recording,
        "dashboards": len(dashboards),
        "panels": count_panels(dashboards),
        "prometheus_rule_files": len(prom_rules),
        "tested_rules": len(tested),
        "untested_rules": prom - len(tested),
        "alloy_agents": count_alloy_agents(),
        "receivers": count_notifying_receivers(),
        "vlans": count_vlans(),
        "adrs": count_adrs(),
        "runbooks": count_runbooks(),
    }


# ---------------------------------------------------------------------------
# 1. Counted claims
# ---------------------------------------------------------------------------
@functools.cache
def _unreadable_pattern(pattern: str) -> re.Pattern[str]:
    """`pattern` with the number token swapped for the ones that cannot be read.

    Memoised because the claim patterns are rebuilt on every call and compiling
    a 99-branch alternation repeatedly is most of what made the ceiling scan
    expensive.
    """
    return re.compile(pattern.replace(COUNT, UNPARSEABLE_NUMBER.pattern))


def check_counts(f: dict) -> list[str]:
    # "N alert rules" is genuinely ambiguous in this repository: README uses it
    # for the total, security.md for the Prometheus half. Both readings are
    # legitimate prose, so both are accepted — the check still catches a number
    # that is neither, which is what stale looks like.
    claims = (
        (rf"{COUNT}" + WS + r"alert rules", {f["prometheus_rules"], f["total_rules"]}, "alert rules"),
        (rf"{COUNT}" + WS + r"rules in total", {f["total_rules"]}, "total rules"),
        (rf"{COUNT}" + WS + r"rules loaded", {f["prometheus_rules"]}, "rules loaded"),
        (rf"{COUNT}" + WS + r"rules across", {f["prometheus_rules"]}, "Prometheus rules"),
        (rf"{COUNT}" + WS + r"metric-based", {f["prometheus_rules"]}, "metric-based rules"),
        (rf"{COUNT}" + WS + r"log-based", {f["loki_rules"]}, "log-based rules"),
        # README says "13 LogQL rules" where observability.md says "log-based".
        # Same number, different prose; the first phrasing matched nothing.
        (rf"{COUNT}" + WS + r"LogQL rules", {f["loki_rules"]}, "LogQL rules"),
        (rf"{COUNT}" + WS + r"(?:provisioned\s+)?dashboards", {f["dashboards"]}, "dashboards"),
        (rf"{COUNT}" + WS + r"panels", {f["panels"]}, "panels"),
        # "39 rules across six files" states two counts. The first was checked
        # and the second was not, so splitting a rule file could not fail here.
        (r"rules across" + WS + COUNT + WS + r"files", {f["prometheus_rule_files"]}, "Prometheus rule files"),
        # How many rules have a unit test, and how many do not. Both were
        # unguarded and both were already stale: the sentence read "Coverage is
        # six rules of 39 ... ContainerHighMemory and Watchdog" while
        # blackbox.test.yaml had covered three more for weeks. This is the
        # figure most likely to drift, because it moves whenever a test lands.
        (r"[Cc]overage is" + WS + COUNT + WS + r"rules", {f["tested_rules"]}, "unit-tested rules"),
        (r"[Oo]ther" + WS + COUNT + WS + r"are still validated", {f["untested_rules"]}, "rules without a unit test"),
        # The same count in the wording #843 left it in, once it reached zero:
        # "the other 0 are still validated" reads as nonsense, and a clearer
        # sentence that nothing checked would drift silently.
        (
            r"leaves" + WS + COUNT + WS + r"rules" + WS + r"without" + WS + r"a" + WS + r"unit" + WS + r"test",
            {f["untested_rules"]},
            "rules without a unit test",
        ),
        # "Coverage is fifteen rules of 45" states two counts and only the
        # first was checked, so the denominator could go stale on its own —
        # the same shape as "39 rules across six files" above, and it did go
        # stale the same way the moment a rule was added (#81).
        (r"rules of" + WS + COUNT + WS + r"so far", {f["prometheus_rules"]}, "rules in the coverage denominator"),
        # deploy-stack.md's Status → Rules step: "The page lists 129: the 127
        # alert rules ... plus the two recording rules". Three counts in one
        # sentence, and only the middle one was guarded. A merge on 2026-10-03
        # recomputed it to 132 and left the total at 129, so the step told the
        # operator to expect a number five short of a healthy page. The total
        # is the alert rules plus the recording rules, both counted from the
        # rule files, so adding either kind fails here; the recording-rule count
        # is guarded beside it because it is the other half of the sum.
        (
            r"page" + WS + r"lists" + WS + COUNT,
            {f["rules_page"]},
            "rules on Prometheus's Rules page (alert + recording)",
        ),
        (rf"{COUNT}" + WS + r"recording rules", {f["recording_rules"]}, "recording rules"),
        # Where the agents run is documented, not deployed from here, so the
        # architecture table is the source and hardware.md's sentence is the
        # claim. See count_alloy_agents.
        (rf"{COUNT}" + WS + r"Alloy agents", {f["alloy_agents"]}, "Alloy agents"),
        # The second time a number in observability.md drifted (#72, then #212):
        # `security` was added and the sentence introducing the routing table
        # still said three. Both halves of "N receivers, N separate
        # destinations" are counted, because they went stale as a pair and
        # guarding only the first would leave the second free to drift alone —
        # the "39 rules across six files" shape above.
        #
        # Derived from alertmanager.yaml the way the rule counts are derived
        # from the rule files, so adding a receiver fails here rather than
        # waiting for someone to reread the paragraph.
        (rf"{COUNT}" + WS + r"receivers", {f["receivers"]}, "notifying receivers"),
        (rf"{COUNT}" + WS + r"separate destinations", {f["receivers"]}, "separate destinations"),
        # Asserted five times and enumerated zero times (#209). The count is
        # the segment table's tag column, so the untagged switch-management LAN
        # stays uncounted here and "seven internal networks" stays sayable.
        (rf"{COUNT}" + WS + r"VLANs", {f["vlans"]}, "VLANs"),
        # Two counts in one README sentence and neither was guarded. "23 ADRs"
        # was stale by four. "19 runbooks" is correct, and is guarded because
        # #265 is in review and adds one — the same drift, caught on the way in
        # rather than four ADRs after the fact.
        #
        # THE HONEST OBJECTION IS THAT NEITHER NUMBER MEANS ANYTHING, and it is
        # worth writing down because it nearly won. Every other count above is
        # load-bearing: alert rules are the coverage ADR-0007 explicitly trades
        # away, Alloy agents say which hosts are observed, a stale panel count
        # broke a verification step inside a deploy runbook (#72). An ADR total
        # says only how long the project has been running. Nothing reads it,
        # nothing branches on it, and deleting both numbers from the prose would
        # have ended the obligation rather than automating it.
        #
        # Guarded anyway, for what the sentence is doing rather than what it
        # counts. README.md is where this repository claims its documents are
        # checked against the things they describe, and a number that is visibly
        # wrong there is a false claim about the checking rather than a stale
        # fact about ADRs. The precedent is also settled: #72, #81 and #367 were
        # each answered by widening the patterns, never once by removing the
        # number, and "delete the claim" is a fix available to every one of them.
        #
        # Plural only, like `receivers` and `VLANs` above. "one ADR" in prose is
        # a reference and not an inventory.
        (rf"{COUNT}" + WS + r"ADRs", {f["adrs"]}, "ADRs"),
        (rf"{COUNT}" + WS + r"runbooks", {f["runbooks"]}, "runbooks"),
        # How many assertions this file runs — and the same shape as #72, #81
        # and #367: a counted claim that no pattern named, going stale where
        # nothing could see it. README.md said "Six assertions" while main()
        # registered ten, and the enumeration beside it had been overtaken by
        # four checks — ADR numbering, firewall posture, guest claims and the
        # buy list were all running and none was mentioned. Drift in the one
        # file whose subject is catching drift, which makes it a false claim
        # about the checking rather than a stale fact about the checks.
        #
        # The ADRs comment above settles the "just delete the number" argument
        # and it applies here with more force: the sentence exists to say how
        # thoroughly these documents are checked, so the number is the claim.
        #
        # Sourced from len(checks) in main() rather than a constant here, so
        # an eleventh assertion fails this line on the way in instead of
        # waiting for someone to reread the bullet. main() writes it into the
        # facts dict after building the registry; facts() cannot compute it,
        # because the registry closes over the dict facts() is still building.
        (rf"{COUNT}" + WS + r"assertions", {f["assertions"]}, "assertions"),
    )
    problems = []
    for rel in PROSE:
        path = REPO / rel
        if not path.exists():
            continue
        text = path.read_text(encoding="utf-8")
        # Whether this file could possibly hold an unreadable count, asked once
        # per file instead of once per claim pattern. Without it the ceiling
        # scan re-walks every document with a 99-branch alternation for each of
        # ~20 patterns, and check_docs went from 1.6s to 7.5s — measured, not
        # guessed. Almost no file contains "hundred" at all, so this skips the
        # whole scan for nearly all of them and the cost returns to noise.
        may_be_unreadable = UNPARSEABLE_NUMBER.search(text) is not None
        for pattern, expected, label in claims:
            for match in re.finditer(pattern, text):
                if number(match.group(1)) in expected:
                    continue
                n = text.count("\n", 0, match.start()) + 1
                want = " or ".join(str(v) for v in sorted(expected))
                claimed = " ".join(match.group(1).split())
                problems.append(f"{rel}:{n} claims {claimed} {label}; the repository has {want}")

            # THE SAME CLAIM, WRITTEN IN A NUMBER THIS FILE CANNOT READ.
            # NUMBER_WORDS reaches ninety-nine and no further, so "a hundred
            # panels" is a counted claim that every assertion above walks
            # straight past — which is exactly how "forty-five rules" stayed
            # stale while the digit beside it was corrected twice in one day
            # (#367). The ceiling is asserted here rather than assumed, so the
            # next time prose outgrows the table it fails loudly instead of
            # going quiet.
            if not may_be_unreadable:
                continue
            unreadable = _unreadable_pattern(pattern)
            for match in unreadable.finditer(text):
                n = text.count("\n", 0, match.start()) + 1
                problems.append(
                    f"{rel}:{n} states {label} as a number this check cannot "
                    f"read ({' '.join(match.group(0).split())!r}). Counts above "
                    f"ninety-nine must be written in digits, or NUMBER_WORDS "
                    f"needs extending — an unreadable count is an unchecked one"
                )
    return problems


# ---------------------------------------------------------------------------
# 2. SNMP targets against the network inventory
# ---------------------------------------------------------------------------
def network_sections(text: str) -> dict[str, list[list[str]]]:
    """Host rows keyed by VLAN id, plus 'wan' and 'lan'."""
    out: dict[str, list[list[str]]] = {}
    for heading in re.finditer(r"^##\s+(.+)$", text, re.MULTILINE):
        title = heading.group(1)
        vlan_match = re.search(r"VLAN\s+(\d+)", title)
        if vlan_match:
            key = vlan_match.group(1)
        elif title.strip().upper() in ("WAN", "LAN"):
            key = title.strip().lower()
        else:
            continue
        found = tables_under(text, re.compile(re.escape(heading.group(0))))
        if found:
            out[key] = found[0]
    return out


def check_snmp_targets() -> list[str]:
    targets_file = STACK / "prometheus/targets/snmp.yaml"
    targets = yaml.safe_load(targets_file.read_text(encoding="utf-8")) or []
    sections = network_sections(NETWORK_MD.read_text(encoding="utf-8"))
    problems = []

    for entry in targets:
        labels = entry.get("labels") or {}
        device = labels.get("device", "")
        vlan = str(labels.get("vlan", ""))
        ips = entry.get("targets") or []
        rows = sections.get(vlan)
        if rows is None:
            problems.append(
                f"snmp.yaml polls {device} with vlan label {vlan!r}, and docs/network.md has no section for it"
            )
            continue
        for ip in ips:
            hit = [r for r in rows if strip_md(r[0]).lower() == device.lower() and ip in strip_md(r[1])]
            if not hit:
                problems.append(
                    f"snmp.yaml polls {device} at {ip} on VLAN {vlan}, and "
                    f"docs/network.md does not list that host at that address "
                    f"in the VLAN {vlan} table"
                )
    return problems


# ---------------------------------------------------------------------------
# 3. Host and stack mapping
# ---------------------------------------------------------------------------
# A stack directory can legitimately exist before the host that runs it does.
# `stacks/lab` was committed complete — compose file, configs, rules, unit
# tests — while the guest that will run it, `alexander`, was still an issue
# (#262, #264). ADR-0004 puts the host-to-stack mapping in this document, so the
# row has to exist; but docs/network.md is the inventory of what is actually on
# the wire, and writing an unbuilt guest into it would be the precise kind of
# false claim this file exists to catch. It would also mean inventing a MAC, a
# device and an OS for a machine whose distribution is explicitly undecided.
#
# So the row is marked, and the marker INVERTS the check rather than switching
# it off. A normal row's host must APPEAR in network.md; a row marked "not built
# yet" must be ABSENT from it. That is what makes the marker self-clearing —
# rack the host, add its network.md row, and this fails saying the marker is
# stale, instead of quietly tolerating a row that claims both things at once.
# The address is still required and still checked for collisions, so a plan is
# held to the same standard as a deployment.
#
# The marker itself is defined next to strip_md(), because count_alloy_agents()
# reads it too.


def check_host_stack_table() -> list[str]:
    text = ARCH_MD.read_text(encoding="utf-8")
    tables = tables_under(text, re.compile(r"^##\s+Host and stack mapping"))
    if not tables:
        return ["docs/architecture.md has no 'Host and stack mapping' table"]

    rows = tables[0][1:]
    sections = network_sections(NETWORK_MD.read_text(encoding="utf-8"))
    problems = []
    named_stacks: set[str] = set()

    for row in rows:
        host_cell = strip_md(row[0])
        host = host_cell.split("(")[0].strip()
        ip_match = re.search(r"(\d+\.\d+\.\d+\.\d+)", host_cell)
        vlan_match = re.search(r"(\d+)", strip_md(row[1]))
        named_stacks.update(re.findall(r"stacks/([a-z0-9-]+)", row[2]))
        planned = bool(len(row) > 3 and NOT_BUILT.search(row[3]))

        if not (ip_match and vlan_match):
            problems.append(
                f"docs/architecture.md host row {host_cell!r} has no address or "
                f"no VLAN, so it cannot be checked against docs/network.md"
            )
            continue

        ip, vlan = ip_match.group(1), vlan_match.group(1)
        rows_for_vlan = sections.get(vlan, [])
        hit = [r for r in rows_for_vlan if strip_md(r[0]).lower() == host.lower() and ip in strip_md(r[1])]

        if planned:
            # The VLAN must be one network.md actually describes, or a typo'd
            # segment would make every assertion below vacuously true.
            if not rows_for_vlan:
                problems.append(
                    f"docs/architecture.md plans {host} on VLAN {vlan}, which docs/network.md has no table for"
                )
            if hit:
                problems.append(
                    f"docs/architecture.md still marks {host} 'not built yet', "
                    f"and docs/network.md now lists it at {ip} on VLAN {vlan} — "
                    f"it has been built, so drop the marker"
                )
            # An unbuilt host planned onto an address something else already
            # holds is a real conflict, and the cheapest possible moment to
            # find it is before anyone racks it.
            clash = [r for r in rows_for_vlan if ip in strip_md(r[1]) and strip_md(r[0]).lower() != host.lower()]
            if clash:
                problems.append(
                    f"docs/architecture.md plans {host} at {ip}, which "
                    f"docs/network.md already gives to "
                    f"{strip_md(clash[0][0])} on VLAN {vlan}"
                )
        elif not hit:
            problems.append(
                f"docs/architecture.md places {host} at {ip} on VLAN {vlan}; docs/network.md does not list it there"
            )

    on_disk = {p.name for p in (REPO / "stacks").iterdir() if p.is_dir() and not p.name.startswith(".")}
    for missing in sorted(on_disk - named_stacks):
        problems.append(
            f"stacks/{missing}/ exists and no row of the host and stack mapping "
            f"in docs/architecture.md names it — ADR-0004 puts that mapping in "
            f"documentation, which only works if it is complete"
        )
    return problems


# ---------------------------------------------------------------------------
# 4. Ports
# ---------------------------------------------------------------------------
BIND_VAR = re.compile(r"\$\{([A-Z_][A-Z0-9_]*)")


def normalise_bind(bind: str) -> str:
    """A bind address as the ports table writes it.

    Loopback is written literally. An address that comes from a variable is
    written as that variable, `${BIND_ADDR}` or `${INGEST_BIND_ADDR}` (#182),
    whatever default follows it in compose.yaml. Anything else is read as
    `${BIND_ADDR}`, which is what every published port was before the ingest
    proxy needed an address of its own.
    """
    if "127.0.0.1" in bind:
        return "127.0.0.1"
    m = BIND_VAR.search(bind)
    return f"${{{m.group(1)}}}" if m else "${BIND_ADDR}"


def published_ports(services: dict) -> dict[str, list[tuple[str, str]]]:
    """service -> [(bind, container port)] for anything bound to the host."""
    out: dict[str, list[tuple[str, str]]] = {}
    for name, svc in services.items():
        for spec in svc.get("ports") or []:
            parts = str(spec).split(":")
            if len(parts) < 2:
                continue  # "9116" — exposed to the compose network only
            bind = normalise_bind(parts[0])
            container = parts[-1]
            out.setdefault(name, []).append((bind, container))
    return out


def check_ports() -> list[str]:
    text = ARCH_MD.read_text(encoding="utf-8")
    tables = tables_under(text, re.compile(r"^##\s+Ports\s*$"))
    if not tables:
        return ["docs/architecture.md has no 'Ports' table"]

    services = compose_services()
    published = published_ports(services)
    problems = []
    documented: set[tuple[str, str]] = set()

    for row in tables[0][1:]:
        service = strip_md(row[0]).lower().split()[0]
        port = strip_md(row[1])
        bind = strip_md(row[2])
        internal_only = "compose network" in bind.lower()

        if service not in services:
            problems.append(
                f"docs/architecture.md documents a port for {service!r}, which is not a service in compose.yaml"
            )
            continue

        if internal_only:
            if service in published:
                problems.append(
                    f"docs/architecture.md says {service} is never published to "
                    f"a host interface; compose.yaml publishes it"
                )
            continue

        want_bind = normalise_bind(bind)
        actual = published.get(service, [])
        match = [a for a in actual if a[1] == port]
        if not match:
            problems.append(
                f"docs/architecture.md says {service} publishes {port}; "
                f"compose.yaml publishes "
                f"{', '.join(p for _, p in actual) or 'nothing'}"
            )
            continue
        if match[0][0] != want_bind:
            problems.append(
                f"docs/architecture.md says {service}:{port} binds to {bind}; compose.yaml binds it to {match[0][0]}"
            )
        documented.add((service, port))

    for service, entries in published.items():
        for _, port in entries:
            if (service, port) not in documented:
                problems.append(
                    f"compose.yaml publishes {service}:{port} and the ports "
                    f"table in docs/architecture.md does not mention it"
                )
    return problems


# ---------------------------------------------------------------------------
# 5. Compute table
# ---------------------------------------------------------------------------
def os_key(text: str) -> tuple[str, str]:
    """('ubuntu', '24.04') from 'Ubuntu Server 24.04 LTS'.

    The two documents word the same OS differently on purpose — network.md
    says 'Ubuntu 24.04 LTS' where hardware.md says 'Ubuntu Server 24.04 LTS',
    and 'FreeBSD 16.0' where the other says 'FreeBSD 16.0 (pfSense)'. Comparing
    the family and the version rather than the string is what keeps this an
    agreement check instead of a house-style check.
    """
    plain = strip_md(text)
    family = re.match(r"([A-Za-z]+)", plain)
    version = re.search(r"(\d+(?:\.\d+)*)", plain)
    return (
        family.group(1).lower() if family else "",
        version.group(1) if version else "",
    )


# A third component is a point release — 24.04.3, 9.2.11 — and a point release
# changes when the host takes an update, which is not an edit to this
# repository. Both laptops were recorded as 24.04.3 while running 24.04.4, and
# the two documents agreed with each other throughout, so the assertion below
# could not see it: it compares the copies, and both copies were stale.
#
# So the tables record the release line, which changes only on a decision to
# rebuild, and the running version stays where it is already collected —
# `node_os_info{pretty_name="Ubuntu 24.04.4 LTS"}` on every Linux host, from
# the same Alloy agents that produce everything else here.
#
# An assertion rather than a house rule in prose, because the rule is invisible
# to whoever adds the next host, and reading it off the existing rows is
# exactly what nobody does.
POINT_RELEASE = re.compile(r"\d+(?:\.\d+){2,}")


def point_release_problem(doc: str, host: str, cell: str) -> str | None:
    if not POINT_RELEASE.search(cell):
        return None
    return (
        f"{doc} gives {host} the OS {cell!r} — that column records the release "
        f"line, not the point release, which goes stale the next time the host "
        f"takes an update. node_os_info carries the running one"
    )


def check_compute_table() -> list[str]:
    tables = tables_under(HARDWARE_MD.read_text(encoding="utf-8"), re.compile(r"^##\s+Compute"))
    if not tables:
        return ["docs/hardware.md has no 'Compute' table"]

    sections = network_sections(NETWORK_MD.read_text(encoding="utf-8"))
    hosts: dict[str, tuple[str, str]] = {}
    # `hosts` keeps the first row per host, because the cross-document
    # comparison below needs one OS per host and every table agrees today.
    # `pinned` must not: `morpheus` has a row in all seven segment tables, so a
    # point release written into the sixth would be invisible to a check that
    # only ever reads the first. Every row is scanned; identical (host, cell)
    # pairs collapse so one mistake is reported once rather than seven times.
    pinned: set[tuple[str, str]] = set()
    for rows in sections.values():
        for row in rows[1:]:
            if len(row) >= 5:
                host, cell = strip_md(row[0]).lower(), strip_md(row[4])
                hosts.setdefault(host, os_key(row[4]))
                if POINT_RELEASE.search(cell):
                    pinned.add((host, cell))

    problems = []
    for host, cell in sorted(pinned):
        problem = point_release_problem("docs/network.md", host, cell)
        if problem:
            problems.append(problem)

    for row in tables[0][1:]:
        host = strip_md(row[0])
        want = os_key(row[5])
        have = hosts.get(host.lower())
        problem = point_release_problem("docs/hardware.md", host, strip_md(row[5]))
        if problem:
            problems.append(problem)
        if have is None:
            problems.append(f"docs/hardware.md lists {host}, which appears nowhere in docs/network.md")
        elif have != want:
            problems.append(
                f"docs/hardware.md says {host} runs {strip_md(row[5])!r}; docs/network.md says {have[0]} {have[1]}"
            )
    return problems


# ---------------------------------------------------------------------------
# 6. Image versions must not appear in prose at all
# ---------------------------------------------------------------------------
def check_image_versions() -> list[str]:
    """compose.yaml is the only place an image version may appear.

    An earlier version of this check asserted that a version quoted in prose
    *matched* compose.yaml. That keeps the documents honest but keeps the
    duplication, and the duplication is the actual defect: Dependabot edits
    only compose.yaml, so a version written anywhere else is stale from the
    next bump onward. All six in the stack README went stale exactly that way
    (#73), and matching them would have made every Dependabot PR red until
    someone hand-edited a table.

    ci.yml already refuses duplicated pins in shell, YAML and the Makefile —
    "hardcoded copies went stale silently" — but its grep does not cover
    Markdown, which is how the six survived. This is that rule, extended to
    prose.
    """
    pinned = set()
    doc = yaml.safe_load(COMPOSE.read_text(encoding="utf-8"))
    for svc in (doc.get("services") or {}).values():
        image = str((svc or {}).get("image", ""))
        if image:
            pinned.add(image.partition("@")[0].rpartition(":")[0])

    # Only inline code spans, and only a repository compose.yaml actually
    # pins. An OS version in a hardware table is not an image pin, and neither
    # is a registry path mentioned without a tag — naming the image is fine,
    # naming its version is what goes stale.
    span = re.compile(r"`([a-z0-9][a-z0-9._-]*/[a-z0-9][a-z0-9._-]*):([^`\s]+)`")
    problems = []
    for rel in PROSE:
        path = REPO / rel
        if not path.exists():
            continue
        for n, line in enumerate(path.read_text(encoding="utf-8").splitlines(), 1):
            for match in span.finditer(line):
                repository = match.group(1)
                if repository in pinned:
                    problems.append(
                        f"{rel}:{n} pins {repository}:{match.group(2)}; only "
                        f"compose.yaml may carry a version — drop the tag and "
                        f"write `{repository}`"
                    )
    return problems


# ---------------------------------------------------------------------------
def check_adr_numbers() -> list[str]:
    """One ADR per number, and the H1 inside each file agreeing with its name.

    Two branches opened in the same afternoon both took 0017 — #96 for `ifrit`
    and #97 for the switch — and git merged them without a murmur, because the
    filenames differ and nothing downstream reads the number. The result is a
    repository where "ADR-0017" in prose has two referents, which is the one
    thing an ADR number exists to prevent. Nothing here caught it: every other
    assertion in this file compares prose against a config, and an ADR number
    is prose all the way down.

    The H1 check is the same failure one step later: renumbering a file is a
    `git mv` plus an edit, and the edit is the half that gets forgotten.
    """
    problems: list[str] = []
    adr_dir = REPO / "docs" / "adr"

    by_number: dict[str, list[str]] = {}
    for path in sorted(adr_dir.glob("[0-9][0-9][0-9][0-9]-*.md")):
        num = path.name[:4]
        by_number.setdefault(num, []).append(path.name)

        heading = ""
        for line in path.read_text().splitlines():
            if line.startswith("# "):
                heading = line
                break
        if not heading:
            problems.append(f"{path.name} has no H1 heading")
        elif not heading.startswith(f"# ADR-{num}:"):
            problems.append(f"{path.name} is numbered {num} but its heading reads {heading[2:].split(':')[0]!r}")

    for num, names in sorted(by_number.items()):
        if len(names) > 1:
            problems.append(
                f"ADR-{num} is claimed by {len(names)} files: {', '.join(names)} — renumber the one that landed second"
            )

    return problems


def check_guest_claims() -> list[str]:
    """A host that says it runs no guests, while another row says it hosts one.

    docs/architecture.md's host table carried both of these at once for two
    days: `Saruman`'s row read "Proxmox VE 9, no guests yet" while the row
    immediately below it read "`alexander` … A guest on `Saruman`". Both are
    free prose in the Contents column, so the host-and-stack assertion had
    nothing to compare — it checks that the hosts and stacks named here match
    network.md and stacks/, and both rows passed that.

    The claim is only checkable against its siblings, which is what this does:
    if any row declares itself a guest on some host, that host may not also
    claim to have none. Narrow on purpose — it asserts one contradiction rather
    than trying to understand the column.
    """
    problems: list[str] = []
    text = ARCH_MD.read_text(encoding="utf-8")

    hosts = tables_under(text, re.compile(r"^##\s+.*[Hh]ost", re.MULTILINE))
    if not hosts:
        return ["docs/architecture.md has no host table under a Host heading"]
    rows = [r for r in hosts[0] if len(r) >= 4]

    # Which hosts something claims to be a guest of. The name is in backticks
    # in every row that says this, which is what keeps the match from picking
    # up prose that merely mentions a hostname.
    guest_of: dict[str, list[str]] = {}
    for row in rows:
        name = row[0].split("`")[1] if "`" in row[0] else row[0]
        for m in re.finditer(r"[Gg]uest on\s+`([^`]+)`", row[-1]):
            guest_of.setdefault(m.group(1), []).append(name)

    for row in rows:
        name = row[0].split("`")[1] if "`" in row[0] else row[0]
        if not re.search(r"no guests(?:\s+yet)?\b", row[-1], re.IGNORECASE):
            continue
        if name in guest_of:
            problems.append(
                f"docs/architecture.md says {name} has no guests, and "
                f"{', '.join(guest_of[name])} says it is a guest on it"
            )
    return problems


def check_firewall_posture() -> list[str]:
    """docs/security.md naming the same segments docs/firewall-claims.yaml does.

    This is the CI half of #363 and it is deliberately the smaller half. The
    claims file is checked against the running firewall by
    check_firewall_claims.py, which cannot run here: the ruleset is not in this
    repository and never will be, for the reason security.md gives two sections
    below. What CAN be checked here is that the prose and the claims file agree,
    so the pair closes — the file cannot drift from the firewall, and the
    section cannot drift from the file.

    Anchored on "Default deny holds for" / "It does not hold for", which is how
    that section has opened since it was written. If those sentences are
    rewritten this stops matching, and it says so rather than passing.
    """
    problems: list[str] = []
    claims_path = REPO / "docs" / "firewall-claims.yaml"
    if not claims_path.is_file():
        return [f"{claims_path.relative_to(REPO)} is missing — check-firewall has nothing to check"]

    claims = yaml.safe_load(claims_path.read_text(encoding="utf-8")) or {}
    interfaces = claims.get("interfaces") or {}
    if not interfaces:
        return [f"{claims_path.name} declares no interfaces"]

    # IPv4 only. The v6 rows are latent — nothing routes over them — and are
    # prose in that section rather than a list, so holding them to this shape
    # would force the paragraph to lie about which half is live.
    deny, open_ = set(), set()
    for spec in interfaces.values():
        segment = (spec or {}).get("segment")
        if not segment:
            problems.append(f"{claims_path.name} has an interface with no segment name")
            continue
        (deny if not ((spec.get("wholesale") or {}).get("inet")) else open_).add(segment)

    text = (REPO / "docs" / "security.md").read_text(encoding="utf-8")

    def named(prefix: str) -> set[str] | None:
        """Bolded segment names in the sentence starting with `prefix`.

        The window starts two characters early and the prefix is then dropped,
        because one of these sentences bolds its own opening words — "**It does
        not hold for Hicks (50)**" — so anchoring on the prefix alone would put
        the run's opening `**` outside the window and pair the closing one with
        whatever bold came next.
        """
        start = text.find(prefix)
        if start == -1:
            return None
        sentence = text[max(0, start - 2) : start + 400].replace(prefix, "", 1)
        # Up to the first sentence-ending period that is not inside the bold
        # runs — the names carry "(99)" and similar, so a naive split on "." is
        # fine here but a split on ", " is not.
        cut = re.search(r"\.\s", sentence)
        if cut:
            sentence = sentence[: cut.start()]
        return {re.sub(r"\s*\(\d+\)$", "", n).strip() for n in re.findall(r"\*\*(.+?)\*\*", sentence)}

    claimed_deny = named("Default deny holds for")
    claimed_open = named("It does not hold for")
    if claimed_deny is None or claimed_open is None:
        return problems + [
            (
                "docs/security.md no longer opens its Segmentation section with "
                "'Default deny holds for' / 'It does not hold for', so this check "
                "has stopped reading it — re-anchor it or the section is unchecked"
            )
        ]

    for segment in sorted(deny - claimed_deny):
        problems.append(
            f"{claims_path.name} says default deny holds for {segment}, and "
            f"security.md's Segmentation section does not say so"
        )
    for segment in sorted(claimed_deny - deny):
        problems.append(
            f"security.md says default deny holds for {segment}, and "
            f"{claims_path.name} says its catch-all still reaches another segment"
        )
    for segment in sorted(open_ ^ claimed_open):
        problems.append(f"{segment} is named as an exception by exactly one of security.md and {claims_path.name}")

    return problems


def check_buy_list() -> list[str]:
    """README's count of outstanding purchases against the roadmap's table.

    `roadmap.md` is a record and not checked as prose — see the module
    docstring. This assertion reads it as a *source* instead: the buy table is
    the one place a purchase is allowed to enter or leave, by that section's
    own rule, so it is the only thing README's count can honestly be checked
    against.

    It exists because the count went stale inside a day. The reconciliation on
    2026-09-10 grew the table from three rows to seven and left README saying
    "three items now", in the sentence that exists *because* a purchase had
    been missed once already. Every other count in this repository is asserted
    here; this one was prose on both ends, so nothing caught it.

    The "one later" half is checked too. `ifrit` is a paragraph rather than a
    row, and a second such paragraph would make the word wrong in exactly the
    same silent way.
    """
    problems: list[str] = []
    roadmap = ROADMAP_MD.read_text()

    tables = tables_under(roadmap, re.compile(r"^## Everything still to buy\b"))
    if not tables:
        return [
            (
                "docs/roadmap.md has no table under 'Everything still to buy' — "
                "README's count of outstanding purchases has nothing to check "
                "against"
            )
        ]

    # The section's first table is the "Buy these" list; rows[0] is its header.
    rows_now = len(tables[0]) - 1
    rows_later = len(re.findall(r"\*\*One more, later", roadmap))

    num = "|".join(["[0-9]+", *NUMBER_WORDS])
    pattern = re.compile(rf"({num}){WS}items?{WS}now,{WS}({num}){WS}later", re.IGNORECASE)
    text = (REPO / "README.md").read_text()
    match = pattern.search(text)
    if not match:
        return [
            (
                "README.md no longer says 'N items now, M later' of the roadmap's "
                "outstanding purchases — the claim moved or was reworded, and this "
                "assertion cannot follow it. Update the pattern or drop the check "
                "deliberately"
            )
        ]

    line = text[: match.start()].count("\n") + 1
    said_now, said_later = number(match.group(1)), number(match.group(2))
    if said_now != rows_now:
        problems.append(
            f"README.md:{line} says {match.group(1)} outstanding purchase(s), "
            f"but docs/roadmap.md's buy table has {rows_now} row(s)"
        )
    if said_later != rows_later:
        problems.append(
            f"README.md:{line} says {match.group(2)} purchase(s) later, but "
            f"docs/roadmap.md has {rows_later} 'One more, later' paragraph(s)"
        )
    return problems


RUNBOOK_BASE = "https://github.com/Gerrrt/HomeLab/blob/main/docs/runbooks/"


def github_anchor(heading: str) -> str:
    """The id GitHub gives a Markdown heading, as its renderer computes it.

    Lower-case; drop everything but letters, digits, spaces, hyphens and
    underscores; each space becomes a hyphen, NOT collapsed, so "Verify — the"
    is `verify--the`. Inline code backticks go with the other punctuation.
    """
    text = heading.strip().lower()
    text = re.sub(r"[^\w\- ]", "", text)
    return text.replace(" ", "-")


def markdown_anchors(text: str) -> set[str]:
    """The anchors GitHub generates for a Markdown file's headings.

    Lines inside fenced code are skipped: a shell comment such as
    `# Debian/Ubuntu` in a bash block is not a heading, and GitHub makes no
    anchor for it (#874 review). A fence opens on three or more backticks or
    tildes and closes on a fence of the same character.
    """
    anchors: set[str] = set()
    fence = ""
    for line in text.splitlines():
        stripped = line.lstrip()
        m = re.match(r"(`{3,}|~{3,})", stripped)
        if m:
            if not fence:
                fence = m.group(1)[0]
            elif m.group(1)[0] == fence:
                fence = ""
            continue
        if fence:
            continue
        h = re.match(r"#{1,6}\s+(.+?)\s*$", line)
        if h:
            anchors.add(github_anchor(h.group(1)))
    return anchors


def check_runbook_urls() -> list[str]:
    """Every critical alert names a runbook that exists, at a section that exists.

    #842. About 95 of the estate's alerts named no runbook, critical ones
    included, and there was no runbook for a security alert at all: the pages
    most likely to arrive at 1am pointed at nothing. A `runbook_url` is a
    claim about a file, so it is checked like every other claim here: the URL
    has the repository's shape, the file is in the tree, and the anchor, if
    any, is a heading in that file. A renamed heading fails here instead of
    sending a phone to the top of the page.
    """
    problems = []
    anchors: dict[pathlib.Path, set[str]] = {}
    rule_files = sorted((STACK / "prometheus/rules").glob("*.yaml")) + sorted((STACK / "loki/rules").glob("*.yaml"))
    for path in rule_files:
        doc = yaml.safe_load(path.read_text(encoding="utf-8")) or {}
        for group in doc.get("groups") or []:
            for rule in group.get("rules") or []:
                name = rule.get("alert")
                if not name or (rule.get("labels") or {}).get("severity") != "critical":
                    continue
                where = f"{path.relative_to(REPO)}: {name}"
                url = (rule.get("annotations") or {}).get("runbook_url")
                if not url:
                    problems.append(f"{where} is critical and has no runbook_url")
                    continue
                if not url.startswith(RUNBOOK_BASE):
                    problems.append(f"{where}: runbook_url must start with {RUNBOOK_BASE}")
                    continue
                file, _, anchor = url[len(RUNBOOK_BASE) :].partition("#")
                target = REPO / "docs/runbooks" / file
                if not target.is_file():
                    problems.append(f"{where}: runbook_url names docs/runbooks/{file}, which does not exist")
                    continue
                if anchor:
                    if target not in anchors:
                        anchors[target] = markdown_anchors(target.read_text(encoding="utf-8"))
                    if anchor not in anchors[target]:
                        problems.append(f"{where}: docs/runbooks/{file} has no heading for #{anchor}")
    return problems


# Every `](target)` — inline links, images, and the outer target of a linked
# image `[![alt](img)](target)`, which a `!?\[...\]\(...\)` form misses because
# it stops at the inner image. Reference definitions are matched separately.
_INLINE_TARGET = re.compile(r"\]\(([^)]+)\)")
# A link reference definition at line start: `[label]: target` with only a
# destination, optionally a quoted title. A footnote (`[^label]:`) or a line
# whose "target" is followed by unquoted prose is not one — per CommonMark it
# is a paragraph, so `[^lan]: True in both directions` is not a link.
_REF_DEF = re.compile(r"""^\s{0,3}\[(?!\^)[^\]]+\]:\s+(\S+)\s*(?:["'(].*)?$""")


def markdown_link_targets(text: str) -> list[str]:
    """Every link target in a Markdown file's prose, skipping fenced code.

    Covers the three forms this repository uses: inline `[text](target)` and
    `![alt](target)`, the outer target of a linked badge `[![alt](img)](target)`
    (both the inner image and the outer link are yielded), and reference
    definitions `[label]: target`. A `](...)` or definition inside a ``` or
    ~~~ block is an example, not a live link — the same fence rule
    markdown_anchors uses.

    A reference definition whose target itself opens with `[` is a footnote
    carrying a nested link (`[^x]: [text](url)`); its real `](url)` is caught by
    the inline pass, so the `[text` fragment is not treated as a target.
    """
    targets: list[str] = []
    fence = ""
    for line in text.splitlines():
        m = re.match(r"(`{3,}|~{3,})", line.lstrip())
        if m:
            if not fence:
                fence = m.group(1)[0]
            elif m.group(1)[0] == fence:
                fence = ""
            continue
        if fence:
            continue
        targets.extend(lm.group(1) for lm in _INLINE_TARGET.finditer(line))
        ref = _REF_DEF.match(line)
        if ref:
            targets.append(ref.group(1))
    return targets


def _tracked_markdown() -> list[pathlib.Path]:
    """Tracked `*.md` paths, from git rather than a walk of the working tree.

    The convention `check_image_pins.py` sets, and for its reasons: a recursive
    walk descends into the git worktrees this repository keeps under
    `.claude/worktrees/` and reports another branch's files, and an untracked
    scratch `.md` is nobody's concern and must not fail a local run.
    """
    out = subprocess.run(
        ["git", "ls-files", "-z", "*.md"],
        cwd=REPO,
        capture_output=True,
        text=True,
        check=True,
    )
    return [REPO / rel for rel in out.stdout.split("\0") if rel]


def relative_link_problems(rel: str, text: str, root: pathlib.Path) -> list[str]:
    """The broken relative links in one Markdown file's text (pure; testable).

    `rel` is the file's path for messages, `text` its contents, `root` the
    repository it lives in. External links (`http(s)`, `mailto`, `tel`) are not
    reached out to. A target that resolves outside `root` fails: GitHub cannot
    render a link that leaves the repository, so an absolute path or a `../`
    that escapes the checkout is as broken as a missing file.
    """
    problems: list[str] = []
    here = (root / rel).parent
    own_anchors = markdown_anchors(text)

    def anchors_of(path: pathlib.Path) -> set[str]:
        return markdown_anchors(path.read_text(encoding="utf-8"))

    for raw in markdown_link_targets(text):
        target = raw.strip()
        if target.startswith("<") and target.endswith(">"):
            target = target[1:-1].strip()
        target = target.split()[0] if target.split() else target  # drop a title
        if not target or target.startswith(("http://", "https://", "mailto:", "tel:")):
            continue
        path_part, _, anchor = target.partition("#")
        if not path_part:  # a same-file anchor, [text](#heading)
            if anchor and anchor not in own_anchors:
                problems.append(f"{rel}: #{anchor} is not a heading in this file")
            continue
        dest = (here / path_part).resolve()
        try:
            dest.relative_to(root.resolve())
        except ValueError:
            problems.append(f"{rel}: link to {target} leaves the repository")
            continue
        if not dest.exists():
            problems.append(f"{rel}: link to {target} — {path_part} does not exist")
            continue
        if anchor and dest.suffix == ".md" and anchor not in anchors_of(dest):
            problems.append(f"{rel}: {path_part} has no heading for #{anchor}")
    return problems


def check_relative_links() -> list[str]:
    """Every relative Markdown link resolves to a file, and its anchor to a heading.

    Two broken links shipped in one week — a template pointing at a mis-typed
    ADR filename, and an ADR using `../adr/` from `docs/` (which resolves to a
    repository-root `/adr/` that does not exist). `check_adr_numbers` checks
    that ADR files are numbered right; nothing checked that a link *to* one
    lands. This does, the same way `check_runbook_urls` checks a critical
    alert's `runbook_url`: the target is a file in the tree, inside the
    repository, and its anchor, if any, is a heading in it.

    Unlike the claim checks, this runs over every tracked Markdown file,
    records included: a link in `changelog.md` or an ADR that points at a
    renamed file is broken whether or not the prose around it is a record, so
    the present-tense scope does not apply to it. The parsing and resolution
    live in `relative_link_problems`, which `--self-test` exercises directly.
    """
    problems: list[str] = []
    for md in sorted(_tracked_markdown()):
        rel = str(md.relative_to(REPO))
        problems += relative_link_problems(rel, md.read_text(encoding="utf-8"), REPO)
    return problems


def main() -> int:
    f = facts()
    checks = (
        ("counted claims", lambda: check_counts(f)),
        ("SNMP targets against docs/network.md", check_snmp_targets),
        ("host and stack mapping", check_host_stack_table),
        ("ports table against compose.yaml", check_ports),
        ("compute table against docs/network.md", check_compute_table),
        ("image versions in prose (compose.yaml owns them)", check_image_versions),
        ("ADR numbering", check_adr_numbers),
        ("firewall posture prose against docs/firewall-claims.yaml", check_firewall_posture),
        ("guest claims against each other", check_guest_claims),
        ("README's outstanding-purchase count against the roadmap's table", check_buy_list),
        ("critical alerts' runbook_url against docs/runbooks/", check_runbook_urls),
        ("relative Markdown links resolve to a file and an anchor", check_relative_links),
    )

    # The registry is the source for README's "N assertions", so adding a
    # check above updates the claim rather than invalidating it. Written here
    # rather than in facts() because the tuple closes over `f`, so the count
    # does not exist until after the dict does; check_counts reads it through
    # the same closure, which runs below this line.
    f["assertions"] = len(checks)

    total = 0
    for label, check in checks:
        problems = check()
        total += len(problems)
        for problem in problems:
            print(f"  {problem}", file=sys.stderr)
        if problems:
            print(f"  ^ {label}\n", file=sys.stderr)

    if total:
        print(
            f"{total} disagreement(s) between the documents and the configs",
            file=sys.stderr,
        )
        return 1

    print(
        f"docs OK — {f['prometheus_rules']} Prometheus + {f['loki_rules']} Loki "
        f"rules, {f['dashboards']} dashboards, {f['panels']} panels, "
        f"{len(checks)} assertions"
    )
    return 0


def self_test() -> int:
    """The anchor rules check_runbook_urls depends on (#842, #874)."""
    doc = (
        "# Title\n## 5. Verify — the certificate\n### `make up`, then\n"
        "```bash\n# Debian/Ubuntu\nsudo apt install x\n```\n"
        "~~~\n# Also not a heading\n~~~\n## After the fence\n"
    )
    got = markdown_anchors(doc)
    cases = [
        ("a heading gets GitHub's anchor", "title" in got),
        ("an em dash leaves a double hyphen, not one", "5-verify--the-certificate" in got),
        ("inline code and commas are dropped", "make-up-then" in got),
        ("a # comment inside a ``` fence is not a heading", "debianubuntu" not in got),
        ("nor inside a ~~~ fence", "also-not-a-heading" not in got),
        ("a heading after a closed fence is one", "after-the-fence" in got),
    ]
    # The link parser and resolver check_relative_links depends on. Built as a
    # temp tree so the resolution, not only the parsing, is exercised.
    import tempfile

    extracted = markdown_link_targets(
        "[a](one.md) and ![i](img.png)\n"
        "[![alt](badge.svg)](outer.md#sec)\n"
        "[ref]: ../two.md\n"
        "[^note]: True, and not a link at all\n"
        "[bad]: also not a link, just prose\n"
        "```\n[x](skipped.md)\n```\n"
    )
    cases += [
        ("an inline link target is found", "one.md" in extracted),
        ("an image target is found", "img.png" in extracted),
        ("a linked badge yields its outer target", "outer.md#sec" in extracted),
        ("and its inner image target", "badge.svg" in extracted),
        ("a reference definition is found", "../two.md" in extracted),
        ("a footnote is not a link", "True," not in extracted),
        ("prose after a label is not a link", "also" not in extracted),
        ("a target inside a fence is skipped", "skipped.md" not in extracted),
    ]
    with tempfile.TemporaryDirectory() as d:
        root = pathlib.Path(d)
        (root / "b.md").write_text("# Top\n## A Section\n")
        (root / "img.png").write_bytes(b"")
        text = (
            "[ok](b.md)\n[anchor-ok](b.md#a-section)\n[img-ok](img.png)\n"
            "[self-ok](#here)\n# Here\n"
            "[missing](nope.md)\n[anchor-bad](b.md#ghost)\n"
            "[self-bad](#ghost)\n[escape](../outside.md)\n"
        )
        probs = relative_link_problems("a.md", text, root)
        joined = " | ".join(probs)
        cases += [
            ("a valid file link passes", "b.md —" not in joined),
            ("a valid cross-file anchor passes", "a-section" not in joined),
            ("a valid same-file anchor passes", "#here" not in joined),
            ("a missing file is caught", "nope.md does not exist" in joined),
            ("a missing cross-file anchor is caught", "no heading for #ghost" in joined),
            ("a missing same-file anchor is caught", "#ghost is not a heading" in joined),
            ("a target leaving the repository is caught", "leaves the repository" in joined),
        ]

    failed = 0
    for name, ok in cases:
        failed += not ok
        print(f"  {'PASS' if ok else 'FAIL'} {name}")
    return 1 if failed else 0


if __name__ == "__main__":
    if "--self-test" in sys.argv[1:]:
        sys.exit(self_test())
    sys.exit(main())