Skip to content

Spend verification where the money is, and leave a map for the next session - #25

Merged
napkinstack-agent[bot] merged 1 commit into
mainfrom
ops/how-we-verify
Sep 22, 2026
Merged

napkinstack-agent[bot] merged 1 commit into
mainfrom
ops/how-we-verify

Conversation

@napkinstack-agent

Copy link
Copy Markdown
Contributor

Spend verification where the money is, and leave a map for the next session


Module

None. Two documents under docs/, plus an index row.

  • This pull request touches a single module — n/a, no module touched
  • ⚠️ Cross-module

Deliverable

Not delivery work: no module changed beyond its description. It answers the decider's point
that too many constraints become a real brake, and leaves the project resumable across an
interruption.

What it changes

ADR-0003 — the verification budget follows criticality

Cycle 02 spent six independent verification rounds on screening's window read: a high
module, read-only, holding no key and placing no order — while the critical module has not
been started. The rounds found eleven real defects, so they were not wasted; the cost was
still paid at the wrong end.

Criticality Budget
high and below, read-only One round. Blockers fixed, everything else a recorded follow-up. A confirmation run at the head for T4 is not a round
critical — key, orders, money Adversarial rounds without a cap, until the verifier signs

Criticality is already in the MANIFEST, so this needs no new field and no judgement in the
moment.

And a second rule, which the numbers made necessary

Reading the eleven defects by origin:

Origin Count
In the original implementation 4
Introduced by the author in a later round, in changes nobody asked for 3
Fixtures of the author's that proved nothing 9 across the cycle

Three of round six's blockers did not exist before round five. Each came from tidying inside a
fix: a guard removed as "unreachable" on a reason that was false, an assertion deleted while
rewriting the test around it, a reorder made because the code read better that way.

The rounds were not finding a module that resisted being fixed. They were finding a fixer
who kept changing more than was asked.
So: a fix changes only what the finding names;
anything else becomes a follow-up line. It is free, and it would have prevented three of this
cycle's blockers.

What this does not reduce: the two things a critical module must show, and they are
tests rather than process — the agent key places orders and nothing else, and revocation
works.

docs/project/handover.md — the map

Where the project stands, what is in flight and what it still needs, where every decision
lives, what belongs to the decider, the next steps in order, and the working conventions that
are not obvious: gh-agent only, never git config --global, mutation work with bytecode
writing off and the patch verified as applied before its result is read, exit codes
measured without a pipe, gates checked by their own exit status rather than by a command
chained after a tail.

Each of those is there because it has already gone wrong here.

It is indexed in docs/project/README.md so it is found rather than remembered, and it states
that where it disagrees with the charter, a cycle, an ADR or a PDR, those win.

Oracle

  • The success criterion was written before the implementation — ADR-0003 carries a dated
    one that can fail: if any high deliverable takes more than one round, or any blocker
    is traced to a change its finding did not name, the rules did not bind
  • It was seen failing, for the right reason — cycle 02 violated both rules, six times and
    three times respectively, which is what produced them
  • It passes now — nstack fitness compliant

Kind of oracle: fitness function · other: the record of the next cycle is what tests this


Review budget

  • Lines changed (excluding generated): 191
  • Files touched: 3
  • Within budget

Contracts

  • No contract affected

Definition of Done

  • Oracle green — nstack fitness compliant, 92 tracked files
  • Lint, format, types — n/a, no code
  • Fitness functions
  • Security analysis — the ADR reduces scrutiny on read-only modules and reduces none
    on critical ones; the two non-negotiable key tests are named explicitly so the
    reduction cannot be read as covering them
  • Affected documentation updated — ADR index, project README
  • Diff self-reviewed line by line
  • everything else — n/a, no code

Test sheet

Not required: no module changed beyond its description.


Summary

DONE
ADR-0003 and handover.md, both indexed.

VERIFIED
nstack fitness compliant. The defect counts are taken from this cycle's own verification
reports and commit history.

ASSUMED
That one round catches most of what matters on a read-only module. Four of the eleven defects
were found in the first round, which is evidence and not proof.

NOT VERIFIED
Whether "no unasked change" is followed. Nothing enforces it; it is a habit with a success
criterion attached, and the next cycle is what tests it.

RISKS
Less verification on high modules means some defects ship there. Taken deliberately: a wrong
figure in a screening report is visible to an administrator who can re-run it; a wrong order
in the copying module is a member's money. The removal condition names the event that would
prove this wrong — a high module shipping a defect that reaches a member's money.

Second: a follow-up list nobody reads. That is where "no unasked change" sends everything it
stops, and it is the failure mode to watch.


Signals to report

  • A rule applied by hand would be worth automating — a verification round that finds a
    blocker introduced by the previous round's fix is a specific, detectable pattern.
    Three of this cycle's blockers were of that shape and only a human reading two reports
    side by side noticed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NGwPYhV6fYZhjrh41GNfJy

…ession

Two things, both written because the work is being interrupted and a session
with no memory of it has to pick it up.

ADR-0003 answers the decider's point that too many constraints are a real
brake. Cycle 02 spent six independent verification rounds on a read-only
module that holds no key and places no order, while the `critical` module has
not been started. The budget now follows criticality: one round for `high` and
below, adversarial rounds without a cap for anything that moves money.

Reading the eleven defects by origin is what made the second rule necessary.
Four were in the original implementation; three I introduced in later rounds,
in changes nobody asked for; nine fixtures of mine proved nothing. The rounds
were not finding a module that resisted being fixed, they were finding a fixer
who kept changing more than was asked. So: a fix changes only what the finding
names, and anything else becomes a follow-up line. It is free, and it would
have prevented three of this cycle's blockers.

What the ADR does not reduce: the two things a `critical` module must show, and
they are tests rather than process — the agent key places orders and nothing
else, and revocation works.

`docs/project/handover.md` is the map: where the project stands, what is in
flight and what it still needs, where every decision lives, what belongs to the
decider, the next steps in order, and the working conventions that are not
obvious — gh-agent only, never `git config --global`, mutation work with
bytecode writing off and the patch verified as applied before its result is
read, exit codes measured without a pipe, and gates checked by their own exit
status rather than by a command chained after a `tail`.

Each of those is there because it has already gone wrong once here.

It is indexed in `docs/project/README.md` so it is found rather than
remembered, and it says plainly that where it disagrees with the charter, a
cycle, an ADR or a PDR, those win.

Agent-Session: session_01NGwPYhV6fYZhjrh41GNfJy
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NGwPYhV6fYZhjrh41GNfJy
@napkinstack-agent
napkinstack-agent Bot merged commit b639939 into main Sep 22, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant