Skip to content

Record the DASH7 SNR sweep as run once and unusable, and add convention 10 - #24

Open
emrefbulut wants to merge 4 commits into
mainfrom
docs/snr-sweep-unusable
Open

emrefbulut wants to merge 4 commits into
mainfrom
docs/snr-sweep-unusable

Conversation

@emrefbulut

Copy link
Copy Markdown
Owner

What changed

Four documents now say that the DASH7 SNR grid was run once and that the
result is unusable, instead of saying it has not been run. No code changes.

  • docs/leakage-real-snr.md — status block replacing "It has not been run"
  • docs/methodology.md §7 — a new entry under its own heading
  • CONTRIBUTING.md + .github/PULL_REQUEST_TEMPLATE.md — convention 10
  • ROADMAP.md — Now item 1 no longer reads as never attempted

Why

The grid ran to completion in a scratch working tree. Its output was written
where the script was told to write it, was read, and was never committed; the
tree was cleaned up afterwards. Nothing survives — no commit on any branch, no
other worktree, no stash, no reachable object — so there is no table, no
per-run JSON and no log, and therefore no environment stamp, seed pairs or run
count to audit.

Two decisions are recorded with it:

  • The figures are not transcribed from memory. A number whose source file
    does not exist is not reproducible from this repository, which is what
    docs/methodology.md promises in its first line. Writing them down would
    satisfy a reader and mislead them (conventions 3 and 9).
  • The grid is not re-run. The plan is pre-registered: n, the SNR list and
    the stem were fixed before any run so the result could not choose its own
    stopping rule. Having read a result removes that property from the reader, so
    a repeat is not the run the plan describes (convention 5). The item stays
    open to someone who has not read the lost table.

The §7 entry is under a separate heading because it is a process failure, not
a code defect; mixing it into that list would blunt both. It carries the lesson
the loss actually teaches: an uncommitted result is worse than a result that
was never produced, because the work is believed to have been done.
That is
convention 10.

How it was verified

Each claim of absence was checked rather than assumed:

git log --all --oneline -- artifacts/leakage_real_runs.json artifacts/leakage_real_table.md
# (no output -- never committed on any branch)

ls artifacts/ | grep real
# leakage_real_cliff_*, leakage_real_pilot_*, leakage_real_probe_*,
# leakage_real_stride_* only -- the two names the recipe promises are absent

git stash list            # the single stash does not contain them
git fsck --lost-found     # the dangling blobs are a 61-line synthetic stride file

Docs-only change, and the test suite reads these files, so:

uv run ruff format --check .   # 82 files already formatted
uv run ruff check .            # All checks passed!
uv run pytest tests/test_repo_hygiene.py tests/test_preflight.py -q
# 74 passed, 1 skipped in 349.57s

The skip is test_the_loraiq_skip_reason_names_the_variable_to_set, which
skips because IQFORGE_LORAIQ is configured on this machine; CI runs it
unconfigured. The full suite runs here in CI.

Conventions it touches

  • 3 Do not claim what you did not measure — the lost figures are not
    written down, and only what the repository can still show is stated
  • 5 Do not enlarge a sample after seeing the result — the reason the grid
    is not simply re-run
  • 6 Report your own mistakes — the entry is a mistake made in this
    project, recorded rather than quietly dropped
  • 9 A claim repeated is not a claim checked — "it has not been run" was
    true when written and had been restated since; each absence here was
    re-checked against git rather than copied forward
  • 10 A measurement is committed the moment it exists — added by this PR,
    because this is the failure that taught it

🤖 Generated with Claude Code

emrefbulut and others added 4 commits October 13, 2026 18:57
The recipe said the grid had not been run. It had: once, to completion, in a
scratch working tree, with its output written where the script was told to
write it and never committed. The tree was cleaned up and nothing survives --
no commit on any branch, no other worktree, no stash, no reachable object, so
no table, no per-run JSON, no log, and therefore no environment stamp, seed
pairs or run count to audit.

The figures are not transcribed here. A number whose source file does not
exist is not reproducible from this repository, which is what methodology
promises in its first line.

The grid is not re-run either. The plan is pre-registered, and having seen a
result removes that property from whoever saw it -- a repeat by them is not
the pre-registered run however exactly it repeats the command. It stays open
for someone who has not read the lost table.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Section 7 collects code defects that produced plausible output. This one is a
process failure, so it goes under a separate heading rather than being mixed
in -- putting it in the same list would blunt both.

The entry says what is verifiable from the repository: the grid was run, the
output was never committed, the tree was cleaned, and nothing survives. It
carries no figures from the lost table. It also says why the loss is not
repairable by re-running: the plan is pre-registered, and that property does
not survive the person who read the result.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ists

Every convention in this file was learned by breaking it first. This one was
broken this month: the DASH7 SNR grid ran to completion in a working
directory, was read, and was never committed. The directory was cleaned and
the result is unrecoverable.

The rule is to write the run file and the table into artifacts/ and commit
them in the step that produces them, before reading them. The convention says
why the pre-registered case is the expensive one: reading a result you cannot
keep closes the question to you, because a repeat is no longer the
pre-registered run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Item 1 read as never tried. It was run once; the output was never committed
and is unrecoverable. The distinction matters to the next reader, because the
plan is pre-registered: the item is open to someone who has not seen that
result, and a repeat by whoever has is not the run the plan describes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant