From 44a17f8fb4f48929fdc3b6496a464da323c02a5f Mon Sep 17 00:00:00 2001 From: Ray Walker Date: Sun, 20 Sep 2026 00:27:18 +1000 Subject: [PATCH 1/3] feat(tools): verify-time twin-equivalence gate keyed on a twin_of declaration (LAB-3967) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit verify() now hard-fails any python-frame vector declaring `twin_of` that differs from its base beyond envelope encoding (value_json, frame-prefix bytes, compressed_data_hex, checksum_hex, original_size, format, inner_msgpack_hex), or whose declaration cannot hold: unknown base, self-reference, same-encoding copy, envelope-less base, duplicate vector names. Until now that check ran only at generate time, as a warning. Keyed on the declaration, not the bytes: from the bytes alone "the wheel drifted" and "the protocol legitimately moved while the legacy vector stayed frozen" are indistinguishable, so mirroring the byte compare into CI would move the generator deadlock one level up. The exit for a legitimate protocol evolution is dropping `twin_of` in the same commit as the regenerated bytes — a reviewable fixture diff — with every byte comparison intact and both encodings still observed. - generate keeps warning (never raising) and names both exits - _upsert carries `twin_of` across rebuilds; generate never adds/drops it - the generator's `_bin` description no longer claims twin-ness in prose, so the exit survives the next generate - fixture: `twin_of` added to the `_bin` vector, description reworded; every frame/payload/envelope byte unchanged (sha256 d8a3756a… → f43eb733…); no repo in the org vendors this fixture - test_python_frame_reference.py rewritten as a stdlib mutation suite that runs verify against a mutated copy of the committed fixture - dead `import msgpack` in generate(); bare `assert` on module import in the test → explicit ImportError --- CHANGELOG.md | 29 ++++ spec/wire-format.md | 10 ++ test-vectors/python-frame.json | 3 +- tools/python-frame-reference.py | 194 +++++++++++++-------- tools/test_python_frame_reference.py | 246 ++++++++++++++++++++++----- 5 files changed, 359 insertions(+), 123 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ae60140..637189c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -92,6 +92,35 @@ All notable changes to the CacheKit Protocol Specification. ### Test Vectors +- **`verify` now enforces the python-frame default-path twin claim** (LAB-3967, + follow-up to LAB-1203). The `bin` twin in + `test-vectors/python-frame.json` carries a new operator-owned + `"twin_of": "default_saas_write_msgpack_bytestorage"` field, and + `python-frame-reference.py verify` hard-fails any declared twin that differs + from its base beyond envelope encoding — `value_json`, frame-prefix bytes + (compared as bytes, so a header key reorder is caught), `compressed_data_hex`, + `checksum_hex`, `original_size`, `format`, `inner_msgpack_hex` — or names a + vector that does not exist. Until now that check ran only at `generate` time, + as a stderr warning nobody in CI reads. The gate is keyed on the declaration, + not on the bytes, because from the bytes alone "the wheel drifted" and "the + protocol legitimately moved while the legacy vector stayed frozen" are + indistinguishable — mirroring the byte compare into CI would have moved the + LAB-1203 generator deadlock one level up. `generate` keeps warning (never + raising) and now names the two exits: fix the wheel/codec, or drop `twin_of` + in the same commit as the regenerated bytes. `_upsert` carries `twin_of` + across rebuilds and never adds or drops it; the generator's `_bin` + description no longer makes the twin claim in prose (the field is the claim), + so the exit survives the next `generate`. A declaration that cannot hold — + pointing at itself, at a same-encoding copy, at an envelope-less vector, or + into a fixture with duplicate vector names — fails rather than passing + vacuously. Pinned by `tools/test_python_frame_reference.py` (mutation suite + over the committed fixture). **Fixture sha256 changes** — JSON metadata only + (`twin_of` added, `_bin` description reworded); every `frame_hex`, + `expected_payload_hex`, `expected_header` and `payload_envelope` byte is + unchanged: `d8a3756a6814971f5a2c3e6573908f60789056ecf4a2f59c28bad525745a4b6c` → + `f43eb733ccccdd9b75e95f9695719733ec78c7e31f06d8db42f0cd12b5160ac7` (no SDK + vendored this fixture when this landed, so nothing downstream re-pins). + - `tools/python-frame-reference.py generate` now **upserts by vector name** (LAB-1203): it rebuilds only the vectors the installed `cachekit` wheel can reproduce and leaves every other committed vector byte-untouched, so dropping diff --git a/spec/wire-format.md b/spec/wire-format.md index 3d98225..653d14b 100644 --- a/spec/wire-format.md +++ b/spec/wire-format.md @@ -515,6 +515,16 @@ round-trip) in both envelope encodings — the legacy array-of-ints original release emitting `bin`) — an Arrow-envelope frame (structural checks), and must-reject error vectors — including a CK frame fed to a strict interop reader. +The `bin` twin carries `"twin_of": "default_saas_write_msgpack_bytestorage"`: an +operator-owned declaration that it differs from the legacy vector **only** in +envelope encoding (same value, frame-prefix bytes, compressed bytes, checksum, +size, format and inner MessagePack). `verify` enforces the declaration as a hard +failure; `generate` never adds or removes it and only warns on divergence. When +the default write path legitimately moves, the exit is to drop `twin_of` from +the regenerated vector in the same commit — a reviewable fixture diff — not to +loosen a byte comparison. The legacy vector stays frozen (no installable wheel +emits the array-of-integers envelope any more) as legacy-read proof. + Verify: ```bash diff --git a/test-vectors/python-frame.json b/test-vectors/python-frame.json index efbd433..52f39ea 100644 --- a/test-vectors/python-frame.json +++ b/test-vectors/python-frame.json @@ -70,7 +70,7 @@ }, { "name": "default_saas_write_msgpack_bytestorage_bin", - "description": "Protocol 1.1 twin of default_saas_write_msgpack_bytestorage: same value, same default @cache write path, but the ByteStorage envelope's compressed_data is msgpack bin (serde_bytes) instead of an array of integers. Readers MUST accept both encodings; the legacy encoding stays pinned by the legacy vector's bytes.", + "description": "Default @cache write (StandardSerializer, integrity on) from a protocol 1.1 wheel: the ByteStorage envelope's compressed_data is msgpack bin (serde_bytes) instead of the legacy array of integers. Readers MUST accept both encodings; the legacy encoding stays pinned by default_saas_write_msgpack_bytestorage. When this vector carries 'twin_of', that field \u2014 not this text \u2014 is the claim that the two differ only in envelope encoding, and verify enforces it.", "value_json": { "user_id": 42, "name": "cachekit", @@ -97,6 +97,7 @@ "format": "msgpack", "inner_msgpack_hex": "83a7757365725f69642aa46e616d65a863616368656b6974a6616374697665c3" }, + "twin_of": "default_saas_write_msgpack_bytestorage", "generator": "cachekit 0.17.0 (PyPI wheel; Rust core via PyO3), generated by tools/python-frame-reference.py generate-bin-twin" } ], diff --git a/tools/python-frame-reference.py b/tools/python-frame-reference.py index 145985d..80bf80a 100644 --- a/tools/python-frame-reference.py +++ b/tools/python-frame-reference.py @@ -26,12 +26,22 @@ legacy (array-of-ints) wheel rebuilds the legacy original — the vector a wheel cannot produce is simply not touched. Every generated frame is round-tripped through the real cachekit-py - deserialization path before being written, and the default-path - pair is checked to differ ONLY in envelope encoding — a - mismatch prints a stderr warning (not a hard failure: the - legacy wheel is gone, so a legitimate write-path change can - never be reflected in that frozen vector, and treating the - divergence as fatal would deadlock `generate` forever). + deserialization path before being written, and every vector + declaring `twin_of` is checked against its base (stderr warning + only — see "Twin declarations" below; `verify` is the gate). + +Twin declarations (LAB-3967): + A frame vector may carry `"twin_of": ""` — the operator's + standing claim that it differs from the named base ONLY in envelope + encoding (value, frame-prefix bytes, compressed bytes, checksum, size, + format and inner msgpack all identical). The claim lives in the fixture, + not in this code, because from the bytes alone "the wheel drifted" and + "the protocol legitimately moved while the legacy vector stayed frozen" + are indistinguishable. `generate` never adds or removes the field; a + rebuild carries it over. When the default write path genuinely moves, + the exit is to drop `twin_of` from the regenerated vector in the same + commit — a reviewable fixture diff — and `verify` stays green with every + byte comparison intact and both encodings still observed. The independent parser below implements exactly the layout documented in spec/wire-format.md: @@ -58,6 +68,17 @@ FRAME_VERSION = 3 PREFIX_LEN = 7 # magic(2) + version(1) + header_len(4) +# Generator-owned text for the `_bin` default-path vector. It describes; it makes +# no twin claim — that claim is the operator-owned `twin_of` field, so dropping +# the field is a durable exit the next `generate` cannot revert by rewriting text. +BIN_DESCRIPTION = ( + "Default @cache write (StandardSerializer, integrity on) from a protocol 1.1 wheel: the " + "ByteStorage envelope's compressed_data is msgpack bin (serde_bytes) instead of the legacy " + "array of integers. Readers MUST accept both encodings; the legacy encoding stays pinned by " + "default_saas_write_msgpack_bytestorage. When this vector carries 'twin_of', that field — not " + "this text — is the claim that the two differ only in envelope encoding, and verify enforces it." +) + def _load_wire_format_codec() -> ModuleType: """Load tools/wire-format-reference.py as a module (hyphenated filename).""" @@ -116,10 +137,61 @@ def parse_frame(frame: bytes) -> tuple[dict, bytes]: return header, frame[header_end:] +_TWIN_ENVELOPE_FIELDS = ("compressed_data_hex", "checksum_hex", "original_size", "format", "inner_msgpack_hex") + + +def _frame_prefix_hex(vec: dict) -> str: + """Everything before the payload: magic, version, header length, header bytes.""" + return vec["frame_hex"][: len(vec["frame_hex"]) - len(vec["expected_payload_hex"])] + + +def _twin_divergence(twin: dict, by_name: dict[str, dict]) -> str | None: + """Why `twin` is not an encoding-only twin of its `twin_of` base; None when the claim holds. + + "Differs ONLY in envelope encoding" entails "differs in envelope encoding": + a declaration pointing at itself, or at a same-encoding copy under another + name, is vacuous and fails here rather than passing green. The frame prefix + is compared at the BYTE level, not as parsed JSON — a wheel that reorders or + reformats the header JSON is a byte-level non-twin that a dict compare would + wave through. Shared by verify() (hard fail) and generate() (warning), so + the two can never drift apart on what "twin" means. + """ + base = by_name.get(twin["twin_of"]) + if base is None: + return f"twin_of names unknown vector {twin['twin_of']!r}" + for side in (twin, base): + missing = [k for k in ("value_json", "expected_payload_hex", "payload_envelope") if k not in side] + if missing: + return f"twin_of requires envelope vectors on both sides; {side['name']!r} lacks {', '.join(missing)}" + twin_env, base_env = twin["payload_envelope"], base["payload_envelope"] + if twin_env.get("envelope_encoding") == base_env.get("envelope_encoding"): + return ( + f"declared twin_of {base['name']!r} but both carry envelope_encoding " + f"{twin_env.get('envelope_encoding')!r} — a twin must differ from its base in encoding" + ) + mismatches: list[str] = [] + if twin["value_json"] != base["value_json"]: + mismatches.append("value_json") + if _frame_prefix_hex(twin) != _frame_prefix_hex(base): + mismatches.append("frame prefix (magic/version/header bytes)") + mismatches += [ + f"payload_envelope.{field}" for field in _TWIN_ENVELOPE_FIELDS if twin_env[field] != base_env[field] + ] + if not mismatches: + return None + return f"declared twin_of {base['name']!r} but differs beyond envelope encoding: " + ", ".join(mismatches) + + def verify() -> int: doc = _load_fixture() failures = 0 observed_encodings: set[str] = set() + by_name = {v["name"]: v for v in doc["frame_vectors"]} + if len(by_name) != len(doc["frame_vectors"]): + # twin_of resolves by name; a duplicate would silently shadow the real + # base. _upsert refuses duplicates at generate time — verify must too. + print("FAIL fixture: duplicate frame vector names") + failures += 1 for vec in doc["frame_vectors"]: name = vec["name"] @@ -192,6 +264,13 @@ def verify() -> int: if payload[: det["checksum_len"]].hex() != det["checksum_hex"]: print(f"FAIL {name}: Arrow envelope checksum prefix mismatch") vec_failed += 1 + if "twin_of" in vec: + # CI gate for the twin claim (LAB-3967). Hard fail: the operator's + # exit is dropping the declaration, never loosening this compare. + why = _twin_divergence(vec, by_name) + if why: + print(f"FAIL {name}: {why}") + vec_failed += 1 failures += vec_failed if not vec_failed: print(f"ok {name}") @@ -273,12 +352,7 @@ def _build_default_path_vector() -> dict: _require(default_header["m"] == meta and default_header["s"] == ser_name, "frame header disagrees with unwrap metadata") if encoding == "bin": name = "default_saas_write_msgpack_bytestorage_bin" - description = ( - "Protocol 1.1 twin of default_saas_write_msgpack_bytestorage: same value, same " - "default @cache write path, but the ByteStorage envelope's compressed_data is " - "msgpack bin (serde_bytes) instead of an array of integers. Readers MUST accept " - "both encodings; the legacy encoding stays pinned by the legacy vector's bytes." - ) + description = BIN_DESCRIPTION encoding_note = ( "rmp_serde positional fixarray(4); compressed_data encodes as msgpack bin " "(serde_bytes, protocol 1.1); checksum [u8;8] stays an array of integers" @@ -318,18 +392,23 @@ def _upsert(committed: list[dict], built: list[dict], generator_stamp: str) -> l Never removes anything: a vector this run did not rebuild stays exactly as committed, so dropping a committed vector is structurally impossible. A rebuilt vector whose content matches the committed one (ignoring its - per-vector 'generator' provenance) keeps the committed entry byte-untouched - — a no-op `generate` leaves the fixture byte-identical. Returns the names - of the vectors actually rewritten or added. + per-vector 'generator' provenance and any operator-owned 'twin_of' + declaration) keeps the committed entry byte-untouched — a no-op `generate` + leaves the fixture byte-identical. A rewrite carries 'twin_of' over + unchanged: the wheel knows nothing about it, and generate never adds or + drops it — that is the operator's reviewable move. Returns the names of + the vectors actually rewritten or added. """ index = {v["name"]: i for i, v in enumerate(committed)} _require(len(index) == len(committed), "committed fixture has duplicate vector names") changed: list[str] = [] for vec in built: i = index.get(vec["name"]) - if i is not None and {k: v for k, v in committed[i].items() if k != "generator"} == vec: + old = committed[i] if i is not None else {} + if i is not None and {k: v for k, v in old.items() if k not in ("generator", "twin_of")} == vec: continue - stamped = {**vec, "generator": generator_stamp} + carried = {"twin_of": old["twin_of"]} if "twin_of" in old else {} + stamped = {**vec, **carried, "generator": generator_stamp} if i is None: index[vec["name"]] = len(committed) committed.append(stamped) @@ -339,61 +418,29 @@ def _upsert(committed: list[dict], built: list[dict], generator_stamp: str) -> l return changed -def _require_twin_equivalence(frame_vectors: list[dict]) -> None: - """Warn when the default-path pair differs beyond envelope encoding. - - The `_bin` twin's description asserts the encoding is the sole delta from - the legacy vector. Check it rather than trusting the wheel: a wheel that - also changed the LZ4 level, msgpack key order, or the frame header would - otherwise upsert a vector that lies about what it isolates, into a - fixture downstream SDKs pin (LAB-903). - - This is a warning, not a `_require()` invariant: the legacy (array-of-ints) - wheel is gone from every installable release, so `legacy` can never be - regenerated. A hard failure here would mean any FUTURE default-write-path - change — however legitimate — permanently deadlocks `generate`, because - the newly-rebuilt `_bin` twin can then never again match a legacy vector - frozen at the OLD write path. Surfacing the divergence lets a human decide - whether it's a codec/wheel regression (don't commit) or a genuine protocol - evolution (commit, and update the twin's description to stop claiming an - encoding-only delta) — `generate` itself cannot tell those apart. - - No-op when either default-path twin is absent (partial fixture): generate() - only ever rebuilds the encoding the wheel emits, so nothing is comparable - until both exist. Completeness is gated by verify(), not here. +def _warn_twin_divergence(frame_vectors: list[dict]) -> None: + """Warn (never raise) when a declared twin diverges from its base beyond encoding. + + A warning, not a `_require()` invariant: the legacy (array-of-ints) wheel is + gone from every installable release, so the legacy vector can never be + regenerated, and a hard failure here would permanently deadlock `generate` + the first time the default write path legitimately moves (LAB-1203). + verify() is the gate; this is the operator's early sight of what + verify will reject, with the two legitimate exits spelled out. """ by_name = {v["name"]: v for v in frame_vectors} - legacy = by_name.get("default_saas_write_msgpack_bytestorage") - twin = by_name.get("default_saas_write_msgpack_bytestorage_bin") - if legacy is None or twin is None: - # Partial fixture (fresh bootstrap, or a deliberately removed vector). - # verify's coverage floor pins ENCODINGS, not these names — it fails - # the fixture until both int-array and bin are observed, which today - # only this pair carries. _upsert never removes, so an established - # fixture can never regress into this branch. - print("note: default-path twin pair incomplete; equivalence proof skipped", file=sys.stderr) - return - mismatches: list[str] = [] - if twin["value_json"] != legacy["value_json"]: - mismatches.append("value_json differs from the legacy vector") - # Header equality must hold at the BYTE level, not just as parsed JSON — a - # wheel that reorders or reformats the header JSON would otherwise slip a - # byte-level non-twin past a dict compare. The frame prefix is everything - # before the payload: magic, version, header length, header bytes. - legacy_prefix = legacy["frame_hex"][: len(legacy["frame_hex"]) - len(legacy["expected_payload_hex"])] - twin_prefix = twin["frame_hex"][: len(twin["frame_hex"]) - len(twin["expected_payload_hex"])] - if twin_prefix != legacy_prefix: - mismatches.append("frame prefix (magic/version/header bytes) differs from the legacy vector") - for field in ("compressed_data_hex", "checksum_hex", "original_size", "format", "inner_msgpack_hex"): - if twin["payload_envelope"][field] != legacy["payload_envelope"][field]: - mismatches.append(f"payload_envelope.{field} differs from the legacy vector") - if mismatches: - print( - "warning: default-path twin diverges from the legacy vector beyond envelope " - "encoding (legacy wheel is unreproducible, so this cannot be auto-resolved) — " - "review before committing:\n " + "\n ".join(mismatches), - file=sys.stderr, - ) + for vec in frame_vectors: + if "twin_of" not in vec: + continue + why = _twin_divergence(vec, by_name) + if why: + print( + f"warning: {vec['name']}: {why}\n" + " `verify` will FAIL this fixture. Two legitimate exits: fix the wheel/codec so the " + "rebuilt vector matches its base again, or — if the default write path genuinely " + "moved — drop 'twin_of' from this vector in the same commit as the regenerated bytes.", + file=sys.stderr, + ) def _build_error_vectors(raw_frame: bytes) -> list[dict]: @@ -454,11 +501,8 @@ def _build_error_vectors(raw_frame: bytes) -> list[dict]: def generate() -> int: - import msgpack # third-party; generation only - - from cachekit.serializers.wrapper import SerializationWrapper - import cachekit + from cachekit.serializers.wrapper import SerializationWrapper doc = _load_fixture() built: list[dict] = [] @@ -541,7 +585,7 @@ def generate() -> int: unstamped = {v["name"] for v in doc["frame_vectors"] + doc["error_vectors"] if "generator" not in v} changed = _upsert(doc["frame_vectors"], built, generator_stamp) changed += _upsert(doc["error_vectors"], built_errors, generator_stamp) - _require_twin_equivalence(doc["frame_vectors"]) + _warn_twin_divergence(doc["frame_vectors"]) if not changed: print( diff --git a/tools/test_python_frame_reference.py b/tools/test_python_frame_reference.py index 2ce9aa6..302c4e2 100644 --- a/tools/test_python_frame_reference.py +++ b/tools/test_python_frame_reference.py @@ -1,14 +1,12 @@ #!/usr/bin/env python3 -"""Regression test for the twin-equivalence check in python-frame-reference.py. +"""Mutation suite for the `twin_of` machinery in python-frame-reference.py (LAB-3967). -LAB-1203 fix-loop (Cobel G3(b) HIGH, 2026-09-19): _require_twin_equivalence() -used to `_require()` (raise ValueError) on any divergence between the -`_bin` twin and the frozen `legacy` vector. Since the legacy (array-of-ints) -wheel is gone from every installable release, `legacy` can never be -regenerated — so any LEGITIMATE future default-write-path change would -permanently deadlock `generate()`. The fix downgrades the check to a stderr -warning. This test pins that: a real divergence must be reported, but must -never raise. +Design and rationale: python-frame-reference.py, "Twin declarations". This +suite mutates a copy of the COMMITTED fixture and proves verify() fails on +exactly the mutated field, that the drop-`twin_of` exit stays green without +loosening any byte comparison, that generate() warns and never raises +(LAB-1203 deadlock pin), and that _upsert() carries the declaration across +rebuilds. Stdlib only, no framework. Run: python3 tools/test_python_frame_reference.py (exit 1 on any failure) """ @@ -16,19 +14,168 @@ from __future__ import annotations import contextlib +import copy import importlib.util import io +import json import sys +import tempfile from pathlib import Path HERE = Path(__file__).resolve().parent spec = importlib.util.spec_from_file_location("python_frame_reference", HERE / "python-frame-reference.py") -assert spec and spec.loader +if not (spec and spec.loader): + # Explicit, not `assert`: a bare assert is stripped under `python -O`, the + # same trap the generator's own _require() exists to avoid. + raise ImportError(f"cannot load {HERE / 'python-frame-reference.py'}") pfr = importlib.util.module_from_spec(spec) spec.loader.exec_module(pfr) -LEGACY = { - "name": "default_saas_write_msgpack_bytestorage", +LEGACY_NAME = "default_saas_write_msgpack_bytestorage" +BIN_NAME = LEGACY_NAME + "_bin" +COMMITTED = json.loads(pfr.VECTOR_PATH.read_text()) + +FAILURES = 0 + + +def check(name: str, cond: bool) -> None: + global FAILURES + print(f"{'ok ' if cond else 'FAIL'} {name}") + if not cond: + FAILURES += 1 + + +def run_verify(doc: dict) -> tuple[int, str]: + """Run pfr.verify() against `doc` written to a temp file; returns (exit code, stdout).""" + with tempfile.TemporaryDirectory() as tmp: + path = Path(tmp) / "python-frame.json" + path.write_text(json.dumps(doc)) + saved, pfr.VECTOR_PATH = pfr.VECTOR_PATH, path + out = io.StringIO() + try: + with contextlib.redirect_stdout(out): + rc = pfr.verify() + finally: + pfr.VECTOR_PATH = saved + return rc, out.getvalue() + + +def mutated(mutate) -> tuple[dict, dict]: + """Deep copy of the committed fixture with `mutate(bin_vector)` applied; returns (doc, twin).""" + doc = copy.deepcopy(COMMITTED) + twin = next(v for v in doc["frame_vectors"] if v["name"] == BIN_NAME) + mutate(twin) + return doc, twin + + +def flip_last_nibble(h: str) -> str: + return h[:-1] + ("0" if h[-1] != "0" else "1") + + +def reorder_header_keys(twin: dict) -> None: + """Same parsed header, different header BYTES — the ad90a3f finding. + + Re-serialise the header JSON with sorted keys: parse_frame() yields an + identical dict, so verify's header check passes and ONLY the byte-level + twin compare can catch it. Header length is unchanged, so HDR_LEN stays valid. + """ + frame = bytes.fromhex(twin["frame_hex"]) + header, payload = pfr.parse_frame(frame) + hdr = json.dumps(header, sort_keys=True, separators=(", ", ": ")).encode() + old_hdr = frame[pfr.PREFIX_LEN : len(frame) - len(payload)] + if len(hdr) != len(old_hdr) or hdr == old_hdr: + raise RuntimeError("header key-reorder mutation is not a same-length byte change; test is broken") + twin["frame_hex"] = (frame[: pfr.PREFIX_LEN] + hdr + payload).hex() + + +# --- committed fixture: green, and the declaration is actually there to enforce --- +rc, out = run_verify(COMMITTED) +check("committed fixture verifies green", rc == 0) +committed_twin = next(v for v in COMMITTED["frame_vectors"] if v["name"] == BIN_NAME) +check("committed _bin vector declares twin_of the legacy vector", committed_twin.get("twin_of") == LEGACY_NAME) +check( + "committed _bin description is the generator's (a no-op generate rewrites nothing)", + committed_twin["description"] == pfr.BIN_DESCRIPTION, +) + +# --- verify hard-fails on every field the twin claim covers --- +TWIN_FAIL = "differs beyond envelope encoding" + + +def env_mutation(field: str, fn): + def mutate(twin: dict) -> None: + twin["payload_envelope"][field] = fn(twin["payload_envelope"][field]) + + return mutate + + +MUTATIONS = { + "value_json": lambda t: t["value_json"].__setitem__("user_id", 43), + "frame prefix (magic/version/header bytes)": reorder_header_keys, + "payload_envelope.compressed_data_hex": env_mutation("compressed_data_hex", flip_last_nibble), + "payload_envelope.checksum_hex": env_mutation("checksum_hex", flip_last_nibble), + "payload_envelope.original_size": env_mutation("original_size", lambda n: n + 1), + "payload_envelope.format": env_mutation("format", lambda _: "json"), + "payload_envelope.inner_msgpack_hex": env_mutation("inner_msgpack_hex", flip_last_nibble), +} +# Fields NO other verify check covers: here the twin gate is the only thing standing. +ONLY_TWIN_GATE = {"value_json", "frame prefix (magic/version/header bytes)", "payload_envelope.inner_msgpack_hex"} + +for field, mutate in MUTATIONS.items(): + doc, _ = mutated(mutate) + rc, out = run_verify(doc) + twin_lines = [line for line in out.splitlines() if line.startswith(f"FAIL {BIN_NAME}") and TWIN_FAIL in line] + check(f"mutate {field}: verify exits 1", rc == 1) + check(f"mutate {field}: twin gate names the field", len(twin_lines) == 1 and field in twin_lines[0]) + if field in ONLY_TWIN_GATE: + fail_lines = [line for line in out.splitlines() if line.startswith("FAIL")] + check(f"mutate {field}: twin gate is the ONLY check that fires", fail_lines == twin_lines) + +# --- a dangling declaration is a failure, not a silent skip --- +doc, _ = mutated(lambda t: t.__setitem__("twin_of", "no_such_vector")) +rc, out = run_verify(doc) +check("twin_of naming an unknown vector: verify exits 1", rc == 1) +check("twin_of naming an unknown vector: named in the failure", "unknown vector 'no_such_vector'" in out) + +# --- vacuous declarations fail: self-reference, same-encoding decoy, envelope-less base --- +doc, twin = mutated(reorder_header_keys) +twin["twin_of"] = BIN_NAME +rc, out = run_verify(doc) +check("self-referential twin_of on a diverged twin: verify exits 1", rc == 1) +check("self-referential twin_of: failure names the encoding rule", "must differ from its base in encoding" in out) + +doc, twin = mutated(reorder_header_keys) +doc["frame_vectors"].append({**copy.deepcopy(committed_twin), "name": "decoy_bin_copy"}) +del doc["frame_vectors"][-1]["twin_of"] +twin["twin_of"] = "decoy_bin_copy" +rc, out = run_verify(doc) +check("twin_of pointing at a same-encoding copy: verify exits 1", rc == 1 and "must differ from its base" in out) + +doc, twin = mutated(lambda t: t.__setitem__("twin_of", "raw_payload_frame")) +rc, out = run_verify(doc) +check("twin_of pointing at an envelope-less vector: verify exits 1", rc == 1) +check("twin_of pointing at an envelope-less vector: names the missing fields", "lacks" in out and "payload_envelope" in out) + +# --- a duplicate name cannot shadow the base --- +doc, twin = mutated(reorder_header_keys) +doc["frame_vectors"].append({**copy.deepcopy(twin), "name": LEGACY_NAME}) +del doc["frame_vectors"][-1]["twin_of"] +rc, out = run_verify(doc) +check("duplicate legacy name appended (shadows the base): verify exits 1", rc == 1) +check("duplicate legacy name: failure names duplicates", "duplicate frame vector names" in out) + +# --- the protocol-evolution exit: drop the declaration, nothing else loosens --- +# The header-reorder mutation is a byte-level prefix change that every OTHER +# verify check waves through; with twin_of gone it must verify green, and both +# encodings are still observed, so the coverage floor holds without the claim. +doc, twin = mutated(reorder_header_keys) +del twin["twin_of"] +rc, out = run_verify(doc) +check("evolution exit (twin_of dropped): diverged pair verifies green (coverage floor included)", rc == 0) + +# --- generate-time: warns, never raises, spells out the exits --- +SYNTH_LEGACY = { + "name": LEGACY_NAME, "value_json": {"a": 1}, "frame_hex": "aabbccddpayload", "expected_payload_hex": "payload", @@ -38,49 +185,54 @@ "original_size": 3, "format": "msgpack", "inner_msgpack_hex": "33", + "envelope_encoding": "int-array", }, } +SYNTH_TWIN = { + **SYNTH_LEGACY, + "name": BIN_NAME, + "twin_of": LEGACY_NAME, + "payload_envelope": {**SYNTH_LEGACY["payload_envelope"], "envelope_encoding": "bin"}, +} -FAILURES = 0 +def warn_output(vectors: list[dict]) -> tuple[bool, str]: + buf = io.StringIO() + raised = False + try: + with contextlib.redirect_stderr(buf): + pfr._warn_twin_divergence(vectors) + except ValueError: + raised = True + return raised, buf.getvalue() -def check(name: str, cond: bool) -> None: - global FAILURES - print(f"{'ok ' if cond else 'FAIL'} {name}") - if not cond: - FAILURES += 1 +raised, err = warn_output([SYNTH_LEGACY, {**SYNTH_LEGACY, "name": BIN_NAME}]) +check("generate: no declaration -> silent", not raised and err == "") +raised, err = warn_output([SYNTH_LEGACY, SYNTH_TWIN]) +check("generate: identical declared twin -> silent", not raised and err == "") +diverged = {**SYNTH_TWIN, "payload_envelope": {**SYNTH_LEGACY["payload_envelope"], "inner_msgpack_hex": "ff"}} +raised, err = warn_output([SYNTH_LEGACY, diverged]) +check("generate: diverged declared twin -> does not raise (no generator deadlock)", not raised) +check("generate: warning names the diverging field", "inner_msgpack_hex" in err) +check("generate: warning spells out the drop-twin_of exit", "drop 'twin_of'" in err) +raised, err = warn_output([diverged]) +check("generate: dangling twin_of -> warns, does not raise", not raised and "unknown vector" in err) -# --- identical pair: no warning, no raise --- -twin_ok = {**LEGACY, "name": "default_saas_write_msgpack_bytestorage_bin"} -buf = io.StringIO() -with contextlib.redirect_stderr(buf): - pfr._require_twin_equivalence([LEGACY, twin_ok]) -check("identical twin: no warning emitted", buf.getvalue() == "") - -# --- diverged pair (legitimate protocol evolution OR a wheel bug): warns, does not raise --- -twin_diverged = { - **LEGACY, - "name": "default_saas_write_msgpack_bytestorage_bin", - "payload_envelope": {**LEGACY["payload_envelope"], "inner_msgpack_hex": "ff"}, -} -buf = io.StringIO() -raised = False -try: - with contextlib.redirect_stderr(buf): - pfr._require_twin_equivalence([LEGACY, twin_diverged]) -except ValueError: - raised = True -check("diverged twin: does not raise (no generator deadlock)", not raised) -check("diverged twin: warning names the diverging field", "inner_msgpack_hex" in buf.getvalue()) - -# --- partial fixture: unchanged no-op-with-note behavior --- -buf = io.StringIO() -with contextlib.redirect_stderr(buf): - pfr._require_twin_equivalence([LEGACY]) -check("partial fixture: skip note, no warning text", "incomplete" in buf.getvalue()) +# --- _upsert: the declaration survives a rebuild and never causes churn --- +committed = [copy.deepcopy(SYNTH_TWIN) | {"generator": "old wheel"}] +built = [{k: v for k, v in SYNTH_TWIN.items() if k != "twin_of"}] +changed = pfr._upsert(committed, built, "new wheel") +check("_upsert: identical content -> no-op despite twin_of/generator on the committed side", changed == []) +check("_upsert: no-op leaves the committed entry untouched", committed[0]["generator"] == "old wheel") +built[0]["payload_envelope"] = {**built[0]["payload_envelope"], "inner_msgpack_hex": "ff"} +changed = pfr._upsert(committed, built, "new wheel") +check("_upsert: rebuilt content -> rewritten", changed == [BIN_NAME] and committed[0]["generator"] == "new wheel") +check("_upsert: rewrite carries twin_of over", committed[0].get("twin_of") == LEGACY_NAME) +changed = pfr._upsert(committed, [{**built[0], "name": "brand_new"}], "new wheel") +check("_upsert: appended vector gains no twin_of", changed == ["brand_new"] and "twin_of" not in committed[1]) if FAILURES: print(f"\n{FAILURES} failure(s)") sys.exit(1) -print("\nall twin-equivalence checks passed") +print("\nall python-frame twin-gate checks passed") From e512157bb7ec393188043b21ef981f16e1256173 Mon Sep 17 00:00:00 2001 From: Ray Walker Date: Sun, 20 Sep 2026 00:28:33 +1000 Subject: [PATCH 2/3] test(tools): diverged synthetic twin keeps the bin envelope encoding (LAB-3967) The diverged generate-time case built its envelope from the legacy vector, so it inherited envelope_encoding "int-array" and the new "a twin must differ from its base in encoding" rule fired before the field compare; the case then asserted the wrong warning text. --- tools/test_python_frame_reference.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/test_python_frame_reference.py b/tools/test_python_frame_reference.py index 302c4e2..267e7c7 100644 --- a/tools/test_python_frame_reference.py +++ b/tools/test_python_frame_reference.py @@ -211,7 +211,7 @@ def warn_output(vectors: list[dict]) -> tuple[bool, str]: check("generate: no declaration -> silent", not raised and err == "") raised, err = warn_output([SYNTH_LEGACY, SYNTH_TWIN]) check("generate: identical declared twin -> silent", not raised and err == "") -diverged = {**SYNTH_TWIN, "payload_envelope": {**SYNTH_LEGACY["payload_envelope"], "inner_msgpack_hex": "ff"}} +diverged = {**SYNTH_TWIN, "payload_envelope": {**SYNTH_TWIN["payload_envelope"], "inner_msgpack_hex": "ff"}} raised, err = warn_output([SYNTH_LEGACY, diverged]) check("generate: diverged declared twin -> does not raise (no generator deadlock)", not raised) check("generate: warning names the diverging field", "inner_msgpack_hex" in err) From e12a8b79119fa06410749dddc41a1aba58451b8e Mon Sep 17 00:00:00 2001 From: Ray Walker Date: Sun, 20 Sep 2026 02:03:22 +1000 Subject: [PATCH 3/3] fix(tools): _twin_divergence reports a missing or malformed field as FAIL, not a traceback (LAB-3967) The presence guard covered value_json, expected_payload_hex and payload_envelope, then strictly indexed frame_hex and the five envelope fields the twin compare reads. A vector whose payload_envelope lacked one of them made verify() die with a KeyError traceback instead of the `FAIL {name}: ...` line the gate promises, and the generate-time warning path raised where its contract is "warn, never raise". The guard now covers every field the compare reads, and tests the value rather than the key: a JSON null on both sides would otherwise compare equal and pass the twin claim vacuously. A non-string `twin_of` (an unhashable list, say) is reported the same way instead of a TypeError. The comparison itself stays strict indexing after the precondition. The mutation suite pins each case against the pre-fix behaviour; the CHANGELOG sentence names partial-envelope vectors alongside envelope-less ones. --- CHANGELOG.md | 7 ++++--- tools/python-frame-reference.py | 11 +++++++++-- tools/test_python_frame_reference.py | 20 ++++++++++++++++++++ 3 files changed, 33 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 637189c..14322f6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -111,9 +111,10 @@ All notable changes to the CacheKit Protocol Specification. across rebuilds and never adds or drops it; the generator's `_bin` description no longer makes the twin claim in prose (the field is the claim), so the exit survives the next `generate`. A declaration that cannot hold — - pointing at itself, at a same-encoding copy, at an envelope-less vector, or - into a fixture with duplicate vector names — fails rather than passing - vacuously. Pinned by `tools/test_python_frame_reference.py` (mutation suite + pointing at itself, at a same-encoding copy, at an envelope-less or + partial-envelope vector, or into a fixture with duplicate vector names — + fails rather than passing vacuously (a missing envelope field is a `FAIL` + line, never a traceback). Pinned by `tools/test_python_frame_reference.py` (mutation suite over the committed fixture). **Fixture sha256 changes** — JSON metadata only (`twin_of` added, `_bin` description reworded); every `frame_hex`, `expected_payload_hex`, `expected_header` and `payload_envelope` byte is diff --git a/tools/python-frame-reference.py b/tools/python-frame-reference.py index 80bf80a..ee7997d 100644 --- a/tools/python-frame-reference.py +++ b/tools/python-frame-reference.py @@ -154,13 +154,20 @@ def _twin_divergence(twin: dict, by_name: dict[str, dict]) -> str | None: is compared at the BYTE level, not as parsed JSON — a wheel that reorders or reformats the header JSON is a byte-level non-twin that a dict compare would wave through. Shared by verify() (hard fail) and generate() (warning), so - the two can never drift apart on what "twin" means. + the two can never drift apart on what "twin" means. Never raises on a + malformed declaration or a vector missing (or null in) a compared field: + the reason names what is lacking, so verify prints a FAIL line, not a + traceback. """ + if not isinstance(twin["twin_of"], str): + return f"twin_of must be a vector-name string, got {twin['twin_of']!r}" base = by_name.get(twin["twin_of"]) if base is None: return f"twin_of names unknown vector {twin['twin_of']!r}" for side in (twin, base): - missing = [k for k in ("value_json", "expected_payload_hex", "payload_envelope") if k not in side] + missing = [k for k in ("value_json", "frame_hex", "expected_payload_hex", "payload_envelope") if side.get(k) is None] + env = side.get("payload_envelope") or {} + missing += [f"payload_envelope.{f}" for f in _TWIN_ENVELOPE_FIELDS if env.get(f) is None] if missing: return f"twin_of requires envelope vectors on both sides; {side['name']!r} lacks {', '.join(missing)}" twin_env, base_env = twin["payload_envelope"], base["payload_envelope"] diff --git a/tools/test_python_frame_reference.py b/tools/test_python_frame_reference.py index 267e7c7..ca46d6d 100644 --- a/tools/test_python_frame_reference.py +++ b/tools/test_python_frame_reference.py @@ -156,6 +156,23 @@ def mutate(twin: dict) -> None: check("twin_of pointing at an envelope-less vector: verify exits 1", rc == 1) check("twin_of pointing at an envelope-less vector: names the missing fields", "lacks" in out and "payload_envelope" in out) +# --- a partial envelope is a clean FAIL line, never a KeyError traceback --- +# inner_msgpack_hex is read by NO other verify check, so only the twin gate can trip on it. +doc, _ = mutated(lambda t: t["payload_envelope"].pop("inner_msgpack_hex")) +rc, out = run_verify(doc) +check("twin lacking an envelope subfield: verify exits 1", rc == 1) +check("twin lacking an envelope subfield: failure names it", "lacks payload_envelope.inner_msgpack_hex" in out) + +# JSON null on BOTH sides must not compare equal and pass the claim vacuously. +doc, twin = mutated(lambda t: t["payload_envelope"].__setitem__("inner_msgpack_hex", None)) +next(v for v in doc["frame_vectors"] if v["name"] == LEGACY_NAME)["payload_envelope"]["inner_msgpack_hex"] = None +rc, out = run_verify(doc) +check("null envelope subfield on both sides: verify exits 1", rc == 1 and "lacks payload_envelope.inner_msgpack_hex" in out) + +doc, _ = mutated(lambda t: t.__setitem__("twin_of", [LEGACY_NAME])) +rc, out = run_verify(doc) +check("non-string twin_of: verify exits 1 with a FAIL line", rc == 1 and "must be a vector-name string" in out) + # --- a duplicate name cannot shadow the base --- doc, twin = mutated(reorder_header_keys) doc["frame_vectors"].append({**copy.deepcopy(twin), "name": LEGACY_NAME}) @@ -218,6 +235,9 @@ def warn_output(vectors: list[dict]) -> tuple[bool, str]: check("generate: warning spells out the drop-twin_of exit", "drop 'twin_of'" in err) raised, err = warn_output([diverged]) check("generate: dangling twin_of -> warns, does not raise", not raised and "unknown vector" in err) +# Reachable only via generate: verify() indexes frame_hex for every vector before the twin gate runs. +raised, err = warn_output([{k: v for k, v in SYNTH_LEGACY.items() if k != "frame_hex"}, SYNTH_TWIN]) +check("generate: base lacking frame_hex -> warns, does not raise", not raised and "lacks frame_hex" in err) # --- _upsert: the declaration survives a rebuild and never causes churn --- committed = [copy.deepcopy(SYNTH_TWIN) | {"generator": "old wheel"}]