Repository navigation
comline-ir-formats: frozen format 3 and the format gate (step 1.2 of #268) - #280
Merged
Merged
Conversation
Kinflou
marked this pull request as ready for review
November 14, 2026 17:40
This was referenced Nov 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step 1.2 of #268 (IR migrations). The format stays 3; nothing here migrates, deletes or changes a store.
What is in it
comline-ir-formats(core/formats/,publish = false, no dependency oncomline-core; deps areserde,ciborium, andarbitrarybehind a feature). One directory per format,src/v3/, with the four files the design names:types.rs: the frozen copy of format 3's types (the WIP commit's file, re-checked against master: no persisted type has changed since 07d86da), plusUnknownRecordandProducedBy.codec.rs: the only code that reads format 3's bytes:decode_unit,decode_config,decode_docs,decode_history,decode_unknown,decode_produced_by. A reader never turns bytes it can't read into an empty value.SCHEMA.md: the document for format 3 (store layout, objects, every blob and entry, absence readings, the ledger table). Source for every format's document; the CAS guide keeps describing only the current one.ledger.toml: 22 rows,3.01to3.22: the twenty additions from the design's table, then the two new entries.The seal.
core/formats/MANIFEST: one line per superseded format,v<N> blake3:<hex> # <why>; the hash is over every file undersrc/v<N>/(relative path, a zero byte, length, bytes, in path order). Empty today: format 3 is current and is sealed when format 4 is introduced.cargo xtask check-sealfails when a seal differs, when a superseded format isn't sealed, and when the current one is.cargo xtask seal-format <N> --why <text>writes or moves a seal and refuses without a reason.live == frozen(current)(core/tests/migrations/live_frozen.rs). (a) every object of the stores built for the fixture packages (with a release and a second build, so commits have parents and arelease): the live type and the frozen copy decode it and each re-encodes it to the same bytes; a root or dependency tree entry the frozen copy doesn't know fails with "this is a ledger row". (b) values of the frozen types drawn withArbitrary(proptest, 512 cases each for schema units, config units, docs, facts, commit, tree, blob): the frozen copy's bytes decode in the live type and re-encode to the same bytes. (c)CURRENT == CURRENT_IR_FORMAT_VERSION, and the entry names agree with core's.Golden fresh-build hashes. Corpus under
core/tests/migrations/corpus/(tour,headers,unverifiedwith@unverifiedat argument, field, type and protocol scope plushandshake = "none"and awire, anddependencies: app, acme with an encoding, base with another) plustests/fixtures/packages/test. A fresh build of each writes the objectscore/tests/migrations/expected/<name>.txtrecords: one line per entry of the root tree, recursively,<hash> <path>. The standard library's schema tree (compiled into a build that usesstd) is not recorded, so a change tostddoesn't move them. Thedependenciesfixture needsdeps. Regenerate withCOMLINE_UPDATE_GOLDEN=1 cargo test -p comline-core --features deps --test mod migrations::golden.ir_roundtrip_testsnow reads itsTOURfrom the corpus file instead of holding a second copy.Reserved entries and fail-closed readers. Root-tree entry
unknown(CBOR[UnknownRecord { path, step, reason, original? }]) andproduced_by(CBORProducedBy { core, spec }), bothdeny_unknown_fields, ledger rows3.21and3.22. Nothing writes them. Core'scas::entriesreads them:read_unknown,require_known(an errorVersionHasUnknownsfor a non-empty list, and an error for an entry that can't be decoded, isn't a blob or has a field this build doesn't know),read_produced_by(Nonemeans unknown, not the reading core).doc::read_docs,compat::read_historyandcomparecallrequire_known, so no consumer takes such a version for complete.cargo xtask check-format-bump [--base <rev>](xtask/, alias in.cargo/config.toml, workflow.github/workflows/format-gate.yml, which passes the PR's base sha). Always: the current format's directory has its four files, and the seals hold. With--base, against the merge base:CURRENT_IR_FORMAT_VERSIONwent up: it must be by one;v<C>/hasstep.rsdefiningUNKNOWN,fn upandfn down;v<C>/SCHEMA.mdstarts# IR format <C>; format C-1 is sealed; every ledger row of C-1 names a fixture that exists undercore/tests/migrations/stores/format-<C-1>/. Format 3's rows have none yet, so the format 4 pull request can't pass untilcargo xtask ir-fixtures(step 1.5) exists and has run, which is the intent.SCHEMA.mdandtypes.rsorcodec.rschanged in the same change; moved golden hashes need a new row.The xtask's own tests build throw-away git repositories and show each failure.
Decisions to confirm
core/formats/(packagecomline-ir-formats), the format directory iscore/formats/src/v3/, the seal iscore/formats/MANIFEST. The design saysformats/v<C>/andformats/MANIFEST; this follows the WIP branch'score/formats/src/v3/. A bump's step lives inv<C>/step.rs.unknownandproduced_by(one blob holding both versions, not two entries); JSON pointers forpath;stepas"3-4". The design left the names open.Known<T>yet.read_produced_byreturnsOption(Noneis unknown).Known<T>belongs with the first format that uses it (format 4).unseen, compare modulo unknown) need the engine and a step that writes one; until then the strict reading is the only safe one.3.NN, in its order); the two new ones are a row each forunknownandproduced_by.fixtureis optional and absent on all of them for now.std). The "moved golden hashes need a ledger row" rule is enforced by the xtask on the PR diff, not by the test: the test only says they moved.# IR format <C>heading ofSCHEMA.mdinstead. Say if the guide check should come back some other way.live == frozenare drawn from the frozen types (Arbitrarybehind the crate'sarbitraryfeature, a dev-dependency of core only), not from the live ones: that catches a frozen copy that is wrong about a field, and the fixtures catch the live side. A value nested deeper than the CBOR reader's recursion limit is skipped.seal-formatandcheck-sealare two xtask commands beyond the one the issue names; the design asks for a seal that moves only by a command that names why.publish = falseas designed, andcomline-coredepends on it with a path.cargo publish -p comline-corefails until the crate is published or the dependency is made optional.