Repository navigation
The same evidence collected twice gets two different bundle ids #19
Description
Found by a CI flake on PR #15, then reproduced on demand. Two collections of byte-identical evidence, one second apart, produce different bundle_ids.
Measured
identical evidence (receipt excluded): True
receipt started_at / ended_at : ...T17:32:20Z ...T17:32:20Z | ...T17:32:21Z ...T17:32:21Z
bundle_id a : 7af00d4060057b3b
bundle_id b : 7b682d40722a9088
SAME IDENTITY : False
Four members differ, and the Vitals are not among them:
observations.json receipt.ended_at 17:32:34Z -> 17:32:35Z
snapshot.json observations_digest
manifest.json bundle_id
report.md the rendered collection time
The chain
GitHubAdapter.collect stamps the receipt from the wall clock — started at entry, ended_at from timeutil.now_utc() at the end. The receipt is embedded in observations.json, whose digest is in snapshot.json, and canonical.digest(receipt_dict) is source_receipts_digest in the identity preimage. So both routes into bundle_id carry the two timestamps.
The identity of a bundle therefore depends on when collection started and how long it took, not only on the evidence.
Why this matters more than a flake
The README's third promise is "Same evidence, same output. Evaluation is a pure function of the recorded observations and a declared policy version. Two implementations computing the same bundle get the same identity." Two implementations given the same evidence cannot get the same identity today unless their clocks agree to the second and they run for equally long.
0.1.6 already decided this question in the other direction for a neighbouring field: devostasis.receipt.v2 dropped request_count specifically so that enabling the conditional cache would not move a bundle id, and the counts went to post-identity run_meta. How long a run took and when it started are the same kind of fact — properties of the collection, not of the evidence — and they are still identity-bearing.
tests/test_rate_limits.py::test_a_cached_run_produces_the_same_bundle_identity states the intended contract in its own docstring ("The whole point: enabling the cache must not move a bundle id") and only passes because both of its collections usually land in the same second. It failed on Python 3.13 in run 34147115154 and passed on 3.12 and 3.14 in the same run.
That test is deliberately being left as it is. It is a correct assertion of the intended contract, and it fails when the contract is violated. Freezing its clock would turn an intermittent real violation into a green build, which is the anti-gaming failure this project exists to name: a retry that erases a failure. Expect it to fail occasionally until this is repaired.
Not new in 0.1.8
The receipt has carried both timestamps since the beginning; nothing in 0.1.8 introduced it. The CI determinism check does not catch it because it builds twice from examples/observations.json, where started_at and ended_at are fixed values in the file rather than readings of a clock.
What is being asked
The decision is which of two readings the artifact contract holds, and it belongs to the contract owner rather than to the implementation:
- Collection timing is provenance.
started_atandended_atjoinrequests,billed_requests,conditional_hitsandretriesin post-identityrun_meta, exactly asrequest_countdid in 0.1.6. The receipt keeps recording what was asked and what came back. Identity then depends on evidence alone, and the promise holds. - A collection is part of what a bundle is. Two runs over the same evidence are genuinely two different bundles, and the README sentence and the cache test are both wrong and should be corrected instead.
Reading 1 is the one 0.1.6 already chose for the neighbouring field, and it is the only one under which a second implementation can reproduce an identity.
Either answer is a change to an accepted contract, so it wants a research judgement:
- Reading 1 removes fields from the identity-bearing receipt, which moves
bundle_idfor every project once more. 0.1.8's notes call the delta contract the last planned identity move, so this is a fifth one and the owner has to want it. - It also has to say what happens to the 218 bundles already in the store: whether verification recomputes the receipt digest under the new rule (which would fail every stored bundle) or the rule is versioned so that a bundle verifies under the contract it was written with. The second is the only one compatible with "history is immutable and self-contained", and it is the harder half of the work.
Acceptance criteria
- Two collections of identical evidence, minutes apart, produce the same
bundle_id, the samesnapshot.jsonand the sameobservations.json. - Whatever the receipt keeps recording is still enough to distinguish "not requested", "requested but unknown" and "observed zero", which is what the receipt exists for.
- Every bundle already in the store still verifies.
- The determinism check in CI covers a live collection path and not only two builds from one observations file.
Activity
andy-zen-dev commentedon Sep 7, 2026
PV-REV-RECEIPT-TIMING-001 — REPAIR REQUIRED, but the broad rule “same evidence collected minutes apart must have the same bundle_id” is not accepted as written.
Accepted PV-BUNDLE-ID semantics make
observed_atintentionally identity-bearing, so two genuine observations at differentobserved_atvalues may and should be different bundles. That decision remains in force.The narrower defect is confirmed: current receipt
run_id,started_at, andended_atare collector-invocation mechanics layered on top ofobserved_at, yet they are hashed throughsource_receipts_digestand can also re-enter throughobservations_digest. That makes bundle identity depend on wall-clock/run mechanics that can vary across implementations, scheduling, retries, or machine speed even when the canonical observation moment and evidence are unchanged.Research therefore requires a named/versioned producer repair before implementation changes identity semantics. The repair must preserve
observed_atas identity-bearing; define an explicit receipt identity projection separating evidence facts from post-identity run mechanics; retainrun_id/start/end for audit provenance without hashing them into the new identity unless a semantic evidence role is separately proven; and remove any indirect leakage throughobservations.jsonor another identity-bearing member.Minimum regression boundary: same
observed_at+ same normalized evidence/semantic receipt facts but different run id/start/end => same new-version identity; differentobserved_at=> different identity; requested/unknown/PARTIAL/observed-zero distinctions stay intact; old bundles remain verifiable under their stored contract/preimage and are never rewritten.The follow-up research chain is
PV-RECEIPT-IDENTITY-001producer, then separatePV-REV-RECEIPT-IDENTITY-001judgement. Public determinism wording should be scoped to “same canonical evidence plus the same identity metadata, including observed_at,” not different observation times.andy-zen-dev commentedon Sep 7, 2026
PV-REV-RECEIPT-IDENTITY-001 — CHANGES_REQUIRED.
The core issue #19 boundary is sound:
observed_atremains identity-bearing;run_id/started_at/ended_atare invocation mechanics and must not enter the successor identity directly or throughobservations_digest/another identity-bearing path. The explicitreceipt_identityprojection, fail-closed status/coverage distinctions, and historical version dispatch are retained.Two adoption blockers remain in the producer candidate:
The candidate permits
manifest.run_metato differ between executions that compute the samePV-BUNDLE-ID-003, then asks persistence not to fail merely because that post-identity metadata differs. Butmanifest.jsonis a required canonical bundle member and the accepted immutable-store contract makes a duplicate id idempotent only when the required persisted canonical bytes are identical. Same id with invocation-dependent manifest bytes would make the canonical bundle first-writer-dependent. The repair must keep invocation-varying run metadata outside required canonical bundle bytes while still emitting the full current-run receipt on an explicitly non-canonical execution/result surface.On the current public repository surface a new manifest schema is mandatory, not optional.
devostasis.manifest.v1hard-codesbundle_identity_contract = PV-BUNDLE-ID-002and requires the current receipt/run_meta shape. A successor must therefore version the manifest contract/schema alongside receipt and observations, with historical verification dispatch.RECEIPT-ID-01..15remain useful candidate regressions but are not accepted while the parent contract is unaccepted.RECEIPT-ID-16is rejected in its current duplicate-persistence wording and must not be silently reassigned. The next producer isPV-RECEIPT-IDENTITY-002, followed by separatePV-REV-RECEIPT-IDENTITY-002; it should add new permanent cases for canonical-member byte invariance and manifest-version dispatch.No Vital rule, threshold/window, gauge, demand, ordering, aggregate, HistoryStore topology, or runtime-model change is authorized by this judgement.
andy-zen-dev commentedon Sep 7, 2026
PV-REV-RECEIPT-IDENTITY-002 — CHANGES_REQUIRED.
The repaired candidate closes both blockers from the previous review: invocation-varying run metadata is now outside required canonical bundle bytes, so one
PV-BUNDLE-ID-003implies byte-identical canonical members; and a fresh manifest schema/contract lineage is now mandatory alongside fresh receipt/observations lineages.One verification blocker remains.
source_receipts_digestis defined as the hash of canonicalreceipt_identity, but the candidate only requiresreceipt_identityinsideobservations.jsonwhen that optional member is enabled. It requires the successor manifest to bind the receipt schema lineage and store the identity preimage, but it does not require the exactreceipt_identitydigest preimage itself to be persisted unconditionally.Therefore, with canonical observations persistence disabled, a successor bundle could contain
source_receipts_digestwithout containing the receipt evidence needed to recompute that digest offline. That weakens the accepted self-contained artifact boundary: receipt digests are defined over receipts persisted with the bundle, and verification must not merely trust an already-stored digest/preimage field.Required next repair:
PV-RECEIPT-IDENTITY-003, then separatePV-REV-RECEIPT-IDENTITY-003. Preserve all of the -002 fixes, but persist exact canonicalreceipt_identityin every successor bundle regardless of the observations toggle (the successor manifest is the narrowest compatible location), recompute/comparesource_receipts_digestfrom it during verification, require the observations copy to match when present, and add a new permanent regression case for the observations-disabled path.RECEIPT-ID-16remains rejected/unreassigned;RECEIPT-ID-01..15and17..20remain candidate cases until the parent contract is accepted.No Vital rule, threshold/window, gauge, demand, ordering, aggregate, HistoryStore topology, or runtime-model change is authorized by this judgement.
andy-zen-dev commentedon Sep 8, 2026
PV-REV-RECEIPT-IDENTITY-003 — ACCEPTED.
Research accepts the repaired cumulative PV-BUNDLE-ID-003 successor receipt/bundle identity contract.
observed_atremains identity-bearing. Withobserved_at, canonical evidence and all other identity metadata fixed,run_id,started_at,ended_at, cache/retry/request accounting, scheduler timing and equivalent invocation mechanics are non-canonical and must not changebundle_idor any required canonical bundle bytes.Every successor bundle must persist the exact canonical
receipt_identityin its required new-version manifest even whenobservations.jsonis disabled. Verification recomputessource_receipts_digestfrom that object and fails closed on absence, malformed shape or mismatch. Ifobservations.jsonis enabled, itsreceipt_identitymust exactly match the authoritative manifest copy; disagreement fails closed and the verifier may not choose or merge authorities.Fresh public receipt, observations and manifest schema lineages are mandatory. Legacy bundles remain byte-for-byte immutable and verify under their stored historical rules; mixed legacy/successor lineage fails closed. Same successor
bundle_idimplies byte-identical required canonical members, so the existing exact-byte HistoryStore collision/idempotency rule remains unchanged and no first-writer exception is authorized.RECEIPT-ID-01..15 and RECEIPT-ID-17..21 are accepted with their existing meanings. RECEIPT-ID-16 remains permanently rejected/unassigned and must not be reused.
This does not accept the broad claim that equivalent provider bytes at different genuine observation times must share identity: a different
observed_atremains an intentional identity difference. Repository adoption still requires implementation, the accepted cases, fresh public schemas, docs/spec + PROVENANCE + changelog updates, historical-version verification, and a later post-adoption PV-SPEC-001 pass. No Vital, threshold/window, gauge, demand, ordering, aggregate, HistoryStore topology or runtime-model semantic changes are authorized by this judgement.abogun-product commentedon Oct 2, 2026
Reproduced during the complete 0.3.0 Evidence to Action validation on
2026-10-02 (Windows, Python 3.13.13), before any change to the core receipt
or bundle-identity implementation:
python -m pytest: 1 failed, 2166 passed in 32.09 s.Failure:
tests/test_rate_limits.py::test_a_cached_run_produces_the_same_bundle_identity.Fresh bundle id:
db18d2d6f63461a0c9f9eff6dc67adf077d30f833926caa9f33b32773364648bCached bundle id:
6784cd8eb47d98fa8ac91372c64f1c606c1ef308760db454fd483c387e93e63eAn isolated rerun passed (1 passed in 0.23 s), confirming that the first
failure must not be erased from the release validation record. The test
assertion and its clock remain unchanged. This is the known receipt identity
adoption debt here, not a new work-scope determinism failure, and is recorded
in this existing issue rather than creating a duplicate. The new companion
has a separate replay contract and proves identical complete inputs/clock
produce byte-identical bundles; it does not claim to implement
PV-BUNDLE-ID-003 for the existing core.
The 0.3.0 PR will link this observation and retain the accepted adoption
criteria above, including historical schema/version verification. No Vital
rule or threshold is changed to accommodate the test.
abogun-product commentedon Oct 6, 2026
Implemented the accepted cumulative receipt repair in PR #64, commit
4d96120.Successor bundle v3 / manifest v2 stores the complete canonical receipt identity. Invocation IDs, start/end times, counters, retries, cache diagnostics and tool build are external execution receipts. Identical canonical evidence including its genuine observation time and material acquisition metadata now reproduces the ID and every canonical member byte; duplicate HistoryStore puts retain the exact-byte collision check. Genuine
observed_atremains material, as required by the accepted contract; the rejected historical RECEIPT-ID-16 is not reassigned.Regression proof covers RECEIPT-ID-01..15 and 17..21, observations-disabled durability, material acquisition distinctions and historical/mixed lineage. Local full suite: 2,255 passed; CLI corpus: 148 passed. Implementation is prepared in the PR, not yet released.