Repository navigation
feat(ci): wire the dirty-tree offload into nself ci --where (P7-AUTO-13) - #556
Merged
Merged
Conversation
Nothing in cmd registered the run's OffloadHook slot, so offload.Take was reachable only through tests. cmd/v15_offload.go now registers run.SnapshotOffload from init(): on a dirty tree, a bare run or `ci runs rerun` whose --where target can reach a registered node takes a snapshot commit under refs/nself-ci/offload/<id>, the ssh driver bundles it, and the ref is released when the run ends. Untracked files left out are named on stderr, never their content; a tracked secret-shaped file refuses the run with E807. Targets that cannot reach a node keep HEAD unchanged: local, hosted, no --where with nothing registered, and pool/auto/all when only hosted providers are registered (a hosted runner fetches from the forge, where a snapshot never exists). Tests drive runV15 end to end against an in-process ssh node: the node stages the bundle and sees the edited and untracked files but not .env, .env.local or ignored files; the repository state and refs are unchanged afterwards; E807 exits 2 naming the path only. (P7-AUTO-13)
… (P7-AUTO-13) offload.Take built its temporary index from HEAD, so a file added with `git add` and not committed never reached the snapshot and the node tested a tree other than the user's. Paths the user's index tracks that HEAD lacks, and that exist in the working tree, are now added too. Git runs with GIT_OPTIONAL_LOCKS=0 (status no longer refreshes the user's index) and GIT_LITERAL_PATHSPECS=1 (a file name is never a pattern). A snapshot replaced the run's revision, so a dirty run, including an `auto` run that stayed local, recorded a commit that no ref keeps once the run ends. That breaks `ci runs reproduce` after gc and contradicts P7-CI-41 (a reproduction never uses uncommitted changes). The snapshot is now only what nodes check out: Session.Checkout feeds the lease and the remote evidence check, while the run records HEAD, dirty, and never posts a status of record. The offload unit tests isolate HOME, XDG_CONFIG_HOME and git's global and system config, so a host's global excludes no longer change them. (P7-AUTO-13)
…O-13) A hosted runner fetches the revision from the forge, where a snapshot commit never exists. When the dispatcher delivers a lease of a snapshot run to a hosted runner, the transport adapter now gives that lease the committed HEAD instead, the run expects evidence naming HEAD, and the run notes "dirty changes not included: hosted runner" once. Leases to enrolled nodes keep the snapshot; runs without one are unchanged. Tests: TestHostedLeaseNeverGetsSnapshot covers node and hosted delivery of one snapshot run and a clean run; TestCLIDirtyRunNodeWithHostedRegistered registers a node and a hosted runner and places on the node through the CLI. (P7-AUTO-13)
Contributor
Author
|
Folded into #543 (land/p7f-plugins) to land in one CI run. |
acamarata
added a commit
that referenced
this pull request
Oct 30, 2026
…#562 trust gate + lease evidence store)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on
land/p7f-plugins(#543). Do not merge before review. Related: #551 (sim legs, unchanged by this PR).What
Nothing under
free/ci/cmdregistered the run'sOffloadHook, sooffload.Takeran only from tests.cmd/v15_offload.goregistersrun.SnapshotOffloadfrominit().Call path:
runV15->runBare->parseBare(whereOptionssetswhereChosen) ->Coordinator.Run->c.offload->SnapshotOffload.Offload->buildOffload->offload.Take(temporary index, private ref) -> sshmakeBundle-> node checkout. The ref is released when the run ends.ci runs rerunuses the same slot, and its clean-tree rule means it never takes a snapshot.When it takes a snapshot:
--where <node>: always (on a dirty tree).autoorall: only when at least one node is registered.local,hosted, no--wherewith nothing registered, or only hosted providers registered: HEAD is used unchanged. A hosted runner fetches from the forge, where a snapshot never exists.Untracked files that are left out are named on stderr, never their content. A tracked file that looks like a secret refuses the run with E807 (exit 2).
Tests (cmd, through the real CLI)
TestCLIWhereNodeOffloadsDirtyTree:nself ci --where n1on a dirty repo, against an in-process ssh node. The node stages the bundle. It sees the editedf.txtand the newnew.txt, and it does not see.env,.env.localorignored.txt. The snapshot's parent is HEAD,.env.localis named and no secret content appears. Status, refs, index and files are identical before and after, and norefs/nself-ci/ref is left.TestCLIWhereNodeTrackedSecretRefused: a trackeddeploy.pemgives E807 and exit 2 naming the path. The content is not printed and the repo is unchanged.TestOffloadOnlyForTargetsThatReachANodeandTestOffloadCleanTreeKeepsHEAD.Run with
CGO_ENABLED=0 go test -mod=vendor -count=1:./cmd/(full package),./architecture/...,./internal/run/...,./test/sim/auto/...and-run TestCodeRegistry ./cmd/...all pass.go build ./...andgo vet ./cmd/are clean.parity-with-core.shwas not run, because this only fills a V15 hook slot.Follow-up fixes (commit 2)
offload.Takenow carries files that aregit add-ed but not committed. Test:TestSnapshotCarriesStagedNewFile, which fails on the old code. Git also runs withGIT_OPTIONAL_LOCKS=0andGIT_LITERAL_PATHSPECS=1.GIT_CONFIG_NOSYSTEMandGIT_CONFIG_GLOBAL, so they no longer depend on the host.Session.Checkoutfeeds the lease and the remote evidence check, and the run records HEAD, dirty, with no status of record. This follows P7-CI-41: a reproduction never uses uncommitted changes, and no ref outlives the run. Tests:TestCLIAutoDirtyRunStaysLocal(auto, node registered but offline, job runs here, recorded revision is HEAD),requireRecordedHEADin the node test, andTestOffloadCheckoutRevision. The CLI tests fail on the old code.The #551 sim legs pass on this head. The scope change is in a separate hq PR.
Hosted runners (commit 3)
A lease of a snapshot run that goes to a hosted runner now gets HEAD, never the snapshot. The run expects evidence naming HEAD and notes
dirty changes not included: hosted runner. Tests:TestHostedLeaseNeverGetsSnapshot(run package, at the delivery boundary: node lease gets the snapshot, hosted lease gets HEAD, a clean run is unchanged) andTestCLIDirtyRunNodeWithHostedRegistered(CLI, a node and a hosted runner both registered, placed on the node).A foreground run never places on a hosted runner today. Its world drops hosted runners unless
ci serveruns the pipeline, and privacy refuses hosted execution outside a public repository (D-1382). So hosted placement can't be driven end to end, and the hosted test works at the delivery boundary instead.TestRunsWaitfails on mm4 on both this head and the previous one (its 200 ms invoke-deadline subtest), so it isn't caused by this PR. #551's sim legs pass on this head when HOME is private. They have the same global-excludes problem fixed here for the unit tests.