Repository navigation
Add headless sandbox smoke coverage and verified teardown - #5251
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a headless sandbox smoke test that exercises fleet orchestration and one smoke-task worker through a canary edit, configured sandbox test gate, branch push, and a PR closed without merging. The harness stages the recipe, assigns an exact per-run branch before launch, and verifies the PR's complete diff, immutable head/ref agreement, and executable canary.
The workflow removes and verifies the exact model container before starting independent remote cleanup, including failures that push a branch without creating a PR. Recipe expected failures match exact structured envelope identities; cleanup errors remain fatal. The PR also adds catalog triggers, reproduction instructions, lifecycle/fault regressions, and production fleet ingredient-binding coverage.
base_branchfollows fleet's existing config authority using the sandbox project's configuration. This is a necessary adjustment to the plan: fleet strips caller overrides before required-ingredient materialization.Validation:
test_check: 16,284 passed, 207 skipped, 12 xfailed.pre-commit run --all-files: passed.expected_failurefor Fleet L3 recipe bootstrap fails with a retired kitchen generation #5250 and verified post-match cleanup. The assigned branch is absent, no open PR remains, and removal/cleanup exit codes are zero.A clean worker/canary/PR live run remains pending the retired-generation kitchen bootstrap defect in #5250. Its observed exact signature is linked in the catalog and must be removed after the defect is fixed. The separate failure-reporter defect found during validation is tracked in #5249.
Refs #5232.