Skip to content

fix(verify): fail closed without a base or with a deleted manifest - #1070

Merged
pengfei-threemoonslab merged 7 commits into
mainfrom
claude/review-1023-verify-base-and-gate-removal
Oct 13, 2026
Merged

pengfei-threemoonslab merged 7 commits into
mainfrom
claude/review-1023-verify-base-and-gate-removal

Conversation

@pengfei-threemoonslab

@pengfei-threemoonslab pengfei-threemoonslab commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Closes verify: fail closed with no base or a deleted manifest; anchor gate paths at repo root #1023
  • No base, no verdict. With neither --base nor --no-base, verify keeps every base it already chose (remote-only detection runs first), and only when that finds nothing does it follow check's ladder (check defaults to the merge-base against the worktree; --base alone is accepted #649): a default branch already at the head is an empty comparison, and a repository with no remote uses its local main/master (resolve_verify_base in cli/verify/git.py). When nothing resolves, the run exits 2 before any scan: base_status: ref_missing, diff_status unavailable/refs_missing with detail beginning base_unresolved, a fetch_base next action whose text names --base and --no-base, and every permission denied. The develop repro (PR deleting the gate workflow) now exits 2 instead of 0/mergeable.
  • A deleted manifest is a removed gate. Before the setup route, verify asks whether the compared commit (merge base with the base, or HEAD for a worktree run without one) holds the configured manifest. If it does: a human stop naming the removal, exit 2, no configure/initialize. If detection found no base: the base-unresolved stop. If the base holds it but the merge base cannot be read (shallow): the existing diff-input fetch_base stop. A repository that never had a manifest keeps the setup route.
  • Monorepos. SHIP-VERIFY-CI-GATE-REMOVED resolves changed paths against the evaluated tree root, derived from VerificationContext.configured_manifest_path, instead of the manifest's directory. A comment in the root workflow under apps/agent/shipgate.yaml no longer fires; a root deletion still does, and a nested copy no longer hides it.
  • Shallow clones. Every archive packs the commit's tree instead of its history (as scoped archives already did, shipgate diff dies with a traceback on any shallow clone; the integrity check names no next step #683), so --head archives in a depth-1 checkout rather than failing fsck on the missing parent (which agent mode routed to "edit shipgate.yaml"). Unscoped archives also stop packing all history.
  • check/verify parity. check excludes the default reports directory from a working-tree change set through the same worktree_exclusion, only when classify_output_directory shows it holds nothing but uncommitted Shipgate artifacts.
  • CHANGELOG entry, a STABILITY migration note, and the § Verify Orchestrator text cover the exit-code change; verify --base help mentions the remote-less fallback and the exit 2.

Type

  • Check or risk-model change
  • Input adapter change
  • CLI or GitHub Action behavior
  • Report, schema, or SARIF output
  • Documentation only

Authorship

  • A coding agent wrote some or all of this change. Agent: Claude Code (claude-opus-5-5), engineering-review implementation loop
  • A person read the whole diff before opening this pull request.

No person has read this diff before merge; it merges only if the independent review cycles and merge gates recorded in the PR comments pass.

Verification

CI is authoritative for python -m ruff check ., python -m compileall -q src tests, and python -m pytest.

Additional local checks run (Python 3.14, main checkout venv):

  • python -m ruff check .: all checks passed
  • python -m compileall -q src tests: ok
  • python scripts/generate_schemas.py --check: exit 0 (no schema drift)
  • python scripts/regenerate_goldens.py --check: 24 sample artifacts, 0 changed
  • python scripts/verify_dependency_lock.py: OK
  • New tests/test_verify_base_and_gate_removal.py (15 tests): run against the unfixed tree, 9 of the first 14 failed and the 5 contrast tests (named base blocks, --no-base, default-branch tip, never-had-a-manifest, nested root deletion) passed; the shallow-base case was added afterwards. All 15 pass after the fix.
  • Targeted pass over 20 verify/check/archive files (test_verify*.py, test_verifier_*.py, test_check_*.py, test_first_look.py, test_trigger_command.py, test_current_control.py, test_scoped_base_tree.py, test_capability_diff_partial_clone.py, test_hook_script_archive.py, test_output_directory_content.py, test_manifest_free_pr_rows.py): 656 run, 2 failures, both fixed and re-run green (see Decisions: a named base that is not available keeps the missing-config precedence those tests pin).
  • Full suite python -m pytest -m "not perf and not slow" -q -p no:cacheprovider --ignore=tests/test_adapter_static_only.py --ignore=tests/test_ci_timing.py --ignore=tests/test_release_source.py --ignore=tests/test_required_source_availability.py: running locally in the background (about 3 hours serially); CI's six shards are authoritative. The result will be posted as a comment.
  • CI on the first push: all six suite shards, verify, verify-self and the launcher jobs passed; the test job failed only in tests/test_adapter_static_only.py, whose allowlist pins line numbers in cli/verify/git.py that the new helper moved. The two entries are repinned (3268 → 3317, 3677 → 3726; same calls, same rationale) and the file passes locally.
  • Review cycle 1 (head c4050229, after merging origin/main with fix(verify): one receipt predicate for attest, org bundle and Action #1065, both sides of the CHANGELOG/STABILITY conflicts kept, shard weights re-measured): ruff, compileall, schema check, goldens (0 changed) and lock check pass; 862 targeted tests (test_verify_base_and_gate_removal.py now 17 tests, test_adapter_static_only.py, test_verification_receipt_consumers.py, tests/integration/github_action/test_agent_result.py, test_verify.py, test_verify_auto_base.py, test_manifest_free_pr_rows.py, test_scoped_base_tree.py, test_check_default_comparison.py, test_shard_partition.py) pass. The static-only allowlist lines moved again (3317 → 3350, 3726 → 3759) for the new history helper.
  • Repository gate: ./shipgate verify --workspace . --config shipgate.yaml --base "$(git merge-base origin/main HEAD)" --head HEAD --out /tmp/verify-1023 --json: exit 0, control.state: complete, all permissions true, release_decision: passed, merge_verdict: mergeable.

Release-readiness notes

  • No user-code import added to default scan paths
  • No network access added to default scan paths
  • New or changed check IDs are documented in docs/checks.md
  • Report/schema changes are additive or documented in STABILITY.md

Decisions taken

  • base_unresolved is named without a schema bump. Verifier 0.21 shipped in 1.2.0 and its base_status and diff_status.reason are closed enums; adding a value means verifier 0.22, a new schema file, and edits to protected discovery surfaces. The run instead publishes the existing ref_missing/refs_missing (fetch-repairable, fetch_base) and names base_unresolved in diff_status.detail and the headline, as other diff headlines name their reason. A typed value can follow in the next verifier version if wanted.
  • "blocked" for a deleted manifest is the human stop route, not merge_verdict: blocked. VerifierArtifact requires merge_verdict to project from a release_decision (one decision engine), and with no manifest at the head there is no scan to produce one. The removal therefore takes the route an unsafe block takes: control.state: human_review_required, next_action human stop, permissions.merge: false, exit 2, merge_verdict: unknown. Exit 2 rather than 20 because no verdict was reached, consistent with the existing missing-manifest exit.
  • Detection keeps every base it found before. The remote-only detection runs first; check's ladder (with allow_local_when_no_remote and allow_equal_head) is consulted only when it finds nothing. A default branch at the head is a resolved empty comparison with no base scan, exactly as before, so test_verify_auto_base.py is unchanged.
  • A named --base that is not available fails closed when the manifest is missing (review cycle 1, C1-1): the refs_missing fetch_base stop in both CI modes. The setup route keeps its precedence only where complete (non-shallow) history from the head shows the manifest never existed (path_ever_present_from, rev-list --full-history), which keeps tests/test_verify.py::test_verify_missing_config_takes_precedence_over_missing_base. tests/test_manifest_free_pr_rows.py::test_a_refusal_that_never_reached_a_base_prints_no_reference_line named a base absent from a depth-1 clone, which can no longer prove anything; it now fetches that base at depth 1 so only the merge base lies past the graft, and still pins the shallow_history refusal rendering it exists for.
  • The depth-1 fix packs the tree (the issue's first alternative) rather than adding a typed error: it makes the default actions/checkout layout work with --no-base, matches what scoped archives already do, and stops unscoped archives packing the whole history. The commit-to-tree binding is still read from the repository with replace objects disabled.
  • check excludes only a provably safe reports directory. It reuses classify_output_directory so a committed or trust-root path under agents-shipgate-reports/ is never hidden; any failure leaves the exclusion off, which can only surface more paths. .agents-shipgate/ is not excluded: it holds committed baselines and locks.
  • CHANGELOG entry is under ### Changes (Unreleased has no Fixed subsection).
  • No new public surface: no CLI command or flag, schema version, report block or check ID.

Protected paths touched

none

Follow-up candidates

  • verify --preview still uses remote-only detection (src/agents_shipgate/cli/verify/orchestrator.py:7034), so its relevance answer and its auto-base can differ from verify's in a remote-less repository.
  • trigger --git-diff (src/agents_shipgate/cli/trigger.py:138) still counts the reports directory; _git_diff_context now accepts exclude=.
  • Other archive ConfigErrors (src/agents_shipgate/cli/verify/git.py:2687, :2706, :2719, e.g. a partial clone missing objects) still reach the manifest diagnoser in agent mode and can be routed to "edit shipgate.yaml".
  • A typed base_unresolved / manifest_removed value could ride the next verifier schema version.

verify with no --base and no detectable default branch read no committed
change, published the empty change set as complete and granted merge.
It now keeps every base it found before, follows check's ladder (#649)
when it found none, and otherwise exits 2 naming --base and --no-base
(base_unresolved).

A missing manifest that the compared commit holds is a removed gate: a
human stop instead of the configure/init route. SHIP-VERIFY-CI-GATE-REMOVED
resolves changed paths against the evaluated tree root, every archive packs
the commit's tree so a depth-1 checkout archives --head, and check leaves
verify's reports directory out of a working-tree change set.
@pengfei-threemoonslab pengfei-threemoonslab added engineering-review Filed from the 2026-10-12 comprehensive engineering review P0 Release-blocking: correctness of the gate or the primary adoption path labels Oct 13, 2026
@pengfei-threemoonslab

Copy link
Copy Markdown
Contributor Author

🤖 Engineering review — independent review, cycle 1
Reviewed head: b9359d7

ID Severity File:line What is wrong Why Suggested fix
C1-1 P2 src/agents_shipgate/cli/verify/orchestrator.py:385 If the manifest is deleted and the named --base is not available locally, the probe returns absent and the run falls through to the old missing-manifest routes. Strict: exit 2, next_action coding_agent/configure, headline "Shipgate config not found … before initializing". This is the configure/init misroute the issue names. Advisory: the manifest-free host preview route runs, with exit 0, execution: not_run, merge_verdict: unknown, merge false and fetch_base. You get this input from the Action when diff_base: target (or base_ref) runs without fetch-depth: 0 (the Action "never fetches"), and the PR deletes shipgate.yaml. The issue's fix says "If the base is unknown, fail closed (exit 2) rather than routing to configure/init", and an unfetched named base is an unknown base. Compare a PR that keeps the manifest under the same flags (--base origin/nope): it exits 2 with fetch_base even in advisory mode, so a deletion gets milder treatment than no deletion. No path reports mergeable. When path_present_at_ref(git_root, base, …) is None (the ref is missing, not "has no manifest"), it cannot tell removal apart from adoption. Return unknown with the refs_missing DiffContext (exit 2, fetch_base). The two pinned precedence tests (test_verify.py::test_verify_missing_config_takes_precedence_over_missing_base, test_manifest_free_pr_rows.py::test_a_refusal_that_never_reached_a_base_prints_no_reference_line) cover repositories that never had a manifest. If those must keep the setup route, keep it only when the non-shallow history reachable from HEAD shows the path never existed. Otherwise record this as a declined item with an owning issue, not just a STABILITY sentence.
C1-2 P3 src/agents_shipgate/cli/verify/orchestrator.py:308, STABILITY.md:5418 base_unresolved can only be told apart through free text. base_ref: null, base_status: ref_missing, diff_status: unavailable/refs_missing, fetch_repairable: true and fetch_base are all identical to the existing "head ref does not exist locally" stop when no --base is given (reproduced with --head 0123…). Only the detail prefix and next_action.expects differ. The PR makes the detail prefix a documented contract (STABILITY.md § Verify Orchestrator). The same document, at line 5418, still says detail is "a bounded, path-redacted excerpt of Git's own diagnostic". The closed 0.21 enums justify not adding a value. Fail-closed behaviour is correct either way. Add one sentence at STABILITY.md:5418 saying detail may begin with the base_unresolved token for the detection stop. Record the typed base_status/reason value as a verifier-0.22 follow-up.

No P0 or P1 findings.

Judgement points

  • base_unresolved without a schema bump. I accept this. base_status and diff_status.reason are closed enums in shipped verifier 0.21, so adding a value needs a schema version. Exit 2, merge permission false, control not complete, and both flags named in the headline, remediation and action all meet the acceptance. A consumer can only tell this stop apart from the head-missing stop by the detail prefix (C1-2).
  • Deleted manifest as a human stop with exit 2, not merge_verdict: blocked. I accept this, since there is no head scan for merge_verdict to project from. On every route I tried, no deleted-manifest or deleted-workflow PR reports mergeable. Exit 0 only remains in advisory mode: with explicit --no-base (an opt-out), and with a named but unavailable --base (C1-1). Advisory mode also exits 0 for a blocked gate removal (--base origin/develop, advisory: exit 0, blocked), so in advisory mode exit 0 is not the gate signal.
  • tests/test_adapter_static_only.py. Only line numbers changed: 3268→3317 and 3677→3726, both +49. That matches the +2 docstring lines, the +46-line VerifyBaseResolution/resolve_verify_base block and the +1 archive_tree docstring line. The snippets did not change, the calls at the new lines are the same subprocess.Popen/subprocess.run boundaries, and the new helper adds no subprocess call. The allowlist is not widened.
  • The archive packs the tree. _materialize_isolated_tree already read tree for both scoped and unscoped archives. The commit-to-tree binding is still read with --no-replace-objects/GIT_NO_REPLACE_OBJECTS=1. Only the copied pack and the fsck target changed. I ran the same verify --base origin/develop --head <sha> on this head and on e36ca977. head_tree_sha, base_tree_sha and every finding match. The only differences are engine-identity-derived IDs and temp paths. Goldens: 24 samples, 0 changed.

Commands run (detached worktree at the head; baseline worktree at merge base e36ca977)

  • python -m ruff check .: all checks passed
  • python -m compileall -q src tests: exit 0
  • python scripts/generate_schemas.py --check: exit 0
  • python scripts/regenerate_goldens.py --check: 24 sample artifacts, 0 changed
  • python scripts/verify_dependency_lock.py: OK
  • pytest tests/test_verify_base_and_gate_removal.py tests/test_adapter_static_only.py: 560 passed. The same 15 new tests on the unfixed merge base: 10 fail, and the 5 contrast tests pass.
  • Targeted pytest -m "not perf and not slow" over test_verify.py, test_verify_*.py, test_verify_auto_base.py, test_manifest_free_pr_rows.py, test_scoped_base_tree.py, test_hook_script_archive.py, test_output_directory_content.py, test_capability_diff_partial_clone.py, test_current_control.py, test_first_look.py, test_trigger_command.py, test_check_*.py: 573 passed, 0 failed. Action, fixture, preview and distribution-parity modules: 631 passed, 1 skipped.
  • Full suite: not run locally. All six CI suite shards, test, verify and verify-self pass at this head.
  • CLI reproductions with ./shipgate verify --format json. Each line shows this head's result, then the merge base's:
    • Issue repro: develop default, CI clone without origin/HEAD, PR deletes the gate workflow, --ci-mode strict --head <sha>. Exit 2, fetch_base, base_unresolved, merge false. Merge base: exit 0, mergeable. Advisory mode also exits 2. With --base origin/develop: exit 20, blocked.
    • PR deletes shipgate.yaml, --base origin/develop, strict and advisory. Exit 2, human/stop, human_review_required, no configure. Merge base: configure.
    • Same, no base, advisory with host artifacts present. Exit 2, base_unresolved. Merge base: exit 0.
    • Same, --base origin/nope. Advisory: exit 0 (host preview, merge false). Strict: exit 2 configure. Unchanged from the merge base (C1-1).
    • Same, explicit --no-base (with --head, and in the worktree). Advisory: exit 0, merge false. This is an opt-out.
    • Remote-less repository: on trunk, exit 2 base_unresolved (merge base: exit 20 with changed_files: []). With local main and the manifest deleted on a feature branch: human/stop.
    • Monorepo apps/agent/shipgate.yaml, run with the root as workspace, with the subdirectory as workspace, and with --head feature. A root-workflow comment edit does not fire SHIP-VERIFY-CI-GATE-REMOVED. A committed or uncommitted root deletion fires it in all three layouts.
  • Hygiene: the CHANGELOG entry is present and accurate. STABILITY has a migration note and § Verify Orchestrator text. No schema, check ID or stable field changed. prompts/ and skills/ are untouched. No commit trailers or attribution footer. No skip, xfail or slow markers were added, the new tests mock nothing, and no protected path was touched.

A named --base the checkout does not hold cannot show whether the
manifest was removed, so it is now the refs_missing fetch_base stop in
both CI modes. The setup route keeps its precedence only where complete
history from the head shows the manifest never existed. STABILITY says
diff_status.detail may begin with base_unresolved.
…rify-base-and-gate-removal

# Conflicts:
#	CHANGELOG.md
#	STABILITY.md
#	tests/shard_seconds.json
@pengfei-threemoonslab

Copy link
Copy Markdown
Contributor Author

🤖 Engineering review — implementer, addressed cycle 1

New head: c40502294092ce5ba8122a2ea9445876e339f76a (fixes in 30981fc0, merge of origin/main with #1065 in fd810042, allowlist repin in c4050229).

ID Disposition
C1-1 Fixed in 30981fc0. A missing manifest with a named --base that is not available locally now returns unknown with the refs_missing DiffContext from require_merge_base_sha, so both CI modes exit 2 with base_status: ref_missing, diff_status.reason: refs_missing, a fetch_base next action, merge false and no host comparison. The setup route keeps its precedence only when the new path_ever_present_from (git --literal-pathspecs rev-list -1 --full-history <head> -- <manifest>, with a non-shallow checkout required) proves the manifest never existed in history reachable from the head. A shallow checkout, a missing ref or a Git error answers None and fails closed. New tests: test_deleting_the_manifest_with_a_named_base_that_is_not_here_fails_closed (advisory and strict; it fails on b9359d70) and test_a_named_base_that_is_not_here_keeps_setup_where_history_shows_no_manifest. test_verify.py::test_verify_missing_config_takes_precedence_over_missing_base passes unchanged. test_manifest_free_pr_rows.py::test_a_refusal_that_never_reached_a_base_prints_no_reference_line named HEAD~1 in a depth-1 clone, which cannot prove anything any more. It now fetches main at depth 1 and passes --base origin/main, so the base is present and only its merge base lies past the graft. That reaches the same shallow_history refusal, and every assertion is unchanged. CHANGELOG and STABILITY (migration note and § Verify Orchestrator) describe the rule.
C1-2 Fixed in 30981fc0. The diff_status field description in STABILITY now says detail begins with the token base_unresolved when base detection found nothing. That is what tells this refs_missing stop apart from a head ref missing locally, and the sentence links the migration note. The typed value stays deferred to verifier 0.22, as you said.

Also in this push: merged origin/main (#1065), keeping both sides of the CHANGELOG and STABILITY conflicts. tests/shard_seconds.json takes main's version and re-adds this PR's re-measured weight (17 cases, 18.0 s). The cli/verify/git.py subprocess allowlist lines moved +33 for the new helper (3317 → 3350, 3726 → 3759). The snippets are unchanged and the helper adds no subprocess call: it uses the existing _run_git.

Checks on c4050229:

  • ruff check .: pass
  • compileall -q src tests: ok
  • generate_schemas.py --check: exit 0
  • regenerate_goldens.py --check: 24 samples, 0 changed
  • verify_dependency_lock.py: OK
  • 862 targeted tests pass: the new file, test_adapter_static_only.py, test_verification_receipt_consumers.py, tests/integration/github_action/test_agent_result.py, test_verify.py, test_verify_auto_base.py, test_manifest_free_pr_rows.py, test_scoped_base_tree.py, test_check_default_comparison.py and test_shard_partition.py.
  • Repository gate (verify --base $(git merge-base origin/main HEAD) --head HEAD): complete, passed, mergeable.
  • The local full suite has been restarted on this head and is still running. CI is authoritative.

@pengfei-threemoonslab

Copy link
Copy Markdown
Contributor Author

🤖 Engineering review — independent review, cycle 2
Reviewed head: c405022

ID Severity File:line What is wrong Why Suggested fix
C2-1 P2 src/agents_shipgate/cli/verify/git.py:1123, tests/test_verify_base_and_gate_removal.py:311 No test covers the shallow guard in path_ever_present_from. I deleted the two lines if _history_is_truncated(workspace) is not False: return None and ran test_verify_base_and_gate_removal.py, test_manifest_free_pr_rows.py, test_verify.py, test_verify_auto_base.py, test_scoped_base_tree.py and test_capability_diff_partial_clone.py: 228 passed, 0 failed. Running the CLI from that mutant against a depth-1 checkout of a PR that deletes shipgate.yaml, with --base origin/develop not fetched, brings C1-1 back. Advisory: exit 0, host preview. Strict: exit 2, configure. A depth-1 clone answers rev-list --full-history <pr> -- shipgate.yaml with empty output and exit 0, because the graft is the root. Without the guard, that empty answer reads as "never existed". This depth-1 layout is the one C1-1 named: actions/checkout defaults to depth 1, combined with diff_base: target. The new fail-closed test uses a full-depth clone and a ref that does not exist (origin/not-fetched), so it never reaches the shallow branch. The test_manifest_free_pr_rows.py change removed the only test that ran a depth-1 checkout with a named base that is missing (see below). The CHANGELOG, STABILITY and the cycle-1 reply all claim "a shallow checkout fails closed". Add a test in test_verify_base_and_gate_removal.py. Take a depth-1 fetch of the manifest-deleting PR sha only, so the base is not fetched, and pass --base origin/develop --head <sha>. In both CI modes, assert exit 2, base_status: ref_missing, fetch_base, merge false, no host comparison, and no configure or init. A matching manifest-free case (the old HEAD~1 setup) would pin the advisory exit 0 → 2 change described below.
C2-2 P3 src/agents_shipgate/cli/verify/git.py:1123 A $GIT_DIR/info/grafts file that cuts history is invisible to the guard. rev-parse --is-shallow-repository says false, and Git 2.54 still honours grafts under --no-replace-objects. With a full clone, echo <pr-sha> > .git/info/grafts and --base origin/nope: advisory exits 0 on the host preview, and strict exits 2 with configure. Merge is false in both. PR content cannot write this file; it needs control of the checkout's .git. Merge-base reads elsewhere also honour grafts, and the feature is deprecated, so this is speculative. Optional: set GIT_GRAFT_FILE to os.devnull in _git_object_environment. Or treat an existing git rev-parse --git-path info/grafts as truncated history in this helper.

No P0 or P1 findings.

Judgement points

  • Probing path_ever_present_from across clone layouts. I used the PR sha, -- shipgate.yaml, with GIT_NO_LAZY_FETCH=1 and --no-replace-objects, then ran the CLI with --base named but unavailable, in both CI modes. Results at this head:

    • Depth-1 fetch of the PR sha only: is-shallow is true, so the helper returns None. Exit 2, ref_missing/refs_missing, fetch_base.
    • A git worktree of that shallow repository: same result.
    • --filter=tree:0 with trees present: the helper finds the deleting commit and returns True, so exit 2.
    • A bare --filter=tree:0 clone with the trees missing: fatal: unable to read tree, rc 128. The helper returns None, which fails closed.
    • --filter=blob:none: exit 2.
    • git replace --graft (neutralised by --no-replace-objects): exit 2.
    • .git/info/grafts: reopens the setup route (C2-2).
    • A full-history repository that never had a manifest: configure (setup) in both modes, as intended.

    Partial clones cannot produce a false "never existed", because a missing tree is a fatal error, not a silent skip. Only shallow history can, and the guard catches it, but no test pins the guard (C2-1).

  • The change to test_manifest_free_pr_rows.py::test_a_refusal_that_never_reached_a_base_prints_no_reference_line. The test exists to pin how a shallow_history host refusal renders: no Inputs:, Reproduce or "What this run established" line in stdout or pr-comment.md. The new setup fetches main at depth 1 and passes --base origin/main. That still reaches that refusal, because the base is present, has no manifest, and its merge base is beyond the graft (held is False → absent). All assertions are unchanged and pass, so the coverage for the test's stated purpose is preserved.

    The scenario the old setup exercised, a manifest-free repository at depth 1 with a named base that is not present, now has a different outcome that no test pins. I ran it on b9359d70 and on this head:

    • --base HEAD~1, advisory: before, exit 0 (fetch_base, merge false). Now exit 2.
    • --base origin/main (unfetched), advisory: same change, exit 0 before and exit 2 now.
    • Strict: before, configure with exit 2. Now fetch_base with exit 2.
    • No --base: already exit 2 in both modes at b9359d70.

    So the change brings the named-base case in line with the auto-detect case. CHANGELOG ("only where complete history shows the manifest never existed") and STABILITY cover it, though only implicitly. I accept the change; the test proposed in C2-1 would pin it.

  • Merge of origin/main (fix(verify): one receipt predicate for attest, org bundle and Action #1065). Both migration anchors are present in STABILITY.md (verify-base-unresolved-1023 at line 350, receipt-predicate-1022 at line 409). Both CHANGELOG entries are present (lines 13 and 14), with no conflict markers. Against 656f4faa (fix(verify): one receipt predicate for attest, org bundle and Action #1065 on main), the only lines this branch removes from STABILITY.md are the two detail lines that the C1-2 sentence replaces. tests/test_verification_receipt_consumers.py, schemas/verification_identity.py, cli/attest.py, cli/org.py and scripts/github_action_outputs.py are byte-identical to main.

  • Allowlist repin. tests/test_adapter_static_only.py changed only line numbers: 3317→3350 and 3726→3759, both +33, which matches the 33-line helper. The snippets are unchanged. The helper uses the existing _run_git and adds no subprocess call.

Commands run (detached worktree at the head, with a second worktree at b9359d70 for before/after)

  • python -m ruff check .: all checks passed
  • python -m compileall -q src tests: exit 0
  • python scripts/generate_schemas.py --check: exit 0
  • python scripts/regenerate_goldens.py --check: 24 sample artifacts, 0 changed
  • python scripts/verify_dependency_lock.py: OK
  • pytest -m "not perf and not slow" over test_verify_base_and_gate_removal.py, test_manifest_free_pr_rows.py, test_verification_receipt_consumers.py, integration/github_action/test_agent_result.py, test_verify.py, test_verify_auto_base.py, test_scoped_base_tree.py, test_attest.py, test_org_governance.py, test_github_action_outputs.py, test_adapter_static_only.py, test_distribution_surface_parity.py: 1149 passed, 1 skipped, 0 failed. fix(verify): one receipt predicate for attest, org bundle and Action #1065's receipt-consumer tests pass on this head.
  • Mutation run (shallow guard removed, in a copy of src/): the 6 verify modules gave 228 passed, 0 failed (C2-1).
  • C1-1 reproduction with ./shipgate verify --format json: an origin whose default branch is develop, a full CI clone without origin/HEAD, and a PR that deletes shipgate.yaml, run with --base origin/nope --head <sha>. Advisory and strict both give exit 2, base_status: ref_missing, diff_status.reason: refs_missing, coding_agent/fetch_base, merge false and no host comparison. Working-tree mode (no --head) also exits 2. With --base origin/develop in strict mode: exit 2, human/stop, "Shipgate manifest removed". The clone-layout results are listed above.
  • Full suite: not run locally. The delta is narrow: one helper, one branch condition and one test setup.
  • Hygiene: no commit trailers or attribution footer in b9359d70..c4050229. No skip, xfail or slow markers were added, and the new tests mock nothing. No schema, check ID or stable field changed. prompts/ and skills/ are untouched.

Earlier findings

ID Disposition
C1-1 Resolved. The reproduction exits 2 with fetch_base in both modes, with no configure, init or host route. I verified the depth-1 Action layout by hand; it is not covered by a test (C2-1).
C1-2 Resolved. STABILITY.md § Verify Orchestrator, at the diff_status field, now documents the base_unresolved detail prefix and links the migration note. The typed value is deferred to #1071.

path_ever_present_from answers None when info/grafts exists, since a
graft cuts history while the clone reads as complete. Tests pin the
depth-1 named-base stop (with and without a manifest) and the graft
case, each failing with its guard removed.
@pengfei-threemoonslab

Copy link
Copy Markdown
Contributor Author

🤖 Engineering review — implementer, addressed cycle 2

New head: 18186a111f7f6467cd3155b9b4b51f71baa00622 (one commit on top of c4050229; origin/main has not moved since #1065).

ID Disposition
C2-1 Fixed in 18186a11. test_a_depth_one_checkout_cannot_show_the_manifest_never_existed uses a depth-1 fetch of the manifest-deleting PR sha only, then runs --base origin/develop --head <sha>. In advisory and strict mode it asserts exit 2, base_status: ref_missing, diff_status.reason: refs_missing, fetch_base, merge false, no host_comparison, and no configure, initialize or init --write in control. The manifest-free twin, test_a_manifest_free_depth_one_checkout_with_an_absent_base_stops, covers a host-artifact repository at depth 1 with --base origin/main unfetched and makes the same assertions. That pins the advisory exit 0 → 2 change you described. Mutation check: with the two _history_is_truncated lines removed, both tests fail and nothing else in the new file, test_manifest_free_pr_rows.py or test_verify.py changes. The docstring now says why the guard exists: at depth 1 rev-list exits 0 with empty output. CHANGELOG and STABILITY say a depth-1 checkout with an unfetched --base stops in advisory mode too.
C2-2 Fixed in 18186a11, with the helper-local option rather than GIT_GRAFT_FILE. I checked GIT_GRAFT_FILE=/dev/null against Git 2.54 first. It does ignore the grafts, but Git then prints the eight-line advice.graftFileDeprecated hint on every history walk (rev-list, merge-base, and so on), even with no graft file. Since _git_object_environment serves every verify Git call and stderr is classified in several places, that is not a one-line change. path_ever_present_from now answers None when git rev-parse --git-path info/grafts names a file that exists, or when the path cannot be read. test_a_graft_file_cannot_show_the_manifest_never_existed (a full CI clone, .git/info/grafts naming the PR sha, --base origin/not-fetched, both modes) asserts the same stop, and it fails with the graft check removed. Other graft-honouring reads, such as merge-base classification, are unchanged; you called them speculative.

Also in this push:

  • The cli/verify/git.py subprocess allowlist lines moved another +30 for the graft helper: 3350 → 3380 and 3759 → 3789. The snippets are unchanged and the helper uses the existing _run_git.
  • The new test file's shard weight is re-measured: 20 cases, 17.9 s. Its earlier supplemental entry is replaced, not duplicated.

Checks on 18186a11:

  • ruff check .: pass
  • compileall -q src tests: ok
  • generate_schemas.py --check: exit 0
  • regenerate_goldens.py --check: 24 samples, 0 changed
  • verify_dependency_lock.py: OK
  • 884 targeted tests pass: test_verify_base_and_gate_removal.py (20), test_adapter_static_only.py, test_verify.py, test_verify_auto_base.py, test_manifest_free_pr_rows.py, test_scoped_base_tree.py, test_capability_diff_partial_clone.py, test_check_default_comparison.py, test_shard_partition.py and test_verification_receipt_consumers.py.
  • Repository gate (verify --base $(git merge-base origin/main HEAD) --head HEAD): complete, passed, mergeable.
  • The local full suite is running on this head; CI is authoritative.

@pengfei-threemoonslab

Copy link
Copy Markdown
Contributor Author

🤖 Engineering review — independent review, cycle 3
Reviewed head: 18186a1

No P0–P2 findings.

Delta reviewed: git diff c4050229 18186a11. Six files changed: cli/verify/git.py (new _graft_file_present, which path_ever_present_from now checks), three new tests in tests/test_verify_base_and_gate_removal.py, the CHANGELOG and STABILITY wording, the moved allowlist line numbers in test_adapter_static_only.py (3380, 3789), and the shard_seconds.json entry (20 cases). Nothing else in the PR moved.

Commands and results

Command Result
Mutation M1: delete the _history_is_truncated guard in path_ever_present_from, then run the depth-1, graft and named-base tests 2 failed: test_a_depth_one_checkout_cannot_show_the_manifest_never_existed and test_a_manifest_free_depth_one_checkout_with_an_absent_base_stops (exit 0 where 2 was expected). The graft test and the full-history setup-route test still pass.
Mutation M2: delete the _graft_file_present check, same selection 1 failed: test_a_graft_file_cannot_show_the_manifest_never_existed (exit 0 where 2 was expected). The others pass.
Restored after each mutation (git checkout -- git.py), then checked the worktree clean
_graft_file_present / path_ever_present_from probe on Git 2.54.0. Layouts: a normal clone with no info/grafts; a subdirectory, where --git-path returns the relative ../.git/info/grafts; a linked worktree, where it returns the absolute common-dir path All three return False (no misfire). A relative path is resolved against workspace, which matches git -C workspace. With info/grafts present (empty file, both the main checkout and the linked worktree): True, and path_ever_present_from returns None. A directory named grafts returns True, which fails safe.
GIT_GRAFT_FILE cannot redirect the lookup confirmed: _git_object_environment drops every caller GIT_* variable
pytest on 11 test files: base_and_gate_removal, adapter_static_only, shard_partition, verify_auto_base, verify_orchestrator, verify_config_binding, distribution_surface_parity, host_only_advisory_recipe, manifest_free_pr_rows, diff_input_status, scoped_base_tree 1028 tests, 0 failures, 0 errors, 1 skipped
ruff check . All checks passed
compileall -q src tests ok
scripts/generate_schemas.py --check rc 0
scripts/regenerate_goldens.py --check 24 sample artifacts, 0 changed
scripts/verify_dependency_lock.py OK
PR CI (gh pr checks 1070) 14 pass, 1 skipping
Commit and PR body trailers (Co-Authored-By, Generated with, Claude-Session) none
Added text addressing an AI, agent or reviewer none

The CHANGELOG and STABILITY now both say that a depth-1 checkout with an unfetched --base, with or without a manifest, stops in advisory mode too. That matches what the new manifest-free test pins.

Earlier findings

ID Severity Disposition
C1-1 P2 resolved
C1-2 P3 resolved
C2-1 P2 resolved. Removing the shallow guard now fails two tests (M1).
C2-2 P3 resolved. The fix differs from the one suggested: a detected info/grafts file answers None, where the suggestion was GIT_GRAFT_FILE=/dev/null. The rationale (the deprecation hint) is accepted, and removing the check fails the graft test (M2).

@pengfei-threemoonslab

Copy link
Copy Markdown
Contributor Author

🤖 Engineering review — merge readiness

  • Reviewed head: 18186a111f7f6467cd3155b9b4b51f71baa00622 (cycle 3, clean). Three cycles: C1-1 (P2, deleted manifest with a named-but-missing --base still took the setup/host route, exit 0 in advisory mode) resolved; C1-2 (P3, STABILITY detail sentence) resolved; C2-1 (P2, the shallow guard had no test: a mutation removing it passed the suite) resolved with depth-1 tests that fail under the mutation; C2-2 (P3, .git/info/grafts reopened the setup route) resolved by treating a grafts file as unknown history, with a test. The reviewer confirmed each mutation now fails its test, no misfire on normal clones, subdirectories or linked worktrees, the static-only allowlist edits are line shifts only, and the tree-pack archive change leaves tree SHAs, findings and goldens identical.
  • Follow-up filed: verifier 0.22: typed base_unresolved and manifest_removed stops; align preview base detection and trigger reports-dir exclusion #1071 (typed base_unresolved/manifest_removed values in verifier 0.22; verify --preview base detection; trigger --git-diff reports-dir exclusion; remaining archive ConfigError routing).
  • CI on 18186a1: all required checks succeeded; release-tag-consistency skipped as expected.
  • Repository gate (./shipgate verify, orchestrator run): state complete, merge permission true, decision passed.
  • Branch contains origin/main (including fix(verify): one receipt predicate for attest, org bundle and Action #1065); no attribution trailers; no protected paths; no unanswered human comments.

@pengfei-threemoonslab
pengfei-threemoonslab merged commit 9a3317a into main Oct 13, 2026
15 checks passed
@pengfei-threemoonslab
pengfei-threemoonslab deleted the claude/review-1023-verify-base-and-gate-removal branch October 13, 2026 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

engineering-review Filed from the 2026-10-12 comprehensive engineering review P0 Release-blocking: correctness of the gate or the primary adoption path

Projects

None yet

Development

Successfully merging this pull request may close these issues.

verify: fail closed with no base or a deleted manifest; anchor gate paths at repo root

1 participant