Skip to content

docs: STABILITY exit-code text says 20 is strict-only; advisory exits 20 on a fail_on match (and the base floor after #1074) #1080

Open

Description

@pengfei-threemoonslab

Deferred by the engineering-review implementation loop from #1074 (issue #1024).

Problem

STABILITY.md's § Exit codes says exit code 20 is strict-mode only, and the 0.x note says "In advisory mode every decision — blocked included — exits 0". ci/exit_policy.py exit_code_for_report also exits 20 in advisory mode when an explicit fail_on severity matches (reproduced by the #1074 reviewer: advisory with fail_on: [high] and a high finding exits 20). The code is the intended behaviour (the GitHub Action's fail_on input depends on it); the contract text is stale. After #1074 the exit code in advisory mode is also floored by the base's fail_on, which the same paragraphs must describe.

Evidence

  • STABILITY.md § Exit codes (row for 20) and the 0.x advisory note; src/agents_shipgate/ci/exit_policy.py (fail_on severity check in advisory mode).
  • Related confirmed finding Bump actions/github-script from 7.1.0 to 9.0.0 #2 of the 2026-10-12 review (strict-mode exit 20 without a fail_on match contradicts STABILITY.md:5821).

Proposed fix

Rewrite the exit-code paragraphs to state the actual rule in both modes: strict exits 20 on blocked, insufficient_evidence or any blocks_release finding regardless of fail_on; advisory exits 20 only on a fail_on severity match, where fail_on is the head's declared list, --fail-on when given, floored by the base's list (#1074). Add a doc-parity assertion beside the verdict-contract test so the paragraphs cannot drift again. Part of #1045 (single-source contract tables) if that lands first.

Acceptance criteria

  • STABILITY's exit-code text matches exit_code_for_report for every (mode, fail_on, decision) combination in the truth table.
  • A test asserts the documented rule against the function.

Source

Engineering review 2026-10-12 follow-up (PR #1074 review cycle 2). Tracked in #1060.

Activity

  1. added
    documentationImprovements or additions to documentation
    P2Queued; valuable but not blocking
    area:verifyverify / preview orchestration and SHIP-VERIFY-* checks
    engineering-reviewFiled from the 2026-10-12 comprehensive engineering review
    on Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Queued; valuable but not blockingarea:verifyverify / preview orchestration and SHIP-VERIFY-* checksdocumentationImprovements or additions to documentationengineering-reviewFiled from the 2026-10-12 comprehensive engineering review

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions