Repository navigation
feat(report-back): fleet members report back to the Hypatia verdict grid #626
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
68858e0
feat(report-back): fleet members report back to the Hypatia verdict grid
claude 4c9ea43
fix(report-back): survive large finding lists; no caller-chosen fleetβ¦
claude c66b679
Merge branch 'main' into claude/gitbot-fleet-feedback-0vdo64
hyperpolymath e582c5f
chore(report-back): canonicalise the enrolment manifest to I-JSON/JCS
claude e85b71b
fix(report-back): keep the fleet checkout out of the scanned tree; reβ¦
claude 5f04466
docs(report-back): document shell helper behavior and error handling
coderabbitai[bot] 225ee8d
test(report-back): cover reporting edge cases and build helper failurβ¦
coderabbitai[bot] 4a55497
feat(report-back): fix-fleet-enrolment.sh rolls the report-backs out β¦
claude 6344af5
fix(report-back): envelopes outside the scanned tree; a build budget;β¦
claude bf5993a
fix(report-back): executable fixer; job limit covers both phases; wirβ¦
claude bf421a9
fix(report-back): member names must be slugs; no zero adapter timeoutβ¦
claude e6042f5
fix(report-back): shape guards, bounded titles, capped adapter timeouβ¦
claude 8ac4dd3
test(report-back): expand coverage for enrolment, build limits, and aβ¦
coderabbitai[bot] 8a8be16
Merge branch 'main' into claude/gitbot-fleet-feedback-0vdo64
hyperpolymath File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| {"reporters":{"accessibilitybot":{"enrolled":false,"reason":"no served HTML surface; the HTML in the tree is accessibilitybot's own test fixtures and gsbot's templates"},"cipherbot":{"enrolled":true,"title":"cipherbot (cryptographic hygiene)"},"echidnabot":{"enrolled":false,"reason":"the only proof files here are echidnabot's own dogfood fixtures (bots/echidnabot/proofs); no proofs about the fleet"},"finishingbot":{"enrolled":true,"title":"finishingbot (release readiness)"},"glambot":{"enrolled":true,"title":"glambot (presentation quality)"},"panicbot":{"enrolled":true,"title":"panicbot (panic recovery, error handling)"},"rhodibot":{"enrolled":true,"title":"rhodibot (RSR compliance)"},"seambot":{"enrolled":false,"reason":"no seam register is declared (spec/seams/seam-register.json)"},"sustainabot":{"enrolled":false,"reason":"reserved fleet slot; no adapter exists (bots/sustainabot/README.adoc)"}},"schema":1} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| # gitbot-fleet report-backs | ||
|
|
||
| One JSON envelope per enrolled fleet member, `<member>.json`, in the shape | ||
| `docs/VERDICT-GRID-SPEC.adoc` of metadatastician/standards defines and | ||
| `docs/REPORT-BACK.adoc` in this repository explains. | ||
|
|
||
| **This directory is empty in git on purpose, and stays so.** Envelopes are | ||
| produced per run by `scripts/fleet-report-back.sh` (through | ||
| `.github/workflows/fleet-report-back-reusable.yml`), uploaded as the | ||
| `fleet-report-backs` artifact, and placed here by the Hypatia lane only at | ||
| render time. A committed envelope would be stale evidence about an older | ||
| commit, so none is ever committed. An enrolled member with no envelope | ||
| renders **orange** ("enrolled, but no report-back was received"); only a real | ||
| run may change that. | ||
|
|
||
| Only `*.json` files are read, so this README is inert. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,280 @@ | ||
| # This workflow is managed by gh actions-lock. | ||
| # SPDX-License-Identifier: MPL-2.0 | ||
| # SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell (hyperpolymath) <j.d.a.jewell@open.ac.uk> | ||
| # | ||
| # fleet-report-back-reusable.yml β produce the gitbot-fleet report-backs for | ||
| # one pull request, as an artifact the Hypatia verdict grid reads. | ||
| # | ||
| # WHY. The verdict grid (standards: docs/VERDICT-GRID-SPEC.adoc) gives every | ||
| # fleet member a repository enrols in `.gitbot-fleet/enrolment.json` one row, | ||
| # and that row is orange ("enrolled, but no report-back was received") until a | ||
| # `.gitbot-fleet/report-backs/<member>.json` envelope exists when the grid | ||
| # renders. Until this workflow existed nothing in the estate wrote one, so the | ||
| # fleet reported nothing on any PR. This is the producer. | ||
| # | ||
| # WHAT IT DOES. Checks out the caller's tree, checks out gitbot-fleet at the | ||
| # commit the caller pinned, builds only the adapters the manifest enrols | ||
| # (cached with actions/cache), runs `scripts/fleet-report-back.sh`, and | ||
| # uploads the envelopes as the `fleet-report-backs` artifact. The Hypatia lane | ||
| # (standards: hypatia-scan-reusable.yml, input `report-backs-artifact`) | ||
| # downloads that artifact into `.gitbot-fleet/report-backs/` before rendering. | ||
| # | ||
| # WHAT IT DOES NOT DO. It posts nothing and gates nothing: an adapter that is | ||
| # missing, crashes or times out becomes an `error` envelope with its reason, | ||
| # and the grid renders that orange. The job fails only when the driver itself | ||
| # cannot run. It writes nothing into the caller's repository; the envelopes | ||
| # live in the artifact for the run, never in git. | ||
| # | ||
| # Permissions: `contents: read` only. A called workflow may not request more | ||
| # than its caller grants, and this one needs nothing more. | ||
| # | ||
| # Caller sketch (see docs/REPORT-BACK.adoc for the full adoption note): | ||
| # | ||
| # jobs: | ||
| # fleet: | ||
| # uses: hyperpolymath/gitbot-fleet/.github/workflows/fleet-report-back-reusable.yml@<SHA> | ||
| # permissions: { contents: read } | ||
| # hypatia: | ||
| # needs: [fleet] | ||
| # if: ${{ always() }} | ||
| # uses: <standards>/.github/workflows/hypatia-scan-reusable.yml@<SHA> | ||
| # ... | ||
| # | ||
| # A repository whose Actions policy refuses reusable workflows from another | ||
| # owner (metadatastician/* calling hyperpolymath/*) runs the same steps inline | ||
| # instead: check out gitbot-fleet at a pinned SHA and call the driver. The | ||
| # standards canon's own fork-governance.yml is the worked example of that. | ||
|
|
||
| name: Fleet report-backs (reusable) | ||
|
|
||
| on: | ||
| workflow_call: | ||
| inputs: | ||
| enrolment: | ||
| description: Path of the enrolment manifest inside the caller's tree. | ||
| type: string | ||
| required: false | ||
| default: ".gitbot-fleet/enrolment.json" | ||
| members: | ||
| description: >- | ||
| Comma-separated member names to run instead of every enrolled | ||
| member. A member the manifest marks `enrolled: false` is still | ||
| skipped: the repository's statement wins. | ||
| type: string | ||
| required: false | ||
| default: "" | ||
| artifact-name: | ||
| description: Name of the uploaded artifact holding the envelopes. | ||
| type: string | ||
| required: false | ||
| default: "fleet-report-backs" | ||
| adapter-timeout-seconds: | ||
| description: >- | ||
| Seconds each adapter may run before its row reads error. At most 300: | ||
| the job limit is sized for seven adapters at 300 s after the build | ||
| budget, so a larger value is capped (with a warning) and a value that | ||
| is not a positive integer falls back to 300. | ||
| type: string | ||
| required: false | ||
| default: "300" | ||
| outputs: | ||
| members: | ||
| description: >- | ||
| JSON object mapping each member that ran to its envelope status | ||
| (ran | not_run | error). Empty object when nothing was enrolled. | ||
| value: ${{ jobs.report-back.outputs.members }} | ||
| artifact-name: | ||
| description: The artifact name, echoed for the consumer's download step. | ||
| value: ${{ jobs.report-back.outputs.artifact-name }} | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| report-back: | ||
| name: Fleet report-backs | ||
| runs-on: ubuntu-latest | ||
| # Warm (cache hit): about two minutes. Cold: up to six Rust adapters plus | ||
| # panic-attack from source (~15 min). The limit is the worst case, so a | ||
| # stalled phase never kills the job before the upload: the build budget | ||
| # (fleet-report-back-build.sh --build-budget, 2400 s) + up to seven adapters | ||
| # at adapter-timeout-seconds each (7 Γ 300 s = 2100 s) + checkout, cache and | ||
| # upload (~5 min) = 80 min. GNU timeout bounds each command on its own; only | ||
| # this limit bounds their sum. | ||
| timeout-minutes: 85 | ||
| permissions: | ||
| contents: read | ||
| outputs: | ||
| members: ${{ steps.run.outputs.members }} | ||
| artifact-name: ${{ inputs.artifact-name }} | ||
| env: | ||
| # panic-attack, built from source at this commit for panicbot. Kept in | ||
| # step with the standards Hypatia lane's PANIC_ATTACK_SHA so the two | ||
| # panic-attack-derived rows on one grid describe the same tool. | ||
| PANIC_ATTACK_SHA: b689a74d2762a1c19839ae3fbd59b7fe5ef920ea | ||
| steps: | ||
| - name: Checkout the repository to report on | ||
| uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| persist-credentials: false | ||
| # The PR head, not the merge ref: the grid's change scope is computed | ||
| # from the same head, so paths and line numbers agree with it. | ||
| ref: ${{ github.event.pull_request.head.sha || github.sha }} | ||
|
|
||
| - name: Checkout gitbot-fleet at the pinned commit | ||
| uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| repository: hyperpolymath/gitbot-fleet | ||
| # `job.workflow_sha` is the commit of THIS file, which is what the | ||
| # caller pinned. It is not `github.workflow_sha` (the caller's file) | ||
| # and `github.job_workflow_sha` does not exist. | ||
| # It is deliberately NOT an input: a ref a caller can choose is untrusted | ||
| # code run with a cache the default branch shares (CodeQL | ||
| # actions/cache-poisoning, three alerts on gitbot-fleet PR 626). The pin | ||
| # in `uses:` is the only way to choose the fleet commit, and it is immutable. | ||
| ref: ${{ job.workflow_sha }} | ||
| path: .fleet-src | ||
|
hyperpolymath marked this conversation as resolved.
coderabbitai[bot] marked this conversation as resolved.
|
||
| persist-credentials: false | ||
|
|
||
| # The fleet checkout must not sit inside the tree being reported on: | ||
| # walk-based adapters (glambot, finishing-bot, cipherbot) would scan the | ||
| # fleet's own source and emit findings no PR diff can match. actions/checkout | ||
| # only writes under the workspace, so the checkout is moved out to | ||
| # RUNNER_TEMP before anything reads the caller's tree. The lockfile hash | ||
| # for the cache key is taken first, while hashFiles() can still see it. | ||
| - name: Move the fleet checkout out of the scanned tree | ||
| id: fleet | ||
| run: | | ||
| set -euo pipefail | ||
| lockhash=$(cat .fleet-src/bots/*/Cargo.lock .fleet-src/shared-context/Cargo.lock 2>/dev/null | sha256sum | cut -c1-16) | ||
| mv .fleet-src "$RUNNER_TEMP/fleet-src" | ||
| echo "dir=$RUNNER_TEMP/fleet-src" >> "$GITHUB_OUTPUT" | ||
| echo "lockhash=$lockhash" >> "$GITHUB_OUTPUT" | ||
|
|
||
| - name: Plan which adapters to build | ||
| id: plan | ||
| env: | ||
| FLEET_DIR: ${{ steps.fleet.outputs.dir }} | ||
| ENROLMENT: ${{ inputs.enrolment }} | ||
| MEMBERS: ${{ inputs.members }} | ||
| run: | | ||
| set -euo pipefail | ||
| args=(--plan --enrolment "$ENROLMENT") | ||
| IFS=',' read -r -a _m <<< "${MEMBERS:-}" | ||
| for m in "${_m[@]}"; do m="${m// /}"; [ -n "$m" ] && args+=(--member "$m"); done | ||
| plan=$(bash "$FLEET_DIR/scripts/fleet-report-back.sh" "${args[@]}") | ||
| echo "plan:"; printf '%s\n' "$plan" | sed 's/^/ /' | ||
| # Crates to build: the non-empty second column, deduplicated. | ||
| crates=$(printf '%s\n' "$plan" | awk -F'\t' '$2 != "" {print $2}' | sort -u | tr '\n' ' ') | ||
| echo "crates=${crates% }" >> "$GITHUB_OUTPUT" | ||
| if printf '%s\n' "$plan" | awk -F'\t' '$1 == "panicbot" {found=1} END {exit !found}'; then | ||
| echo "panicbot=true" >> "$GITHUB_OUTPUT" | ||
| else | ||
| echo "panicbot=false" >> "$GITHUB_OUTPUT" | ||
| fi | ||
| { | ||
| echo "workspaces<<WS" | ||
| for c in $crates; do echo "$FLEET_DIR/bots/$c -> target"; done | ||
| echo "$FLEET_DIR/shared-context -> target" | ||
| echo "WS" | ||
| } >> "$GITHUB_OUTPUT" | ||
| [ -n "${crates% }" ] && echo "build=true" >> "$GITHUB_OUTPUT" || echo "build=false" >> "$GITHUB_OUTPUT" | ||
|
|
||
| # Rust comes with the GitHub-hosted runner image (rustup, stable), so no | ||
| # third-party toolchain action is needed; the cache is actions/cache, keyed | ||
| # on the fleet commit's lockfiles. Only GitHub-owned actions, pinned by | ||
| # SHA, appear in this file: a caller whose Actions policy requires SHA pins | ||
| # or forbids unverified creators can still call it (metamanifold-webui | ||
| # PR 296 died at plan time with zero jobs on the tag-pinned version). | ||
| - name: Cache adapter builds | ||
| if: steps.plan.outputs.build == 'true' | ||
| uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 | ||
| with: | ||
| path: | | ||
| ~/.cargo/registry/index | ||
| ~/.cargo/registry/cache | ||
| ~/.cargo/git/db | ||
| ${{ runner.temp }}/fleet-src/bots/*/target | ||
| key: fleet-report-back-v3-${{ runner.os }}-${{ runner.arch }}-${{ steps.fleet.outputs.lockhash }}-${{ steps.plan.outputs.crates }} | ||
| restore-keys: | | ||
| fleet-report-back-v3-${{ runner.os }}-${{ runner.arch }}-${{ steps.fleet.outputs.lockhash }}- | ||
| fleet-report-back-v3-${{ runner.os }}-${{ runner.arch }}- | ||
|
|
||
| - name: Report the Rust toolchain in use | ||
| if: steps.plan.outputs.build == 'true' || steps.plan.outputs.panicbot == 'true' | ||
| run: | | ||
| set -euo pipefail | ||
| command -v cargo >/dev/null || { echo "::error::cargo is not on this runner; the adapters cannot be built"; exit 1; } | ||
| cargo --version; rustc --version | ||
|
|
||
| - name: Restore panic-attack build cache | ||
| if: steps.plan.outputs.panicbot == 'true' | ||
| uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 | ||
| with: | ||
| path: ~/.panic-attack | ||
| key: panic-attack-v1-${{ runner.os }}-${{ runner.arch }}-${{ env.PANIC_ATTACK_SHA }} | ||
|
|
||
| - name: Build the enrolled adapters | ||
| if: steps.plan.outputs.build == 'true' || steps.plan.outputs.panicbot == 'true' | ||
| env: | ||
| FLEET_DIR: ${{ steps.fleet.outputs.dir }} | ||
| CRATES: ${{ steps.plan.outputs.crates }} | ||
| PANICBOT: ${{ steps.plan.outputs.panicbot }} | ||
| run: | | ||
| set -euo pipefail | ||
| # One build recipe for the reusable and for inline callers; a crate | ||
| # that fails to build is a warning here and an `error` row later. | ||
| args=(--fleet "$FLEET_DIR" --bin-dir "$RUNNER_TEMP/fleet-bin" --crates "$CRATES") | ||
| [ "$PANICBOT" = true ] && args+=(--panic-attack "$PANIC_ATTACK_SHA") | ||
| bash "$FLEET_DIR/scripts/fleet-report-back-build.sh" "${args[@]}" | ||
|
|
||
| - name: Run the enrolled adapters and write the envelopes | ||
|
github-advanced-security[bot] marked this conversation as resolved.
Fixed
|
||
| id: run | ||
| # The envelopes are written under RUNNER_TEMP, never into the scanned tree: | ||
| # each member's JSON would otherwise be scanned by the walk-based adapters | ||
| # that run after it, and a tracked report-backs/ directory in the caller | ||
| # would leak into the artifact. | ||
| env: | ||
| FLEET_DIR: ${{ steps.fleet.outputs.dir }} | ||
| ENROLMENT: ${{ inputs.enrolment }} | ||
| MEMBERS: ${{ inputs.members }} | ||
| ADAPTER_TIMEOUT: ${{ inputs.adapter-timeout-seconds }} | ||
| FLEET_REF: ${{ job.workflow_sha }} | ||
| run: | | ||
| set -euo pipefail | ||
| # The job limit (timeout-minutes) assumes at most 300 s per adapter; a | ||
| # larger input would let the scans outrun it and lose every envelope. | ||
| case "$ADAPTER_TIMEOUT" in | ||
| ''|*[!0-9]*|0) echo "::warning title=adapter-timeout-seconds::'$ADAPTER_TIMEOUT' is not a positive integer; using 300"; ADAPTER_TIMEOUT=300 ;; | ||
| esac | ||
| if [ "$ADAPTER_TIMEOUT" -gt 300 ]; then | ||
| echo "::warning title=adapter-timeout-seconds::$ADAPTER_TIMEOUT s exceeds the 300 s the job limit is sized for; capped at 300" | ||
| ADAPTER_TIMEOUT=300 | ||
| fi | ||
| mkdir -p "$RUNNER_TEMP/fleet-bin" "$RUNNER_TEMP/report-backs" | ||
| args=(--repo . --out "$RUNNER_TEMP/report-backs" --bin-dir "$RUNNER_TEMP/fleet-bin" | ||
| --enrolment "$ENROLMENT" --fleet-ref "$FLEET_REF" | ||
| --timeout "$ADAPTER_TIMEOUT" --summary "$GITHUB_STEP_SUMMARY") | ||
| IFS=',' read -r -a _m <<< "${MEMBERS:-}" | ||
| for m in "${_m[@]}"; do m="${m// /}"; [ -n "$m" ] && args+=(--member "$m"); done | ||
| bash "$FLEET_DIR/scripts/fleet-report-back.sh" "${args[@]}" | ||
| # Status map for the caller, and a non-empty directory for the | ||
| # upload: a run that enrolled nobody still uploads the note below so | ||
| # the consumer can tell "nothing enrolled" from "upload failed". | ||
| if ls "$RUNNER_TEMP"/report-backs/*.json >/dev/null 2>&1; then | ||
| members=$(jq -s -c 'map({(.reporter): .status}) | add' "$RUNNER_TEMP"/report-backs/*.json) | ||
| else | ||
| members='{}' | ||
| printf 'No member is enrolled in %s; no report-back was produced.\n' "$ENROLMENT" > "$RUNNER_TEMP/report-backs/NONE-ENROLLED.txt" | ||
| fi | ||
| echo "members=$members" >> "$GITHUB_OUTPUT" | ||
| echo "members: $members" | ||
|
|
||
| - name: Upload the report-backs | ||
|
github-advanced-security[bot] marked this conversation as resolved.
Fixed
|
||
| if: ${{ always() && steps.run.conclusion != 'skipped' }} | ||
| uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 | ||
| with: | ||
| name: ${{ inputs.artifact-name }} | ||
| path: ${{ runner.temp }}/report-backs | ||
| if-no-files-found: error | ||
| retention-days: 90 | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.