Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 26 additions & 17 deletions .github/workflows/k9-conformance.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,10 @@ name: k9-conformance
# the PR base: an entry the PR adds exempts nothing unless a commit message or
# the PR body carries `Ratchet-exception: .machine_readable/k9-contract-debt.txt
# — <why>` (the governance exemption ratchet's form; it watches this ledger
# too). On a pull request the checkout is full so the changed-file list is the
# merge-base diff and the PR's commit messages are readable.
# too). The checkout is full so the changed-file list is the merge-base diff
# and the commit messages are readable; on a push the range is the push's own
# (`before`..`sha`), so a direct push to a ledgered contract, or a direct
# ledger addition, is ratcheted exactly as a pull request is.
#
# Outputs: the `k9-grid` artifact (report.json and badge.json as I-JSON/JCS,
# report.md, the validator logs) and the `verdict` output. The caller posts
Expand All @@ -49,7 +51,6 @@ on:
paths:
- '**.k9.ncl'
- '**.k9'
- '**.k9.lock'
- '1-formats/k9/**'
- 'scripts/k9-grid.sh'
- 'scripts/lib/k9-grid.jq'
Expand All @@ -60,7 +61,6 @@ on:
paths:
- '**.k9.ncl'
- '**.k9'
- '**.k9.lock'
- '1-formats/k9/**'
- 'scripts/k9-grid.sh'
- 'scripts/lib/k9-grid.jq'
Expand Down Expand Up @@ -112,10 +112,10 @@ jobs:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
# Full history on a pull request: the changed-file list is the
# merge-base diff, and the ledger ratchet reads the PR's commit
# messages. Strings, not numbers: a bare 0 is falsy in an expression.
fetch-depth: ${{ github.event_name == 'pull_request' && '0' || '1' }}
# Full history: the changed-file list is the merge-base diff and the
# ledger ratchet reads the commit messages, on a pull request (base
# ..head) and on a push (the push's before..sha) alike.
fetch-depth: 0
persist-credentials: false

- name: checkout the K9 validator, contract and renderer at this workflow's commit
Expand Down Expand Up @@ -186,34 +186,41 @@ jobs:
echo "TYPECHECK_CODE=$?" >> "$GITHUB_ENV"
cat "$RUNNER_TEMP/typecheck.log"

- name: Enumerate the committed K9 files, the PR's changed files and ledger growth
- name: Enumerate the committed K9 files, the changed files and ledger growth
env:
BASE_SHA: ${{ github.event.pull_request.base.sha }}
# A pull request compares with its base; a push with the commit the
# push replaced (all zeros when the ref is new: nothing to compare).
BASE_SHA: ${{ github.event.pull_request.base.sha || github.event.before }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve PR-body exemptions on the merge push

When ledger growth is authorized only by the supported Ratchet-exception in the PR body, the PR run clears the growth, but the post-merge push run now compares before to the merged SHA and detects it again. A push payload has no .pull_request.body, and squash/rebase or configured merge messages need not retain that body or the original trailer, so lines 248–251 leave the growth undeclared and turn main red immediately after an accepted merge. The push path needs to recover the associated PR exemption or otherwise avoid invalidating an exemption already accepted by the PR run.

Useful? React with 👍 / 👎.

HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
LEDGER: .machine_readable/k9-contract-debt.txt
run: |
set -euo pipefail
# Tracked files only: an untracked file is not a committed contract.
# The two extensions K9-CONTRACT-SPEC.adoc reserves, nothing else.
# A caller with no K9 file at all is a legitimate (grey) corpus, so
# grep's no-match exit is not an error here.
git ls-files -z -- '*.k9.ncl' '*.k9' '*.k9.lock' \
git ls-files -z -- '*.k9.ncl' '*.k9' \
| tr '\0' '\n' \
| { grep -v -E '^(\.k9-canon/|9-archive/|1-formats/k9/tools/fixtures/)' || [ $? -eq 1 ]; } \
| sort > "$RUNNER_TEMP/expected"
echo "$(wc -l < "$RUNNER_TEMP/expected") committed K9 file(s)"
: > "$RUNNER_TEMP/changed"
: > "$RUNNER_TEMP/ledger-growth"
echo "BASE_UNRESOLVED=" >> "$GITHUB_ENV"
case "$BASE_SHA" in *[!0]*) ;; *) BASE_SHA="" ;; esac
[ -n "$BASE_SHA" ] || exit 0

# The PR's own changes are those since its merge base, not since the
# base branch tip: a file the base branch changed after the PR
# branched is not this PR's edit. The full checkout already holds
# the base commit; the fetch is a fallback only, and runs without
# credentials (persist-credentials is off), so it serves public
# repositories.
# branched is not this PR's edit. A push compares with `before`
# itself, never a merge base: after a force push the fork point is
# not the state the push replaced, and a ledger entry the old tip
# had removed would come back unseen. The full checkout already
# holds the base commit; the fetch
# is a fallback only, and runs without credentials
# (persist-credentials is off), so it serves public repositories.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
if { git cat-file -e "$BASE_SHA^{commit}" 2>/dev/null || git fetch --quiet origin "$BASE_SHA" 2>/dev/null; } \
&& merge_base=$(git merge-base "$BASE_SHA" "$HEAD_SHA" 2>/dev/null) \
&& merge_base=$(if [ "$GITHUB_EVENT_NAME" = pull_request ]; then git merge-base "$BASE_SHA" "$HEAD_SHA"; else git rev-parse --verify "$BASE_SHA^{commit}"; fi 2>/dev/null) \
&& git diff --name-only "$merge_base" "$HEAD_SHA" > "$RUNNER_TEMP/changed" 2>/dev/null; then
echo "$(wc -l < "$RUNNER_TEMP/changed") file(s) changed since the merge base $merge_base"
else
Expand Down Expand Up @@ -260,7 +267,9 @@ jobs:
worst=0
while IFS= read -r f; do
[ -n "$f" ] || continue
bash "$CANON/1-formats/k9/tools/k9-validate.sh" --strict --layer L2 --json --quiet "$f" >> "$RUNNER_TEMP/results.jsonl" 2>> "$RUNNER_TEMP/corpus.log"
# `--` ends the options: a tracked file whose name starts with `-`
# is a path, not a flag.
bash "$CANON/1-formats/k9/tools/k9-validate.sh" --strict --layer L2 --json --quiet -- "$f" >> "$RUNNER_TEMP/results.jsonl" 2>> "$RUNNER_TEMP/corpus.log"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Harden the validator before accepting dash-leading paths

The added -- only gets a root-level name such as -dash.k9.ncl through the validator's own option parser; the validator then passes that bare name to utilities without their own option terminators. In particular, check_l0 runs cmp -s - "$f", which interprets -dash.k9.ncl as an option and records a false K9-E002 failure, while later head, sed, and dirname calls have the same problem. Thus a conforming tracked file in the exact scenario this change claims to support still fails the lane; either make all validator file consumers option-safe or normalize the filesystem path without changing the logical filename emitted in JSON.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reproduced (cmp, sed and dirname all read -dash.k9.ncl as options, giving a false K9-E002). Fixed at the source in the next commit on this branch: validate_one hands every check a ./-prefixed path when the name starts with -, and the JSON and messages keep the logical name. A copy of a conforming contract under a dash-leading name now reaches the same verdict as an ordinary copy, and the validator self-test still passes.


Generated by Claude Code

Comment thread
coderabbitai[bot] marked this conversation as resolved.
rc=$?
[ "$rc" -gt "$worst" ] && worst=$rc
done < "$RUNNER_TEMP/expected"
Expand Down
2 changes: 1 addition & 1 deletion 1-formats/k9/tools/CONFORMANCE-REPORTING.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ bash 1-formats/k9/tools/k9-validate.sh --self-test > selftest.log; echo $? > sel
nickel typecheck 1-formats/k9/spec/contract/k9_contract.ncl > typecheck.log; echo $? > typecheck.code
# The same exclusions as the lane: the fixture corpus is deliberately invalid
# and the archive is out of scope.
git ls-files -- "*.k9.ncl" "*.k9" "*.k9.lock" \
git ls-files -- "*.k9.ncl" "*.k9" \
| { grep -v -E "^(9-archive/|1-formats/k9/tools/fixtures/)" || [ $? -eq 1 ]; } | sort > expected
# The lane also records the validator's worst exit status over the corpus.
corpus_code=0
Expand Down
28 changes: 23 additions & 5 deletions 1-formats/k9/tools/k9-validate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -663,14 +663,19 @@
validate_one() {
F="$1"; DIALECT=""; HAS_MAGIC=0
ERRORS=0; WARNINGS=0; SKIPS=0; FINDINGS=()
# F is the logical name (JSON, messages); FP is the path the checks hand to
# cmp, head, sed, grep and dirname. A name that starts with `-` would be read
# as an option by those, so it is given a ./ prefix here and nowhere else.
local FP="$F"

Check warning on line 669 in 1-formats/k9/tools/k9-validate.sh

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Local variable 'FP' should use lower_case naming convention.

See more on https://sonarcloud.io/project/issues?id=hyperpolymath_standards&issues=AaFWb_jnmNrmCZWo26AO&open=AaFWb_jnmNrmCZWo26AO&pullRequest=1310
case "$FP" in -*) FP="./$FP" ;; esac

Check failure on line 670 in 1-formats/k9/tools/k9-validate.sh

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Add a default case (*) to handle unexpected values.

See more on https://sonarcloud.io/project/issues?id=hyperpolymath_standards&issues=AaFWb_jnmNrmCZWo26AP&open=AaFWb_jnmNrmCZWo26AP&pullRequest=1310

if [ ! -f "$F" ]; then
if [ ! -f "$FP" ]; then

Check failure on line 672 in 1-formats/k9/tools/k9-validate.sh

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.

See more on https://sonarcloud.io/project/issues?id=hyperpolymath_standards&issues=AaFWb_jnmNrmCZWo26AQ&open=AaFWb_jnmNrmCZWo26AQ&pullRequest=1310
err K9-E000 L0 "no such file"
if [ "$JSON" -eq 1 ]; then emit_json "error"; else emit_findings; fi
return 1
fi

check_l0 "$F"
check_l0 "$FP"
local rc=$?
if [ $rc -eq 2 ]; then
# Routed out of scope: examined, and handed to the spec that governs it.
Expand All @@ -683,9 +688,9 @@
return 1
fi

layer_reached L1 && check_l1 "$F"
layer_reached L2 && check_l2 "$F"
layer_reached L3 && check_l3 "$F"
layer_reached L1 && check_l1 "$FP"
layer_reached L2 && check_l2 "$FP"
layer_reached L3 && check_l3 "$FP"

local verdict="pass"
[ "$ERRORS" -gt 0 ] && verdict="fail"
Expand Down Expand Up @@ -782,6 +787,19 @@
t "schema_version stays on line 5" ' schema_version = "1.0.0",' "$(sed -n '5p' "$stripped")"
rm -f "$tmp" "$facts" "$stripped"

echo "== a name that starts with '-' is a path, not an option =="
# The checks hand file names to cmp, head, sed, grep and dirname. A tracked
# `-x.k9.ncl` must reach the same verdict as `x.k9.ncl`, and the JSON must
# carry the name as given.
local dashdir dashjson
dashdir="$(mktemp -d)"
cp "$SCRIPT_DIR/fixtures/valid/kennel-data.k9.ncl" "$dashdir/-dash.k9.ncl"
dashjson="$(cd "$dashdir" && LAYER=L1 JSON=1 QUIET=1 validate_one "-dash.k9.ncl" 2>/dev/null || true)"
t "dash-leading name: no L0 failure" "" "$(printf '%s' "$dashjson" | grep -o 'K9-E00[0-9]' | head -n 1)"
t "dash-leading name: verdict pass at L1" 1 "$(printf '%s' "$dashjson" | grep -c '"verdict":"pass"')"
t "dash-leading name: JSON keeps the logical name" 1 "$(printf '%s' "$dashjson" | grep -c '"file":"-dash.k9.ncl"')"
rm -rf "$dashdir"

echo "== L3: signature presence is not verification (§10) =="
# No fixture can make a verifier appear, so the three verdicts a host can
# reach are asserted here with a stub verifier instead. What is under test is
Expand Down
12 changes: 10 additions & 2 deletions scripts/lib/k9-grid.jq
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,8 @@ def corpus_items(results; expected; ledger; growth; changed; require_files):
.rows += [item("validator output"; "indeterminate"; "Malformed JSON result.")]
elif (($r.file | type) != "string") or ($expected[$r.file] | not) or (.seen[$r.file] // false)
or (($r.verdict // "") | IN("pass", "fail", "error", "routed") | not)
or (($r.findings | type) != "array") then
or (($r.findings | type) != "array")
or (($r.findings | type) == "array" and ($r.findings | any(type != "object"))) then
.rows += [item((if ($r.file | type) == "string" then $r.file else "validator output" end); "indeterminate"; "Unexpected, duplicate or invalid result.")]
else
.seen[$r.file] = true
Expand Down Expand Up @@ -213,8 +214,15 @@ def state_label: {red: "🔴 failing", orange: "🟠 indeterminate", yellow: "
def colour: {red: "red", orange: "orange", yellow: "yellow", green: "brightgreen", grey: "lightgrey"}[.];
def word: {red: "failing", orange: "indeterminate", yellow: "advisory", green: "conformant", grey: "not_applicable"}[.];
# Escape text for grid cells and finding summaries, flattening line breaks.
# Ids and reasons carry contributor-controlled file names and validator text,
# and the comment is posted by the lane's own account: every character that
# could start Markdown or HTML syntax, or an @-mention, becomes an entity, so a file named
# `![x](https://e.invalid/i).k9.ncl` is shown, not rendered. Entities display as
# the character itself in a GitHub table cell.
def esc: tostring | gsub("&"; "&amp;") | gsub("<"; "&lt;") | gsub(">"; "&gt;") | gsub("\\|"; "&#124;")
| gsub("`"; "&#96;") | gsub("\r?\n"; " ");
| gsub("`"; "&#96;") | gsub("\\\\"; "&#92;") | gsub("\\["; "&#91;") | gsub("\\]"; "&#93;")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Flatten standalone carriage returns

When contributor-controlled text contains a bare \r, this final substitution leaves it intact because it only matches LF and CRLF. For example, an edited debt-ledger entry containing evil\r\r---\r\r# fake reaches the stale-exemption row unchanged; Markdown normalizes those carriage returns as line endings, allowing the bot-authored <details> content to be broken into a horizontal rule or fake heading. Replace standalone CR characters as well as LF so every rendered value remains on one Markdown line.

Useful? React with 👍 / 👎.

| gsub("\\("; "&#40;") | gsub("\\)"; "&#41;") | gsub("\\*"; "&#42;") | gsub("_"; "&#95;")
| gsub("!"; "&#33;") | gsub("~"; "&#126;") | gsub("@"; "&#64;") | gsub("\r?\n"; " ");
def grid_label: "K9 L0–L2";
def badge_url(st): "https://img.shields.io/badge/\(grid_label | @uri)-\(st | word | @uri)-\(st | colour)";

Expand Down
22 changes: 22 additions & 0 deletions scripts/tests/k9-grid-test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,28 @@ check "baseline with trailing junk is one value" 1 "$(bash "$GRID" baseline "$WO
render "$WORK/o11l" --results "$WORK/r-clean" --expected "$WORK/expected" "${good_evidence[@]}" --sha 2222222222222222222222222222222222222222 --pr 7 --previous "$WORK/two-objects"
check "a forged two-object baseline still renders" yes "$(grep -q 'no comparable baseline' "$WORK/o11l/report.md" && echo yes || echo no)"

# 13. A findings array that holds something other than objects is not a
# gradeable record (orange), never a jq error that leaves no report.
{ rec a.k9.ncl pass '[]'; jq -nc '{file: "b.k9.ncl", verdict: "pass", findings: ["bad"]}'; } > "$WORK/r-bad-finding"
render "$WORK/o13a" --results "$WORK/r-bad-finding" --expected "$WORK/expected" "${good_evidence[@]}"
check "a non-object finding element is orange" orange "$(state "$WORK/o13a")"
check "the non-object finding names its file" "Unexpected, duplicate or invalid result." "$(reason "$WORK/o13a" b.k9.ncl)"

# 14. Contributor-controlled ids and validator text cannot write Markdown or
# HTML into the lane's own comment.
evil='![x](https://e.invalid/i)_[y]*.k9.ncl'
printf '%s\n' "$evil" > "$WORK/expected-evil"
rec "$evil" fail '[{"severity":"error","rule":"K9-S002","layer":"L1","message":"see <b>here</b> [z](https://e.invalid) \\ `x`"}]' > "$WORK/r-evil"
render "$WORK/o14" --results "$WORK/r-evil" --expected "$WORK/expected-evil" "${good_evidence[@]}"
check "markdown image syntax is escaped in the id" no "$(grep -q '!\[x\](' "$WORK/o14/report.md" && echo yes || echo no)"
check "the escaped id is still legible" yes "$(grep -q '&#33;&#91;x&#93;&#40;https://e.invalid/i&#41;&#95;&#91;y&#93;&#42;.k9.ncl' "$WORK/o14/report.md" && echo yes || echo no)"
check "markdown link syntax is escaped in the reason" no "$(grep -q '\[z\](https' "$WORK/o14/report.md" && echo yes || echo no)"
check "html and backslashes are escaped in the reason" yes "$(grep -q '&lt;b&gt;here&lt;/b&gt; &#91;z&#93;&#40;https://e.invalid&#41; &#92; &#96;x&#96;' "$WORK/o14/report.md" && echo yes || echo no)"
rec '@hyperpolymath/owners.k9.ncl' fail '[{"severity":"error","rule":"K9-S002","layer":"L1","message":"ping @someone"}]' > "$WORK/r-mention"
printf '%s\n' '@hyperpolymath/owners.k9.ncl' > "$WORK/expected-mention"
render "$WORK/o14b" --results "$WORK/r-mention" --expected "$WORK/expected-mention" "${good_evidence[@]}"
check "an @-mention in an id or a reason is not live" no "$(grep -q '@hyperpolymath\|@someone' "$WORK/o14b/report.md" && echo yes || echo no)"

# 12. Usage errors exit 2.
check "missing --results exits 2" 2 "$(bash "$GRID" render --out "$WORK/o12" >/dev/null 2>&1; echo $?)"
check "unknown subcommand exits 2" 2 "$(bash "$GRID" frobnicate >/dev/null 2>&1; echo $?)"
Expand Down
Loading