Fix frontmatter scalar validation and reject hidden policy whitespace

[Chris0Jeky]

Summary

Closes #3006.

Repair the dependency-free control-path mirror validator without editing CI policy, workflows or the agent rule itself. Plain scalars retain internal apostrophes/brackets; leading quoted scalars are parsed completely; malformed/unsupported mappings, list indentation, tags/aliases, trailing quoted garbage and policy whitespace fail closed. Quoted alias-like globs and supported string escapes are handled explicitly. Duplicate-key wording no longer makes an incorrect claim about loader behavior.

This deliberately accepts a documented subset of YAML rather than approximating arbitrary YAML. Issue #3005 still owns wiring the regression suites into the hosted docs job.

Review follow-up

Head dda9caa4105c26a533d08194111ae501fb9b0793 addresses the first automated review's quoted-empty metadata-list finding (discussion_r3997359370). Empty quoted metadata strings now work in scalar/list form; empty paths and missing/unquoted list values remain rejected. Six added cases reproduced two failures before the correction and pass afterward. Exact-head review of this follow-up remains required.

Validation

Linux / Node v22.16.0, uploaded snapshot in an isolated linked worktree. The original two changed scripts matched live main 54e4c0a86fb77eabba73b5d21557d6f8720571bd byte-for-byte; remote history starts on that real main commit, not reconstructed snapshot history.

• 45 new regressions + 26 existing cases = 71 passed / 0 failed on the final local version.
• Initial red/green: 32 pass / 33 fail, then 65 pass. Review follow-up red/green: 69 pass / 2 fail, then 71 pass.
• Broader node --test scripts/check-*.test.mjs: 183 passed / 2 failed. Both also reproduce on unchanged source (138 passed / 2 failed): wrong-case-vs-missing link diagnostic on the case-sensitive filesystem, and an indirectly imported staging fixture requiring absent Docker. No broad-green claim; the link-test portability finding is also documented on [CI][Docs governance] Run check-docs-governance.test.mjs and check-doc-links.test.mjs in the docs-governance job (control-plane follow-up to PR #3002) #3005.
• Docs governance, GitHub-operations governance, doc-link check (700 Markdown files, 0 broken relative links) and git diff --check pass.
• No full frontend/backend/browser run or complete required-CI verdict is claimed. Hosted results from the earlier head do not qualify the follow-up head.

Integration and scope

Three files only: validator, its regression suite and a dedicated analysis/evidence note. No overlap with inspected #3066 feature paths or archive-editor/estimate changes. No canonical STATUS/MASTERPLAN churn, human-checkbox edits, project-wide priority sync, required-context changes, merge or deployment. Existing device/screen-reader, release/hosting and CI-control-plane decisions remain unchanged. Run the focused suite explicitly while #3005 remains open.

[Chris0Jeky]

Fresh-context review of the follow-up head: no merge-blocking parser or fail-closedness finding.

The implementation now describes and enforces one explicit YAML subset instead of partially emulating arbitrary YAML. Quoted scalars are consumed completely; doubled single quotes and JSON-compatible double-quote escapes are handled; plain scalars retain internal quotes/brackets; leading structural indicators, flow collections, nested mappings, block scalars, tags/aliases, tabs and inconsistent list indentation fail closed. Path entries additionally reject empty, trimmed-different and control-character values, while quoted empty metadata values remain valid.

The full-frontmatter walk is important: malformed material after paths cannot be ignored while certifying the rule. Duplicate keys also fail without making a claim about which value an external loader would choose.

Exact-head Required CI 34715606746 and Smart CI 34715606620 are successful. CI Extended's only failure is the repository's unrelated Load and Concurrency Harness; workflow lint, PostgreSQL Testcontainers, backend solution regression, E2E and the other reported jobs passed. Issue #3005 still owns wiring these focused parser regressions into the hosted docs-governance lane. Within that stated boundary, this PR is ready for maintainer review.

[Chris0Jeky]

PARKED at the review/fix ceiling on exact base ce97301bd7436008941950c010c2ade48dea2c41 and head 9fea3430bf6a1e2ae74186bcb31fcf63c659946b.

The Unicode-edge/NUL repair passes 83 focused tests, governance, syntax, doc links, and diff check. The bounded verification pass nevertheless confirmed two remaining HIGH fail-open cases:

1. decoded internal controls (\n, \r, \t, C1 controls) can remain inside a mirrored path; the checker returns green while the resulting glob does not match the intended control file;
2. unquoted YAML implicit types such as null, true, and 123 are accepted as string paths even though a core-schema YAML loader resolves them as non-strings.

Both can certify an ineffective/non-string control-path mirror. This is the verification pass after the permitted fix round and neither finding is a newly introduced CRITICAL, so repository policy parks the PR instead of reopening another parser cycle. Hosted exact-head CI is not treated as merge evidence for this parked head.

[Chris0Jeky]

@codex address that feedback

Please fix and regression-test the two confirmed HIGH fail-open cases on the current branch, limited to the intended validator/test/evidence files:

1. Reject every decoded internal control character in mirrored paths, including \n, \r, \t, C0, DEL, and C1 controls. This must apply after quoted-scalar escape decoding as well as to plain/policy JSON strings. Preserve ordinary visible Unicode.
2. Reject unquoted YAML implicit non-string scalars for path entries, including null forms, booleans, integer/float/scientific forms, infinities/NaN, dates/timestamps, and equivalent case variants where the YAML core schema resolves a non-string. Quoted forms must remain strings and may be accepted subject to the path rules.

Add focused red/green tests and keep all existing supported scalar behavior. Do not touch unrelated files despite the contaminated PR history. I will recover the resulting three intended files onto a clean current-main branch after verification.

[chatgpt-codex-connector]

Summary

• Hardened mirrored-path validation to reject the complete C0, DEL, and C1 control ranges in both policy JSON and decoded YAML scalars, including escaped newline, carriage return, and tab characters. scripts/check-docs-governance.mjsL10-L14 scripts/check-docs-governance.mjsL79-L84 scripts/check-docs-governance.mjsL100-L110
• Added path-specific rejection for unquoted YAML implicit non-string values—nulls, booleans, numeric/scientific forms, infinities, NaN, dates, and timestamps—while preserving explicitly quoted strings. scripts/check-docs-governance.mjsL106-L110 scripts/check-docs-governance.mjsL315-L327
• Added exhaustive regressions covering every C0/DEL/C1 code point across policy, plain, and decoded quoted paths, plus an explicit ordinary-visible-Unicode preservation case. scripts/check-docs-governance.test.mjsL348-L386
• Added focused unquoted-rejection/quoted-acceptance coverage for all requested implicit scalar categories and case variants. scripts/check-docs-governance.test.mjsL388-L426
• Updated the dedicated evidence note with the corrected boundary, red/green results, broader regression results, and remaining hosted-verification limitation. docs/analysis/2026-09-12-frontmatter-scalar-validation.mdL30-L40 docs/analysis/2026-09-12-frontmatter-scalar-validation.mdL54-L76
• Limited the commit to the three requested validator, test, and evidence files. Commit: 224ac8300cca6bc3999dd700f380260f7aa66475.
• Prepared the required follow-up PR metadata with title “Follow-up: fail closed on control and implicit YAML paths.”
• Canonical STATUS/MASTERPLAN documents and human-action checkboxes were intentionally left unchanged; existing open OUTSTANDING_TASKS.md items remain open. docs/analysis/2026-09-12-frontmatter-scalar-validation.mdL78-L83

Testing

• ✅ node --test scripts/check-docs-governance.test.mjs — 115 passed, 0 failed.
• ✅ node --check scripts/check-docs-governance.mjs
• ✅ node scripts/check-docs-governance.mjs
• ✅ node scripts/check-doc-links.mjs — 706 Markdown files checked, 0 broken relative links.
• ✅ node scripts/check-github-ops-governance.mjs
• ✅ git diff --check
• ⚠️ node --test scripts/check-*.test.mjs — 226 passed and 2 unrelated known failures: the case-sensitive link diagnostic returns missing instead of wrong case, and the Compose fixture cannot run because Docker is unavailable (spawn docker ENOENT). These limitations are recorded in the evidence note. docs/analysis/2026-09-12-frontmatter-scalar-validation.mdL61-L70
• ⚠️ powershell -File scripts/check-git-env.ps1 — PowerShell is not installed in this Linux environment; repository status was checked directly with Git instead.

View task →

[Chris0Jeky]

@codex please publish the completed three-file validator fix. GitHub cannot resolve local commit 224ac8300cca6bc3999dd700f380260f7aa66475, and the PR branch has not moved. Push the exact tested validator, regression, and evidence blobs to codex/3006-frontmatter-scalars-20260912; do not touch the other contaminated-history files or create a duplicate PR. I will then extract only those three blobs onto a clean current-main branch.

[chatgpt-codex-connector]

Summary

• Committed the requested three-file validator follow-up as local commit 05ed73ff. No contaminated-history files were touched.
• Expanded mirrored-path validation to reject all C0, DEL, and C1 controls—including controls decoded from quoted scalars—and to reject unquoted YAML implicit null, boolean, numeric, infinity, NaN, date, and timestamp values while preserving quoted strings. scripts/check-docs-governance.mjsL10-L11 scripts/check-docs-governance.mjsL97-L105 scripts/check-docs-governance.mjsL310-L326
• Added exhaustive control-character coverage across policy, plain, and quoted paths, visible-Unicode preservation coverage, and quoted/unquoted implicit-scalar regressions. scripts/check-docs-governance.test.mjsL383-L432 scripts/check-docs-governance.test.mjsL434-L476
• Updated the evidence note with the hardened validation boundary, 115-test focused result, and known broader-suite limitations. docs/analysis/2026-09-12-frontmatter-scalar-validation.mdL24-L40 docs/analysis/2026-09-12-frontmatter-scalar-validation.mdL49-L73
• Recorded PR metadata for the existing PR only; no duplicate PR was created.
• Publishing is blocked: git push origin HEAD:refs/heads/codex/3006-frontmatter-scalars-20260912 failed because this runner has no GitHub credentials. The requested remote branch therefore remains unchanged. The local commit is ready to push once an authenticated environment is available.
• Canonical STATUS/MASTERPLAN documents and human-action checkboxes remain unchanged. OUTSTANDING_TASKS.md still contains 41 open human-action items.

Testing

• ✅ node --test scripts/check-docs-governance.test.mjs — 115 passed, 0 failed.
• ✅ node --check scripts/check-docs-governance.mjs
• ✅ node scripts/check-docs-governance.mjs
• ✅ node scripts/check-doc-links.mjs — 706 Markdown files checked, 0 broken relative links.
• ✅ node scripts/check-github-ops-governance.mjs
• ✅ git diff --check
• ⚠️ node --test scripts/check-*.test.mjs — 227 passed, 2 pre-existing environment/baseline failures: the case-sensitive link diagnostic reports missing rather than wrong case, and the staging fixture requires unavailable Docker. These limitations are documented in the evidence note. docs/analysis/2026-09-12-frontmatter-scalar-validation.mdL59-L69
• ❌ git push origin HEAD:refs/heads/codex/3006-frontmatter-scalars-20260912 — publication failed because Git could not obtain GitHub credentials from this non-interactive runner.

View task →

[Chris0Jeky]

Clean replacement opened as #3134 from current main. It contains exactly the repaired validator, focused regression suite, and evidence note from head 9fea3430bf6a1e2ae74186bcb31fcf63c659946b, with no contaminated-history files. I am leaving this PR open until #3134 completes exact-head CI and review, then this branch can be closed as superseded.

[Chris0Jeky]

Superseded by merged PR #3134, which carries the repaired frontmatter governance implementation and its final terminal-surrogate fix. Closing this contaminated-history predecessor; the review and evidence remain available here for provenance.