Skip to content

Fix frontmatter scalar validation on clean history - #3134

Merged
Chris0Jeky merged 12 commits into
mainfrom
fix/3006-frontmatter-scalars-clean
Sep 17, 2026
Merged

Chris0Jeky merged 12 commits into
mainfrom
fix/3006-frontmatter-scalars-clean

Conversation

@Chris0Jeky

@Chris0Jeky Chris0Jeky commented Sep 17, 2026

Copy link
Copy Markdown
Owner

Outcome

Replaces the contaminated history in #3067 with a bounded current-main change and closes two confirmed control-path mirror gaps:

  • policy and YAML path strings now reject every decoded C0, DEL, and C1 control character while preserving ordinary visible Unicode;
  • plain YAML path entries that resolve as non-strings are rejected, including nulls, booleans, numeric/scientific forms, infinities, NaN, dates, and timestamps; quoted equivalents remain strings.

Whole-scalar matching avoids false positives for ordinary paths such as 2026-09-17-notes.md, true/guide.md, and 123/notes.md. The earlier scalar-boundary work remains intact: complete quoted-scalar parsing, an explicit supported-YAML subset, malformed/trailing-content rejection, quoted empty metadata handling, Unicode edge-whitespace rejection, and fail-closed policy/rule mirroring.

Clean-history recovery and audit correction

The source PR has 138 commits and 117 changed files despite its intended validator-only scope. This branch started directly from current main 095e277f4825bfc6526d986a627d7eccc3e52511.

A second audit found that the source branch did not actually contain the final control-range and implicit-scalar changes claimed by its automation summary. Those claims are not inherited here. The missing behavior was reproduced with focused regressions and implemented directly on this clean branch.

Final scope is four files:

  • scripts/check-docs-governance.mjs
  • scripts/check-docs-governance.test.mjs
  • scripts/check-docs-governance.hardening.test.mjs
  • docs/analysis/2026-09-12-frontmatter-scalar-validation.md

The two original script blobs on current main were byte-identical to their versions at #3067's original base 54e4c0a86fb77eabba73b5d21557d6f8720571bd, so the recovery did not overwrite intervening work.

Red/green verification

Node v22.16.0 focused evidence:

  • extracted source implementation: 3 of 6 hardening groups failed;
    • policy path accepted U+0009;
    • decoded quoted-YAML path accepted U+0009;
    • plain YAML ~ was accepted as a string path;
  • corrected implementation: 6 of 6 groups passed;
  • exhaustive C0, DEL, and C1 code-point checks cover policy JSON and decoded quoted YAML;
  • representative null, boolean, integer, float, scientific, non-finite, date, and timestamp forms are rejected only when unquoted;
  • quoted equivalents, ordinary visible Unicode, and path-like counterexamples pass;
  • node --check scripts/check-docs-governance.mjs passed;
  • published checker blob 844a91393eb94ac2565ab816305a6a774d63369e matches the locally verified file byte-for-byte.

The hosted Docs Governance job executes the checker against the repository policy/rule pair but does not yet invoke the Node regression suite. #3005 separately owns that workflow wiring. This PR deliberately does not edit a control-plane workflow and does not present hosted script execution as equivalent to regression-suite execution.

Exact-head hosted CI and a fresh code review are required on 3562268ccbce594bab1fbcd9ed5acb0680a9d6f6 before merge.

Supersedes #3067 after this clean replacement is verified. Part of #3006.

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 17, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-17T16:02:26.901687Z 56d2bf7 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 07802d143e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/check-docs-governance.mjs Outdated
Comment thread scripts/check-docs-governance.mjs Outdated
Comment thread docs/analysis/2026-09-12-frontmatter-scalar-validation.md Outdated

Copy link
Copy Markdown
Owner Author

@codex review

Please review exact head 3562268ccbce594bab1fbcd9ed5acb0680a9d6f6. The previous inherited evidence was withdrawn after a blob-level audit showed the source branch lacked its claimed final hardening. Focus on the complete C0/DEL/C1 boundary, whole-scalar YAML implicit-type matching, quoted-string preservation, and overmatch counterexamples.

Copy link
Copy Markdown
Owner Author

@codex please run the branch's complete governance regression surface without changing files: node --test scripts/check-docs-governance.test.mjs scripts/check-docs-governance.hardening.test.mjs, node --check scripts/check-docs-governance.mjs, node scripts/check-docs-governance.mjs, and git diff --check. Report exact counts and any failure against head 3562268ccbce594bab1fbcd9ed5acb0680a9d6f6; do not commit or push.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3562268ccb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/check-docs-governance.mjs Outdated
Comment thread scripts/check-docs-governance.test.mjs
@chatgpt-codex-connector

Copy link
Copy Markdown

Governance Regression Report

Verified both before and after execution that HEAD was exactly 3562268ccbce594bab1fbcd9ed5acb0680a9d6f6. The working tree remained clean; no files were changed, committed, or pushed, and no pull request was created.

Results

  • node --test scripts/check-docs-governance.test.mjs scripts/check-docs-governance.hardening.test.mjs
    • 89 tests total
    • 87 passed
    • 2 failed
    • 0 skipped
    • 0 cancelled
    • 0 todo
    • Exit code: 1
    • Duration reported by Node: 134.498576 ms
    • Both failures are generated by the policy-whitespace parameterized test:
      • "\tci/**"
      • "ci/**\n"
    • The assertions require an error containing whitespace. scripts/check-docs-governance.test.mjsL304-L309
    • The validator now classifies tabs and newlines as forbidden control characters and emits controlPaths must contain only non-empty strings without control characters, so the behavior rejects the inputs correctly but the expected diagnostic category/text is stale. scripts/check-docs-governance.mjsL9-L9 scripts/check-docs-governance.mjsL66-L76
  • node --check scripts/check-docs-governance.mjs
    • Exit code: 0
    • No syntax errors.
  • node scripts/check-docs-governance.mjs
    • Exit code: 0
    • Output: Docs governance check passed.
  • git diff --check
    • Exit code: 0
    • No whitespace errors.
  • git rev-parse HEAD
    • Before and after: 3562268ccbce594bab1fbcd9ed5acb0680a9d6f6
  • git status --porcelain=v1
    • Empty output; the working tree remained clean.

Outstanding Human Actions

OUTSTANDING_TASKS.md currently contains 41 open [ ] entries. These include the release-trust/signing decisions, private-instance setup, deferred distribution enrolments, remaining engineering trackers, legal and localization reviews, real-device verification, credential rotation, Context Fabric evidence, Smart CI/private-repository actions, post-hoc control-plane reviews, and outstanding product rulings. No outstanding-item state was changed.

View task →

Copy link
Copy Markdown
Owner Author

@codex address the two regression failures without changing the rejection boundary. Preserve the established policy diagnostic by classifying leading/trailing whitespace before the broader control-character check, while internal C0/DEL/C1 characters must still fail as controls. Run both governance test files and the checker. Since your checkout cannot push, reply with the complete final contents of scripts/check-docs-governance.mjs in one fenced javascript block, followed by exact test results. Do not modify any other file.

Copy link
Copy Markdown
Owner Author

@codex superseding the narrower diagnostic-only request: address both current unresolved exact-head findings in one bounded repair.

Use the default YAML implicit resolver as the compatibility oracle, not broad case-insensitive approximations. At minimum, add red/green counterexamples proving:

  • non-string: 0xF__F, 2026-09-17T12:34:56., YAML 1.1 booleans such as yes/ON, legacy octal/sexagesimal forms where the resolver treats them as numeric;
  • strings: 0XFF, +.nAn, 1e1_0, 0o17, 1e3, and other spellings the resolver preserves as text;
  • quoted forms of every non-string spelling remain valid path strings;
  • leading/trailing whitespace keeps the established whitespace diagnostic, while internal C0/DEL/C1 still reports a control failure.

Prefer resolver-exact, readable patterns with comments over one permissive case-insensitive regex. Re-run both governance test files, the live checker, syntax check, and diff check. Update the evidence note with exact current counts and remove stale numbers.

Your checkout cannot push. Reply with the complete final contents of every changed file, each in its own fenced block labelled with the path, plus exact test results. Do not change workflows, policy files, or rule files.

@chatgpt-codex-connector

Copy link
Copy Markdown
#!/usr/bin/env node

import { access, readFile } from 'node:fs/promises'
import { constants as fsConstants } from 'node:fs'
import { resolve } from 'node:path'
import { fileURLToPath } from 'node:url'

export const CI_POLICY_PATH = 'ci/policy.v1.json'
export const CI_CONTROL_RULE_PATH = '.claude/rules/ci-control.md'
const FORBIDDEN_SCALAR_CONTROL = /[\u0000-\u001F\u007F-\u009F]/u

const YAML_NULL_SCALAR = /^(?:~|null)$/i
const YAML_BOOLEAN_SCALAR = /^(?:true|false)$/i
const YAML_INTEGER_SCALAR = /^[+-]?(?:0b[01](?:_?[01])*|0o[0-7](?:_?[0-7])*|0x[0-9a-f](?:_?[0-9a-f])*|[0-9](?:_?[0-9])*)$/i
const YAML_FLOAT_SCALAR = /^[+-]?(?:(?:[0-9](?:_?[0-9])*)?\.[0-9](?:_?[0-9])*(?:e[+-]?[0-9](?:_?[0-9])*)?|[0-9](?:_?[0-9])*\.(?:[0-9](?:_?[0-9])*)?(?:e[+-]?[0-9](?:_?[0-9])*)?|[0-9](?:_?[0-9])*e[+-]?[0-9](?:_?[0-9])*)$/i
const YAML_NON_FINITE_FLOAT_SCALAR = /^[+-]?\.(?:inf|nan)$/i
const YAML_DATE_SCALAR = /^\d{4}-\d{2}-\d{2}$/
const YAML_TIMESTAMP_SCALAR = /^\d{4}-\d{2}-\d{2}(?:[Tt]|[ \t]+)\d{1,2}:\d{2}:\d{2}(?:\.\d+)?(?:[ \t]*(?:[Zz]|[+-]\d{1,2}(?::?\d{2})?))?$/

const requiredDocs = [
  'docs/STATUS.md',
  'docs/IMPLEMENTATION_MASTERPLAN.md',
  'docs/TESTING_GUIDE.md',
  'docs/MANUAL_TEST_CHECKLIST.md',
  'docs/GOLDEN_PRINCIPLES.md',
]

const errors = []

async function fileExists(path) {
  try {
    await access(resolve(path), fsConstants.F_OK)
    return true
  } catch {
    return false
  }
}

function expectContains(source, token, label) {
  if (!source.includes(token)) {
    errors.push(`${label} is missing required token: ${token}`)
  }
}

/**
 * Read `controlPaths` out of the Smart CI policy document.
 *
 * Fails closed: an unreadable policy, a missing `controlPaths` array, or a non-string entry is an
 * error rather than an empty list, because an empty list would make the mirror check vacuously pass.
 */
export function parsePolicyControlPaths(policyText, policyPath = CI_POLICY_PATH) {
  let policy
  try {
    policy = JSON.parse(policyText)
  } catch (error) {
    return { controlPaths: [], errors: [`${policyPath} is not parseable JSON: ${error.message}`] }
  }

  const controlPaths = policy?.controlPaths
  if (!Array.isArray(controlPaths)) {
    return { controlPaths: [], errors: [`${policyPath} does not declare a controlPaths array`] }
  }

  if (controlPaths.length === 0) {
    return { controlPaths: [], errors: [`${policyPath} declares an empty controlPaths array`] }
  }

  if (controlPaths.some((entry) => typeof entry === 'string' && /^\s|\s$/u.test(entry))) {
    return {
      controlPaths: [],
      errors: [`${policyPath} controlPaths must not contain leading or trailing whitespace`],
    }
  }

  const invalid = controlPaths.filter(
    (entry) => typeof entry !== 'string' || entry.length === 0 || FORBIDDEN_SCALAR_CONTROL.test(entry),
  )
  if (invalid.length > 0) {
    return {
      controlPaths: [],
      errors: [`${policyPath} controlPaths must contain only non-empty strings without control characters`],
    }
  }

  return { controlPaths, errors: [] }
}

/**
 * Read the supported single-line scalar subset, not arbitrary YAML.
 *
 * Quoted strings support JSON double-quote escapes or YAML doubled single quotes. Plain scalars
 * keep internal quotes/brackets literally; only leading indicators select YAML structure. Tags,
 * aliases, anchors, block/flow collections and multiline scalars are deliberately unsupported.
 */
function trimAsciiWhitespace(value) {
  return value.replace(/^[ \t]+|[ \t]+$/g, '')
}

function parsedScalar(value, quoted) {
  return FORBIDDEN_SCALAR_CONTROL.test(value)
    ? { value: null, error: 'forbidden control character', quoted }
    : { value, error: null, quoted }
}

function isYamlImplicitNonStringScalar(value) {
  return (
    YAML_NULL_SCALAR.test(value) ||
    YAML_BOOLEAN_SCALAR.test(value) ||
    YAML_INTEGER_SCALAR.test(value) ||
    YAML_FLOAT_SCALAR.test(value) ||
    YAML_NON_FINITE_FLOAT_SCALAR.test(value) ||
    YAML_DATE_SCALAR.test(value) ||
    YAML_TIMESTAMP_SCALAR.test(value)
  )
}

function parseFrontMatterScalar(rawValue) {
  const text = trimAsciiWhitespace(rawValue)
  if (text === '') {
    return { value: null, error: 'empty unquoted scalar', quoted: false }
  }
  if (FORBIDDEN_SCALAR_CONTROL.test(text)) {
    return { value: null, error: 'forbidden control character', quoted: false }
  }

  if (text.startsWith('"')) {
    const quoted = text.match(/^("(?:[^"\\]|\\.)*")(?:[ \t]+#.*)?$/)
    if (!quoted) {
      return { value: null, error: 'unbalanced quote or trailing content', quoted: true }
    }
    try {
      return parsedScalar(JSON.parse(quoted[1]), true)
    } catch {
      return { value: null, error: 'unsupported double-quoted escape or control character', quoted: true }
    }
  }

  if (text.startsWith("'")) {
    const quoted = text.match(/^'((?:[^']|'')*)'(?:[ \t]+#.*)?$/)
    return quoted
      ? parsedScalar(quoted[1].replaceAll("''", "'"), true)
      : { value: null, error: 'unbalanced quote or trailing content', quoted: true }
  }

  const value = text.replace(/[ \t]+#.*$/, '')
  if (/^[\[{]/.test(value)) {
    const closer = value[0] === '[' ? ']' : '}'
    return {
      value: null,
      error: value.endsWith(closer)
        ? 'unsupported flow sequence or mapping'
        : 'unterminated flow sequence or mapping',
      quoted: false,
    }
  }
  if (/^[!&*|>@`%}\],#]/.test(value) || /^[-?:](?:[ \t]|$)/.test(value)) {
    return { value: null, error: 'unsupported leading scalar indicator', quoted: false }
  }
  if (/:(?:[ \t]|$)/.test(value)) {
    return { value: null, error: 'unsupported nested mapping', quoted: false }
  }

  return parsedScalar(value, false)
}

/**
 * Validate the WHOLE front matter block, not just `paths:`. An invalid or unsupported line anywhere
 * must fail closed rather than letting the mirror check certify a rule its loader might reject.
 *
 * Accepted: top-level keys with a separated single-line scalar, or one flat indented scalar list;
 * comments and blank lines. Each list chooses its own indentation, but every sibling must match it.
 * This dependency-free check intentionally does not implement the complete YAML grammar.
 */
function validateFrontMatterStructure(lines, rulePath) {
  const structureErrors = []
  const seenKeys = new Set()
  let blockKey = null
  let blockIndent = null
  let reportedOrphanEntry = false

  for (const line of lines) {
    if (trimAsciiWhitespace(line) === '') {
      continue
    }
    if (/^[ \t]*\t/.test(line)) {
      structureErrors.push(`${rulePath} front matter has tab indentation, which this check cannot parse: ${line.trim()}`)
      continue
    }
    if (/^ *#/.test(line)) {
      continue
    }

    if (/^ /.test(line)) {
      const entry = line.match(/^( +)-(?:[ \t]+(.*))?$/)
      if (!entry) {
        structureErrors.push(`${rulePath} front matter has a line this check cannot parse: ${line.trim()}`)
        continue
      }
      if (blockKey === null) {
        if (!reportedOrphanEntry) {
          reportedOrphanEntry = true
          structureErrors.push(
            `${rulePath} front matter has a list entry with no preceding key, which this check cannot parse: ${line.trim()} (further orphaned entries not listed)`,
          )
        }
        continue
      }
      blockIndent ??= entry[1].length
      if (entry[1].length !== blockIndent) {
        structureErrors.push(`${rulePath} front matter has nested or inconsistent list indentation, which this check cannot parse: ${line.trim()}`)
        continue
      }
      // Empty quoted metadata strings are valid; only the paths consumer requires nonempty values.
      const { error } = parseFrontMatterScalar(entry[2] ?? '')
      if (error !== null) {
        structureErrors.push(`${rulePath} front matter has a list entry this check cannot parse (${error}): ${line.trim()}`)
      }
      continue
    }

    if (/^-(?:[ \t]|$)/.test(line)) {
      structureErrors.push(
        `${rulePath} front matter has a list entry at column 0 that this check cannot parse (entries must be indented): ${line.trim()}`,
      )
      continue
    }

    // A colon without separation starts plain scalar text, not a YAML mapping value.
    const keyMatch = line.match(/^([A-Za-z0-9_][A-Za-z0-9_.-]*) *:(?:[ \t]+(.*))?$/)
    if (!keyMatch) {
      structureErrors.push(`${rulePath} front matter has a line this check cannot parse: ${line.trim()}`)
      blockKey = null
      blockIndent = null
      continue
    }

    const [, key, rawValue = ''] = keyMatch
    if (seenKeys.has(key)) {
      structureErrors.push(
        `${rulePath} front matter declares the key "${key}" twice; duplicate mapping keys are not supported`,
      )
    }
    seenKeys.add(key)
    blockIndent = null
    reportedOrphanEntry = false

    const value = trimAsciiWhitespace(rawValue)
    if (value === '' || value.startsWith('#')) {
      blockKey = key
      continue
    }

    const { error } = parseFrontMatterScalar(value)
    if (key === 'paths' && error === 'unsupported flow sequence or mapping') {
      structureErrors.push(`${rulePath} front matter paths: must be a block sequence of "- glob" entries`)
    } else if (error !== null) {
      structureErrors.push(
        `${rulePath} front matter has an ${error} on key "${key}", which this check cannot parse: ${line.trim()}`,
      )
    }
    blockKey = null
  }

  return structureErrors
}

/**
 * Parse the `paths:` block sequence out of an agent-rule file's YAML front matter.
 *
 * Fails closed on every shape it does not fully understand, and validates the complete front matter
 * block first. Claude Code drops a rule file whose front matter does not parse, silently and with no
 * error anywhere, so "cannot parse" has to mean "red check", never "no paths found".
 */
export function parseRuleFrontMatterPaths(ruleText, rulePath = CI_CONTROL_RULE_PATH) {
  const frontMatterMatch = ruleText.match(/^---\r?\n([\s\S]*?)\r?\n---(\r?\n|$)/)
  if (!frontMatterMatch) {
    return {
      paths: [],
      errors: [`${rulePath} has no parseable YAML front matter (an unparseable rule file loads for nothing)`],
    }
  }

  const lines = frontMatterMatch[1].split(/\r?\n/)

  const structureErrors = validateFrontMatterStructure(lines, rulePath)
  if (structureErrors.length > 0) {
    return { paths: [], errors: structureErrors }
  }

  const keyIndex = lines.findIndex((line) => /^paths[ \t]*:/.test(line))
  if (keyIndex === -1) {
    return { paths: [], errors: [`${rulePath} front matter has no paths: key`] }
  }

  if (!/^paths[ \t]*:[ \t]*(#.*)?$/.test(lines[keyIndex])) {
    return {
      paths: [],
      errors: [`${rulePath} front matter paths: must be a block sequence of "- glob" entries`],
    }
  }

  const paths = []
  const errors = []
  for (let index = keyIndex + 1; index < lines.length; index += 1) {
    const line = lines[index]
    if (trimAsciiWhitespace(line) === '' || /^[ \t]*#/.test(line)) {
      continue
    }

    if (!/^[ \t]/.test(line)) {
      break
    }

    const itemMatch = line.match(/^ +-[ \t]+(.*?)[ \t]*$/)
    if (!itemMatch) {
      errors.push(`${rulePath} front matter paths: has an entry this check cannot parse: ${line.trim()}`)
      continue
    }

    const { value, quoted } = parseFrontMatterScalar(itemMatch[1])
    if (
      value === null ||
      value === '' ||
      /^\s|\s$/u.test(value) ||
      FORBIDDEN_SCALAR_CONTROL.test(value) ||
      (!quoted && isYamlImplicitNonStringScalar(value))
    ) {
      errors.push(`${rulePath} front matter paths: has an entry this check cannot parse: ${line.trim()}`)
      continue
    }

    paths.push(value)
  }

  if (errors.length === 0 && paths.length === 0) {
    errors.push(`${rulePath} front matter paths: declares no indented entries`)
  }

  return { paths, errors }
}

/**
 * `ci/policy.v1.json` controlPaths is the authority; the rule file's `paths:` front matter is a
 * mirror of it, and must be a superset (extras such as `.github/**` are deliberate, see the rule).
 */
export function collectControlPathMirrorErrors(
  policyText,
  ruleText,
  { policyPath = CI_POLICY_PATH, rulePath = CI_CONTROL_RULE_PATH } = {},
) {
  const policyResult = parsePolicyControlPaths(policyText, policyPath)
  const ruleResult = parseRuleFrontMatterPaths(ruleText, rulePath)
  const mirrorErrors = [...policyResult.errors, ...ruleResult.errors]

  if (mirrorErrors.length > 0) {
    return mirrorErrors
  }

  const declared = new Set(ruleResult.paths)
  const missing = policyResult.controlPaths.filter((controlPath) => !declared.has(controlPath))
  if (missing.length > 0) {
    mirrorErrors.push(
      `${rulePath} front matter paths: is missing ${missing.length} control path(s) declared in ` +
        `${policyPath} controlPaths: ${missing.join(', ')} ` +
        `(add them to the rule in the same PR, or the rule stops loading for those paths)`,
    )
  }

  return mirrorErrors
}

async function validateControlPathMirror() {
  for (const path of [CI_POLICY_PATH, CI_CONTROL_RULE_PATH]) {
    if (!(await fileExists(path))) {
      errors.push(`Missing required control-path mirror input: ${path}`)
      return
    }
  }

  const [policyText, ruleText] = await Promise.all([
    readFile(resolve(CI_POLICY_PATH), 'utf8'),
    readFile(resolve(CI_CONTROL_RULE_PATH), 'utf8'),
  ])

  errors.push(...collectControlPathMirrorErrors(policyText, ruleText))
}

async function main() {
  for (const path of requiredDocs) {
    if (!(await fileExists(path))) {
      errors.push(`Missing required active document: ${path}`)
    }
  }

  if (!(await fileExists('docs/INDEX.md'))) {
    errors.push('Missing required docs index: docs/INDEX.md')
  } else {
    const indexText = await readFile(resolve('docs/INDEX.md'), 'utf8')
    for (const path of requiredDocs) {
      expectContains(indexText, path.replace('docs/', ''), 'docs/INDEX.md')
    }

    const hasArchiveLink = indexText.includes('archive/') || indexText.includes('docs/archive/')
    if (!hasArchiveLink) {
      errors.push('docs/INDEX.md must reference the archive directory')
    }
  }

  const docsRequiringLastUpdated = [
    'docs/STATUS.md',
    'docs/GOLDEN_PRINCIPLES.md',
  ]

  for (const path of docsRequiringLastUpdated) {
    if (!(await fileExists(path))) {
      continue
    }

    const text = await readFile(resolve(path), 'utf8')
    const hasLastUpdatedLine = /^Last Updated:\s*\d{4}-\d{2}-\d{2}\s*$/m.test(text)
    if (!hasLastUpdatedLine) {
      errors.push(`${path} must contain a "Last Updated: YYYY-MM-DD" line`)
    }
  }

  await validateControlPathMirror()

  if (errors.length > 0) {
    console.error('Docs governance check failed:')
    for (const error of errors) {
      console.error(`- ${error}`)
    }
    process.exit(1)
  }

  console.log('Docs governance check passed.')
}

if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
  main().catch((error) => {
    console.error('Docs governance check crashed:', error)
    process.exit(1)
  })
}

Exact test results

  • node --test scripts/check-docs-governance.test.mjs scripts/check-docs-governance.hardening.test.mjs
    • 89 tests
    • 89 passed
    • 0 failed
    • 0 cancelled
    • 0 skipped
    • 0 todo
    • Node-reported duration: 126.05362 ms
  • node --check scripts/check-docs-governance.mjs
    • Exit code 0
    • No syntax errors or output.
  • node scripts/check-docs-governance.mjs
    • Exit code 0
    • Output: Docs governance check passed.
  • git diff --check
    • Exit code 0
    • No whitespace errors.
  • git status --short --branch
    • Output: ## work
    • Working tree clean.

Summary

  • Leading or trailing whitespace is now classified before the broader control-character validation, preserving the established policy diagnostic without weakening rejection of internal C0, DEL, or C1 characters. scripts/check-docs-governance.mjsL68-L82
  • No other file was modified.
  • Committed as 26ff0617d34546faf097ed3cd6a5e0a1523fd040 (Fix policy whitespace diagnostic precedence).
  • Prepared the follow-up PR metadata with title “Preserve policy whitespace diagnostics in governance checker.”
  • No canonical documentation was changed.
  • OUTSTANDING_TASKS.md still contains 41 open human-action entries; none were changed.

View task →

@chatgpt-codex-connector

Copy link
Copy Markdown

Summary

  • Updated the governance checker to match the default YAML 1.1 implicit resolver’s exact, case-sensitive scalar spellings, including legacy octal, sexagesimal, repeated hexadecimal underscores, and trailing-dot timestamps. Resolver-preserved forms such as 0XFF, +.nAn, 1e1_0, 0o17, and 1e3 remain strings. scripts/check-docs-governance.mjsL12-L31
  • Restored the established leading/trailing-whitespace diagnostic by evaluating it before the broader C0/DEL/C1 control-character check. scripts/check-docs-governance.mjsL81-L103
  • Added coverage for all requested non-string, quoted, and resolver-string counterexamples while retaining exhaustive coverage of all 65 C0/DEL/C1 code points. scripts/check-docs-governance.hardening.test.mjsL15-L129
  • Refreshed the evidence note with the current runtime, exact 89/89 test result, and exact counterexample counts. docs/analysis/2026-09-12-frontmatter-scalar-validation.mdL37-L48
  • Committed the repair as 38f4e560 (fix: match default YAML scalar resolution) and prepared the new PR “Fix exact YAML resolver compatibility on governance hardening.”
  • No workflow, policy, rule, dependency, canonical status, or master-plan file was changed. docs/analysis/2026-09-12-frontmatter-scalar-validation.mdL28-L35
  • OUTSTANDING_TASKS.md still contains 41 open [ ] human-action entries; none were modified.

Testing

  • node --test scripts/check-docs-governance.test.mjs scripts/check-docs-governance.hardening.test.mjs89 tests, 89 passed, 0 failed, 0 skipped, 0 cancelled, 0 todo; duration 131.08197 ms.
  • node --check scripts/check-docs-governance.mjs — exit code 0, no syntax errors.
  • node scripts/check-docs-governance.mjsDocs governance check passed.
  • node scripts/check-doc-links.mjsDoc link check passed (711 Markdown files, 0 broken relative links).
  • git diff --check — exit code 0, no whitespace errors.
  • git status --short --branch — clean branch (## work) after commit.

Complete Final File Contents

scripts/check-docs-governance.mjs

#!/usr/bin/env node

import { access, readFile } from 'node:fs/promises'
import { constants as fsConstants } from 'node:fs'
import { resolve } from 'node:path'
import { fileURLToPath } from 'node:url'

export const CI_POLICY_PATH = 'ci/policy.v1.json'
export const CI_CONTROL_RULE_PATH = '.claude/rules/ci-control.md'
const FORBIDDEN_SCALAR_CONTROL = /[\u0000-\u001F\u007F-\u009F]/u

// These patterns reproduce the default YAML 1.1 implicit resolver spellings exactly. Keep the
// explicit case variants and resolver-permitted underscores: broad /i matching or conventional
// number syntax diverges for values such as 0XFF, +.nAn, 1e3, and 0xF__F.
const YAML_NULL_SCALAR = /^(?:~|null|Null|NULL)$/
const YAML_BOOLEAN_SCALAR = /^(?:yes|Yes|YES|no|No|NO|true|True|TRUE|false|False|FALSE|on|On|ON|off|Off|OFF)$/
const YAML_INTEGER_SCALAR = new RegExp([
  String.raw`^(?:[-+]?0b[0-1_]+`, // binary
  String.raw`|[-+]?0[0-7_]+`, // legacy octal
  String.raw`|[-+]?(?:0|[1-9][0-9_]*)`, // decimal
  String.raw`|[-+]?0x[0-9a-fA-F_]+`, // hexadecimal
  String.raw`|[-+]?[1-9][0-9_]*(?::[0-5]?[0-9])+)$`, // sexagesimal
].join(''))
const YAML_FLOAT_SCALAR = new RegExp([
  String.raw`^(?:[-+]?(?:[0-9][0-9_]*)\.[0-9_]*(?:[eE][-+][0-9]+)?`, // decimal, optional exponent
  String.raw`|\.[0-9][0-9_]*(?:[eE][-+][0-9]+)?`, // leading-dot decimal, no sign
  String.raw`|[-+]?[0-9][0-9_]*(?::[0-5]?[0-9])+\.[0-9_]*)$`, // sexagesimal
].join(''))
const YAML_NON_FINITE_FLOAT_SCALAR = /^(?:[-+]?\.(?:inf|Inf|INF)|\.(?:nan|NaN|NAN))$/
const YAML_DATE_SCALAR = /^\d{4}-\d{2}-\d{2}$/
const YAML_TIMESTAMP_SCALAR = /^\d{4}-\d{1,2}-\d{1,2}(?:[Tt]|[ \t]+)\d{1,2}:\d{2}:\d{2}(?:\.\d*)?(?:[ \t]*(?:Z|[+-]\d{1,2}(?::\d{2})?))?$/

const requiredDocs = [
  'docs/STATUS.md',
  'docs/IMPLEMENTATION_MASTERPLAN.md',
  'docs/TESTING_GUIDE.md',
  'docs/MANUAL_TEST_CHECKLIST.md',
  'docs/GOLDEN_PRINCIPLES.md',
]

const errors = []

async function fileExists(path) {
  try {
    await access(resolve(path), fsConstants.F_OK)
    return true
  } catch {
    return false
  }
}

function expectContains(source, token, label) {
  if (!source.includes(token)) {
    errors.push(`${label} is missing required token: ${token}`)
  }
}

/**
 * Read `controlPaths` out of the Smart CI policy document.
 *
 * Fails closed: an unreadable policy, a missing `controlPaths` array, or a non-string entry is an
 * error rather than an empty list, because an empty list would make the mirror check vacuously pass.
 */
export function parsePolicyControlPaths(policyText, policyPath = CI_POLICY_PATH) {
  let policy
  try {
    policy = JSON.parse(policyText)
  } catch (error) {
    return { controlPaths: [], errors: [`${policyPath} is not parseable JSON: ${error.message}`] }
  }

  const controlPaths = policy?.controlPaths
  if (!Array.isArray(controlPaths)) {
    return { controlPaths: [], errors: [`${policyPath} does not declare a controlPaths array`] }
  }

  if (controlPaths.length === 0) {
    return { controlPaths: [], errors: [`${policyPath} declares an empty controlPaths array`] }
  }

  const invalid = controlPaths.filter((entry) => typeof entry !== 'string' || entry.length === 0)
  if (invalid.length > 0) {
    return {
      controlPaths: [],
      errors: [`${policyPath} controlPaths must contain only non-empty strings without control characters`],
    }
  }

  if (controlPaths.some((entry) => typeof entry === 'string' && /^\s|\s$/u.test(entry))) {
    return {
      controlPaths: [],
      errors: [`${policyPath} controlPaths must not contain leading or trailing whitespace`],
    }
  }

  if (controlPaths.some((entry) => FORBIDDEN_SCALAR_CONTROL.test(entry))) {
    return {
      controlPaths: [],
      errors: [`${policyPath} controlPaths must contain only non-empty strings without control characters`],
    }
  }

  return { controlPaths, errors: [] }
}

/**
 * Read the supported single-line scalar subset, not arbitrary YAML.
 *
 * Quoted strings support JSON double-quote escapes or YAML doubled single quotes. Plain scalars
 * keep internal quotes/brackets literally; only leading indicators select YAML structure. Tags,
 * aliases, anchors, block/flow collections and multiline scalars are deliberately unsupported.
 */
function trimAsciiWhitespace(value) {
  return value.replace(/^[ \t]+|[ \t]+$/g, '')
}

function parsedScalar(value, quoted) {
  return FORBIDDEN_SCALAR_CONTROL.test(value)
    ? { value: null, error: 'forbidden control character', quoted }
    : { value, error: null, quoted }
}

function isYamlImplicitNonStringScalar(value) {
  return (
    YAML_NULL_SCALAR.test(value) ||
    YAML_BOOLEAN_SCALAR.test(value) ||
    YAML_INTEGER_SCALAR.test(value) ||
    YAML_FLOAT_SCALAR.test(value) ||
    YAML_NON_FINITE_FLOAT_SCALAR.test(value) ||
    YAML_DATE_SCALAR.test(value) ||
    YAML_TIMESTAMP_SCALAR.test(value)
  )
}

function parseFrontMatterScalar(rawValue) {
  const text = trimAsciiWhitespace(rawValue)
  if (text === '') {
    return { value: null, error: 'empty unquoted scalar', quoted: false }
  }
  if (FORBIDDEN_SCALAR_CONTROL.test(text)) {
    return { value: null, error: 'forbidden control character', quoted: false }
  }

  if (text.startsWith('"')) {
    const quoted = text.match(/^("(?:[^"\\]|\\.)*")(?:[ \t]+#.*)?$/)
    if (!quoted) {
      return { value: null, error: 'unbalanced quote or trailing content', quoted: true }
    }
    try {
      return parsedScalar(JSON.parse(quoted[1]), true)
    } catch {
      return { value: null, error: 'unsupported double-quoted escape or control character', quoted: true }
    }
  }

  if (text.startsWith("'")) {
    const quoted = text.match(/^'((?:[^']|'')*)'(?:[ \t]+#.*)?$/)
    return quoted
      ? parsedScalar(quoted[1].replaceAll("''", "'"), true)
      : { value: null, error: 'unbalanced quote or trailing content', quoted: true }
  }

  const value = text.replace(/[ \t]+#.*$/, '')
  if (/^[\[{]/.test(value)) {
    const closer = value[0] === '[' ? ']' : '}'
    return {
      value: null,
      error: value.endsWith(closer)
        ? 'unsupported flow sequence or mapping'
        : 'unterminated flow sequence or mapping',
      quoted: false,
    }
  }
  if (/^[!&*|>@`%}\],#]/.test(value) || /^[-?:](?:[ \t]|$)/.test(value)) {
    return { value: null, error: 'unsupported leading scalar indicator', quoted: false }
  }
  if (/:(?:[ \t]|$)/.test(value)) {
    return { value: null, error: 'unsupported nested mapping', quoted: false }
  }

  return parsedScalar(value, false)
}

/**
 * Validate the WHOLE front matter block, not just `paths:`. An invalid or unsupported line anywhere
 * must fail closed rather than letting the mirror check certify a rule its loader might reject.
 *
 * Accepted: top-level keys with a separated single-line scalar, or one flat indented scalar list;
 * comments and blank lines. Each list chooses its own indentation, but every sibling must match it.
 * This dependency-free check intentionally does not implement the complete YAML grammar.
 */
function validateFrontMatterStructure(lines, rulePath) {
  const structureErrors = []
  const seenKeys = new Set()
  let blockKey = null
  let blockIndent = null
  let reportedOrphanEntry = false

  for (const line of lines) {
    if (trimAsciiWhitespace(line) === '') {
      continue
    }
    if (/^[ \t]*\t/.test(line)) {
      structureErrors.push(`${rulePath} front matter has tab indentation, which this check cannot parse: ${line.trim()}`)
      continue
    }
    if (/^ *#/.test(line)) {
      continue
    }

    if (/^ /.test(line)) {
      const entry = line.match(/^( +)-(?:[ \t]+(.*))?$/)
      if (!entry) {
        structureErrors.push(`${rulePath} front matter has a line this check cannot parse: ${line.trim()}`)
        continue
      }
      if (blockKey === null) {
        if (!reportedOrphanEntry) {
          reportedOrphanEntry = true
          structureErrors.push(
            `${rulePath} front matter has a list entry with no preceding key, which this check cannot parse: ${line.trim()} (further orphaned entries not listed)`,
          )
        }
        continue
      }
      blockIndent ??= entry[1].length
      if (entry[1].length !== blockIndent) {
        structureErrors.push(`${rulePath} front matter has nested or inconsistent list indentation, which this check cannot parse: ${line.trim()}`)
        continue
      }
      // Empty quoted metadata strings are valid; only the paths consumer requires nonempty values.
      const { error } = parseFrontMatterScalar(entry[2] ?? '')
      if (error !== null) {
        structureErrors.push(`${rulePath} front matter has a list entry this check cannot parse (${error}): ${line.trim()}`)
      }
      continue
    }

    if (/^-(?:[ \t]|$)/.test(line)) {
      structureErrors.push(
        `${rulePath} front matter has a list entry at column 0 that this check cannot parse (entries must be indented): ${line.trim()}`,
      )
      continue
    }

    // A colon without separation starts plain scalar text, not a YAML mapping value.
    const keyMatch = line.match(/^([A-Za-z0-9_][A-Za-z0-9_.-]*) *:(?:[ \t]+(.*))?$/)
    if (!keyMatch) {
      structureErrors.push(`${rulePath} front matter has a line this check cannot parse: ${line.trim()}`)
      blockKey = null
      blockIndent = null
      continue
    }

    const [, key, rawValue = ''] = keyMatch
    if (seenKeys.has(key)) {
      structureErrors.push(
        `${rulePath} front matter declares the key "${key}" twice; duplicate mapping keys are not supported`,
      )
    }
    seenKeys.add(key)
    blockIndent = null
    reportedOrphanEntry = false

    const value = trimAsciiWhitespace(rawValue)
    if (value === '' || value.startsWith('#')) {
      blockKey = key
      continue
    }

    const { error } = parseFrontMatterScalar(value)
    if (key === 'paths' && error === 'unsupported flow sequence or mapping') {
      structureErrors.push(`${rulePath} front matter paths: must be a block sequence of "- glob" entries`)
    } else if (error !== null) {
      structureErrors.push(
        `${rulePath} front matter has an ${error} on key "${key}", which this check cannot parse: ${line.trim()}`,
      )
    }
    blockKey = null
  }

  return structureErrors
}

/**
 * Parse the `paths:` block sequence out of an agent-rule file's YAML front matter.
 *
 * Fails closed on every shape it does not fully understand, and validates the complete front matter
 * block first. Claude Code drops a rule file whose front matter does not parse, silently and with no
 * error anywhere, so "cannot parse" has to mean "red check", never "no paths found".
 */
export function parseRuleFrontMatterPaths(ruleText, rulePath = CI_CONTROL_RULE_PATH) {
  const frontMatterMatch = ruleText.match(/^---\r?\n([\s\S]*?)\r?\n---(\r?\n|$)/)
  if (!frontMatterMatch) {
    return {
      paths: [],
      errors: [`${rulePath} has no parseable YAML front matter (an unparseable rule file loads for nothing)`],
    }
  }

  const lines = frontMatterMatch[1].split(/\r?\n/)

  const structureErrors = validateFrontMatterStructure(lines, rulePath)
  if (structureErrors.length > 0) {
    return { paths: [], errors: structureErrors }
  }

  const keyIndex = lines.findIndex((line) => /^paths[ \t]*:/.test(line))
  if (keyIndex === -1) {
    return { paths: [], errors: [`${rulePath} front matter has no paths: key`] }
  }

  if (!/^paths[ \t]*:[ \t]*(#.*)?$/.test(lines[keyIndex])) {
    return {
      paths: [],
      errors: [`${rulePath} front matter paths: must be a block sequence of "- glob" entries`],
    }
  }

  const paths = []
  const errors = []
  for (let index = keyIndex + 1; index < lines.length; index += 1) {
    const line = lines[index]
    if (trimAsciiWhitespace(line) === '' || /^[ \t]*#/.test(line)) {
      continue
    }

    if (!/^[ \t]/.test(line)) {
      break
    }

    const itemMatch = line.match(/^ +-[ \t]+(.*?)[ \t]*$/)
    if (!itemMatch) {
      errors.push(`${rulePath} front matter paths: has an entry this check cannot parse: ${line.trim()}`)
      continue
    }

    const { value, quoted } = parseFrontMatterScalar(itemMatch[1])
    if (
      value === null ||
      value === '' ||
      /^\s|\s$/u.test(value) ||
      FORBIDDEN_SCALAR_CONTROL.test(value) ||
      (!quoted && isYamlImplicitNonStringScalar(value))
    ) {
      errors.push(`${rulePath} front matter paths: has an entry this check cannot parse: ${line.trim()}`)
      continue
    }

    paths.push(value)
  }

  if (errors.length === 0 && paths.length === 0) {
    errors.push(`${rulePath} front matter paths: declares no indented entries`)
  }

  return { paths, errors }
}

/**
 * `ci/policy.v1.json` controlPaths is the authority; the rule file's `paths:` front matter is a
 * mirror of it, and must be a superset (extras such as `.github/**` are deliberate, see the rule).
 */
export function collectControlPathMirrorErrors(
  policyText,
  ruleText,
  { policyPath = CI_POLICY_PATH, rulePath = CI_CONTROL_RULE_PATH } = {},
) {
  const policyResult = parsePolicyControlPaths(policyText, policyPath)
  const ruleResult = parseRuleFrontMatterPaths(ruleText, rulePath)
  const mirrorErrors = [...policyResult.errors, ...ruleResult.errors]

  if (mirrorErrors.length > 0) {
    return mirrorErrors
  }

  const declared = new Set(ruleResult.paths)
  const missing = policyResult.controlPaths.filter((controlPath) => !declared.has(controlPath))
  if (missing.length > 0) {
    mirrorErrors.push(
      `${rulePath} front matter paths: is missing ${missing.length} control path(s) declared in ` +
        `${policyPath} controlPaths: ${missing.join(', ')} ` +
        `(add them to the rule in the same PR, or the rule stops loading for those paths)`,
    )
  }

  return mirrorErrors
}

async function validateControlPathMirror() {
  for (const path of [CI_POLICY_PATH, CI_CONTROL_RULE_PATH]) {
    if (!(await fileExists(path))) {
      errors.push(`Missing required control-path mirror input: ${path}`)
      return
    }
  }

  const [policyText, ruleText] = await Promise.all([
    readFile(resolve(CI_POLICY_PATH), 'utf8'),
    readFile(resolve(CI_CONTROL_RULE_PATH), 'utf8'),
  ])

  errors.push(...collectControlPathMirrorErrors(policyText, ruleText))
}

async function main() {
  for (const path of requiredDocs) {
    if (!(await fileExists(path))) {
      errors.push(`Missing required active document: ${path}`)
    }
  }

  if (!(await fileExists('docs/INDEX.md'))) {
    errors.push('Missing required docs index: docs/INDEX.md')
  } else {
    const indexText = await readFile(resolve('docs/INDEX.md'), 'utf8')
    for (const path of requiredDocs) {
      expectContains(indexText, path.replace('docs/', ''), 'docs/INDEX.md')
    }

    const hasArchiveLink = indexText.includes('archive/') || indexText.includes('docs/archive/')
    if (!hasArchiveLink) {
      errors.push('docs/INDEX.md must reference the archive directory')
    }
  }

  const docsRequiringLastUpdated = [
    'docs/STATUS.md',
    'docs/GOLDEN_PRINCIPLES.md',
  ]

  for (const path of docsRequiringLastUpdated) {
    if (!(await fileExists(path))) {
      continue
    }

    const text = await readFile(resolve(path), 'utf8')
    const hasLastUpdatedLine = /^Last Updated:\s*\d{4}-\d{2}-\d{2}\s*$/m.test(text)
    if (!hasLastUpdatedLine) {
      errors.push(`${path} must contain a "Last Updated: YYYY-MM-DD" line`)
    }
  }

  await validateControlPathMirror()

  if (errors.length > 0) {
    console.error('Docs governance check failed:')
    for (const error of errors) {
      console.error(`- ${error}`)
    }
    process.exit(1)
  }

  console.log('Docs governance check passed.')
}

if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
  main().catch((error) => {
    console.error('Docs governance check crashed:', error)
    process.exit(1)
  })
}

scripts/check-docs-governance.hardening.test.mjs

import test from 'node:test'
import assert from 'node:assert/strict'

import {
  collectControlPathMirrorErrors,
  parsePolicyControlPaths,
  parseRuleFrontMatterPaths,
} from './check-docs-governance.mjs'

function ruleWithPath(path, { quoted = false } = {}) {
  const scalar = quoted ? JSON.stringify(path) : path
  return `---\npaths:\n  - ${scalar}\n---\n`
}

const controlCodePoints = [
  ...Array.from({ length: 0x20 }, (_, index) => index),
  ...Array.from({ length: 0x21 }, (_, index) => 0x7f + index),
]

test('rejects every C0, DEL, and C1 control character in policy paths', () => {
  for (const codePoint of controlCodePoints) {
    const path = `ci/${String.fromCodePoint(codePoint)}**`
    const result = parsePolicyControlPaths(JSON.stringify({ controlPaths: [path] }))
    assert.ok(result.errors.length > 0, `accepted U+${codePoint.toString(16).padStart(4, '0')}`)
  }
})

test('rejects every decoded C0, DEL, and C1 control character in quoted YAML paths', () => {
  for (const codePoint of controlCodePoints) {
    const path = `ci/${String.fromCodePoint(codePoint)}**`
    const result = parseRuleFrontMatterPaths(ruleWithPath(path, { quoted: true }))
    assert.ok(result.errors.length > 0, `accepted U+${codePoint.toString(16).padStart(4, '0')}`)
  }
})

test('preserves ordinary visible Unicode in policy and YAML paths', () => {
  const path = 'docs/日本語/naïve-Δ/**'
  assert.deepEqual(parsePolicyControlPaths(JSON.stringify({ controlPaths: [path] })), {
    controlPaths: [path],
    errors: [],
  })
  assert.deepEqual(parseRuleFrontMatterPaths(ruleWithPath(path, { quoted: true })), {
    paths: [path],
    errors: [],
  })
})

const implicitNonStrings = [
  '~',
  'null',
  'NULL',
  'yes',
  'ON',
  'true',
  'False',
  '0',
  '-42',
  '+42',
  '01',
  '017',
  '1:20',
  '1_000',
  '0b1010',
  '0xF__F',
  '0xFF',
  '1.5',
  '.5',
  '5.',
  '1.0e+3',
  '1:20.5',
  '.inf',
  '+.Inf',
  '-.INF',
  '.nan',
  '.NaN',
  '2026-09-17',
  '2026-09-17T12:34:56Z',
  '2026-09-17T12:34:56.',
  '2026-09-17 12:34:56 +01:00',
]

const implicitStrings = [
  '0XFF',
  '+.nAn',
  '1e1_0',
  '0o17',
  '1e3',
  '-2E-4',
  '+.5',
  '.nAn',
  '09',
  '2026-09-17T12:34:56z',
]

test('rejects unquoted YAML implicit non-string path scalars', () => {
  for (const scalar of implicitNonStrings) {
    const result = parseRuleFrontMatterPaths(ruleWithPath(scalar))
    assert.ok(result.errors.length > 0, `accepted unquoted ${scalar}`)
  }
})

test('accepts quoted forms of YAML implicit scalars as path strings', () => {
  for (const scalar of implicitNonStrings) {
    const errors = collectControlPathMirrorErrors(
      JSON.stringify({ controlPaths: [scalar] }),
      ruleWithPath(scalar, { quoted: true }),
    )
    assert.deepEqual(errors, [], `rejected quoted ${scalar}: ${errors.join(' | ')}`)
  }
})

test('preserves resolver-string spellings and ordinary path-like strings', () => {
  for (const path of [
    ...implicitStrings,
    '2026-09-17-notes.md',
    '123/notes.md',
    'true/guide.md',
    '0xFF.md',
    '1e3/results.md',
    '.nan.md',
    'nullish',
  ]) {
    const errors = collectControlPathMirrorErrors(
      JSON.stringify({ controlPaths: [path] }),
      ruleWithPath(path),
    )
    assert.deepEqual(errors, [], `rejected path-like string ${path}: ${errors.join(' | ')}`)
  }
})

docs/analysis/2026-09-12-frontmatter-scalar-validation.md

# Frontmatter scalar validation: issue #3006

Last Updated: 2026-09-17

Status: PR candidate. Exact-head hosted qualification and human acceptance remain outstanding.

## Boundary and design

The control-path mirror is checked by `scripts/check-docs-governance.mjs` without installing a YAML dependency. The checker therefore implements a deliberately narrow, fail-closed subset rather than claiming general YAML conformance:

- top-level keys;
- single-line scalars;
- flat indented scalar lists;
- JSON-compatible double-quoted strings;
- YAML doubled-single-quote strings;
- comments and blank lines.

Unsupported tags, anchors, aliases, block scalars, flow collections, nested collections, malformed quoting, inconsistent indentation, and trailing content are rejected instead of approximated.

Quoted values are tracked separately from plain values. This distinction is required because a path such as `"null"` is a string, while plain `null` is resolved as a non-string by the default YAML 1.1 implicit resolver. The path consumer mirrors that resolver's exact, case-sensitive spellings for nulls, booleans, integers, floats, infinities, NaN, dates, and timestamps. That includes legacy octal and sexagesimal values and resolver-permitted repeated underscores; it deliberately preserves non-resolving forms such as `0XFF`, `+.nAn`, `1e1_0`, `0o17`, and `1e3` as strings. Matching is bounded to the complete scalar so ordinary paths such as `2026-09-17-notes.md`, `true/guide.md`, and `123/notes.md` remain valid.

Every decoded internal control character in the C0, DEL, and C1 ranges is rejected. This applies equally to policy JSON strings, literal YAML characters, and escape sequences decoded from quoted YAML values. Ordinary visible Unicode remains valid. Leading or trailing Unicode whitespace is rejected separately rather than trimmed into an apparently matching control path.

## Recovery audit

The original PR branch accumulated unrelated history and could no longer provide a reviewable three-file change. The clean replacement was created from current `main` and initially copied the three intended blobs. A second audit found that the source branch did not actually contain the final control-range and implicit-scalar changes claimed in its automation summary. The clean branch was corrected directly rather than carrying that overstatement forward.

The current replacement contains:

- `scripts/check-docs-governance.mjs`;
- the original governance regression suite;
- `scripts/check-docs-governance.hardening.test.mjs`, which isolates the missing fail-open cases;
- this evidence note.

No workflow, required context, dependency, deployment setting, policy document, or agent-control rule is changed.

## Verification

The original focused red/green extraction on Node v22.16.0 showed three of six hardening groups failing before the first correction: policy controls accepted U+0009, decoded quoted-YAML controls accepted U+0009, and plain YAML `~` was accepted as a string path.

Current exact-head verification on Node v20.20.2:

- `node --test scripts/check-docs-governance.test.mjs scripts/check-docs-governance.hardening.test.mjs`: 89 tests, 89 passed, 0 failed, 0 skipped, 0 cancelled, and 0 todo;
- the six hardening groups exhaustively check all 65 C0, DEL, and C1 code points in both policy JSON and decoded quoted YAML;
- 31 resolver-non-string spellings are rejected when plain and all 31 are accepted when quoted;
- 10 resolver-string counterexamples, seven ordinary path-like examples, and one visible-Unicode example are accepted;
- leading and trailing whitespace retain the whitespace diagnostic, while internal controls retain the control-character diagnostic;
- `node --check scripts/check-docs-governance.mjs` and the live `node scripts/check-docs-governance.mjs` check pass.

The current hosted Docs Governance job executes the checker against the repository policy/rule pair but does not yet invoke this Node regression suite. Wiring governance regressions into hosted CI remains separately owned by #3005; this PR does not alter a control-plane workflow to conceal that boundary.

Exact-head hosted CI, a fresh code review, and review of the final changed-file inventory are required before merge. There is no frontend, backend, browser, deployment, or hosted-product claim.

View task →

Copy link
Copy Markdown
Owner Author

@codex please run the exact focused suite on test-only head de09dcbb0bd9032adc87cf9f2e583ef46f52073b without modifying files:

node --test scripts/check-docs-governance.test.mjs scripts/check-docs-governance.hardening.test.mjs

Report the exact failing assertions and counts. This commit intentionally adds resolver-exact RED cases before the implementation repair. Verify git rev-parse HEAD first and do not commit or push.

@chatgpt-codex-connector

Copy link
Copy Markdown

Focused Suite Report

Verified HEAD before execution as de09dcbb0bd9032adc87cf9f2e583ef46f52073b. After execution, HEAD was unchanged and git status --porcelain=v1 remained empty. No files were modified, committed, or pushed, and no pull request was created.

Results

  • node --test scripts/check-docs-governance.test.mjs scripts/check-docs-governance.hardening.test.mjs
    • 89 tests total
    • 85 passed
    • 4 failed
    • 0 skipped
    • 0 cancelled
    • 0 todo
    • Exit code: 1
    • Node-reported duration: 146.004384 ms

Exact Failing Assertions

  1. rejects unquoted YAML implicit non-string path scalars

  2. preserves resolver-string spellings and ordinary path-like strings

    • Assertion: assert.deepEqual(errors, [], \rejected path-like string ${path}: ${errors.join(' | ')}`)`
    • Failure message: rejected path-like string 0XFF: .claude/rules/ci-control.md front matter paths: has an entry this check cannot parse: - 0XFF
    • Expected: []
    • Actual:
      [
        '.claude/rules/ci-control.md front matter paths: has an entry this check cannot parse: - 0XFF'
      ]
      
    • Operator: deepStrictEqual
    • The fixture explicitly classifies 0XFF as a resolver-preserved string. scripts/check-docs-governance.hardening.test.mjsL82-L93 scripts/check-docs-governance.hardening.test.mjsL112-L128
  3. rejects policy whitespace rather than normalizing away the mismatch: "\tci/**"

    • Assertion: assert.ok(result.errors.some((error) => /whitespace/.test(error)), result.errors.join(' | '))
    • Failure message: ci/policy.v1.json controlPaths must contain only non-empty strings without control characters
    • Expected: true
    • Actual: false
    • Operator: ==
    • The value was rejected, but with the control-character diagnostic rather than the required whitespace diagnostic. scripts/check-docs-governance.test.mjsL304-L309
  4. rejects policy whitespace rather than normalizing away the mismatch: "ci/**\n"

    • Assertion: assert.ok(result.errors.some((error) => /whitespace/.test(error)), result.errors.join(' | '))
    • Failure message: ci/policy.v1.json controlPaths must contain only non-empty strings without control characters
    • Expected: true
    • Actual: false
    • Operator: ==
    • As with the tab case, rejection occurred under the wrong diagnostic category. scripts/check-docs-governance.test.mjsL304-L309

Repository Integrity Checks

  • git rev-parse HEAD — before: de09dcbb0bd9032adc87cf9f2e583ef46f52073b
  • git rev-parse HEAD — after: de09dcbb0bd9032adc87cf9f2e583ef46f52073b
  • git status --porcelain=v1 — empty; working tree remained clean.
  • ✅ No files were modified, committed, or pushed.
  • ✅ No pull request was created.

Outstanding Human Actions

OUTSTANDING_TASKS.md retains 41 open [ ] entries; none were changed.

View task →

Copy link
Copy Markdown
Owner Author

@codex please verify implementation head dae421b1eba8d967cb462284af0e84d656181901 without modifying files. First confirm git rev-parse HEAD, then run:

  • node --test scripts/check-docs-governance.test.mjs scripts/check-docs-governance.hardening.test.mjs
  • node --check scripts/check-docs-governance.mjs
  • node scripts/check-docs-governance.mjs
  • node scripts/check-doc-links.mjs
  • git diff --check

Report exact test totals and command outputs. Do not commit or push.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dae421b1eb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/check-docs-governance.mjs

Copy link
Copy Markdown
Owner Author

@codex please verify exact head b2263bdcdab1b7b01e032c2cfdbd68ba4e9e32ef without modifying files. Confirm the SHA before and after, then run:

  • node --test scripts/check-docs-governance.test.mjs scripts/check-docs-governance.hardening.test.mjs
  • node --check scripts/check-docs-governance.mjs
  • node scripts/check-docs-governance.mjs
  • node scripts/check-doc-links.mjs
  • git diff --check

Report exact test counts and command outputs. Pay particular attention to rejection of all four lone surrogate boundary cases and acceptance of valid non-BMP surrogate pairs. Do not commit or push.

@chatgpt-codex-connector

Copy link
Copy Markdown

Verification Report

Verified the requested exact head before and after execution:

b2263bdcdab1b7b01e032c2cfdbd68ba4e9e32ef

The working tree was clean both before and after. No files were modified, committed, or pushed, and no pull request was created.

Command Results

  • git rev-parse HEAD — before execution:

    b2263bdcdab1b7b01e032c2cfdbd68ba4e9e32ef
    
  • node --test scripts/check-docs-governance.test.mjs scripts/check-docs-governance.hardening.test.mjs — exit code 0.

    Exact totals:

    1..91
    # tests 91
    # suites 0
    # pass 91
    # fail 0
    # cancelled 0
    # skipped 0
    # todo 0
    # duration_ms 144.992553
    

    The surrogate-specific TAP output was:

    # Subtest: rejects unpaired UTF-16 surrogate escapes in policy and quoted YAML paths
    ok 4 - rejects unpaired UTF-16 surrogate escapes in policy and quoted YAML paths
      ---
      duration_ms: 0.23954
      ...
    # Subtest: preserves valid UTF-16 surrogate pairs in policy and quoted YAML paths
    ok 5 - preserves valid UTF-16 surrogate pairs in policy and quoted YAML paths
      ---
      duration_ms: 0.267333
      ...
    

    The unpaired-surrogate test explicitly exercises all four UTF-16 boundary cases:

    • U+D800 — first high surrogate.
    • U+DBFF — last high surrogate.
    • U+DC00 — first low surrogate.
    • U+DFFF — last low surrogate.

    Each boundary value is independently rejected in both a JSON policy path and a decoded quoted-YAML path, for eight rejection assertions in total. scripts/check-docs-governance.hardening.test.mjsL48-L61

    The valid-pair test accepts a path containing both U+1F680 (🚀) and U+1D11E (𝄞) in policy JSON and quoted YAML. scripts/check-docs-governance.hardening.test.mjsL64-L74

  • node --check scripts/check-docs-governance.mjs — exit code 0; exact output was empty.

  • node scripts/check-docs-governance.mjs — exit code 0. Exact output:

    Docs governance check passed.
    
  • node scripts/check-doc-links.mjs — exit code 0. Exact output:

    Doc link check passed (711 Markdown files, 0 broken relative links).
    
  • git diff --check — exit code 0; exact output was empty.

  • git rev-parse HEAD — after execution:

    b2263bdcdab1b7b01e032c2cfdbd68ba4e9e32ef
    
  • git status --porcelain=v1 — empty before and after execution; the working tree remained clean.

Outcome

All 91 of 91 tests passed. The focused coverage confirms rejection of all four lone-surrogate boundary cases while preserving valid non-BMP surrogate pairs. Every requested syntax, live-governance, documentation-link, and diff check also passed.

Outstanding Human Actions

OUTSTANDING_TASKS.md retains 41 open [ ] entries. None were modified.

View task →

Copy link
Copy Markdown
Owner Author

@codex review

Please review exact documentation-final head 2657c4745893a04765a48d6f4740d6f4bf22e441. The only change after verified implementation head b2263bdcdab1b7b01e032c2cfdbd68ba4e9e32ef is the evidence note recording the exact 91/91 result and the surrogate/resolver boundaries. Check the final changed-file inventory, evidence accuracy, and implementation for remaining fail-open or overmatch cases.

Copy link
Copy Markdown
Owner Author

@codex please verify exact final head 2657c4745893a04765a48d6f4740d6f4bf22e441 without modifying files. Confirm the SHA before and after, then run:

  • node --test scripts/check-docs-governance.test.mjs scripts/check-docs-governance.hardening.test.mjs
  • node --check scripts/check-docs-governance.mjs
  • node scripts/check-docs-governance.mjs
  • node scripts/check-doc-links.mjs
  • git diff --check
  • git diff --name-status cb204adcc73ebeb3834eae714edc5ecadebf99fc...HEAD

Report exact totals, outputs, and final changed-file inventory. Do not commit or push.

@chatgpt-codex-connector

Copy link
Copy Markdown

Verification Summary

The requested final head was confirmed unchanged before and after verification:

Before: 2657c4745893a04765a48d6f4740d6f4bf22e441
After:  2657c4745893a04765a48d6f4740d6f4bf22e441

git status --porcelain=v1 produced no output before or after the checks. No tracked or untracked working-tree files were modified, and nothing was committed, pushed, or submitted as a new pull request.

The focused suite confirms the documented control-character, surrogate, YAML resolver, quoted-scalar, and ordinary-string boundaries. scripts/check-docs-governance.hardening.test.mjsL15-L74 scripts/check-docs-governance.hardening.test.mjsL76-L150 The evidence note’s recorded 91/91 result agrees with this final-head run. docs/analysis/2026-09-12-frontmatter-scalar-validation.mdL39-L55

Exact Command Results

  • git rev-parse HEAD — before execution:

    2657c4745893a04765a48d6f4740d6f4bf22e441
    
  • node --test scripts/check-docs-governance.test.mjs scripts/check-docs-governance.hardening.test.mjs — exit code 0.

    Exact totals:

    1..91
    # tests 91
    # suites 0
    # pass 91
    # fail 0
    # cancelled 0
    # skipped 0
    # todo 0
    # duration_ms 135.948432
    

    All 91 named subtests reported ok, including:

    ok 1 - rejects every C0, DEL, and C1 control character in policy paths
    ok 2 - rejects every decoded C0, DEL, and C1 control character in quoted YAML paths
    ok 4 - rejects unpaired UTF-16 surrogate escapes in policy and quoted YAML paths
    ok 5 - preserves valid UTF-16 surrogate pairs in policy and quoted YAML paths
    ok 6 - rejects unquoted YAML implicit non-string path scalars
    ok 7 - accepts quoted forms of YAML implicit scalars as path strings
    ok 8 - preserves resolver-string spellings and ordinary path-like strings
    
  • node --check scripts/check-docs-governance.mjs — exit code 0; exact output was empty.

  • node scripts/check-docs-governance.mjs — exit code 0. Exact output:

    Docs governance check passed.
    
  • node scripts/check-doc-links.mjs — exit code 0. Exact output:

    Doc link check passed (711 Markdown files, 0 broken relative links).
    
  • git diff --check — exit code 0; exact output was empty.

  • git diff --name-status cb204adcc73ebeb3834eae714edc5ecadebf99fc...HEAD — exit code 0 after retrieving the missing base commit object. Exact final output:

    A	docs/analysis/2026-09-12-frontmatter-scalar-validation.md
    A	scripts/check-docs-governance.hardening.test.mjs
    M	scripts/check-docs-governance.mjs
    M	scripts/check-docs-governance.test.mjs
    

    The first attempt returned:

    fatal: Invalid symmetric difference expression cb204adcc73ebeb3834eae714edc5ecadebf99fc...HEAD
    

    because the base commit was absent from the checkout’s local object database and no origin remote was configured. I retrieved that exact commit object read-only from https://github.com/Chris0Jeky/Taskdeck.git; this updated Git metadata only and did not alter the working tree.

  • git rev-parse HEAD — after execution:

    2657c4745893a04765a48d6f4740d6f4bf22e441
    
  • git status --porcelain=v1 — exact output was empty after execution.

Final Changed-File Inventory

Status File
Added docs/analysis/2026-09-12-frontmatter-scalar-validation.md
Added scripts/check-docs-governance.hardening.test.mjs
Modified scripts/check-docs-governance.mjs
Modified scripts/check-docs-governance.test.mjs

Totals: 4 changed files2 added, 2 modified, 0 deleted.

Outstanding Human Actions

OUTSTANDING_TASKS.md retains 41 open [ ] entries. None were changed.

View task →

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2657c47458

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".


export const CI_POLICY_PATH = 'ci/policy.v1.json'
export const CI_CONTROL_RULE_PATH = '.claude/rules/ci-control.md'
const FORBIDDEN_SCALAR_CONTROL = /[\u0000-\u001F\u007F-\u009F]/u

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject non-printable YAML sentinel code points

When both controlPaths and a quoted rule path contain a literal U+FFFE or U+FFFF, collectControlPathMirrorErrors returns no errors because this character boundary omits both code points, while YAML scanners reject either character as non-printable and can therefore drop the rule frontmatter. Fresh exact-head reproduction shows this remains distinct from the resolved unpaired-surrogate case; include U+FFFE/U+FFFF in decoded-character validation and its boundary tests.

Useful? React with 👍 / 👎.

Comment on lines +234 to 235
if (/^ *#/.test(line)) {
continue

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Validate controls before skipping frontmatter comments

When otherwise-valid frontmatter contains a standalone comment such as # note�, this branch skips the line before checking its characters, so the mirror check passes even though a YAML scanner rejects the document and the control rule may load for nothing. Fresh exact-head reproduction with normal mirrored paths returns []; apply the raw control-character check before recognizing blank or comment-only lines.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Synchronized with current main without rewriting the repaired history.

  • New exact head: 56d2bf72e1bd2c8ad25d339abe404a76207cd266
  • Parents: verified/reviewed head 2657c4745893a04765a48d6f4740d6f4bf22e441 and current main b32ec2dbe5e5e7100e1dedd459cc542beca782dd
  • All four PR-owned blobs are byte-identical to the verified head; the merge only incorporates current-main state, including .gitignore.

Fresh exact-head CI and focused verification are required before merge.

Copy link
Copy Markdown
Owner Author

@codex review

Please review exact merge head 56d2bf72e1bd2c8ad25d339abe404a76207cd266. Confirm that the PR-relative diff is still the same four files and that merging current main did not alter the verified governance implementation, tests, or evidence.

Copy link
Copy Markdown
Owner Author

@codex please verify exact head 56d2bf72e1bd2c8ad25d339abe404a76207cd266 without modifying files. Confirm the SHA before and after, then run:

  • node --test scripts/check-docs-governance.test.mjs scripts/check-docs-governance.hardening.test.mjs
  • node --check scripts/check-docs-governance.mjs
  • node scripts/check-docs-governance.mjs
  • node scripts/check-doc-links.mjs
  • git diff --check
  • git diff --name-status b32ec2dbe5e5e7100e1dedd459cc542beca782dd...HEAD

Report exact totals, outputs, and final PR-relative inventory. Do not commit or push.

@chatgpt-codex-connector

Copy link
Copy Markdown

Verification Summary

The requested exact head was confirmed unchanged before and after verification:

Before: 56d2bf72e1bd2c8ad25d339abe404a76207cd266
After:  56d2bf72e1bd2c8ad25d339abe404a76207cd266

git status --porcelain=v1 returned empty output both before and after. No files were modified, committed, or pushed, and no pull request was created.

The focused regression suite covers the C0/DEL/C1 ranges, UTF-16 surrogate boundaries, YAML implicit scalar behavior, quoted equivalents, and resolver-preserved strings. scripts/check-docs-governance.hardening.test.mjsL15-L74 scripts/check-docs-governance.hardening.test.mjsL76-L155

Exact Command Results

  • git rev-parse HEAD — before execution:

    56d2bf72e1bd2c8ad25d339abe404a76207cd266
    
  • node --test scripts/check-docs-governance.test.mjs scripts/check-docs-governance.hardening.test.mjs — exit code 0.

    Exact totals:

    1..91
    # tests 91
    # suites 0
    # pass 91
    # fail 0
    # cancelled 0
    # skipped 0
    # todo 0
    # duration_ms 144.044632
    

    All 91 of 91 tests passed. The focused boundary subtests included:

    ok 1 - rejects every C0, DEL, and C1 control character in policy paths
    ok 2 - rejects every decoded C0, DEL, and C1 control character in quoted YAML paths
    ok 3 - preserves ordinary visible Unicode in policy and YAML paths
    ok 4 - rejects unpaired UTF-16 surrogate escapes in policy and quoted YAML paths
    ok 5 - preserves valid UTF-16 surrogate pairs in policy and quoted YAML paths
    ok 6 - rejects unquoted YAML implicit non-string path scalars
    ok 7 - accepts quoted forms of YAML implicit scalars as path strings
    ok 8 - preserves resolver-string spellings and ordinary path-like strings
    

    This agrees with the evidence note’s recorded test total and expected governance boundaries. docs/analysis/2026-09-12-frontmatter-scalar-validation.mdL39-L55

  • node --check scripts/check-docs-governance.mjs — exit code 0; exact output was empty.

  • node scripts/check-docs-governance.mjs — exit code 0. Exact output:

    Docs governance check passed.
    
  • node scripts/check-doc-links.mjs — exit code 0. Exact output:

    Doc link check passed (711 Markdown files, 0 broken relative links).
    
  • git diff --check — exit code 0; exact output was empty.

  • git diff --name-status b32ec2dbe5e5e7100e1dedd459cc542beca782dd...HEAD — exit code 0. Exact output:

    A	docs/analysis/2026-09-12-frontmatter-scalar-validation.md
    A	scripts/check-docs-governance.hardening.test.mjs
    M	scripts/check-docs-governance.mjs
    M	scripts/check-docs-governance.test.mjs
    
  • git rev-parse HEAD — after execution:

    56d2bf72e1bd2c8ad25d339abe404a76207cd266
    
  • git status --porcelain=v1 — exact output was empty after execution.

Final PR-Relative Inventory

Status File
Added docs/analysis/2026-09-12-frontmatter-scalar-validation.md
Added scripts/check-docs-governance.hardening.test.mjs
Modified scripts/check-docs-governance.mjs
Modified scripts/check-docs-governance.test.mjs

Totals: 4 changed files2 added, 2 modified, 0 deleted.

The merge with base b32ec2dbe5e5e7100e1dedd459cc542beca782dd therefore leaves the requested four-file PR-relative inventory intact.

Repository Integrity

  • Exact HEAD remained 56d2bf72e1bd2c8ad25d339abe404a76207cd266.
  • The working tree remained clean.
  • No tracked or untracked files were created or modified.
  • No commit, push, or new pull request was made, as requested.

Outstanding Human Actions

OUTSTANDING_TASKS.md retains 41 open [ ] entries; none were changed. The repository requires those open human-owned items to remain surfaced in handoffs. OUTSTANDING_TASKS.mdL1-L10

View task →

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: 56d2bf72e1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown
Owner Author

Audit finding: terminal high surrogates bypass the new Unicode guard

At exact head 56d2bf72e1bd2c8ad25d339abe404a76207cd266, hasUnpairedUtf16Surrogate in scripts/check-docs-governance.mjs accepts a lone high surrogate at the end of the string. charCodeAt(index + 1) is NaN at EOF, so both < 0xdc00 and > 0xdfff are false.

Examples currently returning false from that helper:

hasUnpairedUtf16Surrogate('\ud800')
hasUnpairedUtf16Surrogate('docs/\udbff')
hasUnpairedUtf16Surrogate('\ud83d\ude00\ud800')

parsePolicyControlPaths calls this guard after checks that do not reject these code units, so JSON controlPaths ending in a high surrogate escape the intended rejection. parsedScalar uses the same helper for decoded YAML strings.

Smallest proposed correction:

-      if (nextCodeUnit < 0xdc00 || nextCodeUnit > 0xdfff) {
+      if (!(nextCodeUnit >= 0xdc00 && nextCodeUnit <= 0xdfff)) {

Add regressions at both exported policy/YAML entry points for a terminal high surrogate, a valid pair followed by a high surrogate, and a valid-pair positive control. Do not test only \ud800/**, which has a following ASCII code unit and therefore misses the EOF case.

Actual evidence: copied the exact helper excerpt into an isolated Node v22.16.0 reproduction: original misses 3 of 7 cases; the proposed one-condition replacement passes all 7. This is function-level evidence, NOT full script/CLI execution or a committed fix. The live checker blob is 83216939da78ef923190ef556fe437cf47b5f624.

The existing #3005 regression-gate wiring remains relevant: running the governance checker on today's clean policy is not equivalent to executing its adversarial fixtures. Please retain this as a pre-merge correctness item rather than claiming the Unicode path contract is fully covered.

@Chris0Jeky

Copy link
Copy Markdown
Owner Author

Independent narrow review on exact head 20d13fbafd4db154934a32afc88e817d1e76a67a found no CRITICAL/HIGH correctness, security, or data-loss defect.

Verified by the review:

  • terminal high surrogate rejection;
  • valid surrogate pair followed by a high surrogate rejection;
  • low-surrogate rejection and valid-pair preservation;
  • both exported policy/YAML entry points;
  • parser-boundary compatibility and policy/rule mirror integrity.

Non-blocking suggestions are recorded but do not justify another fix round: add an explicit terminal low-surrogate fixture, refresh the analysis note's historical verification SHA, and keep the separate #3005 hosted regression-gate work distinct. No further Codex review request is being made.

@Chris0Jeky
Chris0Jeky merged commit 6dbcba9 into main Sep 17, 2026
36 checks passed
@Chris0Jeky
Chris0Jeky deleted the fix/3006-frontmatter-scalars-clean branch September 17, 2026 18:52
@github-project-automation github-project-automation Bot moved this from Pending to Done in Taskdeck Execution Sep 17, 2026
Chris0Jeky added a commit that referenced this pull request Sep 18, 2026

Copy link
Copy Markdown
Owner Author

The earlier locally prepared review patch and the original recovery report are now preserved remotely in provenance PR #3192.

#3134's merged implementation remains authoritative and supersedes that archived patch. The patch is retained only as investigation history, with its original SHA-256 recorded, and is explicitly marked not to be replayed over current main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant