diff --git a/AGENTS.md b/AGENTS.md index f87667ffd..4ff686b3b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -33,7 +33,7 @@ The app supports a multi-provider AI stack (Gemini, OpenAI, Claude, Grok, OpenRo ```bash pnpm run ci:prepush ``` - This gate is mandatory before every push and after every local correction before re-pushing; it runs sequentially with a single-checker project typecheck, i18n parity/quality and bundle checks, release/doc truth, and lightweight native guardrails. The pre-commit hook separately runs staged-file Biome checks. Full repository lint, coverage, E2E, Storybook, Lighthouse, and mutation checks belong to cloud CI. If branch switching or a lockfile/package-manifest change makes pnpm report dependency verification errors, run `node scripts/dependency-state.mjs reconcile` and rerun the complete pre-push gate. + This gate is mandatory before every push and after every local correction before re-pushing; it classifies the changed files first, runs only applicable local checks, and reports `DEFERRED_TO_REQUIRED_CI` for provably non-TypeScript changes. TypeScript-impacting, dependency, build, native-contract, mixed, ambiguous, and TypeScript test changes run bounded single-checker `tsgo`. The pre-commit hook separately runs staged-file Biome checks. Full repository lint, coverage, E2E, Storybook, Lighthouse, and mutation checks belong to cloud CI. If branch switching or a lockfile/package-manifest change makes pnpm report dependency verification errors, run `node scripts/dependency-state.mjs reconcile` and rerun the complete pre-push gate. Optional targeted smoke test: `pnpm exec vitest run ` **without** `--coverage`. **Hard rule:** Never invoke `pnpm test`, `npm run test`, or a bare Vitest wrapper; always use an explicit `pnpm exec vitest run ` command to avoid watch-mode hangs on constrained hardware. Never start multiple heavyweight processes concurrently. 4. **Audit cloud CI logs, fix locally, then re-push** – If the cloud CI run fails, inspect the logs via GitHub web UI or `gh run watch`, reproduce the specific failing test or lint error in isolation, fix it locally (quick tier to verify), commit, and push again for another cloud CI run. diff --git a/README.md b/README.md index ceaf3c849..ffbcc0162 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,7 @@ IndexedDB v8 PWA v3.0 i18n 19 locales — 2925 keys - 6954+ tests / 575 files + 6957+ tests / 576 files Codecov Coverage License MIT CI Status @@ -512,7 +512,7 @@ The Settings → AI panel shows a live GPU status badge with adapter details and | **Document Export** | docx + jszip | Word-compatible `.docx` generation (lazy-loaded) | | **PWA** | Service Worker + Web App Manifest v3 | Offline support, installability, Workbox chunking | | **i18n** | Custom React Context (`I18nContext.tsx`) | 2925 keys × 19 locales (de/en/es/fr/it + ar/he/fa RTL Beta + ja/zh/pt/el/fi/sv/hu/is/eu/ru/ko Beta); EN fallback; `localStorage` persistence | -| **Testing** | Vitest 4.x (6954+ tests / 575 files) + Playwright E2E | Unit/integration + cross-browser E2E; Stryker mutation (manual workflow) | +| **Testing** | Vitest 4.x (6957+ tests / 576 files) + Playwright E2E | Unit/integration + cross-browser E2E; Stryker mutation (manual workflow) | | **Code Quality** | Biome (lint + format) + TypeScript 7 (tsgo) strict | `--error-on-warnings` in CI; zero `any` policy | | **Visualization** | Force-directed graph | Interactive character relationship network | | **Desktop** | Tauri v2 | Cross-platform installer; auto-updater via `latest.json` | @@ -550,7 +550,7 @@ WorldScript-Studio/ │ ├── sw.js # PWA Service Worker │ └── manifest.json # PWA Web App Manifest v3 ├── tests/ -│ ├── unit/ # Vitest unit tests (6954+ tests, 575 files) — count spans tests/, components/, packages/*/tests/, not just this folder +│ ├── unit/ # Vitest unit tests (6957+ tests, 576 files) — count spans tests/, components/, packages/*/tests/, not just this folder │ │ ├── ai/ # aiSmallModules, aiCoreFallbackPaths │ │ └── settings/ # WebLlmPanel, AiSections │ └── e2e/ # Playwright specs + helpers.ts @@ -712,7 +712,7 @@ The main pipeline is [`.github/workflows/ci.yml`](.github/workflows/ci.yml). Opt | `scorecard` | weekly + `main` push | OpenSSF Scorecard — SARIF uploaded to GitHub Code Scanning | **Current test metrics (2026-08-21, source-synchronized; CI remains authoritative for pass/fail):** -- **6954+ unit tests** across **575 test files** — CI is authoritative for pass/fail +- **6957+ unit tests** across **576 test files** — CI is authoritative for pass/fail - Coverage thresholds: lines ≥ 80 · branches ≥ 66 · functions ≥ 72 · statements ≥ 78 — enforced in CI (see Codecov badge for live metrics) - i18n: **2925 keys × 19 locales** (en/de/fr/es/it + ar/he/fa RTL Beta + ja/zh/pt/el/fi/sv/hu/is/eu/ru/ko Beta) diff --git a/docs/CI.md b/docs/CI.md index ad059aac6..afeb71d49 100644 --- a/docs/CI.md +++ b/docs/CI.md @@ -14,7 +14,7 @@ For historical optimization notes (targets may predate the live workflow), see [ | Tier | Where | Commands / scope | |------|--------|------------------| -| **Quick (local)** | Developer laptop | `pnpm run ci:prepush` (single-checker typecheck, i18n quality, release/doc truth, and lightweight guardrails); the pre-commit hook runs staged Biome checks; optional targeted `pnpm exec vitest run ` for a fast smoke | +| **Quick (local)** | Developer laptop | `pnpm run ci:prepush` (change-aware bounded admission, applicable policy guards, and targeted TypeScript); non-TypeScript-only changes report `DEFERRED_TO_REQUIRED_CI`; optional targeted `pnpm exec vitest run ` for a fast smoke | | **Heavy (CI)** | `ci.yml` | Vitest **with** `--coverage` and thresholds, Playwright E2E (`CI=true`) including **mobile emulation** (Pixel 5 / Chromium), Lighthouse CI, Storybook static build, bundle budget + analyze. Mutation testing (Stryker) is **not** part of this pipeline — see [Mutation testing status](#mutation-testing-status). | **Merge readiness:** A green workflow run on the PR/branch matters more than reproducing every E2E or LHCI step locally. Use CI **artifacts** (Playwright HTML report, coverage, Lighthouse output) to debug failures. @@ -294,7 +294,7 @@ longer runs a root `prepare` command. `pnpm-workspace.yaml` sets `verifyDepsBefo ## Local checks (without Act) -On **low-resource** machines, stop at the **Quick** tier (see [Cloud CI-first vs local development](#cloud-ci-first-vs-local-development)): **`pnpm run ci:prepush`**, and optionally targeted **`pnpm exec vitest run `**. Never run multiple heavyweight local processes concurrently. Treat **`CI=true pnpm run test:e2e`** (desktop + mobile projects in CI), **Lighthouse**, coverage, Storybook, and mutation testing as **CI-owned**. +On **low-resource** machines, stop at the **Quick** tier (see [Cloud CI-first vs local development](#cloud-ci-first-vs-local-development)): **`pnpm run ci:prepush`**, and optionally targeted **`pnpm exec vitest run `**. The gate classifies the change set before running expensive checks; unknown or mixed changes use the broader safe class, while non-TypeScript-only changes report `DEFERRED_TO_REQUIRED_CI`. Never run multiple heavyweight local processes concurrently. Treat **`CI=true pnpm run test:e2e`** (desktop + mobile projects in CI), **Lighthouse**, coverage, Storybook, and mutation testing as **CI-owned**. ```bash pnpm install --frozen-lockfile diff --git a/scripts/ci-prepush-check-registry.d.mts b/scripts/ci-prepush-check-registry.d.mts new file mode 100644 index 000000000..64cda4dd0 --- /dev/null +++ b/scripts/ci-prepush-check-registry.d.mts @@ -0,0 +1 @@ +export function shouldRunAdmissionCheck(name: string, files: readonly string[]): boolean; diff --git a/scripts/ci-prepush-check-registry.mjs b/scripts/ci-prepush-check-registry.mjs new file mode 100644 index 000000000..bc5280252 --- /dev/null +++ b/scripts/ci-prepush-check-registry.mjs @@ -0,0 +1,27 @@ +const routingAuthority = 'scripts/ci-prepush-check-registry.mjs'; +const i18nPolicyFiles = new Set(['scripts/check-i18n-keys.mjs', 'scripts/i18n-locales.mjs']); + +export const admissionCheckRegistry = Object.freeze([ + { + name: 'i18n', + matches: (file) => + file.startsWith('locales/') || + file.startsWith('public/locales/') || + i18nPolicyFiles.has(file), + implementationFiles: new Set([routingAuthority]), + }, + { + name: 'contentGuard', + matches: (file) => + file === 'scripts/content-guard.mjs' || + file.startsWith('community-templates/') || + file.startsWith('public/community-templates/'), + implementationFiles: new Set([routingAuthority]), + }, +]); + +export function shouldRunAdmissionCheck(name, files) { + const entry = admissionCheckRegistry.find((candidate) => candidate.name === name); + if (!entry) throw new Error(`unknown local admission check: ${name}`); + return files.some((file) => entry.matches(file) || entry.implementationFiles.has(file)); +} diff --git a/scripts/ci-prepush-classifier.d.mts b/scripts/ci-prepush-classifier.d.mts new file mode 100644 index 000000000..90e0188fa --- /dev/null +++ b/scripts/ci-prepush-classifier.d.mts @@ -0,0 +1,26 @@ +export type ChangeKind = + | 'NO_CHANGES' + | 'DOCS_ONLY' + | 'WORKFLOW_ONLY' + | 'NON_CODE_ONLY' + | 'RUST_TAURI' + | 'TOOLING' + | 'TEST_ONLY' + | 'TYPESCRIPT_APPLICATION' + | 'DEPENDENCY_TOOLCHAIN' + | 'BUILD_CONFIGURATION' + | 'AMBIGUOUS' + | 'MIXED'; + +export interface ChangeClassification { + readonly kind: ChangeKind; + readonly categories: readonly string[]; + readonly files: readonly string[]; +} + +export function classifyFile(file: string): string; +export function classifyChangedFiles(files: readonly string[]): ChangeClassification; +export function requiresTypecheck( + classification: ChangeClassification, + options?: { readonly full?: boolean }, +): boolean; diff --git a/scripts/ci-prepush-classifier.mjs b/scripts/ci-prepush-classifier.mjs new file mode 100644 index 000000000..338a3d465 --- /dev/null +++ b/scripts/ci-prepush-classifier.mjs @@ -0,0 +1,114 @@ +const DOC_FILE = /\.(?:md|mdx)$/i; +const TS_FILE = /\.(?:c|m)?tsx?$|\.(?:c|m)?jsx?$/i; +const WORKFLOW_ROOTS = ['.github/workflows/', '.github/actions/']; +const RUST_ROOTS = ['src-tauri/', 'crates/']; +const TOOLING_ROOTS = ['scripts/']; +const TOOLING_FILES = new Set(['.gitleaks.toml']); +const DEPENDENCY_FILES = new Set([ + 'package.json', + 'pnpm-lock.yaml', + 'pnpm-workspace.yaml', + '.npmrc', + '.nvmrc', + 'rust-toolchain', + 'rust-toolchain.toml', +]); +const BUILD_CONFIG_FILES = new Set([ + 'biome.json', + 'index.html', + 'playwright.config.ts', + 'postcss.config.js', + 'postcss.config.mjs', + 'tailwind.config.js', + 'tailwind.config.ts', + 'turbo.json', + 'vite.config.ts', + 'vitest.config.ts', +]); + +function startsWithRoot(file, roots) { + return roots.some((root) => file.startsWith(root)); +} + +function normalizePath(file) { + return file.replaceAll('\\', '/').replace(/^\.\//, ''); +} + +function isInstructionFile(file) { + return ( + file === 'AGENTS.md' || + file === 'CLAUDE.md' || + file === '.cursorrules' || + file === '.github/copilot-instructions.md' || + file.startsWith('.cursor/rules/') + ); +} + +export function classifyFile(file) { + const normalized = normalizePath(file); + const base = normalized.split('/').at(-1) ?? normalized; + + if (startsWithRoot(normalized, WORKFLOW_ROOTS)) return 'WORKFLOW'; + if (DOC_FILE.test(normalized) || isInstructionFile(normalized)) return 'DOCS'; + if ( + RUST_ROOTS.some((root) => normalized.startsWith(root)) || + /(?:^|\/)(?:Cargo\.toml|Cargo\.lock)$/.test(normalized) || + normalized.endsWith('.rs') + ) { + return 'RUST_TAURI'; + } + if (normalized.startsWith('tests/')) + return TS_FILE.test(normalized) ? 'TYPESCRIPT_APPLICATION' : 'TEST_ONLY'; + if (TOOLING_FILES.has(normalized) || startsWithRoot(normalized, TOOLING_ROOTS)) return 'TOOLING'; + if ( + DEPENDENCY_FILES.has(base) || + normalized.startsWith('patches/') || + (normalized.startsWith('packages/') && base === 'package.json') + ) { + return 'DEPENDENCY_TOOLCHAIN'; + } + if (BUILD_CONFIG_FILES.has(base)) return 'BUILD_CONFIGURATION'; + if (TS_FILE.test(normalized)) return 'TYPESCRIPT_APPLICATION'; + return 'UNKNOWN'; +} + +// QNBS-v3: classify the outgoing impact before starting expensive local checks. +export function classifyChangedFiles(files) { + const normalizedFiles = [...new Set(files.map(normalizePath).filter(Boolean))].sort(); + const categories = [...new Set(normalizedFiles.map(classifyFile))]; + + if (normalizedFiles.length === 0) + return { kind: 'NO_CHANGES', categories, files: normalizedFiles }; + if (categories.every((category) => category === 'DOCS')) { + return { kind: 'DOCS_ONLY', categories, files: normalizedFiles }; + } + if (categories.every((category) => category === 'WORKFLOW')) { + return { kind: 'WORKFLOW_ONLY', categories, files: normalizedFiles }; + } + if (categories.length === 1) { + if (categories[0] === 'UNKNOWN') + return { kind: 'AMBIGUOUS', categories, files: normalizedFiles }; + return { kind: categories[0], categories, files: normalizedFiles }; + } + if ( + categories.every((category) => ['DOCS', 'WORKFLOW', 'TOOLING', 'TEST_ONLY'].includes(category)) + ) { + return { kind: 'NON_CODE_ONLY', categories, files: normalizedFiles }; + } + if (categories.includes('UNKNOWN')) + return { kind: 'AMBIGUOUS', categories, files: normalizedFiles }; + return { kind: 'MIXED', categories, files: normalizedFiles }; +} + +export function requiresTypecheck(classification, { full = false } = {}) { + if (full) return true; + return ![ + 'NO_CHANGES', + 'DOCS_ONLY', + 'WORKFLOW_ONLY', + 'NON_CODE_ONLY', + 'RUST_TAURI', + 'TOOLING', + 'TEST_ONLY', + ].includes(classification.kind); +} diff --git a/scripts/ci-prepush-lowend.mjs b/scripts/ci-prepush-lowend.mjs index 20ba9d4bb..5a0a3196d 100644 --- a/scripts/ci-prepush-lowend.mjs +++ b/scripts/ci-prepush-lowend.mjs @@ -1,43 +1,83 @@ +import { spawnSync } from 'node:child_process'; import process from 'node:process'; +import { shouldRunAdmissionCheck } from './ci-prepush-check-registry.mjs'; +import { classifyChangedFiles, requiresTypecheck } from './ci-prepush-classifier.mjs'; import { ensureDependencyState, runLocalBinary, runNodeScript } from './hooks/shared.mjs'; -const checks = [ - ['toolchain', () => runNodeScript('scripts/check-pnpm-toolchain.mjs', ['--hook'])], - [ - 'typecheck (single checker)', - // QNBS-v3: Make the low-end resource contract explicit; tsgo's default checker count is not a safe local default. - () => - runLocalBinary('tsgo', ['--project', 'tsconfig.tsgo.json', '--noEmit', '--checkers', '1']), - ], - ['i18n key parity', () => runNodeScript('scripts/check-i18n-keys.mjs')], - ['i18n bundle rebuild', () => runNodeScript('scripts/build-i18n.mjs')], - ['i18n content guard', () => runNodeScript('scripts/content-guard.mjs')], - [ - 'i18n translation quality', - () => - runNodeScript('scripts/i18n-quality-report.mjs', [ - '--strict', - '--min-coverage', - '75', - '--max-length-outliers', - '8', - ]), - ], - ['release/doc truth', () => runNodeScript('scripts/check-doc-metrics.mjs')], - ['CSP policy', () => runNodeScript('scripts/check-csp-policy.mjs')], - ['desktop import boundary', () => runNodeScript('scripts/check-tauri-import-boundary.mjs')], - ['native readiness', () => runNodeScript('scripts/check-native-readiness.mjs')], -]; - -if (!ensureDependencyState()) process.exit(1); - -for (const [name, run] of checks) { - console.log(`[local-lowend] ${name}`); +const full = process.argv.includes('--full'); + +function gitRaw(args) { + const result = spawnSync('git', args, { encoding: 'utf8' }); + if (result.status !== 0) return ''; + return result.stdout ?? ''; +} + +function parseNulDelimitedPaths(output) { + return output.split('\0').filter(Boolean); +} + +function changedFilesFromWorkingTree() { + return parseNulDelimitedPaths( + gitRaw(['diff', '--no-renames', '--name-only', '-z', 'HEAD']), + ).concat(parseNulDelimitedPaths(gitRaw(['ls-files', '--others', '--exclude-standard', '-z']))); +} + +function report(name, status, detail = '') { + console.log(`[local-admission] ${name.padEnd(26)} ${status}${detail ? ` — ${detail}` : ''}`); + return status; +} + +function runCheck(name, run) { const status = run(); - if (status !== 0) { - console.error(`[local-lowend] failed: ${name}`); - process.exit(status); - } + report(name, status === 0 ? 'PASS' : 'FAIL'); + if (status !== 0) process.exit(status ?? 1); +} + +const files = changedFilesFromWorkingTree(); +const classification = classifyChangedFiles(files); +const typecheckRequired = requiresTypecheck(classification, { full }); + +console.log(`[local-admission] change class: ${classification.kind}`); +console.log(`[local-admission] files considered: ${classification.files.length}`); + +if (!ensureDependencyState()) { + report('Dependency state', 'FAIL'); + process.exit(1); +} +report('Dependency state', 'PASS'); + +runCheck('Toolchain', () => runNodeScript('scripts/check-pnpm-toolchain.mjs', ['--hook'])); +runCheck('Docs/release truth', () => runNodeScript('scripts/check-doc-metrics.mjs')); +runCheck('CSP policy', () => runNodeScript('scripts/check-csp-policy.mjs')); +runCheck('Desktop import boundary', () => runNodeScript('scripts/check-tauri-import-boundary.mjs')); +runCheck('Native readiness', () => runNodeScript('scripts/check-native-readiness.mjs')); + +if (shouldRunAdmissionCheck('i18n', classification.files) || full) { + runCheck('i18n key parity', () => runNodeScript('scripts/check-i18n-keys.mjs')); + runCheck('i18n bundle rebuild', () => runNodeScript('scripts/build-i18n.mjs')); + runCheck('i18n translation quality', () => + runNodeScript('scripts/i18n-quality-report.mjs', [ + '--strict', + '--min-coverage', + '75', + '--max-length-outliers', + '8', + ]), + ); +} + +if (shouldRunAdmissionCheck('contentGuard', classification.files) || full) + runCheck('Content guard', () => runNodeScript('scripts/content-guard.mjs')); + +if (typecheckRequired) { + runCheck('TypeScript (single checker)', () => + runLocalBinary('tsgo', ['--project', 'tsconfig.tsgo.json', '--noEmit', '--checkers', '1']), + ); +} else { + report('TypeScript', 'DEFERRED_TO_REQUIRED_CI', 'no TypeScript-impacting changes detected'); } -console.log('[local-lowend] pre-push checks passed sequentially.'); +console.log('\nLOCAL ADMISSION RESULT'); +console.log('Local checks completed sequentially.'); +console.log('Cloud validation required YES'); +console.log(`Classification ${classification.kind}`); diff --git a/tests/unit/tooling/ciPrepushClassifier.test.ts b/tests/unit/tooling/ciPrepushClassifier.test.ts new file mode 100644 index 000000000..1999470f7 --- /dev/null +++ b/tests/unit/tooling/ciPrepushClassifier.test.ts @@ -0,0 +1,48 @@ +import { describe, expect, it } from 'vitest'; +import { shouldRunAdmissionCheck } from '../../../scripts/ci-prepush-check-registry.mjs'; +import { + classifyChangedFiles, + classifyFile, + requiresTypecheck, +} from '../../../scripts/ci-prepush-classifier.mjs'; + +describe('change-aware local admission classifier', () => { + it.each([ + ['README.md', 'DOCS'], + ['.github/workflows/ci.yml', 'WORKFLOW'], + ['tests/unit/example.test.ts', 'TYPESCRIPT_APPLICATION'], + ['tests/fixtures/example.json', 'TEST_ONLY'], + ['src-tauri/src/lib.rs', 'RUST_TAURI'], + ['scripts/check-example.mjs', 'TOOLING'], + ['unknown-extension.data', 'UNKNOWN'], + ])('classifies %s as %s', (file, expected) => { + expect(classifyFile(file)).toBe(expected); + }); + + it('runs TypeScript for TypeScript tests but defers non-TypeScript tests', () => { + const ts = classifyChangedFiles(['tests/unit/example.test.ts']); + const fixture = classifyChangedFiles(['tests/fixtures/example.json']); + + expect(ts.kind).toBe('TYPESCRIPT_APPLICATION'); + expect(requiresTypecheck(ts)).toBe(true); + expect(fixture.kind).toBe('TEST_ONLY'); + expect(requiresTypecheck(fixture)).toBe(false); + }); + + it('uses the broad safe class for unknown or mixed changes', () => { + const unknown = classifyChangedFiles(['README.md', 'new-file.data']); + const mixed = classifyChangedFiles(['components/App.tsx', 'README.md']); + + expect(unknown.kind).toBe('AMBIGUOUS'); + expect(requiresTypecheck(unknown)).toBe(true); + expect(mixed.kind).toBe('MIXED'); + expect(requiresTypecheck(mixed)).toBe(true); + }); + + it('routes only locale-owned files to i18n and keeps content guard path-aware', () => { + expect(shouldRunAdmissionCheck('i18n', ['locales/en/common.json'])).toBe(true); + expect(shouldRunAdmissionCheck('i18n', ['README.md'])).toBe(false); + expect(shouldRunAdmissionCheck('contentGuard', ['community-templates/index.json'])).toBe(true); + expect(shouldRunAdmissionCheck('contentGuard', ['README.md'])).toBe(false); + }); +});