From d4702c93ab0f8d937bb8ab0547bbd260e9f19cde Mon Sep 17 00:00:00 2001 From: Nick Sullivan Date: Tue, 28 Jul 2026 22:08:42 -0500 Subject: [PATCH] ci: stop formatters fighting generated data files [no-deploy] The pre-commit gate has been red on `main` for six consecutive runs since 2026-07-25. The failures were not caused by the PRs they appeared on: a clean `main` reproduces them exactly. PR #23 sat blocked for five review passes on a check that had nothing to do with it. Root cause is two formatters disagreeing with two generators, permanently: - `scripts/fetch-model.py:559` writes model-data.json via `json.dump(..., indent=2)`, expanding arrays one element per line. Prettier collapses short arrays. Each pipeline run re-breaks the hook. - `generate_llms_txt()` re-emits upstream OpenRouter descriptions verbatim, which carry smart quotes and trailing whitespace. The trailing-whitespace and fix-smartquotes hooks rewrite them on every run. These files are machine-written and machine-read. Their formatting is decided by the generator, so a style hook there cannot catch a human mistake -- it only reports the generator being itself. Excludes the three generated data files from Prettier via a new .prettierignore, and excludes llms.txt from the two text-fixup hooks. Verified, including the counterfactuals -- an exclusion that silences a check is worse than the red it replaces: - `pre-commit run --all-files` now exits 0 and rewrites nothing. - Deliberately malformatting js/app.js still FAILS Prettier, so real code is still gated. - Appending garbage to model-data.json still FAILS the pipeline test, so the ignored files are still checked for content, just not for style. The security hook (forbid-bidi-controls) is deliberately left applying to everything, including llms.txt. --- .pre-commit-config.yaml | 8 ++++++++ .prettierignore | 19 +++++++++++++++++++ 2 files changed, 27 insertions(+) create mode 100644 .prettierignore diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index a5d5f3a..825eddf 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -22,6 +22,11 @@ repos: args: [--allow-multiple-documents] - id: trailing-whitespace name: Trim whitespace + # llms.txt is generated by scripts/fetch-model.py from upstream + # OpenRouter descriptions, which carry trailing whitespace and smart + # quotes. Every pipeline run reintroduces them, so a formatting hook + # here fails permanently rather than catching anything. + exclude: ^model-benchmarks/llms\.txt$ - repo: https://github.com/Lucas-C/pre-commit-hooks rev: v1.5.5 @@ -34,6 +39,9 @@ repos: hooks: - id: fix-smartquotes name: Fix smart quotes + # Same reason as trailing-whitespace above: llms.txt is regenerated + # from upstream model descriptions that contain smart quotes. + exclude: ^model-benchmarks/llms\.txt$ - id: forbid-bidi-controls name: Forbid bidirectional text (security) diff --git a/.prettierignore b/.prettierignore new file mode 100644 index 0000000..df74a01 --- /dev/null +++ b/.prettierignore @@ -0,0 +1,19 @@ +# Prettier ignore +# +# Script-generated JSON data files. These are written by +# `model-benchmarks/scripts/*.py` via `json.dump(..., indent=2)`, which expands +# every array to one element per line. Prettier collapses short arrays onto a +# single line, so the two formatters disagree permanently: every pipeline run +# rewrites the file, Prettier reformats it, and the pre-commit hook fails. +# +# That conflict left `main` red on six consecutive pre-commit runs (2026-07-25 +# through 2026-07-28) and the failures were misattributed to whichever PR +# happened to be open, which is how PR #23 sat blocked for five review passes on +# a check that had nothing to do with it. +# +# These files are machine-written and machine-read; their formatting is decided +# by the generator, not by a human editing them. Prettier has no useful opinion +# here. +model-benchmarks/data/model-data.json +model-benchmarks/data/eqbench-upstream-keys.json +model-benchmarks/data/eqbench-raw.json