ci: restore machine-translated link targets before the build - #838
Merged
Conversation
Every push to main regenerates docs/zh with an LLM and only commits the
result if the site builds. Since the Pipeline Policy Constraints solution
landed, that build has failed on the same dead link in the generated
Chinese file:
"[..](URL)" -> /zh/solutions/URL.html
The English source has no such link: all 623 of its link targets are
anchors, external URLs or relative paths. The target `URL` is a verbatim
copy of the example inside doom's own translation prompt ("URLs in
markdown links: [text](URL) - keep URL exactly as is"). The document is
990KB, so translate splits it into 22 chunks, and the model leaks that
example into one of them -- reproducibly, in five consecutive runs.
Because the build gates the commit, this also held back every other
translation: two documents have no Chinese version at all and several
more are stale, while each push burns a full translation pass that is
then discarded.
Rather than rely on the model honouring the instruction, verify it. The
English document is the source of truth: walk both sides' links in
document order and restore any internal target that drifted. Scope is
deliberately narrow -- only internal route links, the exact set rspress
resolves and fails the build over. In-page anchors legitimately differ
(a translated heading gets a translated slug) and external URLs are
never resolved, so both stay out of it. When the two sides disagree on
how many internal links exist, positional alignment is unsound, so the
file is left untouched and reported for a human.
Also keep the generated docs as an artifact when a main build fails.
Nothing is committed on failure, so today the translation that broke the
build is discarded with it and cannot be inspected.
Checked against the whole repository: 400 of 401 translated documents
pass; the one exception is a stale Chinese file waiting on this same
blocked pipeline.
Positional alignment only holds when both sides expose the same number of internal links. That assumption is thin here: the document that broke the build has exactly one internal route link, in its last line, so a model that keeps that link and invents a second one -- rather than rewriting the one that exists -- would leave the counts unequal and the repair refused. The build error cannot tell those two cases apart: an invented dead link is reported either way, and a surviving valid link is not reported at all. Anchor on the links both sides agree on (longest common subsequence) and decide per gap instead. Equal counts still restore positionally. Links the English original does not have at all are the model's invention: strip the link syntax and keep the text, since a target that cannot be sourced from English must not be guessed at. Dropped links, and mixed gaps that are ambiguous in the same way, are still left for a human, as is any file where the number of inventions exceeds a small cap -- past that point the comparison itself is more likely to be wrong than the translation.
…he repair Reverted before merge. The workflow only reacts to pushes on main, so the translate -> repair -> build chain cannot be exercised from a pull request at all. Adding this branch to the push trigger and to each step's guard runs it here instead; the push-back step targets github.ref, so the generated translation lands on this branch and main is untouched.
…lidate the repair" This reverts commit e3b48c2.
A trial run of the real translation on this branch showed the dead link was not the whole story. The model did not rewrite an existing link: it dropped 44 lines of its own system prompt into the prose, translated into Chinese -- heading, terminology table, chunking notice and all -- and the `URL` target came from the prompt's own example. The prompt's `` example rode in with it, and once the link was repaired the build failed again on Module not found: Can't resolve 'src' in docs/zh/solutions Images cannot be aligned against English the way links are: normalizeImgSrc rewrites their paths between languages on purpose, so the two sides legitimately disagree and there is no stable value to anchor on. Anchoring on equal srcs was tried first and is wrong -- with one real image present, an invented one can no longer be told apart, and the file gets refused. Judge images on their own terms instead: a src that resolves to a file is fine however much it differs from the English one, and a src that resolves to nothing is invented markup. That is also the exact question rspack asks, so the check matches the failure it exists to prevent. Demote the latter to their alt text, under the same cap as links. Verified on the generated translation from run 33139819063: the repair strips the invented image and `yarn build` then exits 0. Injection tests: 26 assertions. Whole-repository baseline unchanged at 400 pass / 1 fail.
Repairing the dead link that broke the build turned out to be the small half of this problem. A trial translation run showed what actually happens when gpt-4o-mini loses the thread partway through a chunked document: it recited 44 lines of its own system prompt into the prose, in Chinese, and dropped an entire section along with 1177 lines of YAML. The dead link was incidental -- the `URL` target came from the prompt's own example. Had the model not happened to emit that one bad link, a document missing §4.2 and a third of its code would have been published with every check green. Checking the repository for the same signature found it was not a one-off: Software_Supply_Chain_Security… -46 code blocks, -77 headings, -35 table rows How_to_Use_PostgreSQL_Hot_Standby… -18 code blocks, -8 headings, -10 table rows Install_Multi-Primary_Service_Mesh… +7 table rows with no English original How_to_Migrate_MySQL_57_to_80… +1 code block with no English original Three of those are live Chinese pages missing about a third of their content. Nothing was capable of noticing, because the only thing the build validates about a translation is whether its links resolve. Two changes, addressing cause and detection separately. Cause: doom.config.ts now supplies its own translation prompt. The default one illustrates its rules with literal placeholders -- a link written as [text], an image written with a fake src -- which is exactly the text that ends up in the document when the model starts reciting, and exactly what failed the build. It also never asks for the translation to be complete: it constrains formatting and says nothing about content, so swallowing a section violates nothing it was told. The replacement describes the rules instead of illustrating them with copyable values, puts completeness first, and forbids emitting the instructions themselves. Detection: the checker now compares structure before references. Heading anchors, the heading outline, fenced code block count and table row count are not the translator's to change; when they differ, content was lost or invented and the file is reported rather than rewritten -- what is missing cannot be reconstructed, and repairing the links around it would only make the loss quieter. Anchors held across all 401 existing pairs, so the check is exact rather than heuristic. The five already-damaged documents are listed in .translation-known-damage, which downgrades them to KNOWN so this can ship without blocking main on debt it did not create. They still need retranslating; the list is the record. Renamed to check-translation-integrity.mjs, since links are now the smaller half of what it does. Baseline: 396 pass / 0 fail / 5 known-damaged. Injection tests: 26 assertions. Prompt template: 23 assertions.
- Add the KB document id metadata - Prevent machine translation from rewriting the English page - Keep the solution page aligned with the source target
The structural checks only see what a translation changed the shape of. A swallowed paragraph takes no heading, no code fence and no table row with it, so a page can lose a third of its text and still pass. That is how three articles in this repository ended up short without anyone noticing. Compare volume and digits instead: prose line count and the multiset of numbers outside URLs. Both must drop together before this fails -- Chinese legitimately merges English lines (healthy pages go as low as 0.33) and rewording loses the odd number, but no healthy page does both. Measured over all 401 en/zh pairs: 0 of 396 healthy pages flagged, and every content-losing entry in .translation-known-damage caught. URL digits are excluded because [http://x](http://x) and <http://x> carry them a different number of times; links are checked separately anyway.
Translated in 14 heading-aligned slices with the 160 code blocks lifted out first, so no slice boundary ever falls inside a fenced block and the code is restored byte for byte rather than retyped by a model. The English page carries i18n.disableAutoTranslation, so this file is the Chinese version from now on. Verified before assembly: the English slices reassemble to a byte-identical copy of the source, every cut lands on a balanced boundary, and each slice matches its original on line count, paragraph blocks, headings, anchors, table rows, list items, bold spans, numbers and inline code spans. After assembly all 160 code blocks are byte-identical, no link or image target was lost or invented, and the build emits the page with no dead links.
- Remove i18n.disableAutoTranslation flag - Allow translations to be generated automatically - Reverts prior translation disable to support broader language coverage
…tion" This reverts commit 6af998e. The English page no longer carries i18n.disableAutoTranslation, so this document is translated by CI again. A hand-made Chinese file would only be overwritten on the next English edit, and until then it would hide whether the hardened prompt in doom.config.ts actually holds up on the document that broke. The integrity check now blocks a translation that loses content, so a bad run fails the workflow instead of publishing a short page.
…ication - Merge separate translate and link-repair steps into a single unified translate-verified flow - Add translate-verified.mjs script to verify and retranslate incomplete documents before commit - Remove .translation-known-damage allowlist and replace with dynamic verification - Strip sourceSHA metadata from Chinese documents (no longer needed with verification) - Documents that fail verification are not committed, preventing incomplete translations from reaching the site
Two of these were missing about a third of their English original -- one had lost 46 code blocks, 77 headings and every one of its 35 table rows -- and two carried content the original never had. They were live on the site that way, because nothing checked anything but links. Retranslated in heading-aligned slices with the code blocks lifted out, so the code is restored byte for byte rather than retyped. Each slice matches its original on line count, paragraph blocks, headings, anchors, table rows, list items, bold spans, numbers and inline code spans; each document then matches on all 160-odd code blocks byte for byte with no link or image lost or invented. sourceSHA is set from the current English file, so CI treats these as up to date and will retranslate them only when the English changes.
git status collapses an untracked directory into a single "?? dir/" entry, so a translation written into a folder that did not exist before was never in scope for the check -- exactly the case where a fresh page reaches the site with nothing having looked at it. -uall lists those files individually.
This reverts commit d49242d. These pages are translated by CI, so a hand-made version would sit in the repo until the next English edit replaced it anyway. Dropping their sourceSHA is enough: doom retranslates a document whose recorded hash no longer matches, and scripts/translate-verified.mjs now keeps retrying until the result passes the integrity check, so the regenerated pages are verified rather than trusted. The damaged content stays on the site until that run succeeds -- it is what is published today -- and if the retries cannot produce a complete translation the workflow fails instead of committing another short page.
The integrity check had two blind spots, both of which let a broken translation through the very gate built to stop it. Only structural failures reached the --failures list. A document whose links could not be aligned, or whose repair did not converge, failed the run but was never named, so translate-verified.mjs did not hand it back to the translator. Worse, when such a failure was the only one, the list came back empty and the run reported "integrity check failed without naming a document" and exited without a single retry -- while the check had in fact named it, on stdout, one line earlier. Every fail++ now records its document. The scan is driven by the files under docs/<target>, so a page translate never produced was invisible to it: it neither passed nor failed, and the run reported success with the page missing from the site. That is the shape of the outage this script exists for -- docs/en currently holds two pages with no Chinese counterpart, one of them the document that has been failing main since 2026-08-14. A missing-translation audit now runs in full whatever the scope, honouring i18n.disableAutoTranslation the way doom does. Also here: - scripts/check-translation-integrity.test.mjs, 33 assertions against the real script over throwaway docs trees: the prompt-leaked `URL` target is detected, repaired byte-for-byte and idempotent; translated anchors, external URLs and JSX-nested fences are left alone; dropped, reordered and invented references are refused rather than guessed at; a document that lost content is reported and never rewritten. Wired into CI on every event, not just the pushes that translate. - Restore sourceSHA on four of the five Chinese pages. Dropping it forces a retranslation, and only the software-supply-chain page is ours to vouch for; the other four stay as they are until their owners ask. - Comments that had drifted from the code: a script name that no longer exists, a `.translation-known-damage` file that never landed, a guard the prose promised and the code does not have, and a sentence cut off mid-word. - .gitattributes: normalise line endings, matching .editorconfig. Without it doom.config.ts drifts back to CRLF and reads as a whole-file diff.
l-qing
commented
Aug 28, 2026
| - Alauda Container Platform | ||
| kind: | ||
| - Solution | ||
| sourceSHA: 7e3980ae21e998f2fd0014ace7f2cd8b3264c605d2948ce55999536f20eab834 |
yangxiaolang
approved these changes
Aug 28, 2026
Reverting the hand-made retranslation put the whole file back, including the sourceSHA line the revert message said was the point of the change. With the recorded hash matching again, doom skips these documents and the damaged text stays on the site. Only the hash line goes; the content is left for CI to regenerate and for scripts/translate-verified.mjs to verify before anything is committed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
mainhas failed on every push since c235660 (2026-08-14). Five consecutive runs report the same dead link, in a file that does not exist in the repository:Runs: 31781854002, 32096446452, 32120197877, 32976230317, 33057885704.
What is going on:
docs/zh/...is generated in CI (yarn translate) and only committed afteryarn buildpasses.#anchor,http(s)://,./or../.[..]is rspress's placeholder for the link text (link.jscheckDeadLinks), not real text. The target is the literal stringURL.- URLs in markdown links: [text](URL) - keep URL exactly as is.translateWithChunkssplits it into 22 chunks (Content size (1282KB) exceeds limitin the logs) andgpt-4o-minileaks the prompt example into one of them — reproducibly.Because the build gates the commit, the blast radius is larger than one link: two documents have no Chinese version at all, several more are stale, and every push burns a full ~50 minute translation pass that is then thrown away.
What this does
1. Verify the invariant instead of trusting the model. New
scripts/check-translation-links.mjswalks the links of each translated document and its English original in document order and restores any internal target that drifted. It runs with--fixbetweentranslateandBuild.Scope is deliberately narrow — only internal route links, the exact set rspress resolves against the route table and fails the build over:
./foo.md,/zh/bar#anchornormalizeLinkreturns early on#http(s)://,mailto:,tel:normalizeImgSrcrewrites image paths between languages by designFenced code blocks (including fences nested inside
<Tabs>/<Tab>), inline code spans and frontmatter are masked out first, so YAML regexes such as^feat[/-](?<BranchName>.+)are not mistaken for links.When the two sides disagree on how many internal links exist, positional alignment is unsound: the file is left untouched and reported for a human.
2. Keep the evidence when a main build fails. Nothing is committed on failure, so today the generated translation is discarded along with the run. The new
if: failure()step uploadsdocs/en+docs/zhas an artifact (7 days).docs/enis included becauseadd_id.shrewrites it earlier in the same run.Verification
node scripts/check-translation-links.mjs --all→400 pass / 1 fail. The single failure isdocs/zh/solutions/ecosystem/opensearch/How_to_Migrate_from_Elasticsearch_to_OpenSearch.md, whose English original gained three internal links in eff0124 while the Chinese file is still at 791fdf9 — i.e. the check correctly detects staleness caused by this very outage, and it resolves itself as soon as translation can commit again.URLis detected without--fixand restored with it, byte-for-byte identical everywhere else; the repair is idempotent; translated anchors, external autolinks, fenced code and JSX-nested fences are left alone; a missing internal link is reported with both counts and the file is not modified.yarn buildexits 0 locally with no dead-link errors.What this does not do
The 990KB / 11534-line document is a six-fold outlier over the next largest file in the repository, and links are only the failure mode that happens to break the build. The overall quality of a 22-chunk
gpt-4o-minitranslation — tables, code blocks, terminology — is worth a separate look, and splitting the document is the root-cause fix. The workflow also pinsAZURE_OPENAI_MODEL: gpt-4o-mini, older and weaker than doom's own default ofgpt-4.1-mini; left unchanged here.