Repository navigation
test(parsers): check linear growth instead of fixed wall-clock ceilings in the timing tests - #787
Conversation
… the parser timing tests The timing tests asserted medians under fixed millisecond ceilings next to a growth-ratio check. A slow CI runner exceeds a fixed ceiling and the fixed child deadlines with no regression in the code. The growth check alone catches the superlinear behaviour these tests guard, so the ceilings go, the packed-hashtag test gains a growth check scaled by line length, and every timing child shares one deadline with room for slow runners. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
umm-actually re-reviewed at 1 new finding(s) posted (8 tracked finding(s) across all runs). umm-actually · z-ai/glm-5.3-flash |
…d field-grammar timing tests The heading timing test checked only the variant names, and the field-grammar timing test matched each fixture name with expect.any(String). Both outputs are deterministic apart from the medians, so each test now matches the whole result: the variant, sizes and three medians per heading series, and the 24 fixture names in the order the child times them. Ship-Check: test-audit · claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ld's deadline Vitest cannot interrupt a test blocked in execFileSync, and a killed child's own error is what vitest reports whatever the test timeout. The margin keeps a child that finishes inside its deadline from failing as a vitest timeout, which vitest checks once a synchronous test returns. Ship-Check: bug-check · claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he status-stamp timings The heading and tag-deduplication timing tests passed a bare 10 as the growth check's noise floor; it is now a named constant like the other two files' floor, with the same value. The status-stamp test was the only timing test that never printed its medians. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t their deadline Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Give the corpus regression test an explicit timeout like its siblings
This PR gives the 110,000-case Dataview corpus test an explicit 30 s timeout because 'the 5 s default ran out under local load', but the 10,000-case fixed-seed traversal corpus test in this same file still rides vitest's 5 s default. Its per-case work (a legacy-oracle parse, the current parse, a deep equality check, and per-task re-extraction over up to 24 lines) is far heavier than the regex execs the raised test runs, so a loaded runner can push it past 5 s and fail it spuriously with no parser change — the failure class this PR removes from the timing children. Failure scenario: A CI runner under load — the condition this PR exists for, where the old children 'fail CI at random with no change to the parsers' — runs the 10,000-case corpus test slowly enough to exceed vitest's 5 s default; the test red-balls with a timeout although the parsers are correct, the same '5 s default ran out under local load' failure the PR fixes for the sibling Dataview corpus test, and a local deep run with PARSER_CORPUS_CASES raised hits it sooner. Suggested fixAdd an explicit timeout matching the sibling corpus tests (e.g. close the test with `}, 30_000)`), or hoist a shared CORPUS_TEST_TIMEOUT_MS into helpers/linear-timing.ts beside the other timing constants.umm-actually · z-ai/glm-5.3-flash |
…and correct the floor comments' arithmetic The four fixed-seed corpus tests that still relied on vitest's 5 s default get the same 30 s as their siblings. The 10 ms floor comments now state each file's real margin: the heading traversal's quadratic would pass a 25 ms floor, while the tag-dedup fixture would fail it by only about 10%. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Issue comment 6840872688 (umm-actually, corpus test timeout): fixed in 32339fc. Every fixed-seed corpus test in the four timing files now carries an explicit 30 s timeout, the heading ranges test it named and the trailing-tag, emoji-recurrence and tag-removal-order corpora with it. 🔍 ship-check · pr-monitor · claude-opus-5-5 |
The 10 ms and 14 ms smallest medians hold on an idle fast machine; a loaded or slower runner measures several times more, and there the smallest median clears the 25 ms floor anyway. The arithmetic is unchanged. Ship-Check: test-audit · claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ut as its siblings Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… its two facts Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Ship-Check: code-quality · claude-opus-5-5
What does this PR do?
The parser timing tests (
tasks-linear,tasks-clear-linear,headings-linear,tasks-tag-dedup) fail CI at random with no change to the parsers. The timed child processes asserted two things about their medians:A slow runner breaks the first with no regression in the code, and a fixed per-test child deadline (10 to 30 s) breaks the same way. Failures seen on CI:
status stamp clear exceeded ceiling 259, 559, 1176(while that ceiling was 1,000 ms, before #755 raised it to 2,000 ms) andpacked hashtag edits exceeded ceiling 64, 226, 1025, and the status-stamp child killed at its 15 s deadline.This PR keeps the growth check as the speed test:
__tests__/helpers/linear-timing.ts), with the vitest timeout 10 s above it so a child that finishes inside its deadline never fails as a vitest timeout. The slowest child takes about 7 s on a fast laptop.LINEAR_GROWTH_ALLOWANCE, copied in four files, moves to the same helper. The two files that passed a bare 10 as the growth check's noise floor name it, with the same value.Type of change
Verification
tasks-clear-lineartook 27.6 s against 11.5 s on a fast laptop, which is the margin the old fixed deadlines lacked.tasks.tsandheadings.tsfrom before fix(parsers): keep task, heading, link, frontmatter and canvas parsing linear #753 (995118e^), all nine timing tests fail.Checklist
npm testpasses (CI)npm run buildsucceeds (CI)🤖 Generated with Claude Code