Tell a Timed-Out Node Probe From an Engine Disagreement in test_spec_validate by ptr727 · Pull Request #3192 · ptr727/ProjectTemplate

ptr727 · 2026-10-17T23:39:59Z

Summary

For #3182. The corpus test in test_spec_validate runs node under a 60 second timeout. A cold start on a slow hosted Windows runner raised TimeoutExpired and failed the suite as though the schema patterns disagreed.

A TimeoutExpired from the probe now skips the test with that reason, the same way a host without node already skips.
Every other failure is unchanged: a disagreement still fails assertEqual, and a pattern new RegExp rejects still fails through check=True.
Runaway backtracking can't hide behind the skip. Python's re.search evaluates every pattern over the same corpus before node starts, so the run hangs there instead.

Verification

A forced TimeoutExpired from the node call: the test reports skipped=1, with no failures or errors.
python3 -m unittest tests.test_spec_validate passes with node present.
ruff, mypy, the prose gate, and the eol gate are clean.
A local strict review found nothing.

🤖 Generated with Claude Code

…validate The corpus probe ran node under a 60 second timeout, and a cold start on a slow hosted Windows runner raised TimeoutExpired, failing the suite as though the schema patterns disagreed. A timeout now skips the test with that reason, as a missing node already does. A pattern that backtracks without end still fails rather than skipping, since Python matches the same corpus before node starts. Fixes #3182 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ptr727 · 2026-10-17T23:40:04Z

A recorded local strict-review pass covers head b6f5fbc44b0c2211d3560f5be94ca4579b5ee634, the content this pull request carries at that commit against develop, and it recorded 0 findings.

coderabbitai · 2026-10-17T23:40:08Z

Important

🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Repository: ptr727/ProjectTemplate/.coderabbit.yaml
Review profile: CHILL
Plan: Advanced
Run ID: 504b6ecc-d7df-454f-85dc-2c68f6d1194f

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

Copilot

🟡 Changes recommended

Add regression coverage for the timeout-skip path before approval.

1 open issue

Add regression test for timeout skip path · New

What changed in this PR

Updates the schema corpus test to distinguish Node timeouts from engine disagreements.

Changes:

Skips the test when the Node probe times out.
Preserves existing disagreement and probe-failure behavior.
Documents Python-side evaluation ordering.

File	Summary	Review
`tests/test_spec_validate.py`	Adds timeout handling to the Node probe test.	[P2] nit (3 votes): Add a regression test covering the timeout-skip path.

_{Review effort: Lite}

The skip was reachable only through a real 60 second timeout, so nothing protected it from being removed or mis-scoped. The new test makes the probe raise TimeoutExpired and asserts the corpus test skips with that reason rather than failing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The skip test fed in only TimeoutExpired, so an except widened to every subprocess error would still pass it while turning node refusing a pattern into a skip. It now also feeds in CalledProcessError and asserts that one errors rather than skipping. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

codecov · 2026-10-17T23:45:32Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 65.46%. Comparing base (0b5954f) to head (1eba44a).

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #3192      +/-   ##
===========================================
+ Coverage    65.44%   65.46%   +0.02%     
===========================================
  Files           17       17              
  Lines         9750     9750              
===========================================
+ Hits          6381     6383       +2     
+ Misses        3369     3367       -2

Flag	Coverage Δ
python-3.13	`65.46% <ø> (+0.02%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

ptr727 · 2026-10-17T23:45:40Z

A recorded local strict-review pass covers head 1eba44ada19ab7e67c7522bdb25ea56e6ac15dec, the content this pull request carries at that commit against develop, and it recorded 0 findings.

…o Main (#3195) - #3194 makes `marker_blocks` in `pr_review.py` visit every section bound in a wrapper, opening a block only where its own marker matches. A suppressed section after a previously-missed one in the same wrapper used to go unread under `shapes=ok`. The suppressed reader's missing vetted-but-unread check, which kept that silent, is #3193. - #3192 makes `test_spec_validate`'s corpus test skip when the `node` probe times out, the same way it skips when `node` is missing, so a slow hosted runner no longer reads as an engine disagreement. A new test pins the skip and proves that `node` refusing a pattern still errors. The change is to scripts and tests only, so no installed Skill changes at release. Closes #3180 Closes #3182 🤖 Generated with [Claude Code](https://claude.com/claude-code)

Copilot AI lite review requested due to automatic review settings October 17, 2026 23:40

Copilot started reviewing on behalf of ptr727 October 17, 2026 23:40

View session

Copilot AI reviewed Oct 17, 2026

View reviewed changes

Comment thread tests/test_spec_validate.py

ptr727 and others added 2 commits October 17, 2026 16:43

ptr727 merged commit 9351de5 into develop Oct 17, 2026
13 checks passed

ptr727 deleted the fix/3182-node-probe-timeout branch October 17, 2026 23:57

ptr727 mentioned this pull request Oct 18, 2026

Promote Alternating-Section Reading and the Node Probe Timeout Skip to Main #3195

Merged

ptr727 mentioned this pull request Oct 18, 2026

Session Handoff [review-tooling]: Alternating Sections and the Node Probe Timeout Skip Released as 2.0.1085, Suppressed Read Check Next #3196

Open

Navigation Menu

Repository navigation

Tell a Timed-Out Node Probe From an Engine Disagreement in test_spec_validate - #3192

Tell a Timed-Out Node Probe From an Engine Disagreement in test_spec_validate#3192
ptr727 merged 3 commits into
developfrom
fix/3182-node-probe-timeout

ptr727 commented Oct 17, 2026

Uh oh!

ptr727 commented Oct 17, 2026

Uh oh!

coderabbitai Bot commented Oct 17, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

codecov Bot commented Oct 17, 2026 •

edited

Loading

Uh oh!

ptr727 commented Oct 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

ptr727 commented Oct 17, 2026

Summary

Verification

Uh oh!

ptr727 commented Oct 17, 2026

Uh oh!

coderabbitai Bot commented Oct 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

codecov Bot commented Oct 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ptr727 commented Oct 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

coderabbitai Bot commented Oct 17, 2026 •

edited

Loading

codecov Bot commented Oct 17, 2026 •

edited

Loading