feat(ci): crude test-presence gate (untested-query MVP)#169
Merged
Conversation
The MVP tracer bullet for the CI-MCP verdict loop. In a PR, when data-access code changes but no real-DB (repository/integration) test changes alongside it, flag the run: Query Doctor never saw those queries (nothing runs them against Postgres), so a green check would misread as "safe". A pure diff heuristic over the PR's changed files (GitHub API) -- no capture, no query-to-site mapping. Conservative by design: it asserts *unverified*, never *bad*, and only fires when data-access changed with zero data-layer tests. Emits an inline v0 verdict payload (why + next step + triage hint), renders it as a "could not verify" banner in the PR comment, and fails the check. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Query Doctor Analysis
3 queries analyzed
0 regressed · 0 improved · 0 new · 0 removed
2 pre-existing issues
SELECT "guests"."id", "guests"."session_id", "guests"."username", "guests"."avatar_path", "guests"."color", "guests"."side", "guests"."audio_recording_path", "guests"."audio_recording_public", "gue...
indexassets(event_id, inserted_at desc)
cost 31,003,449 → 1,498 (100% reduction)SELECT * FROM guest_ip_addresses WHERE ip_address = '127.0.0.1';
indexguest_ip_addresses(ip_address)
cost 154,402 → 8 (100% reduction)
Using assumed statistics (10000000 rows/table). For better results, sync production stats.
More detail → get_ci_run({ runId: "019f3d3e-313a-7b51-a182-3b0eaa7de21e" }) · view run · docs
Raises the gate's accuracy from a filename guess to reading the diff. - Content detection: classify a change as data-access when its added diff lines actually contain query code (ORM/query-builder calls or raw SQL), not when the filename looks like a repository. This kills the dominant false positives (a comment-only or whitespace edit to a repository file, a `UserRepositoryCard.tsx` component) and the dominant false negatives (a query added to a plainly-named service.ts / db.ts). Filename is now only a fallback for files whose patch is unavailable (large/binary). - Comment lines are stripped before matching, and only source files are inspected, so prose or docs mentioning SQL keywords don't trip it. - Relate test to code: a query change is satisfied only by a data-layer test in the same directory or carrying the file's stem -- you can no longer silence it by touching an unrelated repository test, and the comment names exactly which files lack a related test. - Warn-only: surfaces via core.warning, not core.setFailed. The heuristic is still crude, so it warns loudly while the capture-based rungs (#3502/#3503) are built; flipping it to a hard failure is opt-in via the per-repo policy (#3500). Still a pure diff heuristic -- no runtime capture, no query-to-site mapping. Those remain #3502/#3503. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the MVP tracer bullet for the CI⇄MCP verdict loop — Query-Doctor/Site#3496 (parent epic Query-Doctor/Site#3493).
What & why
Query Doctor only analyses SQL that a real-DB test actually runs against Postgres. A PR that changes data-access code with no such test produces no captured query, so CI goes green having never seen it — and that silence reads as "safe." This gate reports the blind spot honestly instead.
In a PR, when data-access code changes but no real-DB (repository/integration) test changes alongside it, the run is flagged: a
[!WARNING]banner in the PR comment (why + next step + flagged files + triage hint) and a failed check. It never claims a query is bad — only that it is unverified, flagged conservatively.How
src/gate/test-presence.ts— pure diff heuristic. Buckets the PR's changed files into data-access code vs data-layer tests, and returns a verdict only when data-access changed with zero data-layer tests. No capture, no query-to-site mapping. A repository test counts as a data-layer test, so changing a repository and its test together passes; changing the repository alone flags. Removed/unchanged files never trip it.src/gate/changed-files.ts— fetches the changed files from the GitHub API using theGITHUB_TOKENthe action already receives (more reliable thangit diffunder a shallowactions/checkout).main.ts— runs the gate independent of the baseline comparison (fires even on a brand-new PR), threads the verdict intoReportContext, andcore.setFaileds on it.success.md.j2— renders the v0 verdict payload as a conservative "could not verify" banner.Heuristic (deliberately conservative)
Default "data-access" = paths matching
*repository*/*.repo.*/dal//data-access/; "data-layer test" = a test file whose path signals repository/integration/dal. Tuned to under-fire (epic design principle 3) and to match this project's own*.repository.tsconvention. Kept as overridable config data so the per-repo policy config (Query-Doctor/Site#3500) can tune it without touching gate logic.Tests
src/gate/test-presence.test.ts(13) covers every acceptance criterion — flag / pass-with-test / no-change-unaffected / deletion / framing / diff-introduced-only — plus 2 render tests ingithub.test.ts. All existing reporter tests still pass.typecheckandbuildclean.Scope
The versioned/shared verdict contract, failure taxonomy, and policy engine (Query-Doctor/Site#3497 / #3498 / #3500) live in the Site repo and are separate PRs; this ships the inline v0 payload the issue specifies.
🤖 Generated with Claude Code