Skip to content

feat(ci): crude test-presence gate (untested-query MVP)#169

Merged
veksen merged 2 commits into
mainfrom
feat-untested-query-gate
Jul 7, 2026
Merged

feat(ci): crude test-presence gate (untested-query MVP)#169
veksen merged 2 commits into
mainfrom
feat-untested-query-gate

Conversation

@veksen

@veksen veksen commented Jul 7, 2026

Copy link
Copy Markdown
Member

Implements the MVP tracer bullet for the CI⇄MCP verdict loop — Query-Doctor/Site#3496 (parent epic Query-Doctor/Site#3493).

What & why

Query Doctor only analyses SQL that a real-DB test actually runs against Postgres. A PR that changes data-access code with no such test produces no captured query, so CI goes green having never seen it — and that silence reads as "safe." This gate reports the blind spot honestly instead.

In a PR, when data-access code changes but no real-DB (repository/integration) test changes alongside it, the run is flagged: a [!WARNING] banner in the PR comment (why + next step + flagged files + triage hint) and a failed check. It never claims a query is bad — only that it is unverified, flagged conservatively.

How

  • src/gate/test-presence.ts — pure diff heuristic. Buckets the PR's changed files into data-access code vs data-layer tests, and returns a verdict only when data-access changed with zero data-layer tests. No capture, no query-to-site mapping. A repository test counts as a data-layer test, so changing a repository and its test together passes; changing the repository alone flags. Removed/unchanged files never trip it.
  • src/gate/changed-files.ts — fetches the changed files from the GitHub API using the GITHUB_TOKEN the action already receives (more reliable than git diff under a shallow actions/checkout).
  • main.ts — runs the gate independent of the baseline comparison (fires even on a brand-new PR), threads the verdict into ReportContext, and core.setFaileds on it.
  • success.md.j2 — renders the v0 verdict payload as a conservative "could not verify" banner.

Heuristic (deliberately conservative)

Default "data-access" = paths matching *repository* / *.repo.* / dal/ / data-access/; "data-layer test" = a test file whose path signals repository/integration/dal. Tuned to under-fire (epic design principle 3) and to match this project's own *.repository.ts convention. Kept as overridable config data so the per-repo policy config (Query-Doctor/Site#3500) can tune it without touching gate logic.

Tests

src/gate/test-presence.test.ts (13) covers every acceptance criterion — flag / pass-with-test / no-change-unaffected / deletion / framing / diff-introduced-only — plus 2 render tests in github.test.ts. All existing reporter tests still pass. typecheck and build clean.

Scope

The versioned/shared verdict contract, failure taxonomy, and policy engine (Query-Doctor/Site#3497 / #3498 / #3500) live in the Site repo and are separate PRs; this ships the inline v0 payload the issue specifies.

🤖 Generated with Claude Code

The MVP tracer bullet for the CI-MCP verdict loop. In a PR, when
data-access code changes but no real-DB (repository/integration) test
changes alongside it, flag the run: Query Doctor never saw those queries
(nothing runs them against Postgres), so a green check would misread as
"safe". A pure diff heuristic over the PR's changed files (GitHub API) --
no capture, no query-to-site mapping.

Conservative by design: it asserts *unverified*, never *bad*, and only
fires when data-access changed with zero data-layer tests. Emits an
inline v0 verdict payload (why + next step + triage hint), renders it as
a "could not verify" banner in the PR comment, and fails the check.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Query Doctor Analysis

3 queries analyzed

0 regressed · 0 improved · 0 new · 0 removed

2 pre-existing issues

Using assumed statistics (10000000 rows/table). For better results, sync production stats.

More detail → get_ci_run({ runId: "019f3d3e-313a-7b51-a182-3b0eaa7de21e" }) · view run · docs

Raises the gate's accuracy from a filename guess to reading the diff.

- Content detection: classify a change as data-access when its added diff
  lines actually contain query code (ORM/query-builder calls or raw SQL),
  not when the filename looks like a repository. This kills the dominant
  false positives (a comment-only or whitespace edit to a repository file,
  a `UserRepositoryCard.tsx` component) and the dominant false negatives
  (a query added to a plainly-named service.ts / db.ts). Filename is now
  only a fallback for files whose patch is unavailable (large/binary).
- Comment lines are stripped before matching, and only source files are
  inspected, so prose or docs mentioning SQL keywords don't trip it.
- Relate test to code: a query change is satisfied only by a data-layer
  test in the same directory or carrying the file's stem -- you can no
  longer silence it by touching an unrelated repository test, and the
  comment names exactly which files lack a related test.
- Warn-only: surfaces via core.warning, not core.setFailed. The heuristic
  is still crude, so it warns loudly while the capture-based rungs
  (#3502/#3503) are built; flipping it to a hard failure is opt-in via the
  per-repo policy (#3500).

Still a pure diff heuristic -- no runtime capture, no query-to-site
mapping. Those remain #3502/#3503.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@veksen veksen merged commit d8a8fa0 into main Jul 7, 2026
6 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant