Skip to content

fix(curation): count parsed queries during dry-run - #736

Merged
kstonekuan merged 1 commit into
Hebbian-Robotics:mainfrom
Lohith248:fix/curate-dry-run-sql
Oct 27, 2026
Merged

kstonekuan merged 1 commit into
Hebbian-Robotics:mainfrom
Lohith248:fix/curate-dry-run-sql

Conversation

@Lohith248

@Lohith248 Lohith248 commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Make curate(..., output=None) and CLI --dry-run accept saved SQL ending in ;, a final line comment, or both. These queries already write manifests successfully; the dry-run count wrapper introduces their syntax errors.

 dry_run(sql)
   _reject_tenant_sql(sql)
-  execute("SELECT count(*) FROM (" + sql + ")")
+  connection.sql(sql).aggregate("count(*)").fetchone()

Count the parsed DuckDB relation after the unchanged statement gate. Extend existing SQL-surface and CLI tests, including --sql-file.

Evidence

  • Before: five regression cases fail on main 43380ac; both CLI input modes exit 2. Four selected controls pass, including manifest writes.
    After: 183 catalog/curation tests pass, including existing non-single-SELECT refusal cases.

  • A real synthetic MCAP → App.test(record=True) → catalog → saved SQL-file run previously failed dry-run but wrote a one-row manifest. Afterward, both CLI modes exit 0 with one row and joints: 1/1 (100%). Dry-run creates no manifest.

  • Exact validation from the repository root:

    uv run --locked pytest tests/test_catalog_curation.py -q
    UV_PROJECT_ENVIRONMENT="$HOME/.cache/hflow-onboarding/venv311" uv run --python 3.11 --locked pytest tests/test_catalog_curation.py -q -k 'sql_surface or dry_run or reports_duckdb_execution_errors'
    uv run --locked pytest -q -n 4 -rs
    uv run --locked ruff check --fix
    uv run --locked ruff format
    uv run --locked ty check
    git diff --check

    Python 3.14 full suite: 2,564 passed, 8 skipped. Python 3.11 focused suite: 42 passed, 141 deselected in a separate environment. Skips require network opt-in, model assets, Docker, or a live bucket. Quality checks passed; full-suite prerequisites included FFmpeg 8.1 and a C compiler.

Merge Danger

Door: two-way

Revertible without data migration.

Blast Radius: curation

Only report-only counts change. Statement validation and constrained connection settings remain in place. No dependencies, stored formats, canonical bytes, or manifest-publication changes.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

Copilot AI balanced review requested due to automatic review settings October 24, 2026 17:06

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@greptile-apps

greptile-apps Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no actionable issues were found.

Summary

Changes dry-run counting to use the parsed DuckDB relation instead of wrapping SQL text.

  • Dry-run curation counts queries with trailing semicolons or comments.

No actionable issues found. Tests were inspected, not run; DuckDB is unavailable in the review environment.

Reviews (1) · Last reviewed commit: "fix(curation): count parsed queries duri..." · Reviewed by Greptile

@kstonekuan kstonekuan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix. Ruff, formatting, ty, and the full suite passed (2564 passed, 8 skipped). LGTM.

@kstonekuan
kstonekuan merged commit 7c9ced5 into Hebbian-Robotics:main Oct 27, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants