Skip to content

fix(datafusion): preserve regexp_match null checks - #9698

Open
lance-gatefixer[bot] wants to merge 1 commit into
mainfrom
gatekeeper/fix-9695-1
Open

lance-gatefixer[bot] wants to merge 1 commit into
mainfrom
gatekeeper/fix-9695-1

Conversation

@lance-gatefixer

Copy link
Copy Markdown
Contributor

An explicit regexp_match(...) IS NOT NULL filter became IsNotNull(IsNotNull(regexp_match(...))), selecting every row, including NULL inputs. The corresponding IS NULL filter selected no rows. Filter coercion also rewrote regex results inside comparisons.

Restrict coercion to the filter root and AND/OR/NOT operands. Explicit null checks and comparisons now retain DataFusion's nullable list result, while bare regex filters keep their existing behavior. Extend the planner tests with matching, nonmatching, and NULL rows, boolean combinations, and value-context preservation.

Validation:

  • Before the fix, eight regression cases failed, including the reported six-row reproduction.
  • cargo test --locked --profile ci -p lance-datafusion: 198 unit tests and five doctests passed; three existing doctests were ignored.
  • cargo fmt --all passed.
  • cargo clippy --locked --profile ci --all --tests --benches -- -D warnings passed.

Fixes #9695

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

Tip: disable this comment in your organization's Code Review settings.

@github-actions github-actions Bot added the bug Something isn't working label Oct 3, 2026

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Gate recommendation: approve.

The shared filter-planner fix addresses the incorrect row selection in #9695. Explicit null checks and comparisons retain the regex result’s nullable list semantics, while direct boolean filters retain their existing coercion.

@lance-gatekeeper lance-gatekeeper Bot added the K-approved Latest Gatekeeper recommendation permits acceptance. label Oct 3, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working K-approved Latest Gatekeeper recommendation permits acceptance.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Filter regexp_match(col, pat) IS NOT NULL matches every row including NULLs

0 participants