Skip to content

Close per-query querier in the remote read handler#7672

Open
anxkhn wants to merge 1 commit into
cortexproject:masterfrom
anxkhn:fix/remote-read-querier-close
Open

Close per-query querier in the remote read handler#7672
anxkhn wants to merge 1 commit into
cortexproject:masterfrom
anxkhn:fix/remote-read-querier-close

Conversation

@anxkhn

@anxkhn anxkhn commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

What this PR does:

The remote read handler creates a storage.Querier for each sub-query but never calls Close(). It is the only storage.Querier call site in Cortex that does not close the querier, so it does not honor the interface contract.

The concrete queriers in the current read path have Close() methods that return nil, so this is not an observed resource leak today. It is a correctness and consistency fix that ensures future or wrapped queriers which release resources are closed deterministically.

This adds defer querier.Close() immediately after successful creation, covering both success and error paths. A regression test verifies every created querier is closed.

Which issue(s) this PR fixes:

No existing issue; found by code inspection.

Checklist

  • Tests updated
  • Documentation added (n/a, no user-facing docs or flags changed)
  • CHANGELOG.md updated
  • docs/configuration/v1-guarantees.md updated if this PR introduces experimental flags (n/a, no flags)

This change was prepared with AI assistance per the project policy. I reviewed every line, verified the interface and concrete implementations, and validated the tests and linters.

@dosubot dosubot Bot added component/querier go Pull requests that update Go code type/bug labels Jul 5, 2026
@siddarth2810

Copy link
Copy Markdown
Contributor

Good catch, seems good to me.

One minor thing in the description, in tenant-federation merge queryable Close() returns just nil, so I'm thinking if its a weak example for "releases the resources of the Querier" ?

The remote read handler creates a storage.Querier for each sub-query in
its own goroutine but never calls Close() on it. It is the only
storage.Querier call site in the codebase that does not close the
querier, so it does not honor the interface contract.

The concrete queriers wired into the read path today all have a Close()
that returns nil, so this is not an observed leak in the current code. It
is a correctness and consistency fix: any querier that does release
resources in Close() (a future implementation, a wrapper, or an upstream
Prometheus change) is then released deterministically instead of relying
on GC.

Defer querier.Close() inside the goroutine so it runs on every path,
matching every other querier call site.

Add a regression test asserting that each per-query querier is closed.

Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
@anxkhn anxkhn force-pushed the fix/remote-read-querier-close branch from e405a42 to 89e69b8 Compare July 16, 2026 08:18
@anxkhn anxkhn changed the title Close querier in remote read handler to fix querier leak Close per-query querier in the remote read handler Jul 16, 2026
@anxkhn

anxkhn commented Jul 16, 2026

Copy link
Copy Markdown
Contributor Author

Good point. You are right that mergeQuerier.Close() is a weak example. I checked the concrete queriers in the current read path, and their Close() methods are effectively no-ops today, so the original description overstated this as an existing resource leak. I corrected the title, description, CHANGELOG, and commit message to frame this as honoring the storage.Querier contract and keeping the handler consistent with other call sites.

I also fixed both CI failures: the test now uses go.uber.org/atomic to satisfy faillint, and polls for deferred closes because they can finish just after ServeHTTP returns.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component/querier go Pull requests that update Go code size/M type/bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants