Repository navigation
Conversation
|
Reviewed end to end against the live stack (Loki 3.7.7 direct vs this branch's binary vs current main). This is the strongest of the four and the fix is confirmed live. Confirmed fixed (Loki / main / this branch):
Before merge:
Merge order across the four: this one first, then #570, then #574. #571 should not merge as it stands (see my review there). |
szibis
left a comment
There was a problem hiding this comment.
Decision: accept, merge first of the four, after three small changes.
The fix is real and confirmed live ({app=~"otel"}: Loki empty, main 8 streams, this branch empty).
Please do:
- Reword the
service_nameline in the CHANGELOG: the selector fan-out is not fixed here ({service_name=~"otel"}still returns 1 stream where Loki returns 0). PR #579 fixes the derivation. - Emit
^(?s:…)$instead of^(?:…)$to match Prometheus exactly, and make the CHANGELOG and the code comment agree. - Add one proxy-vs-Loki parity case for
{app=~"<prefix>"}wired into a CI shard, and a line indocs/real-window-compatibility-gaps.mdsince results shrink for existing queries.
Also remove Co-Authored-By: Claude … from the commits and the "Generated with Claude Code" line from the PR body: this repo allows no AI attribution. We squash-merge, so we can strip it from the squash message if it stays, but please drop it at the source.
A LogQL label matcher is a full-value match: Loki builds it with
Prometheus' labels.NewMatcher, which compiles a MatchRegexp as
"^(?s:" + v + ")$". VictoriaLogs' field:~"re" is an unanchored match
and the proxy passed the pattern through verbatim, so every =~ / !~
matcher was silently widened: {namespace=~"nch"} also selected "anch"
and "nch-b", {app=~"a|b"} matched "ab".
AnchorLabelMatcherRegex wraps the pattern once, in internal/logsql, and
every emitter of a label-matcher regexp uses it: the stream selector,
pipeline label filters, the derived service_name matcher and label
filter (which already anchored with the same form) and the multi-tenant
__tenant_id__ fan-out, which compiled the bare pattern with Go regexp.
The form is Prometheus' own ^(?s:...)$, so the Go-compiled call site is
literally what Prometheus compiles too.
Line filters (|~, !~) are substring regexps in Loki and are left alone.
A parity case with a prefix matcher ({app=~"api"}, empty on Loki) joins
the query-semantics matrix, and the docs record that results shrink for
existing substring-style matchers.
ed3fa40 to
4e6afbe
Compare
|
Thanks for the live confirmation. Rebased onto current
Local: gofmt, build, vet, golangci-lint (0 issues), changelog and conformance gates, |
|
Thank you @clain23. Your commit landed in #909 as authored: regexp stream selectors are now anchored to the whole value, as Loki's Prometheus matchers are. #909 also completes it for label filters after a parser, which Loki first simplifies (an equality check, a substring check, or anchored otherwise), and registers a fuzz target for both. Closing this one as merged through #909. |
Summary
A LogQL label matcher is a full-value match. Loki builds its matchers with Prometheus'
labels.NewMatcher, andMatchRegexpcompiles the pattern as"^(?s:" + v + ")$"(prometheus/prometheus,model/labels/regexp.go), so{namespace=~"nch"}selects exactly the valuench.VictoriaLogs'
field:~"re"is an unanchored match, and the proxy passed the pattern through verbatim. Measured against the pinned VictoriaLogs v1.52.0, over a row whoseappisanchor-test-xyz:app:~"anchor"anchor-test-xyzrow)app:~"^(?s:anchor)$"So every
=~/!~matcher the proxy emitted was silently widened. A widened matcher inflates results rather than erroring, which is why nothing surfaced it:{namespace=~"nch"}on valuesnch,anch,nch-bnchnch{app=~"a|b"}onab{ns=~"prod"}onproductionAnchorLabelMatcherRegexwraps the pattern once, ininternal/logsql, and every emitter that carries a label-matcher regexp uses it:internal/logqltranslateLabelMatcher(MatchRe,MatchNotRe);internal/translatorstreamMatcherToFieldFilterfor the stream selector, andtranslateSingleLabelFilterfor a pipeline label filter such as| status =~ "5..";service_namematcher and label filter from fix: derive service_name as Loki assigns it, for selectors and grouping #579, which already anchored their patterns with the same form and now share the helper;__tenant_id__=~fan-out, which compiled the bare pattern with Goregexpand so also matchedprod-euforprod.The emitted form is Prometheus'
^(?s:…)$. On VictoriaLogs thesflag changes nothing (it compiles with DotNL on, verified on v1.52.0), but the multi-tenant path compiles the same helper's output with Goregexp, where(?s)is not the default, so both call sites are now literally what Prometheus compiles. The non-capturing group is load-bearing:^a|b$parses as(^a)|(b$), so an alternation has to be grouped before it is anchored.Line filters (
|~,!~) are deliberately left alone: those are substring regexps in Loki, which is what VictoriaLogs already does. The pipeline label filter anchors after the empty-value branch, so| level =~ ""keeps its absent-or-empty translation.Changelog
One
### Fixedentry (label matchers,__tenant_id__) and one### Changedentry (substring-style=~stops matching) under## [Unreleased].Docs
docs/translation-reference.md: the stream-selector and label-filter rows show the anchored form, with one sentence on why.docs/real-window-compatibility-gaps.md: one correction line, since results shrink for existing queries.Validation
TestAnchorLabelMatcherRegexMatchesLokiSemantics(new): a(pattern, value, shouldMatch)table, including the reportednch/anch/nch-bcase, the alternation leak and the empty pattern.TestAnchorLabelMatcherRegex(new) pins the emitted shape.TestLabelMatchersAreAnchoredLineFiltersAreNot(new,internal/translator) covers the boundary end to end, including one query carrying both kinds:{app=~"a|b"} |~ "substring"→app:~"^(?s:a|b)$" ~"substring". It fails onmainon every anchored case.TestRegexpMatchersReachVictoriaLogsUnescapedandFuzzRegexpMatcherEscapingnow assert the anchored pattern as the one Loki compiles; the other updated expectations acrossinternal/logql,internal/translatorandinternal/proxyall encoded the widened form.selector_regex_prefix_is_not_substringintest/e2e-compat/query-semantics-matrix.json({app=~"api",namespace="prod",env="production"}, line count compared with Loki, empty on both sides), tracked underselector_regexin the operations inventory and run byTestQuerySemanticsMatrix.gofmt -s -lempty,go build ./...,go vet ./...,golangci-lint run(v2.13.2, 0 issues),scripts/ci/check_changelog_pr.pyandscripts/ci/check_conformance.pyok,go test ./... -count=1green across all packages, on top of currentmain(v1.86.0).