Repository navigation
Conversation
|
Reviewed end to end. The core claim is correct and both halves were confirmed against the live stack. Entry at
So on main an unwrapped Before merge:
Merge order: land this before #574. #574 puts evaluation points on whole-second grid positions, which turns the boundary case fixed here from rare into systematic on second-precision ingest. |
szibis
left a comment
There was a problem hiding this comment.
Decision: accept, merge second, right after #572 and before #574, once two things are fixed.
Confirmed live in both directions: on main an unwrapped sum_over_time reads 8x low at the upper edge (4724 vs Loki 39196) and returns data where Loki returns none at the lower edge.
Please do:
- Delete or fix the two dead reference implementations (
buildBareParserMetricMatrix,buildBareParserMetricVector) that still includewindowStart;TestBoundedBareParserMetricOutputcurrently pins the live evaluator against the wrong one and passes only because its samples never touch a window edge. - State in the CHANGELOG that
rate_counterstays non-Loki (PrometheusextrapolatedRate), or drop it from the list of fixed functions; the new expectation is a different wrong value, as the code comment admits. - Update the PR body: the second commit changes the upper bound for every function, which is the larger behaviour change and is only in the CHANGELOG.
Also remove Co-Authored-By: Claude … from the commits: no AI attribution in this repo.
Merge order matters: this must land before #574, which puts evaluation points on whole-second grid positions and would turn this boundary case from rare into systematic.
362cff6 to
41f571f
Compare
|
Rebased onto current
Local: gofmt, build, vet, golangci-lint (0 issues), changelog and conformance gates, |
Summary
A LogQL range vector is
(start, end]for every range function, not only the log-line ones. The proxy got both edges wrong for the unwrapped functions, in opposite directions.Lower bound.
aggregateManualWindowcomputedexcludeStart := isLogRangeWindowFunc(functionName), sosum_over_time,avg_over_time,min_over_time,max_over_time,first_over_time,quantile_over_time,stddev_over_timeandstdvar_over_timekept the sample sitting exactly onwindowStart, the one Loki drops, and answered one sample high on every step. Loki v3.7.1,pkg/logql/range_vector.go,batchRangeVectorIterator.load:That iterator is what unwrapped ranges evaluate over too, so the bound is not a log-line special case.
Upper bound. VictoriaLogs'
/select/logsql/querytreatsendas exclusive where Loki's window is inclusive at its end. The proxy compensated with a 1 ns extension of the fetch end, but only forquantileand the log-range functions. Correcting the lower bound alone would have left every other unwrapped function open at both ends, one sample short instead of one sample high. The extension is now unconditional. In measurement this is the larger change: on the e2e stack an unwrappedsum_over_timeover[1s]read 8x low at the upper edge (4724 against Loki's 39196) and returned data at the lower edge where Loki returns none.Concretely, over samples
{0:1, 10:2, 20:3}and window(0, 20]:sum_over_timeavg_over_timemin_over_timefirst_over_timecount_over_time,rate,bytes_over_timeandbytes_rateare unchanged; they already excluded the lower bound.rate_counterstays non-Loki. It takes the same(start, end]window, but Loki evaluates it through Prometheus'extrapolatedRate, which needs the sample timestamps this evaluator discards. Its value is still(last - first) / rangeover the samples in the window, and the CHANGELOG says so instead of listing it among the fixed functions.Changes
aggregateManualWindow: the lower bound is exclusive unconditionally, and the slice-building branch (quantile,stddev,stdvar,rate_counter) passes the same flag instead offunctionName == "quantile".buildBareParserMetricMatrixandbuildBareParserMetricVector, which still includedwindowStartfor every function, are deleted.TestBoundedBareParserMetricOutputpinned the live evaluator against them and only passed because its samples never touched a window edge; it now pins the livebuildBoundedBareParserMetricagainst explicit(t-range, t]results with a sample on each edge, and fails if either bound moves (checked by mutating the lower bound).TestProxyHelpers_BuildBareParserMetricMatrixtested the deleted function and goes with it.TestLowerBoundIsExclusiveForEveryRangeFunction(new) fails onmain;firstandquantile(q=0)carry the proof,maxis there for coverage of the branch.TestCompatHelpers_AggregateManualWindowencoded the old bounds forsum/avg/min/first/stddev/stdvar/rate_counter; updated to Loki's, with the citation in the comment.Validation
gofmt -s -lempty,go build ./...,go vet ./...,golangci-lint run(v2.13.2, 0 issues),scripts/ci/check_changelog_pr.pyandscripts/ci/check_conformance.pyok,go test ./... -count=1green across all packages, on top of currentmain(v1.86.0).Found while syncing a downstream fork onto v1.81.0 and checking it against a real Loki 3.7.1 in
test/e2e-compat.