Skip to content

Avoid a full miner_predictions scan when loading scores - #337

Merged
Thykof merged 2 commits into
mainfrom
fix/miner-scores-query-plan
Oct 9, 2026
Merged

Thykof merged 2 commits into
mainfrom
fix/miner-scores-query-plan

Conversation

@stas-synth

@stas-synth stas-synth commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

get_miner_scores never bounds validator_requests in time, so the planner can't use the asset/time_length predicates to restrict the join and falls back to a full scan of miner_predictions before filtering. A prediction is scored exactly when its window closes — scored_time = vr.start_time + time_length — so the added bound is implied by the existing scored_time filter and removes no rows. Plan cost 5.59M → 2.09M; 110s → 30s on a 10-day window; verified identical over a fixed window: 491,520 rows both sides, zero rows in either EXCEPT, matching md5 across the full result set including computed window values. The invariant was checked separately: 0 violations across 3.7M rows over 14 days.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

Tip: disable this comment in your organization's Code Review settings.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The added predicate matches the score timestamp construction and preserves existing query results.

0 open findings

What changed in this PR

Optimizes score loading by adding a logically equivalent time bound that enables indexed request and prediction lookups.

Changes:

  • Adds an indexed validator_requests.start_time constraint.
  • Documents the query-plan improvement and validation results.
File Description
synth/​validator/​miner_data_handler.py Avoids a full miner_predictions scan when loading scores.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@Thykof
Thykof merged commit f18efc6 into main Oct 9, 2026
5 checks passed
@Thykof
Thykof deleted the fix/miner-scores-query-plan branch October 9, 2026 13:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants