Repository navigation
Reuse scheduler catalog ranking before publication locks - #71
Conversation
PR reader API performanceResult: INCONCLUSIVE 7 inconclusive · 1 within threshold. Zero request failures across all six runs. Base Three independent runs per revision, each on a fresh GitHub-hosted ARM64 runner. 16 users, 60 seconds/run, 1,000 synthetic articles, cache off, five PgBouncer slots. Latency comparisonAll latency values are in milliseconds; lower is better. Values are medians of three runs.
Additional latency and throughput metricsEach cell shows base → PR. Latencies are in milliseconds.
Individual runs
How to interpret this reportRegressed / Improved: a p95 regression needs >20% and >20 ms increase in the medians and at least two head samples versus the base median. Improvements use the inverse threshold. Within threshold means the change did not meet both thresholds. Inconclusive: a within-revision p95 range >35% of its median and >20 ms is too noisy for a confident latency verdict. Inconclusive latency comparisons are non-blocking: the check passes without claiming an improvement. Confirmed regressions and test/report errors fail the PR gate. Throughput is descriptive: paced users do not measure maximum capacity. This uncached public API test does not establish production health, browser performance or worker capacity. Separate runners can differ in hardware or host load; repetitions and noise checks reduce but cannot eliminate that uncertainty. Full HTML/CSV reports and logs · attempt 1 |
CI failed: 1 infrastructure failure in the GitHub Advanced Security workflow due to an exceeded monthly model API quota.OverviewOne failure pattern was found across 1 analyzed log, entirely related to an infrastructure limitation with the AI model quota. FailuresGitHub Advanced Security Quota Exceeded (confidence: high)
Summary
Code Review ✅ Approved🔴 High risk · Migration 0022 changes catalog revision triggers and the required application schema. Reuses scheduler catalog ranking within batch transactions to eliminate redundant catalog loads and rankings while maintaining correctness. Synthetic benchmarks show 95.6% CPU reduction (40.39s to 1.77s for 50 articles) with identical outputs; production validation recommended post-release. Migration Tip Comment OptionsAuto-apply is off → Gitar will not commit updates to this branch. Comment with these commands to change the behavior for this request:
Was this helpful? React with 👍 / 👎 | Gitar |
|
1 participant
What changes and why?
Scheduler ticks repeatedly loaded the full catalog and ranked the same article during validity checks, publication evaluation and policy application. The audit attributed about 84% of sampled CPU to catalog validity. A fresh production check measured a 105.7-second average tick over one hour, about 0.90 scheduler CPU cores and a 470 MiB working set. Ranking also ran while source/article rows and the topic catalog were locked.
The production PostgreSQL audit window recorded 57 topic write-lock calls averaging 541 ms. The activity snapshot also showed an idle transaction waiting on
ClientRead, consistent with the sampled Python catalog work occurring between database calls. These are baseline observations; this PR makes no claim of a measured production improvement.Article automation now reuses a catalog snapshot and prepared retrieval indexes within each batch. PostgreSQL UUID revisions remain authoritative, including when Redis caching is disabled. Each article is ranked after the preparation transaction closes; the locked transaction verifies the current article input and catalog revisions before reusing the result. A changed input or catalog leaves the article due for the next tick. Source eligibility, editorial revisions, job state, negative classifications and paywall checks retain their existing guards. Candidate results are bounded to eight articles and released at the end of the batch. Publication evaluation is reused only within its existing locked transaction.
Migration
0022includes topicdescriptionandai_descriptionin revision invalidation because ranking reads both. It invalidates existing topic snapshots once. The application now requires schema revision0022.On a synthetic catalog matching the measured production size (4,450 topics and 34,057 tags), the catalog hot path for 50 articles improved from 40.39 seconds to 1.77 seconds, with identical candidate outputs and 95.6% less CPU time. The baseline used the functions from
ba167ee7, one JSON catalog decode per article and three candidate checks per article; the optimized run used one decode per batch, an additional preparation call and the same three checks. Both runs started with cold retrieval caches. This measures the catalog hot path locally; production tick latency and lock waits still need post-release verification.This branch contains only scheduler/catalog reuse, its correctness migration, regression tests and documentation. Authentication-log redaction (#69) and OpenSSF Scorecard (#70) remain on separate branches.
Validation
0022downgrade/re-upgrade passed, including snapshot invalidation and restored/upgraded description-trigger behavior. The migration introduces no model-schema differences; the existing migration-only search statistics index is retained.Rollout notes
Apply migration
0022through the normal migration-first release before starting updated services. It replaces the revision trigger function and bumps the topics revision; it changes no tables, indexes or stored content. A source rollback to the previous exact-schema version also requires the migration downgrade. Downgrade restores the earlier description-invalidation behavior and invalidates snapshots again.No production mutation or deployment was performed. After release, compare scheduler tick duration and CPU together with PostgreSQL topic-lock waits over matching windows. The existing 15-second scheduler pause remains unchanged.
Merge queue validation
Updated against current master after #69. The only conflict was in
CHANGELOG.md; both user-facing fixes are retained. After the update, 46 catalog/CI unit regressions and 19 catalog/scheduler integration tests passed against disposable PostgreSQL 18 and Redis 8, with migration0022applied. Full hosted CI,CI required, CodeQL and Sonar passed on updated commit84649215. Hosted suites passed with 2,619 backend unit tests, 1,111 integration tests, 785 reader tests and 647 admin tests, with no skips. Populated-schema migration upgrades, web/Chrome/Edge parity and all ARM64 image build/scan/smoke checks passed. Sonar MCP confirms 89.4% new-code coverage, 0.0% duplication, and zero new bugs, vulnerabilities or code smells. The separate GitHub-managed AI review remains quota-blocked, independently of these passing checks.Merge this PR before #72, then retarget #72 to master. Merge the worker-lock fix before updating #79, preserving its computation outside catalog locks and rerunning its lock regressions. Auto-merge remains disabled; draft #59 is excluded.
Ready for review