Skip to content

feat(antivirus): prioritize scans from light uploaders - #3659

Open
suse-coder wants to merge 4 commits into
opencloud-eu:mainfrom
suse-coder:feature/antivirus-upload-priority
Open

suse-coder wants to merge 4 commits into
opencloud-eu:mainfrom
suse-coder:feature/antivirus-upload-priority

Conversation

@suse-coder

@suse-coder suse-coder commented Oct 5, 2026 •

Copy link
Copy Markdown

Summary

Implements high/low antivirus scheduling for issue #3110.

  • Copies antivirus scan requests from the existing antivirus durable consumer into durable JetStream priority lanes, acknowledging source events only after the job/order state is persisted.
  • Classifies per-user scan rates in shared JetStream KV. Stable input event IDs avoid re-counting retries while they remain in the bounded recent-rate state; existing timestamp-only rate records remain readable.
  • Workers check the high lane before low work; a configurable worker reservation protects high-priority capacity. Intake concurrency is independent of scan concurrency.
  • Serializes jobs per resource by the StartPostprocessingStep stream sequence seen by antivirus, preserving retry position through UploadReady/CleanUpload.
  • Publishes stable-ID completion events to the existing main stream and uses its two-minute JetStream duplicate window.
  • Adds queue metrics, config validation/docs, and embedded-NATS coverage for priority, retries, context cancellation, deduplication, and connection startup behavior.

Important limits

  • Per-resource ordering starts at the antivirus StartPostprocessingStep event. It does not guarantee original upload/version order: postprocessing workers can emit start events out of upload order.
  • Priority is preferential across shared lanes, not a cluster-wide atomic barrier against a high job published concurrently with a low claim. Running scans cannot be preempted; strict high-first behavior can starve low work.
  • The per-resource KV value contains the pending list and is rewritten with CAS. A very large backlog for one resource may increase write load or reach JetStream's value-size limit; this PR does not claim a 100k-same-resource load result.
  • Completion deduplication is bounded to two minutes. Processing is at-least-once beyond that window.
  • Intake backlog can affect how much of a burst is classified before a high-lane scan starts.

For upgrades, stop old antivirus replicas before starting the new version, then start the new replicas together. The implementation reuses the existing antivirus durable consumer cursor.

Validation

  • go test ./...
  • go test -race ./services/antivirus/pkg/queue ./services/antivirus/pkg/service
  • go vet ./services/antivirus/...
  • .make/check-env-var-annotations.sh

Closes #3110

@codacy-production

codacy-production Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 458 complexity

Metric Results
Complexity 458

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] antivirus container massive files to antivirus not fair distributed

1 participant