Repository navigation
feat: admit submissions and carry out review decisions - #37
Conversation
Submissions need a verified email and are limited per user (5 per 10 minutes by default, counted from the cache). Each submission gets a SubmissionReview owned by it. The review reconciler carries out Rejected, Merged and Approved decisions: merges and approvals give the submitter a follow and a vote, and approvals link the submission once feedback-sync syncs a post annotated with its name. Merged decisions must name a visible, unmerged post. Field indexes move to a shared package, and generation now points the webhook config at the operator's Service.
|
Non-blocking follow-up: the submission limit is a soft limit, not a strict one. Parallel requests can all read the same cached count and be accepted. Please document that trade-off, or enforce the limit with a shared atomic counter if it needs to resist abuse. The workflow otherwise looks good; this PR still depends on the merge fix in #34. |
The limit counted submissions in the operator's cache, so parallel requests could all read the same count and be admitted. Each replica now also remembers what it admitted until the cache shows it, deciding one request at a time, so a burst to one replica gets at most the limit. Document that the limit is still soft across replicas (replicas x limit per window at worst), and why that's acceptable: submissions are never published without staff review.
creationTimestamp has whole seconds, so a submission created in the same second as its admission looked older than the admission, and both counted towards the limit: users got 429 after 3 or 4 of 5. Count instead how many of this replica's admissions the cache already shows since the oldest one, rounded down to the second.
# Conflicts: # internal/controller/setup.go
|
scotwells
left a comment
There was a problem hiding this comment.
The new rate-limit guard still counts one submission twice when admission crosses a second boundary (internal/webhook/submission.go, unseen). The API server can stamp the object at 12:00:00 and call the webhook at 12:00:01.020; the cached object then fails the time-based match. We reproduced this with a limit of two: the second submission was refused after only one had been created.
Please match pending admissions to cached objects by the submission's UID rather than timestamps, and add a regression test across a second boundary. Also skip recording dry-run requests so they don't consume real quota.
The documented soft limit across replicas is fine; this is a separate bug that rejects legitimate submissions too early.
The submission limit matched the admissions it remembered to cached submissions by time. creationTimestamp is set before admission and has whole seconds, so an admission at 12:00:01.020 for a submission stamped 12:00:00 counted twice, and the next submission was refused early. The webhook validates after the API server sets the UID, so pending admissions are now matched to cached submissions by UID. Also: - Dry runs are limited but not recorded, so they don't use up the limit. - An admission the cache never shows (the create failed later) stops counting after a minute instead of the whole window. - The same create admitted twice counts once. Tests cover a second boundary, dry runs, failed creates and repeated admissions. The second-boundary, dry-run and repeated-admission tests fail without this change.
|
Thanks, you're right: matching by time can't work when
Tests cover admission across a second boundary (your 12:00:00 / 12:00:01.020 case, limit 2), a same-second case, dry runs, failed creates and repeated admission. The second-boundary, dry-run and repeated-admission tests fail on the previous code. The kind e2e suite in #42 passes with this change. |
scotwells
left a comment
There was a problem hiding this comment.
The UID-based matching fixes the reproduced double-counting bug. Dry runs no longer consume quota, and failed/repeated admissions are handled. Reviewed the change and ran the submission tests ten times with the race detector on this head and the updated stack; tests and vet passed. The earlier change request is resolved.
The base branch was changed.
Part of #8. Stacked on #34. Attachments come in the next PR, with the upload service.
Admission
submissionruns on creates.--submission-limit(default 5) per--submission-window(default 10m), counted from the operator's cache through aspec.userRef.nameindex. Over the limit, the answer is 429 with how long to wait.submissionreviewruns on creates and updates. AMergeddecision must name a post that exists, isn't hidden and isn't merged. It reuses the vote check's post lookup.Reconcilers
submissioncreates theSubmissionReviewwith the same name, owned by the submission. Milo runs the garbage collector, so deleting a submission deletes its review. It also sets the phase toPendingReview.submission-reviewcarries out the decision:Rejectedand copies the reason the submitter sees.reason: merged) and votes on it, with the submission's client as the vote source. The submission becomesMergedwithpostRef.Approvedand waits withApplied=False, WaitingForPost. feedback-sync creates the GitHub issue and annotates the post it syncs withfeedback.miloapis.com/submission: <submission>. The reconciler watches posts by that annotation. Once the post appears, it links the submission and has the submitter follow (reason: submitted) and vote.Other changes
internal/index, so webhooks and reconcilers share them.task generatenow points the generated webhook config atfeedback-operator-webhook.feedback-system, the way billing does. Before, each webhook needed a kustomize patch by index.gofmtfixes in two files from feat: feedback CRDs and operator scaffold #32.Not in this PR
Tests
SubmissionCheck: the third submission in ten minutes gets a 429 with the wait time; others' submissions don't count; unverified email is refused; attachments are refused without uploads; a limit of 0 disables it.ReviewCheck: merging into a visible post is allowed; hidden or missing posts are refused; rejecting doesn't look up a post.PendingReviewand a review owned by the submission;merged) and a vote with sourceportal;task test,go vet ./..., andtask generatewith no diff