Repository navigation
test: Test ActivityPolicies with activitytool - #1022
Open
JoseSzycho wants to merge 1 commit into
Open
JoseSzycho wants to merge 1 commit into
JoseSzycho wants to merge 1 commit into
Conversation
Add activitytool unit tests for every Milo ActivityPolicy and fix the two bugs they found. - test/activity-policies: 184 cases covering all 109 rules in the 24 policies - Fix create summaries for groups, roles, service accounts, organizations and projects, which errored (and dead-lettered the event) when the response had no metadata - Fix the UserInvitation accepted, declined and delete rules: the -with-org rules errored when the response had no status.organization, so their fallbacks were unreachable - task test-activity-policies runs the suite in the activitytool image; CI runs it through datum-cloud/actions' reusable workflow Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ecv
approved these changes
Nov 18, 2026
ecv
left a comment
Contributor
There was a problem hiding this comment.
Approve.
The six create-summary guards and the three UserInvitation guards are valid CEL and only add conditions that were already true for events that worked before, so summaries and matching for those events don't change.
- I ran the pinned activitytool image against the head: 184/184 pass, 24 policies, 109/109 rules matched. Reverting the
metadataguard fails 5 tests (group, role, serviceaccount, organization, project, "create without a response body"); reverting the UserInvitation guards fails 3 ("accepted/declined/cancel without the organization"). The tests do guard both fixes. - The workflow pins activitytool by tag and digest (
v0.10.0@sha256:8846...) with a renovate marker, and the Taskfile carries the same pin. The shared workflow defaultsallow-emptyto false andfail-on-untested: trueis set, so empty input or an untested policy fails. - Events that errored before (no
metadata, or nostatus.organization) now take the fallback rule instead of being dead-lettered; that is the intended behavior change, and it is customer-visible only as a summary appearing where there was none.
Follow-ups:
- The shared workflow is referenced by tag
@v1.29.0, not a commit SHA. - The Taskfile pin and the workflow pin must be kept in sync by hand; one Renovate rule covering both would remove that.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
activitytoolunit tests for every Milo ActivityPolicy (milo-os/activity#343), and fixes the two policy bugs they found.Changes
test/activity-policies/: 184 cases covering all 109 rules in the 24 policies. Every rule is reached, with the exact summary. Reads, controller writes and status updates produce no activity.has(audit.responseObject.metadata.name)errored when the response had nometadata, so the event was dead-lettered. Each level is now checked.accepted-with-org,declined-with-organddelete-with-orgrules errored when the response had nostatus.organization, so theaccepted,declinedanddeletefallbacks could never run.task test-activity-policies: runs the suite inghcr.io/milo-os/activitytool:v0.10.0, offline and read-only, and fails on any untested rule..github/workflows/test-activity-policies.yamlcallsdatum-cloud/actions/…/test-activity-policies.yaml@v1.29.0when policies or tests change.Not in this PR
audit.requestURI.contains('dryRun=')), so rejected and dry-run changes still reach the timeline.config/services/identity/policies/serviceaccount-policy.yamlis an unreferenced copy of the IAM service account policy and could be deleted.Testing
task test-activity-policies: 184/184 passed, 109/109 rules matched.go test ./config/services/activity/policies/(owner labels) passes.🤖 Generated with Claude Code