From 79bc6ba2d3103556fd5ff3b4208a1003e0cada72 Mon Sep 17 00:00:00 2001 From: Matt Jenkinson Date: Tue, 17 Nov 2026 13:41:22 -0500 Subject: [PATCH 1/3] feat: index posts, releases and changelog entries in search ResourceIndexPolicies for Post, Release and ChangelogEntry make their text searchable through Milo's search service, which clients use for keyword search and similar posts. Hidden posts are left out, derived fields aren't indexed, and private kinds never are. hack/search-lag.sh measures how long a change takes to reach the index. --- README.md | 20 ++- config/milo/kustomization.yaml | 4 +- config/milo/search/changelogentries.yaml | 18 +++ config/milo/search/kustomization.yaml | 13 ++ config/milo/search/posts.yaml | 33 +++++ config/milo/search/releases.yaml | 20 +++ hack/search-lag.sh | 51 +++++++ test/config/search_test.go | 168 +++++++++++++++++++++++ 8 files changed, 322 insertions(+), 5 deletions(-) create mode 100644 config/milo/search/changelogentries.yaml create mode 100644 config/milo/search/kustomization.yaml create mode 100644 config/milo/search/posts.yaml create mode 100644 config/milo/search/releases.yaml create mode 100755 hack/search-lag.sh create mode 100644 test/config/search_test.go diff --git a/README.md b/README.md index b75774a..1ebb9f1 100644 --- a/README.md +++ b/README.md @@ -18,8 +18,8 @@ webhooks. The CRDs are defined. The operator sets owners and reviewers in admission, reconciles votes, follows, submission reviews and security reports, and -stores attachments. Triage, notifications and digests, audit and search -config are tracked as sub-issues of the epic. +stores attachments. Activity and search are configured. Triage, +notifications and digests are tracked as sub-issues of the epic. | Resource | Who writes it | Who reads it | | --- | --- | --- | @@ -106,6 +106,17 @@ Milo audits feedback writes like any other CRD. Its audit policy (in `datum-clou `config/milo/activity` holds an `ActivityPolicy` per kind, so people's actions read as sentences in activity timelines: voting, following, sending feedback or a report, staff decisions, hiding posts and editing sources. The service's own components are `system:` users and are left out. `test/config` compiles every rule and checks that rules for metadata-only kinds never read request or response bodies. +## Search + +`config/milo/search` has a `ResourceIndexPolicy` for `Post`, `Release` and `ChangelogEntry`. + +- **What's searchable:** title, summary or body, categories, labels and state. Derived fields (`votes`, `trendingScore`, `delivery`) aren't indexed, since they change as side effects and would go stale. +- **Hidden posts:** the policy's condition leaves them out, and hiding a post removes it from the index. +- **How clients use it:** a `ResourceSearchQuery` for keyword search, and for similar posts while someone writes feedback. +- **Never indexed:** submissions, reviews, security reports, votes and follows. +- **What it relies on:** the indexer reads objects from Milo's audit log, so it needs Milo to log feedback writes at `RequestResponse` (see Audit and activity). +- **Measuring lag:** `hack/search-lag.sh ` measures how long a post change takes to reach the index. Run it against staging; it hides the post briefly. + ## Roles | Role | For | @@ -137,9 +148,10 @@ config/base/ operator and uploads Deployments, Services, leader ele config/components/ milo-integration: Milo kubeconfig, client and webhook certificates config/milo/ applied to Milo: IAM (iam/), RBAC for the service's components (rbac/), webhook config (webhook/), - activity rules (activity/), and optional role - bindings (bindings/) + activity rules (activity/), search index policies + (search/), and optional role bindings (bindings/) config/samples/ an example of every resource +hack/search-lag.sh measures how long a post change takes to reach search test/config/ checks config/ against the CRDs ``` diff --git a/config/milo/kustomization.yaml b/config/milo/kustomization.yaml index 4303f2e..cc1ef7a 100644 --- a/config/milo/kustomization.yaml +++ b/config/milo/kustomization.yaml @@ -1,6 +1,7 @@ # What the service registers in Milo: ProtectedResources and Roles (iam/), # Kubernetes RBAC for the service's own components (rbac/), the admission -# webhooks Milo calls (webhook/) and activity timeline rules (activity/). The CRDs are in config/crd. bindings/ is +# webhooks Milo calls (webhook/), activity timeline rules (activity/) and +# search index policies (search/). The CRDs are in config/crd. bindings/ is # separate and optional: who holds the roles is the deployment's choice. apiVersion: kustomize.config.k8s.io/v1alpha1 kind: Component @@ -10,3 +11,4 @@ components: - rbac/ - webhook/ - activity/ + - search/ diff --git a/config/milo/search/changelogentries.yaml b/config/milo/search/changelogentries.yaml new file mode 100644 index 0000000..2e7f7f0 --- /dev/null +++ b/config/milo/search/changelogentries.yaml @@ -0,0 +1,18 @@ +apiVersion: search.miloapis.com/v1alpha1 +kind: ResourceIndexPolicy +metadata: + name: feedback-changelogentries +spec: + targetResource: + group: feedback.miloapis.com + version: v1alpha1 + kind: ChangelogEntry + fields: + - path: .metadata.name + - path: .status.title + searchable: true + - path: .status.bodyHTML + searchable: true + - path: .status.categories[*] + searchable: true + - path: .status.publishedAt diff --git a/config/milo/search/kustomization.yaml b/config/milo/search/kustomization.yaml new file mode 100644 index 0000000..aadd9b5 --- /dev/null +++ b/config/milo/search/kustomization.yaml @@ -0,0 +1,13 @@ +# Search index policies for Milo's search service. Clients search posts by +# keyword and look for similar posts while someone writes feedback; feedback +# is also findable through platform search. The indexer reads objects from +# the audit log, so these need Milo to log feedback writes at +# RequestResponse. Submissions, reviews, security reports, votes and +# follows are never indexed. +apiVersion: kustomize.config.k8s.io/v1alpha1 +kind: Component + +resources: + - posts.yaml + - releases.yaml + - changelogentries.yaml diff --git a/config/milo/search/posts.yaml b/config/milo/search/posts.yaml new file mode 100644 index 0000000..7cc8d3d --- /dev/null +++ b/config/milo/search/posts.yaml @@ -0,0 +1,33 @@ +# Posts are found by their GitHub text, categories, labels and state. The +# indexer keeps the whole object, but only these fields are searchable; +# votes, trendingScore and delivery change as side effects and would go +# stale, so they aren't. Hidden posts are left out, and hiding a post +# removes it from the index. +apiVersion: search.miloapis.com/v1alpha1 +kind: ResourceIndexPolicy +metadata: + name: feedback-posts +spec: + targetResource: + group: feedback.miloapis.com + version: v1alpha1 + kind: Post + conditions: + - name: visible + expression: "!has(spec.hidden) || !spec.hidden" + fields: + - path: .metadata.name + - path: .status.title + searchable: true + - path: .status.summary + searchable: true + - path: .status.bodyHTML + searchable: true + - path: .status.categories[*] + searchable: true + - path: .status.labels[*] + searchable: true + - path: .status.state + searchable: true + - path: .status.github.org + - path: .status.github.repository diff --git a/config/milo/search/releases.yaml b/config/milo/search/releases.yaml new file mode 100644 index 0000000..7b5a784 --- /dev/null +++ b/config/milo/search/releases.yaml @@ -0,0 +1,20 @@ +apiVersion: search.miloapis.com/v1alpha1 +kind: ResourceIndexPolicy +metadata: + name: feedback-releases +spec: + targetResource: + group: feedback.miloapis.com + version: v1alpha1 + kind: Release + fields: + - path: .metadata.name + - path: .status.title + searchable: true + - path: .status.codename + searchable: true + - path: .status.version + searchable: true + - path: .status.descriptionHTML + searchable: true + - path: .status.dueOn diff --git a/hack/search-lag.sh b/hack/search-lag.sh new file mode 100755 index 0000000..a471e5b --- /dev/null +++ b/hack/search-lag.sh @@ -0,0 +1,51 @@ +#!/usr/bin/env bash +# Measures how long a post change takes to reach Milo's search index. +# +# It hides a visible post, waits until search no longer finds it, then +# unhides it and times how long until search finds it again. Hiding is +# visible to users while it lasts, so run it against staging. +# +# KUBECONFIG= hack/search-lag.sh [timeout-seconds] +# +# Needs: kubectl, jq; staff permission to patch posts and create +# resourcesearchqueries. +set -euo pipefail + +post=${1:?usage: search-lag.sh [timeout-seconds]} +timeout=${2:-300} + +title=$(kubectl get post "$post" -o jsonpath='{.status.title}') +[[ -n $title ]] || { echo "post $post has no title" >&2; exit 1; } + +found() { + kubectl create -o json -f - </dev/null +{"apiVersion": "search.miloapis.com/v1alpha1", "kind": "ResourceSearchQuery", + "metadata": {"generateName": "search-lag-"}, + "spec": {"query": $(jq -Rn --arg t "$title" '$t'), "limit": 100, + "targetResources": [{"group": "feedback.miloapis.com", "version": "v1alpha1", "kind": "Post"}]}} +JSON +} + +wait_for() { # wait_for + local start=$SECONDS + while (( SECONDS - start < timeout )); do + if found; then [[ $1 == 1 ]] && return 0; else [[ $1 == 0 ]] && return 0; fi + sleep 1 + done + echo "timed out after ${timeout}s" >&2 + return 1 +} + +trap 'kubectl patch post "$post" --type=merge -p "{\"spec\":{\"hidden\":false}}" >/dev/null' EXIT + +found || { echo "search doesn't find $post by its title; is it indexed?" >&2; exit 1; } + +kubectl patch post "$post" --type=merge -p '{"spec":{"hidden":true}}' >/dev/null +t0=$SECONDS +wait_for 0 +echo "removed from the index after about $((SECONDS - t0))s" + +kubectl patch post "$post" --type=merge -p '{"spec":{"hidden":false}}' >/dev/null +t0=$SECONDS +wait_for 1 +echo "back in the index after about $((SECONDS - t0))s" diff --git a/test/config/search_test.go b/test/config/search_test.go new file mode 100644 index 0000000..30c745b --- /dev/null +++ b/test/config/search_test.go @@ -0,0 +1,168 @@ +package config + +import ( + "encoding/json" + "os" + "path/filepath" + "reflect" + "strings" + "testing" + + "github.com/google/cel-go/cel" + sigsyaml "sigs.k8s.io/yaml" + + v1 "go.miloapis.com/feedback/pkg/apis/feedback/v1alpha1" +) + +type indexPolicy struct { + Metadata struct { + Name string `json:"name"` + } `json:"metadata"` + Spec struct { + TargetResource struct { + Group, Version, Kind string + } `json:"targetResource"` + Conditions []struct { + Name, Expression string + } `json:"conditions"` + Fields []struct { + Path string `json:"path"` + Searchable bool `json:"searchable"` + } `json:"fields"` + } `json:"spec"` +} + +func loadIndexPolicies(t *testing.T) []indexPolicy { + t.Helper() + files, _ := filepath.Glob("../../config/milo/search/*.yaml") + var out []indexPolicy + for _, f := range files { + if strings.HasSuffix(f, "kustomization.yaml") { + continue + } + data, err := os.ReadFile(f) + if err != nil { + t.Fatal(err) + } + var p indexPolicy + if err := sigsyaml.Unmarshal(data, &p); err != nil { + t.Fatalf("%s: %v", f, err) + } + out = append(out, p) + } + if len(out) != 3 { + t.Fatalf("want policies for Post, Release and ChangelogEntry, got %d", len(out)) + } + return out +} + +// resolve walks a policy field path (.a.b, [*]) through Go types by JSON +// name, so a typo or renamed field fails here rather than indexing nothing. +func resolve(typ reflect.Type, path string) (reflect.Type, bool) { + for _, seg := range strings.Split(strings.ReplaceAll(strings.TrimPrefix(path, "."), "[*]", ".[*]"), ".") { + for typ.Kind() == reflect.Pointer { + typ = typ.Elem() + } + if seg == "[*]" { + if typ.Kind() != reflect.Slice { + return nil, false + } + typ = typ.Elem() + continue + } + f, ok := fieldByJSON(typ, seg) + if !ok { + return nil, false + } + typ = f + } + return typ, true +} + +func fieldByJSON(typ reflect.Type, name string) (reflect.Type, bool) { + if typ.Kind() != reflect.Struct { + return nil, false + } + for i := 0; i < typ.NumField(); i++ { + f := typ.Field(i) + tag := strings.Split(f.Tag.Get("json"), ",")[0] + if f.Anonymous && tag == "" { + if t, ok := fieldByJSON(f.Type, name); ok { + return t, true + } + continue + } + if tag == name { + return f.Type, true + } + } + return nil, false +} + +func TestSearchPolicies(t *testing.T) { + types := map[string]reflect.Type{ + "Post": reflect.TypeOf(v1.Post{}), + "Release": reflect.TypeOf(v1.Release{}), + "ChangelogEntry": reflect.TypeOf(v1.ChangelogEntry{}), + } + for _, p := range loadIndexPolicies(t) { + tr := p.Spec.TargetResource + typ, ok := types[tr.Kind] + if tr.Group != group || tr.Version != "v1alpha1" || !ok { + t.Errorf("%s targets %+v; only posts, releases and changelog entries are indexed", p.Metadata.Name, tr) + continue + } + for _, f := range p.Spec.Fields { + if _, ok := resolve(typ, f.Path); !ok { + t.Errorf("%s: %s isn't a field of %s", p.Metadata.Name, f.Path, tr.Kind) + } + for _, derived := range []string{".status.votes", ".status.delivery", ".status.posts", ".status.reactions"} { + if strings.HasPrefix(f.Path, derived) { + t.Errorf("%s indexes derived field %s", p.Metadata.Name, f.Path) + } + } + } + } +} + +// TestHiddenPostCondition evaluates the posts policy's condition the way the +// indexer does: metadata, spec and status as dynamic variables. +func TestHiddenPostCondition(t *testing.T) { + var cond string + for _, p := range loadIndexPolicies(t) { + if p.Spec.TargetResource.Kind == "Post" { + if len(p.Spec.Conditions) != 1 { + t.Fatalf("want one condition, got %v", p.Spec.Conditions) + } + cond = p.Spec.Conditions[0].Expression + } + } + env, err := cel.NewEnv(cel.Variable("metadata", cel.DynType), cel.Variable("spec", cel.DynType), cel.Variable("status", cel.DynType)) + if err != nil { + t.Fatal(err) + } + ast, iss := env.Compile(cond) + if iss.Err() != nil { + t.Fatal(iss.Err()) + } + prg, err := env.Program(ast) + if err != nil { + t.Fatal(err) + } + for _, c := range []struct { + hidden bool + want bool + }{{false, true}, {true, false}} { + p := v1.Post{Spec: v1.PostSpec{Hidden: c.hidden}} + raw, _ := json.Marshal(p) + var obj map[string]any + _ = json.Unmarshal(raw, &obj) + out, _, err := prg.Eval(map[string]any{"metadata": obj["metadata"], "spec": obj["spec"], "status": obj["status"]}) + if err != nil { + t.Fatalf("hidden=%v: %v", c.hidden, err) + } + if out.Value() != c.want { + t.Errorf("hidden=%v: indexed=%v, want %v", c.hidden, out.Value(), c.want) + } + } +} From 05b8a1556d2316959dce1a05c4a0fcac85dc2866 Mon Sep 17 00:00:00 2001 From: Matt Jenkinson Date: Tue, 17 Nov 2026 16:46:34 -0500 Subject: [PATCH 2/3] fix: restore post visibility in search-lag, default post spec, document search scope search-lag.sh records the post's original spec.hidden and restores exactly that on every exit once it has changed it; it refuses hidden or unindexed posts and tells failed queries apart from misses. The Post CRD defaults spec to {} because search's indexer can't evaluate spec.hidden on a post with no spec. Posts are indexed as platform documents, so only platform-root queries find them; the README says who can search and with what access, and hack/search-scope.sh checks it against an environment. --- README.md | 32 +++- .../bases/feedback.miloapis.com_posts.yaml | 5 +- config/milo/search/kustomization.yaml | 13 +- config/milo/search/posts.yaml | 9 + go.mod | 5 +- go.sum | 32 ++++ hack/search-lag.sh | 97 ++++++++--- hack/search-scope.sh | 68 ++++++++ pkg/apis/feedback/v1alpha1/crd_test.go | 25 +++ pkg/apis/feedback/v1alpha1/post_types.go | 3 + test/config/search_test.go | 102 +++++++++-- test/hack/search_lag_test.go | 161 ++++++++++++++++++ 12 files changed, 508 insertions(+), 44 deletions(-) create mode 100755 hack/search-scope.sh create mode 100644 test/hack/search_lag_test.go diff --git a/README.md b/README.md index 1ebb9f1..f322e75 100644 --- a/README.md +++ b/README.md @@ -111,11 +111,36 @@ Milo audits feedback writes like any other CRD. Its audit policy (in `datum-clou `config/milo/search` has a `ResourceIndexPolicy` for `Post`, `Release` and `ChangelogEntry`. - **What's searchable:** title, summary or body, categories, labels and state. Derived fields (`votes`, `trendingScore`, `delivery`) aren't indexed, since they change as side effects and would go stale. -- **Hidden posts:** the policy's condition leaves them out, and hiding a post removes it from the index. -- **How clients use it:** a `ResourceSearchQuery` for keyword search, and for similar posts while someone writes feedback. +- **Hidden posts:** the policy's condition leaves them out, and hiding a post removes it from the index. The condition reads `spec.hidden`, and the indexer can't evaluate it on an object with no `spec` (it logs an error and treats the post as not matching, which also deletes it from the index), so the `Post` CRD defaults `spec` to `{}`. - **Never indexed:** submissions, reviews, security reports, votes and follows. - **What it relies on:** the indexer reads objects from Milo's audit log, so it needs Milo to log feedback writes at `RequestResponse` (see Audit and activity). -- **Measuring lag:** `hack/search-lag.sh ` measures how long a post change takes to reach the index. Run it against staging; it hides the post briefly. +- **Measuring lag:** `hack/search-lag.sh ` measures how long a post change takes to reach the index. Run it against staging with a platform-root kubeconfig; it hides a visible post briefly and always puts back its original `spec.hidden`. + +### Who can search posts + +The search service filters every `ResourceSearchQuery` by the caller's Milo context, taken from the `iam.miloapis.com/parent-type` and `parent-name` user extras (`internal/registry/resourcesearchquery` in `milo-os/search`): + +- **Platform root** (no parent context): no filter. The query sees every indexed document from every tenant. +- **User, organization or project control plane**: only documents indexed from that same user, organization or project. + +feedback-sync and the operator write posts, releases and changelog entries at the platform root, so they're indexed as platform documents. **Only platform-root queries find them.** A query in a user, organization or project context returns no posts and no error. That rules out the cloud portal's user-context calls and `datumctl` in an org or project context. + +Similar posts and keyword search therefore run on a server, at the platform root: + +| Caller | How it searches posts | +| --- | --- | +| datum.net | its server's service account, at the platform root | +| feedback-operator (submission triage) | its own identity, at the platform root | +| cloud portal | its server, with a platform service account like the one behind its post cache; not with the user's token in the user context | +| `datumctl`, Patch | can't query with the user's credentials; they need a server-side call (a feedback endpoint or `feedback-mcp`), not built yet | + +The caller needs `search.miloapis.com/resourcesearchqueries.create` at the platform root (search's `search.miloapis.com-searcher` role). That isn't limited to feedback: search doesn't check access per object or per kind, and a platform-root query with no `targetResources` searches every index across every tenant. So: + +- Never grant it to end users. +- Always set `targetResources` to the feedback kinds, and never pass through targets or tenant filters a client supplies. +- Don't filter on the result's tenant. Milo marks root requests as scope `global`, so posts indexed from the audit log carry tenant type `Global` while a reindex stores `platform`. + +Nothing in `config/milo` grants this yet. `hack/search-scope.sh` checks the behavior above against a real environment. ## Roles @@ -152,6 +177,7 @@ config/milo/ applied to Milo: IAM (iam/), RBAC for the service's (search/), and optional role bindings (bindings/) config/samples/ an example of every resource hack/search-lag.sh measures how long a post change takes to reach search +hack/search-scope.sh checks which Milo contexts can find a post in search test/config/ checks config/ against the CRDs ``` diff --git a/config/crd/bases/feedback.miloapis.com_posts.yaml b/config/crd/bases/feedback.miloapis.com_posts.yaml index 3b0af9c..93f068c 100644 --- a/config/crd/bases/feedback.miloapis.com_posts.yaml +++ b/config/crd/bases/feedback.miloapis.com_posts.yaml @@ -69,7 +69,10 @@ spec: metadata: type: object spec: - description: PostSpec holds the fields staff control. + default: {} + description: |- + Spec holds the fields staff control. It defaults to {} so every post + has one, which the search index condition on spec.hidden needs. properties: hidden: description: Hidden removes the post from every client and from search. diff --git a/config/milo/search/kustomization.yaml b/config/milo/search/kustomization.yaml index aadd9b5..32ee83b 100644 --- a/config/milo/search/kustomization.yaml +++ b/config/milo/search/kustomization.yaml @@ -1,9 +1,10 @@ -# Search index policies for Milo's search service. Clients search posts by -# keyword and look for similar posts while someone writes feedback; feedback -# is also findable through platform search. The indexer reads objects from -# the audit log, so these need Milo to log feedback writes at -# RequestResponse. Submissions, reviews, security reports, votes and -# follows are never indexed. +# Search index policies for Milo's search service, for keyword search and +# similar posts while someone writes feedback. Everything here is indexed as +# a platform document, so only queries at Milo's platform root find it: +# servers search on clients' behalf (see "Who can search posts" in +# README.md). The indexer reads objects from the audit log, so these need +# Milo to log feedback writes at RequestResponse. Submissions, reviews, +# security reports, votes and follows are never indexed. apiVersion: kustomize.config.k8s.io/v1alpha1 kind: Component diff --git a/config/milo/search/posts.yaml b/config/milo/search/posts.yaml index 7cc8d3d..6c0cf93 100644 --- a/config/milo/search/posts.yaml +++ b/config/milo/search/posts.yaml @@ -3,6 +3,15 @@ # votes, trendingScore and delivery change as side effects and would go # stale, so they aren't. Hidden posts are left out, and hiding a post # removes it from the index. +# +# The indexer binds metadata, spec and status only when the object has +# them, and a condition that can't be evaluated counts as no match. has() +# can't test for a missing spec, so the Post CRD defaults spec to {}. +# +# Posts are written at Milo's platform root, so they're indexed as platform +# documents and only platform-root ResourceSearchQueries find them; user, +# organization and project contexts get nothing. See "Who can search posts" +# in README.md. apiVersion: search.miloapis.com/v1alpha1 kind: ResourceIndexPolicy metadata: diff --git a/go.mod b/go.mod index bfc3c10..4380e5f 100644 --- a/go.mod +++ b/go.mod @@ -12,6 +12,7 @@ require ( gocloud.dev v0.48.0 golang.org/x/time v0.16.0 k8s.io/api v0.37.2 + k8s.io/apiextensions-apiserver v0.37.0 k8s.io/apimachinery v0.37.2 k8s.io/client-go v0.37.2 sigs.k8s.io/controller-runtime v0.25.3 @@ -50,6 +51,7 @@ require ( github.com/aws/aws-sdk-go-v2/service/sts v1.51.4 // indirect github.com/aws/smithy-go v1.28.5 // indirect github.com/beorn7/perks v1.0.1 // indirect + github.com/blang/semver/v4 v4.0.0 // indirect github.com/cespare/xxhash/v2 v2.3.0 // indirect github.com/cncf/xds/go v0.0.0-20260202195803-dba9d589def2 // indirect github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect @@ -134,8 +136,9 @@ require ( google.golang.org/protobuf v1.36.12 // indirect gopkg.in/evanphx/json-patch.v4 v4.13.0 // indirect gopkg.in/inf.v0 v0.9.1 // indirect - k8s.io/apiextensions-apiserver v0.37.0 // indirect + k8s.io/apiserver v0.37.0 // indirect k8s.io/code-generator v0.37.2 // indirect + k8s.io/component-base v0.37.0 // indirect k8s.io/gengo/v2 v2.0.0-20260408192533-25e2208e0dc3 // indirect k8s.io/klog/v2 v2.140.0 // indirect k8s.io/kube-openapi v0.0.0-20260721132016-d427ff9ee9ad // indirect diff --git a/go.sum b/go.sum index 944853c..05dad13 100644 --- a/go.sum +++ b/go.sum @@ -72,12 +72,20 @@ github.com/aws/smithy-go v1.28.5 h1:OqbqKtzY9NUnr2yAe9+hiTdFb20tNaK/lI0PMOGXX9k= github.com/aws/smithy-go v1.28.5/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc= github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= +github.com/blang/semver/v4 v4.0.0 h1:1PFHFE6yCCTv8C1TeyNNarDzntLi7wMI5i/pzqYIsAM= +github.com/blang/semver/v4 v4.0.0/go.mod h1:IbckMUScFkM3pff0VJDNKRiT6TG/YpiHIM2yvyW5YoQ= +github.com/cenkalti/backoff/v5 v5.0.3 h1:ZN+IMa753KfX5hd8vVaMixjnqRZ3y8CuJKRKj1xcsSM= +github.com/cenkalti/backoff/v5 v5.0.3/go.mod h1:rkhZdG3JZukswDf7f0cwqPNk4K0sa+F97BxZthm/crw= github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/cevatbarisyilmaz/ara v0.0.4 h1:SGH10hXpBJhhTlObuZzTuFn1rrdmjQImITXnZVPSodc= github.com/cevatbarisyilmaz/ara v0.0.4/go.mod h1:BfFOxnUd6Mj6xmcvRxHN3Sr21Z1T3U2MYkYOmoQe4Ts= github.com/cncf/xds/go v0.0.0-20260202195803-dba9d589def2 h1:aBangftG7EVZoUb69Os8IaYg++6uMOdKK83QtkkvJik= github.com/cncf/xds/go v0.0.0-20260202195803-dba9d589def2/go.mod h1:qwXFYgsP6T7XnJtbKlf1HP8AjxZZyzxMmc+Lq5GjlU4= +github.com/coreos/go-semver v0.3.1 h1:yi21YpKnrx1gt5R+la8n5WgS0kCrsPp33dmEyHReZr4= +github.com/coreos/go-semver v0.3.1/go.mod h1:irMmmIw/7yzSRPWryHsK7EYSg09caPQL03VsM8rvUec= +github.com/coreos/go-systemd/v22 v22.7.0 h1:LAEzFkke61DFROc7zNLX/WA2i5J8gYqe0rSj9KI28KA= +github.com/coreos/go-systemd/v22 v22.7.0/go.mod h1:xNUYtjHu2EDXbsxz1i41wouACIwT7Ybq9o0BQhMwD0w= github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= @@ -177,6 +185,12 @@ github.com/googleapis/enterprise-certificate-proxy v0.3.24 h1:G9iCJIY+AYVmUskKdj github.com/googleapis/enterprise-certificate-proxy v0.3.24/go.mod h1:hEjCmnoakQzv5ImLUp6alkY7XgLlNoAxwTXmhHHNIsI= github.com/googleapis/gax-go/v2 v2.27.0 h1:qe2iQOUzxb3fd6x6ewpIEixPQhJ+0Ihc/RRZhKA/2g0= github.com/googleapis/gax-go/v2 v2.27.0/go.mod h1:RofiqDw3ehcG4L0H1zO3D2Itpwd3/uS/7wdmbNDF1DE= +github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus v1.1.0 h1:QGLs/O40yoNK9vmy4rhUGBVyMf1lISBGtXRpsu/Qu/o= +github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus v1.1.0/go.mod h1:hM2alZsMUni80N33RBe6J0e423LB+odMj7d3EMP9l20= +github.com/grpc-ecosystem/go-grpc-middleware/v2 v2.3.3 h1:B+8ClL/kCQkRiU82d9xajRPKYMrB7E0MbtzWVi1K4ns= +github.com/grpc-ecosystem/go-grpc-middleware/v2 v2.3.3/go.mod h1:NbCUVmiS4foBGBHOYlCT25+YmGpJ32dZPi75pGEUpj4= +github.com/grpc-ecosystem/grpc-gateway/v2 v2.31.0 h1:Bd7KaOxzULLxtZ/K5s1aLbWhR0+5RToO65TXHsf3bqQ= +github.com/grpc-ecosystem/grpc-gateway/v2 v2.31.0/go.mod h1:nN7ts3dFXKtCZWc//yfkpcQNKJABg16/uDVAZpLDalo= github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= github.com/johannesboyne/gofakes3 v1.2.0 h1:I9VEzPWvvAUAGzDlhYFoZjF0AXMlkcEyZlmBwiI6Oms= @@ -236,6 +250,12 @@ github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM= github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg= go.etcd.io/bbolt v1.5.0 h1:S7GAl7Fxv12yohbwFfIbQCGDWbQbtDGPET4P/bD4lxU= go.etcd.io/bbolt v1.5.0/go.mod h1:mkltfYE5aUHQxUct9N9V+Kp7aSjFqjgrhcXIS70Lrdk= +go.etcd.io/etcd/api/v3 v3.7.0 h1:WZlGK7pRtYGDB8ti8wkrQ5D2oWGMbtNL9VA5e+vF2Fg= +go.etcd.io/etcd/api/v3 v3.7.0/go.mod h1:EcTihnwAQ0BQNh5dfAdaFVFdchuo7EP0HlX7TV3jz/A= +go.etcd.io/etcd/client/pkg/v3 v3.7.0 h1:sW9njJzS3vXKcAJjjLQ4nk+avNUJ12Bcijcx8ehUskE= +go.etcd.io/etcd/client/pkg/v3 v3.7.0/go.mod h1:cnzZGIUzSfjEwLC6UBVsSXlEK1eepS/JUD7wE6PLRT0= +go.etcd.io/etcd/client/v3 v3.7.0 h1:5MHO37VbPB87VRPKUXEcicjeQWiTSjpPv3Ume8xPx20= +go.etcd.io/etcd/client/v3 v3.7.0/go.mod h1:DJ382WuwjmbowjPDyaaQ0idWXy4dh91XRhe4FOrb9vM= go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64= go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y= go.opentelemetry.io/contrib/detectors/gcp v1.47.0 h1:Ct6+aaiSO6Nk5SOb6PYcR7iWp+83YF/pL1OJcGqOmbg= @@ -246,6 +266,10 @@ go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.72.0 h1:LxwW/9c go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.72.0/go.mod h1:tOsftB4SslBwwErVEPaenU2RpThXWPIU8DoJHEC4dyw= go.opentelemetry.io/otel v1.47.0 h1:j7ALJ/zgkS7Z6aeJW09p8VC9804bC+PpeTfCD4XPnOM= go.opentelemetry.io/otel v1.47.0/go.mod h1:8wS9O2qfXrYrzp6hIF/HOYJJf/wIhFPhR2xLuP+iXQU= +go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.47.0 h1:julhjPeUH/q/7hinbSdDdqt5h7Zw9YWmRlWRhI0jd54= +go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.47.0/go.mod h1:Ao2mz688LH/tFf0yMAenidq6k2YNSx6SIY2q6jDACck= +go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.47.0 h1:UFpxOpYPmMNUtOWhdb+nC1WELIgVf8z9higURrbzYU4= +go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.47.0/go.mod h1:YBGjxe3lt0jtXoEXxGrGhv/jM7oUBUDVK7zhAvNyEjc= go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.46.0 h1:PR9eAf7o0dQs3hshZNZpE9aW2dXWX/KdDf6pJilVD3U= go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.46.0/go.mod h1:2Z4KyNdH1uuzivdinyfGsxzNNT/Rl45pwtVwfYVI0xk= go.opentelemetry.io/otel/log v1.47.0 h1:cOTS1CcLbSQeZKanGJ+0JpF/+t4PELi3O3bbl2lqCcI= @@ -260,6 +284,8 @@ go.opentelemetry.io/otel/sdk/metric v1.47.0 h1:lfISg2j93VT6yqdk9OfUaZmw/GfcZqCCV go.opentelemetry.io/otel/sdk/metric v1.47.0/go.mod h1:ypLp+mW1Nt2x+Szt3b5/i1syodyts49lMOwxpDI3VGw= go.opentelemetry.io/otel/trace v1.47.0 h1:JOjX/Oci8K94QHddo+bbfya/Ai/nf6/dt9ZfrFNWSrM= go.opentelemetry.io/otel/trace v1.47.0/go.mod h1:jNaSLa2PZEYFG6fRjJABAu+bw4FS08uDmPg28lTghu0= +go.opentelemetry.io/proto/otlp v1.11.1 h1:SCrPqH9NE5CmBrOqpjNCiT0X5mGwMyGERXdogfR7Yjw= +go.opentelemetry.io/proto/otlp v1.11.1/go.mod h1:wq8W4aSf6bOy8RZyESOeQFr9Pu7wydWX/TY4sgvrTPg= go.shabbyrobe.org/gocovmerge v0.0.0-20230507111327-fa4f82cfbf4d h1:Ns9kd1Rwzw7t0BR8XMphenji4SmIoNZPn8zhYmaVKP8= go.shabbyrobe.org/gocovmerge v0.0.0-20230507111327-fa4f82cfbf4d/go.mod h1:92Uoe3l++MlthCm+koNi0tcUCX3anayogF0Pa/sp24k= go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= @@ -332,10 +358,14 @@ k8s.io/apiextensions-apiserver v0.37.0 h1:zRMQ3+/LIE5oZ0tVvXwYHC+dIkSP5cjNWju7AZ k8s.io/apiextensions-apiserver v0.37.0/go.mod h1:HU0PfSBwchHL5iDau6jjt9zU6ryWkDDlaVUiq91NK80= k8s.io/apimachinery v0.37.2 h1:G4t9RGoPDbQML38hz/kQQiltttgd3egg2ODCR8aIc2o= k8s.io/apimachinery v0.37.2/go.mod h1:qaAvSNvgU2x07wRM95MSHSJTPQ/ob/LeNAGbeDgqQJg= +k8s.io/apiserver v0.37.0 h1:TXg7OxsOWrAH8J4Zi/gBAZuMw1Dfdd+6cca2h4qjRqo= +k8s.io/apiserver v0.37.0/go.mod h1:OddHDF4gy9qyIb8o/3+qaeP6S0vEObWLgOygVqXksv0= k8s.io/client-go v0.37.2 h1:sgFtyRZukGkOrsu+QgS7LYnbFFJ5ut3MchdBNduD//E= k8s.io/client-go v0.37.2/go.mod h1:bo3LFC8YbCtKFPX21Yj8puMsoJLAv/+SvuOODma0s54= k8s.io/code-generator v0.37.2 h1:wfx7G5OkCTJktFu7HkEDTEa6iBYr3RMi2TTC3LBLfOE= k8s.io/code-generator v0.37.2/go.mod h1:k24CS91bM7Kk5b776drjJYvvfLT5vceiLeOzTxNsEUc= +k8s.io/component-base v0.37.0 h1:3SdSa4+itMdFTDFTeR8CxKGmSTSMXFlKL4ky8OqjguM= +k8s.io/component-base v0.37.0/go.mod h1:LjOebp4R9y6LODWZQv102ZQxGheLcDO2ZJLAw6bbh4I= k8s.io/gengo/v2 v2.0.0-20260408192533-25e2208e0dc3 h1:3L6PNkMLXkU/pz3jWzaaIUz0Rs2V9h+5O51AeRC7poc= k8s.io/gengo/v2 v2.0.0-20260408192533-25e2208e0dc3/go.mod h1:yvyl3l9E+UxlqOMUULdKTAYB0rEhsmjr7+2Vb/1pCSo= k8s.io/klog/v2 v2.140.0 h1:Tf+J3AH7xnUzZyVVXhTgGhEKnFqye14aadWv7bzXdzc= @@ -344,6 +374,8 @@ k8s.io/kube-openapi v0.0.0-20260721132016-d427ff9ee9ad h1:oXImqH8mQNk7PmvzKhmN3d k8s.io/kube-openapi v0.0.0-20260721132016-d427ff9ee9ad/go.mod h1:0/mqHCVhlumdJ3BhCfnjSZQE037nAhNodh1/hK0T8/I= k8s.io/utils v0.0.0-20260626114624-be93311217bd h1:Ea7fgQ5we8Y9T0OX5o0dAHzQOBRI07D/dEYRaB9ZZEs= k8s.io/utils v0.0.0-20260626114624-be93311217bd/go.mod h1:xDxuJ0whA3d0I4mf/C4ppKHxXynQ+fxnkmQH0vTHnuk= +sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.36.0 h1:/YpDJ4vReG7ZmzSpBGxduXgywWkJU9zHubgJG03MT+Y= +sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.36.0/go.mod h1:tJo1aepTXyR+8Xs3sUsGBDk4Ub2AM5dPAPKJx0mpm5c= sigs.k8s.io/controller-runtime v0.25.3 h1:b4OcZSY/aKHY8JG+rBlQmQ1xLs574+ZBEjv1dtfLd80= sigs.k8s.io/controller-runtime v0.25.3/go.mod h1:4QqLdT6z/L6Olj8JJCtvztid4/fnIiYsfaTFScegctc= sigs.k8s.io/json v0.0.0-20250730193827-2d320260d730 h1:IpInykpT6ceI+QxKBbEflcR5EXP7sU1kvOlxwZh5txg= diff --git a/hack/search-lag.sh b/hack/search-lag.sh index a471e5b..a8668b3 100755 --- a/hack/search-lag.sh +++ b/hack/search-lag.sh @@ -1,12 +1,21 @@ #!/usr/bin/env bash # Measures how long a post change takes to reach Milo's search index. # -# It hides a visible post, waits until search no longer finds it, then -# unhides it and times how long until search finds it again. Hiding is -# visible to users while it lasts, so run it against staging. +# It hides a visible post, waits until search no longer finds it, then puts +# back the post's original spec.hidden and times how long until search finds +# it again. Hiding is visible to users while it lasts, so run it against +# staging. # # KUBECONFIG= hack/search-lag.sh [timeout-seconds] # +# The kubeconfig must point at Milo's platform root, not a user, organization +# or project control plane: search only returns posts to platform-context +# queries (see "Who can search posts" in README.md). +# +# The post must be visible and indexed; the script exits without changing +# anything otherwise. Once it has hidden the post, it restores the original +# spec.hidden on every exit, including errors, timeouts and Ctrl-C. +# # Needs: kubectl, jq; staff permission to patch posts and create # resourcesearchqueries. set -euo pipefail @@ -14,38 +23,86 @@ set -euo pipefail post=${1:?usage: search-lag.sh [timeout-seconds]} timeout=${2:-300} -title=$(kubectl get post "$post" -o jsonpath='{.status.title}') +obj=$(kubectl get post "$post" -o json) +title=$(jq -r '.status.title // ""' <<<"$obj") +# true, false, or null when the field is unset; a merge patch with null +# removes the field again, so restoring gives back exactly what was there. +# (Not `.spec.hidden // null`: jq's // would turn false into null.) +original=$(jq -c '.spec // {} | if has("hidden") then .hidden else null end' <<<"$obj") + [[ -n $title ]] || { echo "post $post has no title" >&2; exit 1; } +[[ $original != true ]] || { echo "post $post is hidden, so it isn't indexed; pick a visible post" >&2; exit 1; } +# found exits 0 if search finds the post by its title, 1 if it doesn't, and 2 +# if the query fails or the posts index isn't ready. found() { - kubectl create -o json -f - </dev/null -{"apiVersion": "search.miloapis.com/v1alpha1", "kind": "ResourceSearchQuery", - "metadata": {"generateName": "search-lag-"}, - "spec": {"query": $(jq -Rn --arg t "$title" '$t'), "limit": 100, - "targetResources": [{"group": "feedback.miloapis.com", "version": "v1alpha1", "kind": "Post"}]}} -JSON + local query out denied hit + query=$(jq -cn --arg t "$title" '{ + apiVersion: "search.miloapis.com/v1alpha1", kind: "ResourceSearchQuery", + metadata: {generateName: "search-lag-"}, + spec: {query: $t, limit: 100, + targetResources: [{group: "feedback.miloapis.com", version: "v1alpha1", kind: "Post"}]}}') || return 2 + out=$(kubectl create -o json -f - <<<"$query") || return 2 + denied=$(jq '.status.deniedTargetResources // [] | length' <<<"$out") || return 2 + if ((denied > 0)); then + echo "search has no ready index for posts" >&2 + return 2 + fi + hit=$(jq --arg n "$post" '.status.results // [] | any(.resource.metadata.name == $n)' <<<"$out") || return 2 + [[ $hit == true ]] } -wait_for() { # wait_for - local start=$SECONDS - while (( SECONDS - start < timeout )); do - if found; then [[ $1 == 1 ]] && return 0; else [[ $1 == 0 ]] && return 0; fi +wait_for() { # wait_for + local start=$SECONDS rc + while ((SECONDS - start < timeout)); do + rc=0 + found || rc=$? + ((rc == $1)) && return 0 + ((rc == 2)) && { echo "search query failed" >&2; return 1; } sleep 1 done echo "timed out after ${timeout}s" >&2 return 1 } -trap 'kubectl patch post "$post" --type=merge -p "{\"spec\":{\"hidden\":false}}" >/dev/null' EXIT +set_hidden() { # set_hidden + kubectl patch post "$post" --type=merge -p "$(jq -cn --argjson h "$1" '{spec: {hidden: $h}}')" >/dev/null +} + +changed=0 +restore() { + local rc=$? + if ((changed)); then + if set_hidden "$original"; then + echo "restored spec.hidden to $original" >&2 + else + echo "FAILED to restore $post: set spec.hidden back to $original by hand" >&2 + rc=1 + fi + fi + exit "$rc" +} +trap restore EXIT +trap 'exit 130' INT +trap 'exit 143' TERM -found || { echo "search doesn't find $post by its title; is it indexed?" >&2; exit 1; } +rc=0 +found || rc=$? +case $rc in +0) ;; +1) echo "search doesn't find $post by its title; is it indexed?" >&2; exit 1 ;; +*) echo "search query failed" >&2; exit 1 ;; +esac -kubectl patch post "$post" --type=merge -p '{"spec":{"hidden":true}}' >/dev/null +# Set before patching: if the patch fails partway, restoring is harmless. +changed=1 +set_hidden true t0=$SECONDS -wait_for 0 +wait_for 1 echo "removed from the index after about $((SECONDS - t0))s" -kubectl patch post "$post" --type=merge -p '{"spec":{"hidden":false}}' >/dev/null +set_hidden "$original" +changed=0 t0=$SECONDS -wait_for 1 +wait_for 0 echo "back in the index after about $((SECONDS - t0))s" diff --git a/hack/search-scope.sh b/hack/search-scope.sh new file mode 100755 index 0000000..ecb4938 --- /dev/null +++ b/hack/search-scope.sh @@ -0,0 +1,68 @@ +#!/usr/bin/env bash +# Checks which Milo contexts can find a post in search, as README.md's "Who +# can search posts" describes: a query at the platform root finds a visible, +# indexed post, and a query in a user's own context finds no posts at all. +# +# PLATFORM_KUBECONFIG= \ +# USER_KUBECONFIG= \ +# hack/search-scope.sh +# +# USER_KUBECONFIG's server must end in +# /apis/iam.miloapis.com/v1alpha1/users//control-plane, which is how +# datumctl and the cloud portal reach the user context. Run it against +# staging. It only reads posts and creates ResourceSearchQueries, which +# aren't stored. +# +# Exits 0 when search behaves as documented, 1 when it doesn't (so the +# README is out of date), and 2 when a query fails. +# +# Needs: kubectl, jq. +set -euo pipefail + +post=${1:?usage: search-scope.sh } +: "${PLATFORM_KUBECONFIG:?set PLATFORM_KUBECONFIG}" "${USER_KUBECONFIG:?set USER_KUBECONFIG}" + +server=$(KUBECONFIG=$USER_KUBECONFIG kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}') +[[ $server == */apis/iam.miloapis.com/v1alpha1/users/*/control-plane ]] || { + echo "USER_KUBECONFIG's server $server isn't a user control plane" >&2 + exit 2 +} + +obj=$(KUBECONFIG=$PLATFORM_KUBECONFIG kubectl get post "$post" -o json) +title=$(jq -r '.status.title // ""' <<<"$obj") +[[ -n $title ]] || { echo "post $post has no title" >&2; exit 2; } +[[ $(jq '.spec.hidden == true' <<<"$obj") == false ]] || { echo "post $post is hidden; pick a visible post" >&2; exit 2; } + +# search prints the names of the posts the query returns. +search() { + local query out + query=$(jq -cn --arg t "$title" '{ + apiVersion: "search.miloapis.com/v1alpha1", kind: "ResourceSearchQuery", + metadata: {generateName: "search-scope-"}, + spec: {query: $t, limit: 100, + targetResources: [{group: "feedback.miloapis.com", version: "v1alpha1", kind: "Post"}]}}') + out=$(KUBECONFIG=$1 kubectl create -o json -f - <<<"$query") || return 1 + if [[ $(jq '.status.deniedTargetResources // [] | length' <<<"$out") != 0 ]]; then + echo "search has no ready index for posts" >&2 + return 1 + fi + jq -r '.status.results // [] | .[].resource.metadata.name' <<<"$out" +} + +platform=$(search "$PLATFORM_KUBECONFIG") || exit 2 +user=$(search "$USER_KUBECONFIG") || exit 2 + +ok=1 +if grep -qxF "$post" <<<"$platform"; then + echo "platform root: finds $post" +else + echo "platform root: doesn't find $post; is it indexed?" + ok=0 +fi +if [[ -z $user ]]; then + echo "user context: finds no posts" +else + echo "user context: finds posts ($(paste -sd, - <<<"$user")); README.md's \"Who can search posts\" is out of date" + ok=0 +fi +((ok)) || exit 1 diff --git a/pkg/apis/feedback/v1alpha1/crd_test.go b/pkg/apis/feedback/v1alpha1/crd_test.go index a551bbe..c0d6036 100644 --- a/pkg/apis/feedback/v1alpha1/crd_test.go +++ b/pkg/apis/feedback/v1alpha1/crd_test.go @@ -9,6 +9,7 @@ import ( apierrors "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" "k8s.io/apimachinery/pkg/fields" "k8s.io/apimachinery/pkg/runtime/serializer" "sigs.k8s.io/controller-runtime/pkg/client" @@ -117,6 +118,30 @@ func TestCRDs(t *testing.T) { } }) + t.Run("a post created without spec is served with one", func(t *testing.T) { + // The search index condition reads spec.hidden and can't evaluate it + // on an object with no spec, so the CRD defaults spec to {}. + u := &unstructured.Unstructured{Object: map[string]any{ + "apiVersion": v1.SchemeGroupVersion.String(), + "kind": "Post", + "metadata": map[string]any{"name": "a.b.no-spec"}, + }} + if err := c.Create(ctx, u); err != nil { + t.Fatal(err) + } + if _, ok := u.Object["spec"].(map[string]any); !ok { + t.Errorf("create response has no spec: %v", u.Object) + } + got := &unstructured.Unstructured{} + got.SetGroupVersionKind(v1.SchemeGroupVersion.WithKind("Post")) + if err := c.Get(ctx, client.ObjectKey{Name: "a.b.no-spec"}, got); err != nil { + t.Fatal(err) + } + if _, ok := got.Object["spec"].(map[string]any); !ok { + t.Errorf("stored post has no spec: %v", got.Object) + } + }) + t.Run("review decisions", func(t *testing.T) { r := &v1.SubmissionReview{ObjectMeta: metav1.ObjectMeta{Name: "sub-1"}} if err := c.Create(ctx, r); err != nil { diff --git a/pkg/apis/feedback/v1alpha1/post_types.go b/pkg/apis/feedback/v1alpha1/post_types.go index 36374a7..6799708 100644 --- a/pkg/apis/feedback/v1alpha1/post_types.go +++ b/pkg/apis/feedback/v1alpha1/post_types.go @@ -82,6 +82,9 @@ type Post struct { metav1.TypeMeta `json:",inline"` metav1.ObjectMeta `json:"metadata,omitempty"` + // Spec holds the fields staff control. It defaults to {} so every post + // has one, which the search index condition on spec.hidden needs. + // +kubebuilder:default={} Spec PostSpec `json:"spec,omitempty"` Status PostStatus `json:"status,omitempty"` } diff --git a/test/config/search_test.go b/test/config/search_test.go index 30c745b..88d884f 100644 --- a/test/config/search_test.go +++ b/test/config/search_test.go @@ -9,6 +9,10 @@ import ( "testing" "github.com/google/cel-go/cel" + "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions" + apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" + structuralschema "k8s.io/apiextensions-apiserver/pkg/apiserver/schema" + structuraldefaulting "k8s.io/apiextensions-apiserver/pkg/apiserver/schema/defaulting" sigsyaml "sigs.k8s.io/yaml" v1 "go.miloapis.com/feedback/pkg/apis/feedback/v1alpha1" @@ -125,9 +129,11 @@ func TestSearchPolicies(t *testing.T) { } } -// TestHiddenPostCondition evaluates the posts policy's condition the way the -// indexer does: metadata, spec and status as dynamic variables. -func TestHiddenPostCondition(t *testing.T) { +// postCondition compiles the posts policy's condition in the environment +// the search indexer uses: metadata, spec and status as dynamic variables +// (milo-os/search internal/cel NewEnv). +func postCondition(t *testing.T) cel.Program { + t.Helper() var cond string for _, p := range loadIndexPolicies(t) { if p.Spec.TargetResource.Kind == "Post" { @@ -149,20 +155,90 @@ func TestHiddenPostCondition(t *testing.T) { if err != nil { t.Fatal(err) } + return prg +} + +// indexed evaluates the condition the way the indexer does +// (internal/policy/evaluation CachedPolicy.Evaluate): only the top-level +// keys the object has are bound, and an evaluation error counts as no match. +func indexed(prg cel.Program, obj map[string]any) (bool, error) { + activation := map[string]any{} + for _, k := range []string{"metadata", "spec", "status"} { + if v, ok := obj[k]; ok { + activation[k] = v + } + } + out, _, err := prg.Eval(activation) + if err != nil { + return false, err + } + v, _ := out.Value().(bool) + return v, nil +} + +// postDefaulter applies the Post CRD's schema defaults to a raw object, as +// Milo does to every Post it stores and returns, and so to the audit event +// the indexer reads. +func postDefaulter(t *testing.T) func(map[string]any) { + t.Helper() + data, err := os.ReadFile("../../config/crd/bases/feedback.miloapis.com_posts.yaml") + if err != nil { + t.Fatal(err) + } + var crd apiextensionsv1.CustomResourceDefinition + if err := sigsyaml.Unmarshal(data, &crd); err != nil { + t.Fatal(err) + } + var internal apiextensions.JSONSchemaProps + if err := apiextensionsv1.Convert_v1_JSONSchemaProps_To_apiextensions_JSONSchemaProps(crd.Spec.Versions[0].Schema.OpenAPIV3Schema, &internal, nil); err != nil { + t.Fatal(err) + } + s, err := structuralschema.NewStructural(&internal) + if err != nil { + t.Fatal(err) + } + return func(obj map[string]any) { structuraldefaulting.Default(obj, s) } +} + +// TestHiddenPostCondition checks which posts the indexer keeps, starting from +// raw objects as a client could create them, including one with no spec key. +// The indexer can't evaluate spec.hidden without a spec, so a post with none +// is indexed only because the CRD defaults spec to {}. +func TestHiddenPostCondition(t *testing.T) { + prg := postCondition(t) + defaults := postDefaulter(t) for _, c := range []struct { - hidden bool - want bool - }{{false, true}, {true, false}} { - p := v1.Post{Spec: v1.PostSpec{Hidden: c.hidden}} - raw, _ := json.Marshal(p) + name string + raw string + want bool + }{ + {"no spec", `{"metadata":{"name":"a.b.1"},"status":{"title":"t"}}`, true}, + {"empty spec", `{"metadata":{"name":"a.b.2"},"spec":{}}`, true}, + {"visible", `{"metadata":{"name":"a.b.3"},"spec":{"hidden":false}}`, true}, + {"hidden", `{"metadata":{"name":"a.b.4"},"spec":{"hidden":true}}`, false}, + } { var obj map[string]any - _ = json.Unmarshal(raw, &obj) - out, _, err := prg.Eval(map[string]any{"metadata": obj["metadata"], "spec": obj["spec"], "status": obj["status"]}) + if err := json.Unmarshal([]byte(c.raw), &obj); err != nil { + t.Fatal(err) + } + defaults(obj) + got, err := indexed(prg, obj) if err != nil { - t.Fatalf("hidden=%v: %v", c.hidden, err) + t.Errorf("%s: %v (as served: %v)", c.name, err, obj) + continue + } + if got != c.want { + t.Errorf("%s: indexed=%v, want %v", c.name, got, c.want) } - if out.Value() != c.want { - t.Errorf("hidden=%v: indexed=%v, want %v", c.hidden, out.Value(), c.want) + } + + // Typed clients always send a spec; check they agree. + for _, hidden := range []bool{false, true} { + raw, _ := json.Marshal(v1.Post{Spec: v1.PostSpec{Hidden: hidden}}) + var obj map[string]any + _ = json.Unmarshal(raw, &obj) + if got, err := indexed(prg, obj); err != nil || got == hidden { + t.Errorf("typed post hidden=%v: indexed=%v, err=%v", hidden, got, err) } } } diff --git a/test/hack/search_lag_test.go b/test/hack/search_lag_test.go new file mode 100644 index 0000000..9e562b6 --- /dev/null +++ b/test/hack/search_lag_test.go @@ -0,0 +1,161 @@ +// Package hack tests the scripts under hack/. +package hack + +import ( + "errors" + "os" + "os/exec" + "path/filepath" + "slices" + "strings" + "syscall" + "testing" + "time" +) + +// stubKubectl stands in for kubectl against one post. It keeps the post's +// spec.hidden in $STUB/hidden ("absent" when unset), logs each merge patch to +// $STUB/patches and answers ResourceSearchQuery creates according to +// $STUB/mode: +// +// ok found unless hidden (the index catches up at once) +// missing never found +// stuck always found, so hiding never takes effect +// error the query fails +// error-after-hide the query fails while the post is hidden +// denied the posts index isn't ready +const stubKubectl = `#!/usr/bin/env bash +set -euo pipefail +hidden=$(cat "$STUB/hidden") +case "$1" in +get) + if [[ $hidden == absent ]]; then spec='{}'; else spec="{\"hidden\":$hidden}"; fi + echo "{\"metadata\":{\"name\":\"$3\"},\"spec\":$spec,\"status\":{\"title\":\"Wildcard hostnames\"}}" ;; +patch) + p=$6 + echo "$p" >>"$STUB/patches" + h=$(jq -c '.spec.hidden' <<<"$p") + if [[ $h == null ]]; then echo absent >"$STUB/hidden"; else echo "$h" >"$STUB/hidden"; fi + echo '{}' ;; +create) + cat >/dev/null + mode=$(cat "$STUB/mode") + found=false + case $mode in + ok) [[ $hidden == true ]] || found=true ;; + missing) ;; + stuck) found=true ;; + error) exit 1 ;; + error-after-hide) [[ $hidden != true ]] || exit 1; found=true ;; + denied) echo '{"status":{"deniedTargetResources":[{"kind":"Post"}]}}'; exit 0 ;; + esac + results='[]' + $found && results='[{"resource":{"metadata":{"name":"a.b.1"}}}]' + echo "{\"status\":{\"results\":$results}}" ;; +*) echo "unexpected kubectl $*" >&2; exit 1 ;; +esac +` + +type run struct { + stub string + cmd *exec.Cmd + out strings.Builder +} + +func newRun(t *testing.T, hidden, mode, timeout string) *run { + t.Helper() + for _, bin := range []string{"bash", "jq"} { + if _, err := exec.LookPath(bin); err != nil { + t.Skipf("%s not found", bin) + } + } + stub := t.TempDir() + bin := filepath.Join(stub, "bin") + must(t, os.Mkdir(bin, 0o755)) + must(t, os.WriteFile(filepath.Join(bin, "kubectl"), []byte(stubKubectl), 0o755)) + must(t, os.WriteFile(filepath.Join(stub, "hidden"), []byte(hidden+"\n"), 0o644)) + must(t, os.WriteFile(filepath.Join(stub, "mode"), []byte(mode+"\n"), 0o644)) + + r := &run{stub: stub} + r.cmd = exec.Command("bash", "../../hack/search-lag.sh", "a.b.1", timeout) + r.cmd.Env = append(os.Environ(), "STUB="+stub, "PATH="+bin+string(os.PathListSeparator)+os.Getenv("PATH")) + r.cmd.Stdout = &r.out + r.cmd.Stderr = &r.out + return r +} + +func (r *run) exitCode(err error) int { + var ee *exec.ExitError + if errors.As(err, &ee) { + return ee.ExitCode() + } + if err != nil { + return -1 + } + return 0 +} + +func (r *run) patches() []string { + data, _ := os.ReadFile(filepath.Join(r.stub, "patches")) + return strings.Fields(string(data)) +} + +func (r *run) hidden() string { + data, _ := os.ReadFile(filepath.Join(r.stub, "hidden")) + return strings.TrimSpace(string(data)) +} + +func must(t *testing.T, err error) { + t.Helper() + if err != nil { + t.Fatal(err) + } +} + +func TestSearchLag(t *testing.T) { + const hide = `{"spec":{"hidden":true}}` + for _, c := range []struct { + name, hidden, mode string + wantCode int + wantPatches []string + }{ + {"hidden unset", "absent", "ok", 0, []string{hide, `{"spec":{"hidden":null}}`}}, + {"hidden false", "false", "ok", 0, []string{hide, `{"spec":{"hidden":false}}`}}, + {"already hidden", "true", "ok", 1, nil}, + {"not indexed", "absent", "missing", 1, nil}, + {"search fails at the start", "false", "error", 1, nil}, + {"index not ready", "false", "denied", 1, nil}, + {"never leaves the index", "absent", "stuck", 1, []string{hide, `{"spec":{"hidden":null}}`}}, + {"search fails while hidden", "false", "error-after-hide", 1, []string{hide, `{"spec":{"hidden":false}}`}}, + } { + t.Run(c.name, func(t *testing.T) { + r := newRun(t, c.hidden, c.mode, "2") + code := r.exitCode(r.cmd.Run()) + if code != c.wantCode { + t.Errorf("exit %d, want %d\n%s", code, c.wantCode, r.out.String()) + } + if got := r.patches(); !slices.Equal(got, c.wantPatches) { + t.Errorf("patches %v, want %v\n%s", got, c.wantPatches, r.out.String()) + } + if got := r.hidden(); got != c.hidden { + t.Errorf("spec.hidden ends as %s, want the original %s", got, c.hidden) + } + }) + } + + t.Run("interrupted while hidden", func(t *testing.T) { + r := newRun(t, "false", "stuck", "60") + must(t, r.cmd.Start()) + deadline := time.Now().Add(10 * time.Second) + for len(r.patches()) == 0 && time.Now().Before(deadline) { + time.Sleep(50 * time.Millisecond) + } + must(t, r.cmd.Process.Signal(syscall.SIGTERM)) + if code := r.exitCode(r.cmd.Wait()); code != 143 { + t.Errorf("exit %d, want 143\n%s", code, r.out.String()) + } + if got := r.hidden(); got != "false" { + t.Errorf("spec.hidden ends as %s, want false\n%s", got, r.out.String()) + } + }) +} From fd2fcce6eccfd9f5b9007373543345c23fdd8774 Mon Sep 17 00:00:00 2001 From: Matt Jenkinson Date: Tue, 17 Nov 2026 17:19:43 -0500 Subject: [PATCH 3/3] test: end-to-end tests in a kind cluster against a real Milo Add task e2e:up, e2e:test and e2e:down, following milo-os/billing: a test-infra kind cluster, Milo built and deployed from MILO_DIR, and feedback installed in that Milo with the operator, feedback-uploads and MinIO in the cluster (config/overlays/e2e). The operator and feedback-uploads authenticate to Milo with static tokens for their production usernames, so the admission webhooks' identity checks and config/milo/rbac are exercised as they are. test-infra's Milo uses RBAC, so the IAM roles are granted as equivalent ClusterRoles (hack/e2e/iam-rbac.sh). Chainsaw tests cover votes and owner rules, hidden posts, merge chains, submissions and review decisions, the submission rate limit, attachments through feedback-uploads and MinIO, moving submissions to security, and operator readiness. --- .gitignore | 6 + README.md | 67 ++++++++ Taskfile.yaml | 119 ++++++++++++++ config/overlays/e2e/audit-sink.yaml | 66 ++++++++ config/overlays/e2e/certificates.yaml | 51 ++++++ config/overlays/e2e/kustomization.yaml | 115 ++++++++++++++ config/overlays/e2e/milo-kubeconfig.yaml | 47 ++++++ config/overlays/e2e/milo/kustomization.yaml | 26 ++++ config/overlays/e2e/milo/tokens.csv | 12 ++ config/overlays/e2e/milo/users.yaml | 76 +++++++++ config/overlays/e2e/minio.yaml | 83 ++++++++++ config/overlays/e2e/namespace.yaml | 4 + hack/e2e/deploy.sh | 85 ++++++++++ hack/e2e/iam-rbac.sh | 28 ++++ hack/e2e/milo-identities.sh | 37 +++++ test/e2e/.chainsaw.yaml | 23 +++ test/e2e/attachments/chainsaw-test.yaml | 115 ++++++++++++++ test/e2e/attachments/client-pod.yaml | 23 +++ test/e2e/hidden-posts/chainsaw-test.yaml | 65 ++++++++ test/e2e/kubeconfig-milo | 16 ++ test/e2e/lib.sh | 147 ++++++++++++++++++ test/e2e/merge-chain/chainsaw-test.yaml | 83 ++++++++++ .../e2e/operator-readiness/chainsaw-test.yaml | 62 ++++++++ test/e2e/rate-limit/chainsaw-test.yaml | 54 +++++++ test/e2e/security/chainsaw-test.yaml | 76 +++++++++ test/e2e/submissions/chainsaw-test.yaml | 88 +++++++++++ test/e2e/votes/chainsaw-test.yaml | 110 +++++++++++++ 27 files changed, 1684 insertions(+) create mode 100644 config/overlays/e2e/audit-sink.yaml create mode 100644 config/overlays/e2e/certificates.yaml create mode 100644 config/overlays/e2e/kustomization.yaml create mode 100644 config/overlays/e2e/milo-kubeconfig.yaml create mode 100644 config/overlays/e2e/milo/kustomization.yaml create mode 100644 config/overlays/e2e/milo/tokens.csv create mode 100644 config/overlays/e2e/milo/users.yaml create mode 100644 config/overlays/e2e/minio.yaml create mode 100644 config/overlays/e2e/namespace.yaml create mode 100755 hack/e2e/deploy.sh create mode 100755 hack/e2e/iam-rbac.sh create mode 100755 hack/e2e/milo-identities.sh create mode 100644 test/e2e/.chainsaw.yaml create mode 100644 test/e2e/attachments/chainsaw-test.yaml create mode 100644 test/e2e/attachments/client-pod.yaml create mode 100644 test/e2e/hidden-posts/chainsaw-test.yaml create mode 100644 test/e2e/kubeconfig-milo create mode 100644 test/e2e/lib.sh create mode 100644 test/e2e/merge-chain/chainsaw-test.yaml create mode 100644 test/e2e/operator-readiness/chainsaw-test.yaml create mode 100644 test/e2e/rate-limit/chainsaw-test.yaml create mode 100644 test/e2e/security/chainsaw-test.yaml create mode 100644 test/e2e/submissions/chainsaw-test.yaml create mode 100644 test/e2e/votes/chainsaw-test.yaml diff --git a/.gitignore b/.gitignore index 419c721..e42521f 100644 --- a/.gitignore +++ b/.gitignore @@ -60,3 +60,9 @@ temp/ .tmp # Personal workflow artifacts (specs/plans/runbooks from superpowers skill) docs/superpowers/ + +# test-infra checkout and kubeconfig, and remote Taskfile cache, for task e2e:* +.test-infra +.task +# test-infra installs kind here when the installed version differs from its pin +/kind diff --git a/README.md b/README.md index 1ebb9f1..a4a9520 100644 --- a/README.md +++ b/README.md @@ -153,6 +153,8 @@ config/milo/ applied to Milo: IAM (iam/), RBAC for the service's config/samples/ an example of every resource hack/search-lag.sh measures how long a post change takes to reach search test/config/ checks config/ against the CRDs +test/e2e/ Chainsaw end-to-end tests in kind (see End-to-end tests) +hack/e2e/ deploys the end-to-end environment ``` ## Development @@ -164,6 +166,71 @@ task generate # after changing types in pkg/apis: deepcopy and CRDs task docker-build ``` +### End-to-end tests + +`task e2e:up` builds a kind cluster from +[datum-cloud/test-infra](https://github.com/datum-cloud/test-infra) (cert-manager, +Envoy Gateway, Flux), deploys Milo into it from a Milo checkout, and installs +feedback in that Milo. `task e2e:test` runs the [Chainsaw](https://kyverno.github.io/chainsaw/) +tests in `test/e2e/` against it. Docker, kind, kubectl, kustomize, jq and yq +must be installed. + +```bash +task e2e:up MILO_DIR=../milo # MILO_DIR defaults to a sibling ../milo +task e2e:test # or: task e2e:test -- test/e2e/votes +task e2e:redeploy # rebuild feedback and restart it +task e2e:down # deletes the kind cluster +``` + +`e2e:up` runs `e2e:cluster`, `e2e:milo` (Milo's own `dev:build`, `dev:load` +and `dev:deploy`, then `e2e:milo-identities`), `e2e:build` and `e2e:deploy`; +each can be run on its own. Milo answers on https://localhost:30443. + +What runs where: + +- **In Milo:** the CRDs, `config/milo/rbac` and the admission webhooks + (with the webhook CA as `caBundle`, since cert-manager can't inject into + Milo), and the `post-ref` admission policy from `config/milo/activity`. + `config/milo/iam` is checked with a server-side dry run only. +- **In the cluster** (`config/overlays/e2e`): the operator and + `feedback-uploads` (one replica each), a self-signed webhook certificate + from cert-manager, MinIO as the attachments bucket through the + `s3-attachments` component, and in `milo-system` a stand-in for Milo's + audit log processor that logs the audit events Milo sends it. +- **Identities:** `hack/e2e/milo-identities.sh` adds static tokens to Milo's + token file (`config/overlays/e2e/milo/tokens.csv`) for + `system:feedback-operator@feedback.miloapis.com`, + `system:feedback-uploads@feedback.miloapis.com` and + `system:feedback-sync@feedback.miloapis.com` (production uses client + certificates for the same usernames), and for `other-user`, `limit-user`, + `staff-user` and `security-user` besides Milo's `test-user`. + +test-infra's Milo authorizes with Kubernetes RBAC, not Milo IAM. The tests +grant the IAM roles' permissions as ClusterRoles instead +(`hack/e2e/iam-rbac.sh`): `feedback.miloapis.com-user` to every signed-in +user, `-reviewer` to `staff-user` and `-security-team` to `security-user`. +RBAC can't scope a grant to a user's own context, so the tests don't cover +what Milo IAM decides by context: for example, that a user can't act in +another user's context, or read a submission's attachments in the platform +context. + +| Test | What it checks | +| --- | --- | +| `votes` | owner webhook, `.` names, owner filtering, follows, vote totals, post named in delete audit events | +| `hidden-posts` | hidden posts invisible to users, visible to staff, no votes on them | +| `merge-chain` | votes moved along merge chains; a hidden target keeps them, with a condition and no hot loop | +| `submissions` | review created, reviewer recorded, Rejected and Approved carried out | +| `rate-limit` | 429 after the submission limit (`--submission-window=30s` here) | +| `attachments` | upload, attach, copy, download and delete through `feedback-uploads` and MinIO, called from inside the cluster | +| `security` | MoveToSecurity leaves an unrelated report with the target name alone, then moves once it's gone | +| `operator-readiness` | the operator isn't ready before its webhook server serves TLS | + +Not covered: `feedback-sync` and GitHub (the tests write posts with sync's +identity), Milo IAM, activity and search (their services don't run in +test-infra), notifications and digests. + +### Running the operator by hand + Run the operator against a Milo you can reach, in a container: ```bash diff --git a/Taskfile.yaml b/Taskfile.yaml index 46b584d..49fce50 100644 --- a/Taskfile.yaml +++ b/Taskfile.yaml @@ -8,6 +8,23 @@ vars: CONTROLLER_TOOLS_VERSION: "v0.22.0" ENVTEST_VERSION: "release-0.25" ENVTEST_K8S_VERSION: "1.37.x" + CHAINSAW_VERSION: "v0.2.15" + # End-to-end tests run in a kind cluster from datum-cloud/test-infra, with + # Milo built from MILO_DIR (a checkout of milo-os/milo, by default a + # sibling of this one). Keep the test-infra ref in step with Milo's. + TEST_INFRA_CLUSTER_NAME: "test-infra" + TEST_INFRA_REPO_REF: "v0.5.0" + MILO_DIR: '{{.MILO_DIR | default (printf "%s/../milo" .ROOT_DIR)}}' + E2E_IMAGE_TAG: "e2e" + E2E_OVERLAY: "config/overlays/e2e" + +includes: + # Must set TASK_X_REMOTE_TASKFILES=1 to use this feature. + test-infra: + taskfile: https://raw.githubusercontent.com/datum-cloud/test-infra/{{.TEST_INFRA_REPO_REF}}/Taskfile.yml + checksum: 4995ce0e9bc4290b3a0175dee77af2570d715b435d5f3ff65f9cb6c491837bcb + vars: + REPO_REF: "{{.TEST_INFRA_REPO_REF}}" tasks: default: @@ -90,3 +107,105 @@ tasks: cmds: - kustomize build config/base > /dev/null - kustomize build config/overlays/core-control-plane > /dev/null + + # End-to-end tests. See "End-to-end tests" in README.md. + e2e:up: + desc: Create the kind cluster, deploy Milo from MILO_DIR and deploy feedback into it + cmds: + - task: e2e:cluster + - task: e2e:milo + - task: e2e:build + - task: e2e:deploy + + e2e:cluster: + desc: Create the test-infra kind cluster, or reuse it + cmds: + - task: test-infra:ensure-repo + # cluster-up only writes the kubeconfig when it creates the cluster. + - | + if kind get clusters | grep -qx "{{.TEST_INFRA_CLUSTER_NAME}}"; then + kind get kubeconfig --name "{{.TEST_INFRA_CLUSTER_NAME}}" > .test-infra/kubeconfig + fi + - task: test-infra:cluster-up + + e2e:milo: + desc: Build Milo from MILO_DIR, load it into the cluster and deploy it + cmds: + - | + set -e + MILO_DIR="{{.MILO_DIR}}" + if [ ! -f "$MILO_DIR/Taskfile.yaml" ]; then + echo "No Milo checkout at MILO_DIR=$MILO_DIR; set MILO_DIR to one" >&2 + exit 1 + fi + # Milo's tasks find the cluster through its own .test-infra. + [ -e "$MILO_DIR/.test-infra" ] || ln -s "{{.ROOT_DIR}}/.test-infra" "$MILO_DIR/.test-infra" + kind get kubeconfig --name "{{.TEST_INFRA_CLUSTER_NAME}}" > "$MILO_DIR/.test-infra/kubeconfig" + cd "$MILO_DIR" + echo "Deploying Milo from $MILO_DIR at $(git rev-parse --short HEAD)" + TASK_X_REMOTE_TASKFILES=1 task dev:build dev:load dev:deploy + # dev:deploy doesn't restart Milo when only the image changed. + task test-infra:kubectl -- -n milo-system rollout restart deployment/milo-apiserver deployment/milo-controller-manager + task test-infra:kubectl -- -n milo-system rollout status deployment/milo-apiserver deployment/milo-controller-manager --timeout=300s + - task: e2e:milo-identities + + e2e:milo-identities: + desc: Add feedback's service identities and test users to Milo's static tokens + cmds: + - hack/e2e/milo-identities.sh + + e2e:build: + desc: Build the feedback image and load it into the cluster + cmds: + - task: docker-build + vars: + IMAGE_TAG: "{{.E2E_IMAGE_TAG}}" + - kind load docker-image "{{.IMAGE_NAME}}:{{.E2E_IMAGE_TAG}}" --name "{{.TEST_INFRA_CLUSTER_NAME}}" + + e2e:deploy: + desc: Install feedback in Milo and run the operator, uploads and MinIO in the cluster + cmds: + - hack/e2e/deploy.sh + env: + OVERLAY: "{{.E2E_OVERLAY}}" + IMAGE: "{{.IMAGE_NAME}}:{{.E2E_IMAGE_TAG}}" + + e2e:redeploy: + desc: Rebuild the feedback image and restart the operator and uploads + cmds: + - task: e2e:build + - task: e2e:deploy + - task test-infra:kubectl -- -n feedback-system rollout restart deployment/feedback-operator deployment/feedback-uploads + - task test-infra:kubectl -- -n feedback-system rollout status deployment/feedback-operator deployment/feedback-uploads --timeout=180s + + e2e:test: + desc: Run the chainsaw end-to-end tests (pass test directories after --) + deps: + - install-chainsaw + cmds: + - | + set -e + if ! curl -skf -o /dev/null -H "Authorization: Bearer test-admin-token" https://localhost:30443/readyz; then + echo "Milo isn't answering on https://localhost:30443; run task e2e:up first" >&2 + exit 1 + fi + KUBECONFIG="{{.ROOT_DIR}}/test/e2e/kubeconfig-milo" \ + INFRA_KUBECONFIG="{{.ROOT_DIR}}/.test-infra/kubeconfig" \ + "{{.TOOL_DIR}}/chainsaw" test --config test/e2e/.chainsaw.yaml \ + --cluster infra="{{.ROOT_DIR}}/.test-infra/kubeconfig" \ + {{if .CLI_ARGS}}{{.CLI_ARGS}}{{else}}test/e2e/{{end}} + + e2e:down: + desc: Delete the kind cluster (only when run explicitly) + cmds: + - task: test-infra:cluster-down + + install-chainsaw: + desc: Install chainsaw into bin/ + status: + - test -x {{.TOOL_DIR}}/chainsaw-{{.CHAINSAW_VERSION}} + cmds: + - mkdir -p {{.TOOL_DIR}} + - GOBIN={{.TOOL_DIR}} go install github.com/kyverno/chainsaw@{{.CHAINSAW_VERSION}} + - mv {{.TOOL_DIR}}/chainsaw {{.TOOL_DIR}}/chainsaw-{{.CHAINSAW_VERSION}} + - ln -sf {{.TOOL_DIR}}/chainsaw-{{.CHAINSAW_VERSION}} {{.TOOL_DIR}}/chainsaw diff --git a/config/overlays/e2e/audit-sink.yaml b/config/overlays/e2e/audit-sink.yaml new file mode 100644 index 0000000..b181224 --- /dev/null +++ b/config/overlays/e2e/audit-sink.yaml @@ -0,0 +1,66 @@ +# Stands in for Milo's audit log processor (vector-audit-log-processor in +# milo-system, where test-infra's Milo sends audit events and which +# test-infra doesn't deploy). It answers 200 and writes each batch of events +# to its log, one JSON line per batch, so the e2e tests can check what Milo +# audits. Applied by hack/e2e/deploy.sh, outside this kustomization because +# it lives in milo-system. +apiVersion: v1 +kind: ConfigMap +metadata: + name: vector-audit-log-processor + namespace: milo-system +data: + handle.sh: | + #!/bin/sh + # One HTTP request on stdin: log the body to stderr, answer 200. + len=0 + while IFS= read -r line; do + line=$(printf '%s' "$line" | tr -d '\r') + [ -z "$line" ] && break + case $line in [Cc]ontent-[Ll]ength:*) len=${line#*: } ;; esac + done + if [ "$len" -gt 0 ]; then head -c "$len" | tr -d '\n' >&2; echo >&2; fi + printf 'HTTP/1.1 200 OK\r\nContent-Length: 0\r\nConnection: close\r\n\r\n' +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: vector-audit-log-processor + namespace: milo-system +spec: + replicas: 1 + selector: + matchLabels: + app: vector-audit-log-processor + template: + metadata: + labels: + app: vector-audit-log-processor + spec: + securityContext: + runAsNonRoot: true + runAsUser: 65532 + containers: + - name: sink + image: curlimages/curl:8.11.1 + command: [nc, -lk, -p, "8081", -e, /sink/handle.sh] + ports: + - {name: http, containerPort: 8081} + volumeMounts: + - {name: sink, mountPath: /sink} + volumes: + - name: sink + configMap: + name: vector-audit-log-processor + defaultMode: 0555 +--- +apiVersion: v1 +kind: Service +metadata: + name: vector-audit-log-processor + namespace: milo-system +spec: + selector: + app: vector-audit-log-processor + ports: + - {name: http, port: 8081, targetPort: http} diff --git a/config/overlays/e2e/certificates.yaml b/config/overlays/e2e/certificates.yaml new file mode 100644 index 0000000..737dde9 --- /dev/null +++ b/config/overlays/e2e/certificates.yaml @@ -0,0 +1,51 @@ +# A self-signed CA for the operator's webhook certificate. Milo can't use +# cert-manager's CA injector, so hack/e2e/deploy.sh copies the CA into the +# webhook configurations it registers in Milo. +apiVersion: cert-manager.io/v1 +kind: Issuer +metadata: + name: feedback-e2e-selfsigned +spec: + selfSigned: {} +--- +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: feedback-e2e-ca +spec: + isCA: true + commonName: feedback e2e webhook CA + secretName: feedback-e2e-ca + duration: 87600h + privateKey: + algorithm: ECDSA + size: 256 + issuerRef: + name: feedback-e2e-selfsigned + kind: Issuer +--- +apiVersion: cert-manager.io/v1 +kind: Issuer +metadata: + name: feedback-e2e-ca +spec: + ca: + secretName: feedback-e2e-ca +--- +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: feedback-operator-webhook +spec: + secretName: feedback-operator-webhook-cert + commonName: feedback-operator-webhook + dnsNames: + - feedback-operator-webhook.feedback-system.svc + - feedback-operator-webhook.feedback-system.svc.cluster.local + duration: 8760h + privateKey: + algorithm: ECDSA + size: 256 + issuerRef: + name: feedback-e2e-ca + kind: Issuer diff --git a/config/overlays/e2e/kustomization.yaml b/config/overlays/e2e/kustomization.yaml new file mode 100644 index 0000000..ed7e39a --- /dev/null +++ b/config/overlays/e2e/kustomization.yaml @@ -0,0 +1,115 @@ +# e2e runs feedback in the test-infra kind cluster against the Milo that +# Milo's own `task dev:deploy` puts there (`task e2e:up`). It differs from +# core-control-plane only where the test environment has to: +# +# - Identity. The operator and feedback-uploads authenticate to Milo with +# static tokens for their production usernames +# (system:feedback-operator@feedback.miloapis.com and +# system:feedback-uploads@feedback.miloapis.com), which +# hack/e2e/milo-identities.sh adds to Milo's token file, instead of client +# certificates from the datum-control-plane issuer. Milo's RBAC for them is +# config/milo/rbac, unchanged. +# - Trust. They verify Milo with its test CA (the milo-ca ConfigMap that +# hack/e2e/deploy.sh copies from Milo), and the webhook certificate comes +# from a self-signed CA here, whose bundle deploy.sh writes into the webhook +# configurations it registers in Milo. +# - Attachments live in MinIO in this namespace, through the s3-attachments +# component. +# - One replica each, and a 30s submission window so the rate limit test +# doesn't wait 10 minutes. The limit itself is the default. +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + +namespace: feedback-system + +resources: + - namespace.yaml + - ../../base + - certificates.yaml + - milo-kubeconfig.yaml + - minio.yaml + +components: + - ../../components/s3-attachments + +configMapGenerator: + - name: feedback-config + behavior: merge + literals: + - attachments-bucket=s3://feedback-attachments?endpoint=http://minio.feedback-system.svc.cluster.local:9000®ion=us-east-1&use_path_style=true + +images: + - name: ghcr.io/milo-os/feedback + newTag: e2e + +patches: + - target: + kind: Deployment + name: feedback-operator + patch: |- + - op: replace + path: /spec/replicas + value: 1 + - op: add + path: /spec/template/spec/containers/0/args/- + value: --submission-window=30s + - op: add + path: /spec/template/spec/containers/0/args/- + value: --zap-log-level=debug + # The webhook certificate must be there; base leaves it optional. + - op: replace + path: /spec/template/spec/volumes/0/secret/optional + value: false + - target: + kind: Deployment + name: feedback-uploads + patch: |- + - op: replace + path: /spec/replicas + value: 1 + - target: + kind: Deployment + name: feedback-operator + patch: |- + apiVersion: apps/v1 + kind: Deployment + metadata: + name: feedback-operator + spec: + template: + spec: + containers: + - name: operator + volumeMounts: + - {name: milo-kubeconfig, mountPath: /etc/kubernetes/milo, readOnly: true} + - {name: milo-ca, mountPath: /etc/kubernetes/pki/trust, readOnly: true} + volumes: + - name: milo-kubeconfig + secret: + secretName: feedback-operator-milo-kubeconfig + - name: milo-ca + configMap: + name: milo-ca + - target: + kind: Deployment + name: feedback-uploads + patch: |- + apiVersion: apps/v1 + kind: Deployment + metadata: + name: feedback-uploads + spec: + template: + spec: + containers: + - name: uploads + volumeMounts: + - {name: milo-kubeconfig, mountPath: /etc/kubernetes/milo, readOnly: true} + - {name: milo-ca, mountPath: /etc/kubernetes/pki/trust, readOnly: true} + volumes: + - name: milo-kubeconfig + secret: + secretName: feedback-uploads-milo-kubeconfig + - name: milo-ca + configMap: + name: milo-ca diff --git a/config/overlays/e2e/milo-kubeconfig.yaml b/config/overlays/e2e/milo-kubeconfig.yaml new file mode 100644 index 0000000..9fdbff1 --- /dev/null +++ b/config/overlays/e2e/milo-kubeconfig.yaml @@ -0,0 +1,47 @@ +# Kubeconfigs for the operator and feedback-uploads to reach the Milo in this +# cluster. The tokens are test values that hack/e2e/milo-identities.sh adds to +# Milo's token file for the services' production usernames. milo-ca is copied +# from Milo by hack/e2e/deploy.sh. +apiVersion: v1 +kind: Secret +metadata: + name: feedback-operator-milo-kubeconfig +stringData: + kubeconfig: | + apiVersion: v1 + kind: Config + clusters: + - name: milo + cluster: + server: https://milo-apiserver.milo-system.svc.cluster.local:6443 + certificate-authority: /etc/kubernetes/pki/trust/ca.crt + users: + - name: feedback-operator + user: + token: e2e-feedback-operator-token + contexts: + - name: milo + context: {cluster: milo, user: feedback-operator} + current-context: milo +--- +apiVersion: v1 +kind: Secret +metadata: + name: feedback-uploads-milo-kubeconfig +stringData: + kubeconfig: | + apiVersion: v1 + kind: Config + clusters: + - name: milo + cluster: + server: https://milo-apiserver.milo-system.svc.cluster.local:6443 + certificate-authority: /etc/kubernetes/pki/trust/ca.crt + users: + - name: feedback-uploads + user: + token: e2e-feedback-uploads-token + contexts: + - name: milo + context: {cluster: milo, user: feedback-uploads} + current-context: milo diff --git a/config/overlays/e2e/milo/kustomization.yaml b/config/overlays/e2e/milo/kustomization.yaml new file mode 100644 index 0000000..12742e4 --- /dev/null +++ b/config/overlays/e2e/milo/kustomization.yaml @@ -0,0 +1,26 @@ +# What hack/e2e/deploy.sh applies to Milo besides the CRDs (config/crd): the +# Kubernetes RBAC for feedback's components and the admission webhooks, as in +# production, plus the e2e users and their roles. +# +# test-infra's Milo authorizes with RBAC, not Milo IAM, so config/milo/iam +# (ProtectedResources and Roles) and config/milo/bindings don't grant +# anything there. Instead hack/e2e/iam-rbac.sh turns each IAM role in +# config/milo/iam/roles into a ClusterRole named feedback-e2e: with the +# same permissions, and users.yaml binds them: +# +# feedback.miloapis.com-user every signed-in user (production binds it +# in each user's own context only) +# feedback.miloapis.com-reviewer staff-user +# feedback.miloapis.com-security-team security-user +# +# RBAC can't tell a user's own context from another's or from the platform +# context, so what Milo IAM scopes by context isn't exercised here. +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + +resources: + - users.yaml + +components: + - ../../../milo/rbac + - ../../../milo/webhook diff --git a/config/overlays/e2e/milo/tokens.csv b/config/overlays/e2e/milo/tokens.csv new file mode 100644 index 0000000..e71e9ed --- /dev/null +++ b/config/overlays/e2e/milo/tokens.csv @@ -0,0 +1,12 @@ +# Static tokens the e2e harness adds to Milo's token file (token,user,uid,"groups"). +# Test values for the local kind cluster only. +# +# feedback's components, with their production usernames. +e2e-feedback-operator-token,system:feedback-operator@feedback.miloapis.com,feedback-operator +e2e-feedback-uploads-token,system:feedback-uploads@feedback.miloapis.com,feedback-uploads +e2e-feedback-sync-token,system:feedback-sync@feedback.miloapis.com,feedback-sync +# Users besides Milo's own test-user (test-user-token). +e2e-other-user-token,other-user,other-user,"system:authenticated" +e2e-limit-user-token,limit-user,limit-user,"system:authenticated" +e2e-staff-token,staff-user,staff-user,"system:authenticated" +e2e-security-token,security-user,security-user,"system:authenticated" diff --git a/config/overlays/e2e/milo/users.yaml b/config/overlays/e2e/milo/users.yaml new file mode 100644 index 0000000..c3a436f --- /dev/null +++ b/config/overlays/e2e/milo/users.yaml @@ -0,0 +1,76 @@ +# The e2e users (tokens in tokens.csv) and what they may do with feedback. +# The ClusterRoles come from config/milo/iam/roles; see kustomization.yaml. +apiVersion: iam.miloapis.com/v1alpha1 +kind: User +metadata: + name: other-user +spec: + email: other-user@test.local + givenName: Other + familyName: User +--- +apiVersion: iam.miloapis.com/v1alpha1 +kind: User +metadata: + name: limit-user +spec: + email: limit-user@test.local + givenName: Limit + familyName: User +--- +apiVersion: iam.miloapis.com/v1alpha1 +kind: User +metadata: + name: staff-user +spec: + email: staff-user@test.local + givenName: Staff + familyName: User +--- +apiVersion: iam.miloapis.com/v1alpha1 +kind: User +metadata: + name: security-user +spec: + email: security-user@test.local + givenName: Security + familyName: User +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: feedback-e2e:user +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: feedback-e2e:feedback.miloapis.com-user +subjects: + - apiGroup: rbac.authorization.k8s.io + kind: Group + name: system:authenticated +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: feedback-e2e:reviewer +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: feedback-e2e:feedback.miloapis.com-reviewer +subjects: + - apiGroup: rbac.authorization.k8s.io + kind: User + name: staff-user +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: feedback-e2e:security-team +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: feedback-e2e:feedback.miloapis.com-security-team +subjects: + - apiGroup: rbac.authorization.k8s.io + kind: User + name: security-user diff --git a/config/overlays/e2e/minio.yaml b/config/overlays/e2e/minio.yaml new file mode 100644 index 0000000..8fcbd9a --- /dev/null +++ b/config/overlays/e2e/minio.yaml @@ -0,0 +1,83 @@ +# MinIO as the S3-compatible attachments bucket, with the +# feedback-attachments bucket made by the minio-bucket Job (hack/e2e/deploy.sh +# runs it). The credentials are test values; the s3-attachments component +# gives them to the operator and feedback-uploads. MinIO no longer publishes +# images on Docker Hub or Quay, so these are Chainguard's builds, pinned by +# digest. +apiVersion: v1 +kind: Secret +metadata: + name: feedback-attachments-s3 +stringData: + AWS_ACCESS_KEY_ID: feedback-e2e + AWS_SECRET_ACCESS_KEY: feedback-e2e-secret +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: minio +spec: + replicas: 1 + strategy: + type: Recreate + selector: + matchLabels: + app: minio + template: + metadata: + labels: + app: minio + spec: + securityContext: + runAsNonRoot: true + runAsUser: 65532 + fsGroup: 65532 + containers: + - name: minio + image: chainguard/minio@sha256:68aab277f9748c7c3eb85f178744adba1655c8e33a86987b22d2879ca507c332 + args: [server, /data] + env: + - name: MINIO_ROOT_USER + valueFrom: + secretKeyRef: {name: feedback-attachments-s3, key: AWS_ACCESS_KEY_ID} + - name: MINIO_ROOT_PASSWORD + valueFrom: + secretKeyRef: {name: feedback-attachments-s3, key: AWS_SECRET_ACCESS_KEY} + ports: + - {name: s3, containerPort: 9000} + readinessProbe: + httpGet: {path: /minio/health/ready, port: s3} + volumeMounts: + - {name: data, mountPath: /data} + volumes: + - name: data + emptyDir: {} +--- +apiVersion: v1 +kind: Service +metadata: + name: minio +spec: + selector: + app: minio + ports: + - {name: s3, port: 9000, targetPort: s3} +--- +apiVersion: batch/v1 +kind: Job +metadata: + name: minio-bucket +spec: + backoffLimit: 20 + template: + spec: + restartPolicy: OnFailure + containers: + - name: mc + image: chainguard/minio-client@sha256:43a0d3eae573e611d6b07a13255e3c4f4161220232fbcfb9951d8910b9a72331 + args: [mb, --ignore-existing, local/feedback-attachments] + env: + - name: MC_CONFIG_DIR + value: /tmp/mc + - name: MC_HOST_local + value: http://feedback-e2e:feedback-e2e-secret@minio.feedback-system.svc.cluster.local:9000 diff --git a/config/overlays/e2e/namespace.yaml b/config/overlays/e2e/namespace.yaml new file mode 100644 index 0000000..24fa833 --- /dev/null +++ b/config/overlays/e2e/namespace.yaml @@ -0,0 +1,4 @@ +apiVersion: v1 +kind: Namespace +metadata: + name: feedback-system diff --git a/hack/e2e/deploy.sh b/hack/e2e/deploy.sh new file mode 100755 index 0000000..bb60a53 --- /dev/null +++ b/hack/e2e/deploy.sh @@ -0,0 +1,85 @@ +#!/usr/bin/env bash +# Installs feedback in the Milo in the test-infra kind cluster and runs the +# operator, feedback-uploads and MinIO next to it (task e2e:deploy). Safe to +# run again. +# +# 1. In Milo: the CRDs, RBAC for feedback's components, the e2e users and +# their roles (config/overlays/e2e/milo, hack/e2e/iam-rbac.sh). +# 2. In the cluster: config/overlays/e2e, with Milo's CA as the milo-ca +# ConfigMap, and the attachments bucket in MinIO. +# 3. Once the operator is ready, the admission webhooks in Milo, with the +# webhook CA as their caBundle. +set -euo pipefail +cd "$(dirname "$0")/../.." +INFRA_KUBECONFIG="${INFRA_KUBECONFIG:-.test-infra/kubeconfig}" +MILO_KUBECONFIG="${MILO_KUBECONFIG:-test/e2e/kubeconfig-milo}" +OVERLAY="${OVERLAY:-config/overlays/e2e}" +IMAGE="${IMAGE:-ghcr.io/milo-os/feedback:e2e}" +k() { kubectl --kubeconfig "$INFRA_KUBECONFIG" "$@"; } +m() { kubectl --kubeconfig "$MILO_KUBECONFIG" "$@"; } +webhooks='.kind == "MutatingWebhookConfiguration" or .kind == "ValidatingWebhookConfiguration"' + +echo "➤ CRDs in Milo" +m apply --server-side --force-conflicts -k config/crd +for crd in $(kustomize build config/crd | yq -N '.metadata.name'); do + m wait --for=condition=Established "crd/$crd" --timeout=60s +done + +echo "➤ RBAC and e2e users in Milo" +hack/e2e/iam-rbac.sh | m apply -f - +kustomize build "$OVERLAY/milo" | yq "select(($webhooks) | not)" | m apply -f - + +# test-infra's Milo authorizes with RBAC, so the IAM roles and protected +# resources grant nothing there; check Milo accepts them. (config/milo/bindings +# names groups each environment provides, so it isn't checked.) +echo "➤ Milo accepts config/milo/iam (server dry run)" +iam=$(mktemp -d) +cp -R config/milo "$iam/milo" +printf 'apiVersion: kustomize.config.k8s.io/v1beta1\nkind: Kustomization\ncomponents: [milo/iam]\n' > "$iam/kustomization.yaml" +kustomize build "$iam" | m apply --dry-run=server -f - >/dev/null +rm -rf "$iam" + +# The admission policy that names the post in vote and follow delete audit +# events. The ActivityPolicies and search policies need services test-infra +# doesn't run. +if [ -f config/milo/activity/post-ref.yaml ]; then + echo "➤ post-ref admission policy in Milo" + m apply -f config/milo/activity/post-ref.yaml +fi + +echo "➤ Stand-in audit sink for Milo" +k apply -f "$OVERLAY/audit-sink.yaml" +k -n milo-system rollout status deployment/vector-audit-log-processor --timeout=120s + +echo "➤ feedback in the cluster ($IMAGE)" +k apply -f "$OVERLAY/namespace.yaml" +k -n milo-system get secret milo-apiserver-tls -o jsonpath='{.data.ca\.crt}' | base64 -d > "${TMPDIR:-/tmp}/feedback-e2e-milo-ca.crt" +k -n feedback-system create configmap milo-ca --from-file=ca.crt="${TMPDIR:-/tmp}/feedback-e2e-milo-ca.crt" \ + --dry-run=client -o yaml | k apply -f - +# cert-manager's webhook can lag its rollout on a new cluster. +# A Job's template can't change; it is only there to make the bucket once. +k -n feedback-system delete job minio-bucket --ignore-not-found >/dev/null +until k apply -k "$OVERLAY"; do echo " retrying in 5s"; sleep 5; done +k -n feedback-system set image deployment/feedback-operator operator="$IMAGE" >/dev/null +k -n feedback-system set image deployment/feedback-uploads uploads="$IMAGE" >/dev/null + +echo "➤ Attachments bucket in MinIO" +k -n feedback-system rollout status deployment/minio --timeout=180s +k -n feedback-system wait --for=condition=Complete job/minio-bucket --timeout=180s + +echo "➤ Waiting for the operator and feedback-uploads" +k -n feedback-system wait --for=condition=Ready certificate/feedback-operator-webhook --timeout=120s +k -n feedback-system rollout status deployment/feedback-operator --timeout=300s +# Not fatal: the attachments test reports a feedback-uploads that won't run. +if ! k -n feedback-system rollout status deployment/feedback-uploads --timeout=180s; then + echo "⚠️ feedback-uploads isn't ready; its last log lines:" >&2 + k -n feedback-system logs deployment/feedback-uploads --tail=5 >&2 || true +fi + +echo "➤ Admission webhooks in Milo" +CA=$(k -n feedback-system get secret feedback-e2e-ca -o jsonpath='{.data.ca\.crt}') +export CA +kustomize build "$OVERLAY/milo" | + yq "select($webhooks) | .webhooks[].clientConfig.caBundle = strenv(CA)" | m apply -f - + +echo "✅ feedback is deployed. Run the tests with: task e2e:test" diff --git a/hack/e2e/iam-rbac.sh b/hack/e2e/iam-rbac.sh new file mode 100755 index 0000000..63ce987 --- /dev/null +++ b/hack/e2e/iam-rbac.sh @@ -0,0 +1,28 @@ +#!/usr/bin/env bash +# Prints a ClusterRole for each Milo IAM role in config/milo/iam/roles, named +# feedback-e2e:, with the role's permissions and those of the roles it +# inherits. test-infra's Milo authorizes with RBAC, so the e2e tests grant +# these instead of the IAM roles. A permission is +# feedback.miloapis.com/[/].. +set -euo pipefail +cd "$(dirname "$0")/../.." +dir=config/milo/iam/roles + +perms() { # perms : the role's permissions, inherited ones included + local f + f=$(grep -l "^ name: $1\$" "$dir"/*.yaml) + yq '.spec.includedPermissions[]' "$f" + for parent in $(yq '.spec.inheritedRoles[].name' "$f"); do perms "$parent"; done +} + +for f in "$dir"/*.yaml; do + [ "$(yq '.kind' "$f")" = Role ] || continue + name=$(yq '.metadata.name' "$f") + echo "---" + perms "$name" | sort -u | jq -R -s --arg name "$name" ' + split("\n") | map(select(. != "") | capture("^(?[^/]+)/(?.+)\\.(?[a-z]+)$")) + | group_by([.group, .resource]) + | {apiVersion: "rbac.authorization.k8s.io/v1", kind: "ClusterRole", + metadata: {name: ("feedback-e2e:" + $name), labels: {"app.kubernetes.io/part-of": "feedback-e2e"}}, + rules: map({apiGroups: [.[0].group], resources: [.[0].resource], verbs: map(.verb)})}' | yq -P +done diff --git a/hack/e2e/milo-identities.sh b/hack/e2e/milo-identities.sh new file mode 100755 index 0000000..b945a89 --- /dev/null +++ b/hack/e2e/milo-identities.sh @@ -0,0 +1,37 @@ +#!/usr/bin/env bash +# Adds feedback's service identities and the e2e test users to the static +# token file of the Milo in the test-infra cluster, then restarts Milo's API +# server if that changed anything. Run after Milo's `task dev:deploy`, which +# resets the token file. +set -euo pipefail +cd "$(dirname "$0")/../.." +export KUBECONFIG="${INFRA_KUBECONFIG:-.test-infra/kubeconfig}" + +current=$(kubectl -n milo-system get secret milo-apiserver-auth-tokens -o jsonpath='{.data.tokens\.csv}' | base64 -d) +want="$current" +while IFS= read -r line; do + case "$line" in ''|'#'*) continue ;; esac + grep -qxF "$line" <<<"$want" || want+=$'\n'"$line" +done < config/overlays/e2e/milo/tokens.csv + +if [ "$want" = "$current" ]; then + echo "Milo already has feedback's e2e tokens" +else + kubectl -n milo-system patch secret milo-apiserver-auth-tokens --type merge \ + -p "$(jq -n --arg t "$want"$'\n' '{data: {"tokens.csv": ($t | @base64)}}')" + echo "Added feedback's e2e tokens; restarting Milo's API server" + kubectl -n milo-system rollout restart deployment/milo-apiserver +fi +kubectl -n milo-system rollout status deployment/milo-apiserver --timeout=300s + +# The new API server takes a moment to serve once it is Ready. +for _ in $(seq 1 60); do + code=$(curl -sk -o /dev/null -w '%{http_code}' -H "Authorization: Bearer e2e-feedback-operator-token" https://localhost:30443/apis || true) + if [ "$code" = 200 ]; then + echo "Milo accepts the operator's token" + exit 0 + fi + sleep 2 +done +echo "Milo doesn't accept the operator's token (last answer: $code)" >&2 +exit 1 diff --git a/test/e2e/.chainsaw.yaml b/test/e2e/.chainsaw.yaml new file mode 100644 index 0000000..811e69c --- /dev/null +++ b/test/e2e/.chainsaw.yaml @@ -0,0 +1,23 @@ +# Chainsaw configuration for the feedback e2e tests (task e2e:test). The +# default cluster is Milo (KUBECONFIG=test/e2e/kubeconfig-milo); task e2e:test +# registers "infra", the kind cluster the operator, feedback-uploads and MinIO +# run in, with --cluster infra=.test-infra/kubeconfig. +apiVersion: chainsaw.kyverno.io/v1alpha2 +kind: Configuration +metadata: + name: feedback-e2e +spec: + namespace: + # Feedback's resources are cluster-scoped; don't create a namespace per + # test in Milo. + name: default + execution: + parallel: 4 + failFast: false + timeouts: + apply: 30s + assert: 120s + cleanup: 120s + delete: 60s + error: 30s + exec: 240s diff --git a/test/e2e/attachments/chainsaw-test.yaml b/test/e2e/attachments/chainsaw-test.yaml new file mode 100644 index 0000000..bda0250 --- /dev/null +++ b/test/e2e/attachments/chainsaw-test.yaml @@ -0,0 +1,115 @@ +apiVersion: chainsaw.kyverno.io/v1alpha1 +kind: Test +metadata: + name: attachments +spec: + description: | + Attachments through feedback-uploads, stored in MinIO (s3:// with a + custom endpoint and path-style addressing). The uploads Service is called + from a pod in the cluster. + + - test-user uploads a file; it is type-checked and stored under + uploads/submission/test-user/. + - other-user can't attach test-user's upload; test-user can, and the + attachments webhook sets its name and type from the upload. + - The operator copies it under submissions// and adds its + finalizer. + - test-user downloads it in their own context; other-user gets 404 in + theirs, without a token it's 401; staff-user downloads it in the + platform context. + - Deleting the submission deletes its files. + steps: + - name: client + try: + - script: + shell: bash + content: | + . ../lib.sh + if ! infra -n feedback-system rollout status deployment/feedback-uploads --timeout=60s; then + infra -n feedback-system logs deployment/feedback-uploads --tail=5 >&2 || true + fail "feedback-uploads isn't running" + fi + - apply: + cluster: infra + file: client-pod.yaml + - wait: + cluster: infra + apiVersion: v1 + kind: Pod + name: e2e-attachments-client + namespace: feedback-system + timeout: 120s + for: + condition: + name: Ready + value: "true" + cleanup: + - script: + shell: bash + content: | + . ../lib.sh + milo delete submissions.feedback.miloapis.com e2e-att --ignore-not-found --wait=true --timeout=60s || true + milo delete submissionreviews.feedback.miloapis.com e2e-att --ignore-not-found --wait=false || true + milo delete submissions.feedback.miloapis.com e2e-att-stolen --ignore-not-found --wait=false || true + + - name: upload-attach-download + try: + - script: + shell: bash + content: | + . ../lib.sh + # in_cluster : curl from the client pod; + # prints the HTTP status, the body goes to $OUT. + in_cluster() { + local who=$1; shift + local auth=() + [ "$who" = - ] || auth=(-H "Authorization: Bearer $(token "$who")") + infra -n feedback-system exec e2e-attachments-client -- \ + curl -s -o /tmp/out -w '%{http_code}' "${auth[@]}" "$@" + infra -n feedback-system exec e2e-attachments-client -- cat /tmp/out > "$OUT" + } + UP=http://feedback-uploads.feedback-system.svc.cluster.local + + code=$(in_cluster test-user -X POST -H 'Content-Type: text/html' --data-binary '' "$UP/users/test-user/uploads/submissions?name=x.html") + [ "$code" = 400 ] || fail "HTML upload: HTTP $code: $(cat $OUT)" + echo "ok: HTML refused ($code)" + code=$(in_cluster - -X POST -H 'Content-Type: text/plain' --data-binary 'hi' "$UP/users/test-user/uploads/submissions?name=a.txt") + [ "$code" = 401 ] || fail "upload without a token: HTTP $code" + echo "ok: upload without a token refused ($code)" + + code=$(in_cluster test-user -X POST -H 'Content-Type: text/plain' --data-binary 'hello from the feedback e2e tests' "$UP/users/test-user/uploads/submissions?name=notes.txt") + [ "$code" = 201 ] || fail "upload: HTTP $code: $(cat $OUT)" + ref=$(jqo .uploadRef) + echo "ok: uploaded $ref" + minio_ls uploads/submission/test-user/ | grep -q "$ref" || fail "upload $ref isn't in MinIO" + echo "ok: upload is in MinIO" + + expect 403 "other-user can't attach test-user's upload" other-user POST "$(url other-user submissions)" \ + "{\"apiVersion\":\"feedback.miloapis.com/v1alpha1\",\"kind\":\"Submission\",\"metadata\":{\"name\":\"e2e-att-stolen\"},\"spec\":{\"kind\":\"bug\",\"title\":\"stolen\",\"body\":\"x\",\"attachments\":[{\"uploadRef\":\"$ref\"}]}}" + submit test-user e2e-att "\"attachments\":[{\"uploadRef\":\"$ref\",\"name\":\"evil.html\",\"contentType\":\"text/html\"}]" + [ "$(jqo '.spec.attachments[0] | "\(.name) \(.contentType)"')" = "notes.txt text/plain" ] || fail "attachment not set from the upload: $(jqo .spec.attachments)" + + eventually 60 "operator keeps the files" is feedback.miloapis.com/attachments jget submission e2e-att '{.metadata.finalizers[0]}' + minio_ls submissions/e2e-att/ | grep -q "$ref" || fail "attachment isn't under submissions/e2e-att/" + echo "ok: attachment copied under submissions/e2e-att/" + + path="submissions/e2e-att/attachments/$ref" + code=$(in_cluster test-user "$UP/users/test-user/$path") + [ "$code" = 200 ] || fail "owner download: HTTP $code: $(cat $OUT)" + [ "$(cat $OUT)" = "hello from the feedback e2e tests" ] || fail "downloaded: $(cat $OUT)" + echo "ok: test-user downloads it" + code=$(in_cluster other-user "$UP/users/other-user/$path") + [ "$code" = 404 ] || fail "other-user download in their context: HTTP $code: $(cat $OUT)" + echo "ok: other-user gets 404" + code=$(in_cluster - "$UP/users/test-user/$path") + [ "$code" = 401 ] || fail "download without a token: HTTP $code" + echo "ok: no token gets 401" + code=$(in_cluster staff-user "$UP/$path") + [ "$code" = 200 ] || fail "staff download: HTTP $code: $(cat $OUT)" + echo "ok: staff-user downloads it in the platform context" + + milo delete submissions.feedback.miloapis.com e2e-att --wait=true --timeout=60s + echo "ok: submission deleted" + n=$(minio_ls submissions/e2e-att/ | wc -l | tr -d ' ') + [ "$n" = 0 ] || fail "$n files left under submissions/e2e-att/" + echo "ok: its files are gone" diff --git a/test/e2e/attachments/client-pod.yaml b/test/e2e/attachments/client-pod.yaml new file mode 100644 index 0000000..01c5f7a --- /dev/null +++ b/test/e2e/attachments/client-pod.yaml @@ -0,0 +1,23 @@ +# A client in the cluster, to call feedback-uploads through its Service. +apiVersion: v1 +kind: Pod +metadata: + name: e2e-attachments-client + namespace: feedback-system + labels: + app.kubernetes.io/part-of: feedback-e2e +spec: + terminationGracePeriodSeconds: 0 + securityContext: + runAsNonRoot: true + runAsUser: 65532 + seccompProfile: + type: RuntimeDefault + containers: + - name: curl + image: curlimages/curl:8.11.1 + command: [sleep, "3600"] + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: [ALL] diff --git a/test/e2e/hidden-posts/chainsaw-test.yaml b/test/e2e/hidden-posts/chainsaw-test.yaml new file mode 100644 index 0000000..05b6e62 --- /dev/null +++ b/test/e2e/hidden-posts/chainsaw-test.yaml @@ -0,0 +1,65 @@ +apiVersion: chainsaw.kyverno.io/v1alpha1 +kind: Test +metadata: + name: hidden-posts +spec: + description: | + Hidden posts (spec.hidden, iam.miloapis.com/hidden-field): staff-user, a + reviewer, hides a post. test-user can't list, get or vote on it, in their + own context or the platform context. Staff and admin still see it. + steps: + - name: hide + try: + - script: + shell: bash + content: | + . ../lib.sh + post e2e-hidden-shown + post e2e-hidden-secret + expect 200 "staff-user hides a post" staff-user PATCH "$(url platform posts/e2e-hidden-secret)" '{"spec":{"hidden":true}}' + # Only spec.hidden is for staff to change. + code=$(call staff-user PATCH "$(url platform posts/e2e-hidden-shown/status)" '{"status":{"title":"renamed"}}') + [ "$code" = 403 ] || fail "staff-user wrote a post's status: HTTP $code" + echo "ok: staff-user can't write a post's status (403)" + cleanup: + - script: + shell: bash + content: | + . ../lib.sh + cleanup_posts e2e-hidden-shown e2e-hidden-secret + + - name: users-dont-see-it + try: + - script: + shell: bash + content: | + . ../lib.sh + for ctx in test-user platform; do + expect 200 "test-user lists posts ($ctx)" test-user GET "$(url $ctx posts)" + names=$(jqo '[.items[].metadata.name | select(startswith("e2e-hidden-"))] | sort | join(" ")') + [ "$names" = e2e-hidden-shown ] || fail "test-user listed in $ctx: $names" + expect 200 "test-user lists hidden posts by field selector ($ctx)" test-user GET "$(url $ctx 'posts?fieldSelector=spec.hidden%3Dtrue')" + [ "$(jqo '[.items[].metadata.name | select(. == "e2e-hidden-secret")] | length')" = 0 ] || fail "hidden post listed by field selector in $ctx" + expect 404 "test-user gets the hidden post ($ctx)" test-user GET "$(url $ctx posts/e2e-hidden-secret)" + expect 200 "test-user gets the shown post ($ctx)" test-user GET "$(url $ctx posts/e2e-hidden-shown)" + done + expect 403 "test-user votes on the hidden post" test-user POST "$(url test-user votes)" \ + '{"apiVersion":"feedback.miloapis.com/v1alpha1","kind":"Vote","metadata":{"name":"e2e-hidden-secret.test-user"},"spec":{"postRef":{"name":"e2e-hidden-secret"}}}' + grep -q 'not found' "$OUT" || fail "refusal says more than not found: $(cat $OUT)" + expect 403 "test-user follows the hidden post" test-user POST "$(url test-user subscriptions)" \ + '{"apiVersion":"feedback.miloapis.com/v1alpha1","kind":"Subscription","metadata":{"name":"e2e-hidden-secret.test-user"},"spec":{"postRef":{"name":"e2e-hidden-secret"},"reason":"manual"}}' + + - name: staff-see-it + try: + - script: + shell: bash + content: | + . ../lib.sh + expect 200 "staff-user gets the hidden post" staff-user GET "$(url platform posts/e2e-hidden-secret)" + [ "$(jqo .spec.hidden)" = true ] || fail "staff-user got: $(cat $OUT)" + expect 200 "staff-user lists posts" staff-user GET "$(url platform posts)" + [ "$(jqo '[.items[].metadata.name | select(. == "e2e-hidden-secret")] | length')" = 1 ] || fail "staff-user doesn't list the hidden post" + expect 200 "admin gets the hidden post" admin GET "$(url platform posts/e2e-hidden-secret)" + + expect 200 "staff-user shows the post again" staff-user PATCH "$(url platform posts/e2e-hidden-secret)" '{"spec":{"hidden":false}}' + expect 200 "test-user gets it once shown" test-user GET "$(url test-user posts/e2e-hidden-secret)" diff --git a/test/e2e/kubeconfig-milo b/test/e2e/kubeconfig-milo new file mode 100644 index 0000000..4545df4 --- /dev/null +++ b/test/e2e/kubeconfig-milo @@ -0,0 +1,16 @@ +# Milo in the test-infra cluster, as Milo's test admin (system:masters). +apiVersion: v1 +kind: Config +clusters: + - name: milo + cluster: + server: https://localhost:30443 + insecure-skip-tls-verify: true +users: + - name: admin + user: + token: test-admin-token +contexts: + - name: milo + context: {cluster: milo, user: admin} +current-context: milo diff --git a/test/e2e/lib.sh b/test/e2e/lib.sh new file mode 100644 index 0000000..2474a66 --- /dev/null +++ b/test/e2e/lib.sh @@ -0,0 +1,147 @@ +# Helpers for the feedback e2e scripts. Chainsaw runs each script in its +# test's directory: source this with `. ../lib.sh`. +# +# Milo is https://localhost:30443 (Milo's test-infra gateway). Callers: +# test-user Milo's own test user (token test-user-token) +# other-user, limit-user more users (config/overlays/e2e/milo) +# staff-user feedback reviewer; security-user: Security Team +# sync system:feedback-sync@feedback.miloapis.com +# admin system:masters +set -euo pipefail + +E2E_DIR=$(cd "$(dirname "${BASH_SOURCE[0]:-$0}")" 2>/dev/null && pwd || pwd) +[ -f "$E2E_DIR/kubeconfig-milo" ] || E2E_DIR=$(cd .. && pwd) +REPO_DIR=$(cd "$E2E_DIR/../.." && pwd) +MILO=${MILO_URL:-https://localhost:30443} +FB=apis/feedback.miloapis.com/v1alpha1 +INFRA_KUBECONFIG=${INFRA_KUBECONFIG:-$REPO_DIR/.test-infra/kubeconfig} +OUT=$(mktemp) +trap 'rm -f "$OUT"' EXIT + +token() { + case $1 in + test-user) echo test-user-token ;; + other-user) echo e2e-other-user-token ;; + limit-user) echo e2e-limit-user-token ;; + staff-user) echo e2e-staff-token ;; + security-user) echo e2e-security-token ;; + sync) echo e2e-feedback-sync-token ;; + admin) echo test-admin-token ;; + *) echo "unknown caller $1" >&2; return 1 ;; + esac +} + +# url : is a user name for that user's own +# context, or "platform". +url() { + if [ "$1" = platform ]; then + echo "$MILO/$FB/$2" + else + echo "$MILO/apis/iam.miloapis.com/v1alpha1/users/$1/control-plane/$FB/$2" + fi +} + +# call [json body]: prints the HTTP status; the +# response body is in $OUT. +call() { + local who=$1 method=$2 u=$3 body=${4:-} ctype=application/json + case $method in PATCH) ctype=application/merge-patch+json ;; esac + curl -sk -o "$OUT" -w '%{http_code}' -X "$method" \ + -H "Authorization: Bearer $(token "$who")" -H "Content-Type: $ctype" \ + ${body:+--data "$body"} "$u" +} + +# expect +expect() { + local want=$1 what=$2 got + shift 2 + got=$(call "$@") + if [ "$got" != "$want" ]; then + echo "FAIL: $what: got HTTP $got, want $want" >&2 + cat "$OUT" >&2; echo >&2 + exit 1 + fi + echo "ok: $what ($got)" +} + +# jqo : applies filter to the last response. +jqo() { jq -r "$@" "$OUT"; } + +fail() { echo "FAIL: $*" >&2; exit 1; } + +# milo / infra: kubectl as Milo's admin, or against the kind cluster. +milo() { kubectl --kubeconfig "$E2E_DIR/kubeconfig-milo" "$@"; } +infra() { kubectl --kubeconfig "$INFRA_KUBECONFIG" "$@"; } + +# eventually : retries command every second. +eventually() { + local secs=$1 what=$2 i + shift 2 + for ((i = 0; i < secs; i++)); do + if "$@" >/dev/null 2>&1; then echo "ok: $what"; return 0; fi + sleep 1 + done + echo "FAIL: $what (after ${secs}s); last try:" >&2 + "$@" >&2 || true + exit 1 +} + +# jget : a field of a feedback object, read as admin. +jget() { milo get "$1.feedback.miloapis.com" "$2" -o "jsonpath=$3"; } + +# is : true when the command prints want. +is() { local want=$1; shift; [ "$("$@" 2>/dev/null)" = "$want" ]; } + +# gone : true when the object doesn't exist. +gone() { ! milo get "$1.feedback.miloapis.com" "$2" >/dev/null 2>&1; } + +# post [title]: feedback-sync creates a post, as it does from GitHub. +post() { + expect 201 "sync creates post $1" sync POST "$(url platform posts)" \ + "{\"apiVersion\":\"feedback.miloapis.com/v1alpha1\",\"kind\":\"Post\",\"metadata\":{\"name\":\"$1\"},\"spec\":{}}" + expect 200 "sync sets post $1's title" sync PATCH "$(url platform posts/$1/status)" \ + "{\"status\":{\"title\":\"${2:-$1}\",\"state\":\"backlog\"}}" +} + +# vote : the user votes in their own context. +vote() { + expect 201 "$1 votes on $2" "$1" POST "$(url "$1" votes)" \ + "{\"apiVersion\":\"feedback.miloapis.com/v1alpha1\",\"kind\":\"Vote\",\"metadata\":{\"name\":\"$2.$1\"},\"spec\":{\"postRef\":{\"name\":\"$2\"}}}" +} + +# cleanup_posts : deletes the posts and every vote and follow on +# them, as admin. +cleanup_posts() { + local p + for p in "$@"; do + milo delete votes.feedback.miloapis.com --field-selector "spec.postRef.name=$p" --ignore-not-found --wait=false || true + milo delete subscriptions.feedback.miloapis.com --field-selector "spec.postRef.name=$p" --ignore-not-found --wait=false || true + milo delete posts.feedback.miloapis.com "$p" --ignore-not-found --wait=false || true + done +} + +# submit [extra spec json]: the user sends feedback. +submit() { + local extra=${3:-} + expect 201 "$1 sends submission $2" "$1" POST "$(url "$1" submissions)" \ + "{\"apiVersion\":\"feedback.miloapis.com/v1alpha1\",\"kind\":\"Submission\",\"metadata\":{\"name\":\"$2\"},\"spec\":{\"kind\":\"idea\",\"title\":\"e2e $2\",\"body\":\"Sent by the feedback e2e tests.\"${extra:+,$extra}}}" +} + +# minio_ls : lists the attachments bucket under prefix, with the +# MinIO client in a pod in the cluster. +minio_ls() { + local pod="e2e-mc-$RANDOM$RANDOM" phase i + infra -n feedback-system run "$pod" --restart=Never --quiet \ + --image=chainguard/minio-client@sha256:43a0d3eae573e611d6b07a13255e3c4f4161220232fbcfb9951d8910b9a72331 \ + --env=MC_CONFIG_DIR=/tmp/mc \ + --env=MC_HOST_local=http://feedback-e2e:feedback-e2e-secret@minio.feedback-system.svc.cluster.local:9000 \ + -- ls --recursive "local/feedback-attachments/$1" >/dev/null + for ((i = 0; i < 60; i++)); do + phase=$(infra -n feedback-system get pod "$pod" -o jsonpath='{.status.phase}') + case $phase in Succeeded|Failed) break ;; esac + sleep 1 + done + infra -n feedback-system logs "$pod" + infra -n feedback-system delete pod "$pod" --wait=false >/dev/null + [ "$phase" = Succeeded ] || { echo "mc ls failed ($phase)" >&2; return 1; } +} diff --git a/test/e2e/merge-chain/chainsaw-test.yaml b/test/e2e/merge-chain/chainsaw-test.yaml new file mode 100644 index 0000000..fd7db3c --- /dev/null +++ b/test/e2e/merge-chain/chainsaw-test.yaml @@ -0,0 +1,83 @@ +apiVersion: chainsaw.kyverno.io/v1alpha1 +kind: Test +metadata: + name: merge-chain +spec: + description: | + Merged posts (status.mergedInto, written by feedback-sync). + + - Chain: post B is already merged into C, then A is merged into B. Votes + and follows on A end up on C, never on B (which takes no votes). A + voter who already voted on C keeps that one vote. + - Hidden target: D is merged into E, which is hidden. The vote stays on + D, D says why (VotesMoved=False, TargetUnavailable) and the operator + doesn't keep rewriting D. Showing E moves the vote. + steps: + - name: chain + try: + - script: + shell: bash + content: | + . ../lib.sh + for p in a b c; do post e2e-merge-$p; done + vote test-user e2e-merge-a + vote other-user e2e-merge-a + vote test-user e2e-merge-c + eventually 60 "A counts 2 votes" is 2 jget post e2e-merge-a '{.status.votes.total}' + + expect 200 "sync merges B into C" sync PATCH "$(url platform posts/e2e-merge-b/status)" '{"status":{"mergedInto":"e2e-merge-c"}}' + expect 403 "test-user can't vote on merged B" test-user POST "$(url test-user votes)" \ + '{"apiVersion":"feedback.miloapis.com/v1alpha1","kind":"Vote","metadata":{"name":"e2e-merge-b.test-user"},"spec":{"postRef":{"name":"e2e-merge-b"}}}' + expect 200 "sync merges A into B" sync PATCH "$(url platform posts/e2e-merge-a/status)" '{"status":{"mergedInto":"e2e-merge-b"}}' + + eventually 60 "A's votes are gone" is 0 sh -c "kubectl --kubeconfig $E2E_DIR/kubeconfig-milo get votes.feedback.miloapis.com --field-selector spec.postRef.name=e2e-merge-a -o json | jq '.items | length'" + eventually 60 "other-user's vote is on C" jget vote e2e-merge-c.other-user '{.metadata.name}' + jget vote e2e-merge-c.test-user '{.metadata.name}' >/dev/null || fail "test-user's vote on C is gone" + n=$(milo get votes.feedback.miloapis.com --field-selector spec.postRef.name=e2e-merge-b -o json | jq '.items | length') + [ "$n" = 0 ] || fail "$n votes landed on B, which is merged" + eventually 60 "other-user follows C" jget subscription e2e-merge-c.other-user '{.metadata.name}' + n=$(milo get subscriptions.feedback.miloapis.com --field-selector spec.postRef.name=e2e-merge-a -o json | jq '.items | length') + [ "$n" = 0 ] || fail "$n follows left on A" + + eventually 60 "A says its votes moved" is "True Moved" jget post e2e-merge-a '{range .status.conditions[?(@.type=="VotesMoved")]}{.status} {.reason}{end}' + jget post e2e-merge-a '{.status.conditions[?(@.type=="VotesMoved")].message}' | grep -q e2e-merge-c || fail "A's condition doesn't name C" + eventually 60 "C counts 2 votes" is 2 jget post e2e-merge-c '{.status.votes.total}' + eventually 60 "A counts no votes" is 0 sh -c "kubectl --kubeconfig $E2E_DIR/kubeconfig-milo get posts.feedback.miloapis.com e2e-merge-a -o json | jq '.status.votes.total // 0'" + cleanup: + - script: + shell: bash + content: | + . ../lib.sh + cleanup_posts e2e-merge-a e2e-merge-b e2e-merge-c e2e-merge-d e2e-merge-e + + - name: hidden-target + try: + - script: + shell: bash + content: | + . ../lib.sh + post e2e-merge-d + post e2e-merge-e + vote test-user e2e-merge-d + expect 200 "staff-user hides E" staff-user PATCH "$(url platform posts/e2e-merge-e)" '{"spec":{"hidden":true}}' + expect 200 "sync merges D into hidden E" sync PATCH "$(url platform posts/e2e-merge-d/status)" '{"status":{"mergedInto":"e2e-merge-e"}}' + + eventually 60 "D says its votes can't move" is "False TargetUnavailable" jget post e2e-merge-d '{range .status.conditions[?(@.type=="VotesMoved")]}{.status} {.reason}{end}' + jget vote e2e-merge-d.test-user '{.metadata.name}' >/dev/null || fail "test-user's vote on D was deleted" + gone vote e2e-merge-e.test-user || fail "a vote was created on hidden E" + jget post e2e-merge-d '{.status.conditions[?(@.type=="VotesMoved")].message}' | grep -q hidden && fail "the condition says the target is hidden" + + # No hot loop: once the condition is set, D isn't written again. + rv=$(jget post e2e-merge-d '{.metadata.resourceVersion}') + sleep 20 + rv2=$(jget post e2e-merge-d '{.metadata.resourceVersion}') + [ "$rv" = "$rv2" ] || fail "post D keeps changing (resourceVersion $rv -> $rv2): $(milo get posts.feedback.miloapis.com e2e-merge-d -o yaml)" + echo "ok: D unchanged for 20s" + n=$(infra -n feedback-system logs deployment/feedback-operator --since=20s | grep -c '"controller":"merge"' || true) + echo "merge reconciler log lines in the last 20s: $n" + [ "$n" -le 5 ] || fail "the merge reconciler logged $n lines in 20s with nothing to do" + + expect 200 "staff-user shows E" staff-user PATCH "$(url platform posts/e2e-merge-e)" '{"spec":{"hidden":false}}' + eventually 60 "test-user's vote moves to E" jget vote e2e-merge-e.test-user '{.metadata.name}' + eventually 60 "test-user's vote on D is gone" gone vote e2e-merge-d.test-user + eventually 60 "D says its votes moved" is "True Moved" jget post e2e-merge-d '{range .status.conditions[?(@.type=="VotesMoved")]}{.status} {.reason}{end}' diff --git a/test/e2e/operator-readiness/chainsaw-test.yaml b/test/e2e/operator-readiness/chainsaw-test.yaml new file mode 100644 index 0000000..e827455 --- /dev/null +++ b/test/e2e/operator-readiness/chainsaw-test.yaml @@ -0,0 +1,62 @@ +apiVersion: chainsaw.kyverno.io/v1alpha1 +kind: Test +metadata: + name: operator-readiness +spec: + description: | + The operator reports ready only once its webhook server serves TLS, so + Milo's fail-closed webhooks never reach a pod that can't answer. + + - The running operator's /readyz includes the webhook check. + - A fresh operator pod (the Deployment's pod template, with a label the + webhook Service doesn't select, and standing by for leadership) runs + next to a sampler that polls /readyz and the webhook port from the + moment the pod starts. /readyz must never answer 200 while the webhook + port doesn't complete a TLS handshake, and the pod must become Ready. + steps: + - name: running-operator + try: + - script: + shell: bash + content: | + . ../lib.sh + ip=$(infra -n feedback-system get pod -l app=feedback-operator -o jsonpath='{.items[0].status.podIP}') + out=$(infra -n feedback-system run e2e-readyz-probe --rm -i --restart=Never --quiet \ + --image=curlimages/curl:8.11.1 -- curl -s "http://$ip:8081/readyz?verbose") + echo "$out" + grep -q '\[+\]webhook ok' <<<"$out" || fail "/readyz has no passing webhook check" + + - name: fresh-pod + try: + - script: + shell: bash + timeout: 4m + content: | + . ../lib.sh + infra -n feedback-system get deployment feedback-operator -o json | jq ' + .spec.template as $t | { + apiVersion: "v1", kind: "Pod", + metadata: {name: "e2e-operator-readiness", namespace: "feedback-system", + labels: {app: "feedback-operator-e2e-readiness"}}, + spec: ($t.spec + {restartPolicy: "Never", containers: ([{ + name: "sampler", image: "curlimages/curl:8.11.1", + securityContext: {allowPrivilegeEscalation: false, capabilities: {drop: ["ALL"]}}, + command: ["sh", "-c", "i=0; while [ $i -lt 1500 ]; do r=$(curl -s -o /dev/null -m 1 -w %{http_code} http://localhost:8081/readyz); t=$(curl -sk -o /dev/null -m 1 -w %{http_code} https://localhost:9443/); echo \"sample readyz=$r tls=$t\"; i=$((i+1)); sleep 0.05; done; sleep 3600"] + }] + $t.spec.containers)}) + }' | infra apply -f - + infra -n feedback-system wait --for=condition=Ready pod/e2e-operator-readiness --timeout=180s + echo "ok: the fresh pod became Ready" + logs=$(infra -n feedback-system logs e2e-operator-readiness -c sampler) + total=$(grep -c '^sample' <<<"$logs" || true) + bad=$(grep -c 'readyz=200 tls=000' <<<"$logs" || true) + notready=$(grep -Ec 'readyz=(500|503) tls=000' <<<"$logs" || true) + ready=$(grep -c 'readyz=200 tls=404' <<<"$logs" || true) + echo "samples: $total; not ready while TLS refused: $notready; ready while TLS refused: $bad; ready and serving: $ready" + [ "$bad" = 0 ] || fail "/readyz answered 200 while the webhook port refused TLS" + [ "$ready" -gt 0 ] || fail "never saw /readyz 200 with the webhook serving" + cleanup: + - script: + shell: bash + content: | + . ../lib.sh + infra -n feedback-system delete pod e2e-operator-readiness --ignore-not-found --wait=false diff --git a/test/e2e/rate-limit/chainsaw-test.yaml b/test/e2e/rate-limit/chainsaw-test.yaml new file mode 100644 index 0000000..a4dafff --- /dev/null +++ b/test/e2e/rate-limit/chainsaw-test.yaml @@ -0,0 +1,54 @@ +apiVersion: chainsaw.kyverno.io/v1alpha1 +kind: Test +metadata: + name: rate-limit +spec: + description: | + The submission limit: limit-user (its own user, so other tests' feedback + doesn't count) gets 5 submissions (the operator's default + --submission-limit) and then 429 Too Many Requests. The e2e overlay sets + --submission-window=30s; a run within 30s of the last one waits for the + window to pass first. + steps: + - name: limit + try: + - script: + shell: bash + timeout: 3m + content: | + . ../lib.sh + limit=5 + send() { + call limit-user POST "$(url limit-user submissions)" \ + '{"apiVersion":"feedback.miloapis.com/v1alpha1","kind":"Submission","metadata":{"generateName":"e2e-limit-"},"spec":{"kind":"bug","title":"e2e rate limit","body":"Sent by the feedback e2e tests."}}' + } + code=$(send) + if [ "$code" = 429 ]; then + echo "limit-user is still limited from an earlier run; waiting out the window" + sleep 31 + code=$(send) + fi + admitted=0 + while [ "$code" = 201 ]; do + admitted=$((admitted + 1)) + [ $admitted -le $limit ] || break + code=$(send) + done + [ "$code" = 429 ] || fail "after $admitted submissions: HTTP $code: $(cat $OUT)" + if [ $admitted != $limit ]; then + echo "429 response: $(cat $OUT)" >&2 + echo "limit-user's submissions (created at):" >&2 + milo get submissions.feedback.miloapis.com --field-selector spec.userRef.name=limit-user \ + -o custom-columns=NAME:.metadata.name,CREATED:.metadata.creationTimestamp >&2 + fail "admitted $admitted submissions before 429, want $limit" + fi + [ "$(jqo .reason)" = TooManyRequests ] || fail "429 reason: $(cat $OUT)" + echo "ok: $admitted admitted, then 429: $(jqo .message)" + n=$(milo get submissions.feedback.miloapis.com --field-selector spec.userRef.name=limit-user -o json | jq '.items | length') + [ "$n" = $limit ] || fail "limit-user has $n submissions, want $limit" + cleanup: + - script: + shell: bash + content: | + . ../lib.sh + milo delete submissions.feedback.miloapis.com --field-selector spec.userRef.name=limit-user --ignore-not-found --wait=false diff --git a/test/e2e/security/chainsaw-test.yaml b/test/e2e/security/chainsaw-test.yaml new file mode 100644 index 0000000..5b969c3 --- /dev/null +++ b/test/e2e/security/chainsaw-test.yaml @@ -0,0 +1,76 @@ +apiVersion: chainsaw.kyverno.io/v1alpha1 +kind: Test +metadata: + name: security +spec: + description: | + Moving a submission to the security track (MoveToSecurity), with the + submitter other-user (test-user's feedback is left to the other tests so + the 30s submission limit isn't reached). + + - A report the move didn't create already has the moved report's name + (-), here one the Security Team logged for an + external reporter. The move must leave it alone and keep the + submission: the review says Applied=False, ReportNameTaken. + - Once that report is deleted, the move goes ahead: the report is owned + by the submitter, records where it came from, starts in Received with + its triage, and the submission and its review are deleted. The + submitter sees the report in their own context; test-user doesn't. + steps: + - name: name-taken + try: + - script: + shell: bash + content: | + . ../lib.sh + submit other-user e2e-sec + uid=$(jqo .metadata.uid) + target="e2e-sec-$uid" + echo "$target" > "${TMPDIR:-/tmp}/feedback-e2e-security.target" + eventually 60 "review exists" jget submissionreview e2e-sec '{.metadata.name}' + + expect 201 "security-user logs an unrelated report with the moved report's name" security-user POST "$(url platform securityreports)" \ + "{\"apiVersion\":\"feedback.miloapis.com/v1alpha1\",\"kind\":\"SecurityReport\",\"metadata\":{\"name\":\"$target\"},\"spec\":{\"title\":\"Unrelated report\",\"description\":\"Logged from email.\",\"reporter\":{\"email\":\"someone@example.com\"}}}" + squat_uid=$(jqo .metadata.uid) + + expect 200 "staff-user moves the submission to security" staff-user PATCH "$(url platform submissionreviews/e2e-sec)" \ + '{"spec":{"decision":"MoveToSecurity","affected":"connectors"}}' + eventually 60 "review says the name is taken" is "False ReportNameTaken" jget submissionreview e2e-sec '{range .status.conditions[?(@.type=="Applied")]}{.status} {.reason}{end}' + sleep 10 + jget submission e2e-sec '{.metadata.name}' >/dev/null || fail "the submission was deleted although its report wasn't created" + echo "ok: submission kept" + is "$squat_uid" jget securityreport "$target" '{.metadata.uid}' || fail "the existing report was replaced" + [ -z "$(jget securityreport "$target" '{.spec.userRef.name}')" ] || fail "the existing report got an owner" + [ -z "$(jget securityreport "$target" '{.metadata.annotations.feedback\.miloapis\.com/moved-from-submission}')" ] || fail "the existing report was marked as moved" + echo "ok: existing report untouched" + cleanup: + - script: + shell: bash + content: | + . ../lib.sh + [ -f "${TMPDIR:-/tmp}/feedback-e2e-security.target" ] && milo delete securityreports.feedback.miloapis.com "$(cat "${TMPDIR:-/tmp}/feedback-e2e-security.target")" --ignore-not-found --wait=false || true + milo delete submissions.feedback.miloapis.com e2e-sec --ignore-not-found --wait=false || true + milo delete submissionreviews.feedback.miloapis.com e2e-sec --ignore-not-found --wait=false || true + rm -f "${TMPDIR:-/tmp}/feedback-e2e-security.target" + + - name: move + try: + - script: + shell: bash + content: | + . ../lib.sh + target=$(cat "${TMPDIR:-/tmp}/feedback-e2e-security.target") + expect 200 "security-user deletes the unrelated report" security-user DELETE "$(url platform securityreports/$target)" + eventually 90 "the moved report exists, owned by other-user" is other-user jget securityreport "$target" '{.spec.userRef.name}' + is e2e-sec jget securityreport "$target" '{.metadata.annotations.feedback\.miloapis\.com/moved-from-submission}' || fail "report doesn't record its submission" + is connectors jget securityreport "$target" '{.spec.affected}' || fail "affected not copied" + eventually 60 "submission is deleted" gone submission e2e-sec + eventually 90 "its review is deleted" gone submissionreview e2e-sec + eventually 60 "report is Received" is Received jget securityreport "$target" '{.status.phase}' + is Received jget securityreport "$target" '{.status.timeline[0].phase}' || fail "no Received timeline entry" + eventually 60 "report has its triage" jget securityreporttriage "$target" '{.metadata.name}' + + expect 200 "other-user sees their report" other-user GET "$(url other-user securityreports/$target)" + expect 404 "test-user doesn't" test-user GET "$(url test-user securityreports/$target)" + expect 200 "other-user lists reports" other-user GET "$(url other-user securityreports)" + [ "$(jqo "[.items[].metadata.name] | index(\"$target\") != null")" = true ] || fail "other-user's list misses the report" diff --git a/test/e2e/submissions/chainsaw-test.yaml b/test/e2e/submissions/chainsaw-test.yaml new file mode 100644 index 0000000..f37a059 --- /dev/null +++ b/test/e2e/submissions/chainsaw-test.yaml @@ -0,0 +1,88 @@ +apiVersion: chainsaw.kyverno.io/v1alpha1 +kind: Test +metadata: + name: submissions +spec: + description: | + Feedback submissions and staff decisions. + + - test-user sends feedback in their own context; the operator creates + its SubmissionReview (staff-only) and starts it in PendingReview. + - staff-user rejects one: the reviewer webhook records who decided, the + reason reaches the submitter. + - staff-user approves one: it waits for feedback-sync's post, then links + it and has the submitter vote on and follow it. + steps: + - name: send + try: + - script: + shell: bash + content: | + . ../lib.sh + for s in e2e-sub-reject e2e-sub-approve; do + submit test-user $s + [ "$(jqo .spec.userRef.name)" = test-user ] || fail "webhook didn't set the owner: $(cat $OUT)" + done + for s in e2e-sub-reject e2e-sub-approve; do + eventually 60 "$s has its review" jget submissionreview $s '{.metadata.name}' + eventually 60 "$s is PendingReview" is PendingReview jget submission $s '{.status.phase}' + owner=$(jget submissionreview $s '{.metadata.ownerReferences[0].name}') + [ "$owner" = $s ] || fail "review $s is owned by '$owner'" + done + expect 200 "test-user reads their submission" test-user GET "$(url test-user submissions/e2e-sub-reject)" + [ "$(jqo .status.phase)" = PendingReview ] || fail "test-user sees phase $(jqo .status.phase)" + code=$(call test-user GET "$(url test-user submissionreviews/e2e-sub-reject)") + case $code in 403|404) echo "ok: test-user can't read the review ($code)";; *) fail "test-user read the review: HTTP $code";; esac + expect 404 "other-user can't read test-user's submission" other-user GET "$(url other-user submissions/e2e-sub-reject)" + # The user role has no update (403); the CRD would refuse it too (422). + code=$(call test-user PATCH "$(url test-user submissions/e2e-sub-reject)" '{"spec":{"title":"changed"}}') + case $code in 403|422) echo "ok: test-user can't change a sent submission ($code)";; *) fail "changing a sent submission: HTTP $code";; esac + cleanup: + - script: + shell: bash + content: | + . ../lib.sh + for s in e2e-sub-reject e2e-sub-approve; do + milo delete submissions.feedback.miloapis.com $s --ignore-not-found --wait=false || true + milo delete submissionreviews.feedback.miloapis.com $s --ignore-not-found --wait=false || true + done + cleanup_posts e2e-sub-approved-post + + - name: reject + try: + - script: + shell: bash + content: | + . ../lib.sh + expect 422 "Rejected needs a reason" staff-user PATCH "$(url platform submissionreviews/e2e-sub-reject)" '{"spec":{"decision":"Rejected"}}' + expect 200 "staff-user rejects, naming someone else as reviewer" staff-user PATCH "$(url platform submissionreviews/e2e-sub-reject)" \ + '{"spec":{"decision":"Rejected","reason":"Already on the roadmap.","reviewer":"someone-else"}}' + [ "$(jqo .spec.reviewer)" = staff-user ] || fail "reviewer is $(jqo .spec.reviewer), want staff-user" + [ -n "$(jqo '.spec.reviewedAt // empty')" ] || fail "no reviewedAt" + eventually 60 "submission is Rejected" is Rejected jget submission e2e-sub-reject '{.status.phase}' + is "Already on the roadmap." jget submission e2e-sub-reject '{.status.reason}' || fail "reason not copied: $(jget submission e2e-sub-reject '{.status}')" + eventually 60 "review is applied" is true jget submissionreview e2e-sub-reject '{.status.applied}' + expect 422 "a decision can't change" staff-user PATCH "$(url platform submissionreviews/e2e-sub-reject)" '{"spec":{"decision":"Approved","publicTitle":"x","publicBody":"y"}}' + expect 200 "test-user sees the rejection" test-user GET "$(url test-user submissions/e2e-sub-reject)" + [ "$(jqo .status.reason)" = "Already on the roadmap." ] || fail "test-user sees: $(jqo .status)" + + - name: approve + try: + - script: + shell: bash + content: | + . ../lib.sh + expect 422 "Approved needs a public title and body" staff-user PATCH "$(url platform submissionreviews/e2e-sub-approve)" '{"spec":{"decision":"Approved"}}' + expect 200 "staff-user approves" staff-user PATCH "$(url platform submissionreviews/e2e-sub-approve)" \ + '{"spec":{"decision":"Approved","publicTitle":"An approved idea","publicBody":"From the e2e tests."}}' + eventually 60 "submission is Approved" is Approved jget submission e2e-sub-approve '{.status.phase}' + eventually 60 "review waits for the post" is "False WaitingForPost" jget submissionreview e2e-sub-approve '{range .status.conditions[?(@.type=="Applied")]}{.status} {.reason}{end}' + + # feedback-sync creates the issue and syncs its post. + expect 201 "sync creates the post for the submission" sync POST "$(url platform posts)" \ + '{"apiVersion":"feedback.miloapis.com/v1alpha1","kind":"Post","metadata":{"name":"e2e-sub-approved-post","annotations":{"feedback.miloapis.com/submission":"e2e-sub-approve"}},"spec":{}}' + eventually 60 "submission links the post" is e2e-sub-approved-post jget submission e2e-sub-approve '{.status.postRef.name}' + eventually 60 "submitter votes on the post" jget vote e2e-sub-approved-post.test-user '{.metadata.name}' + eventually 60 "submitter follows the post" is submitted jget subscription e2e-sub-approved-post.test-user '{.spec.reason}' + eventually 60 "review is applied" is "true True" jget submissionreview e2e-sub-approve '{.status.applied} {.status.conditions[?(@.type=="Applied")].status}' + eventually 60 "the post counts the vote" is 1 jget post e2e-sub-approved-post '{.status.votes.total}' diff --git a/test/e2e/votes/chainsaw-test.yaml b/test/e2e/votes/chainsaw-test.yaml new file mode 100644 index 0000000..ba98d12 --- /dev/null +++ b/test/e2e/votes/chainsaw-test.yaml @@ -0,0 +1,110 @@ +apiVersion: chainsaw.kyverno.io/v1alpha1 +kind: Test +metadata: + name: votes +spec: + description: | + Votes: who owns them, how they're named, who sees them, and the totals. + + - test-user votes in their own context without naming an owner; the + owner webhook sets spec.userRef to them. + - The name must be .: a vote named for another user, or + claiming another user as owner, is refused, as is a vote made outside + a user context. + - Listing in their context returns only their votes, even when they ask + for another user's; getting another user's vote is 404. + - The operator follows the post for each voter and tallies the post's + status.votes, and recounts when a vote is deleted. + steps: + - name: owner-and-name + try: + - script: + shell: bash + content: | + . ../lib.sh + post e2e-votes + vote test-user e2e-votes + [ "$(jqo .spec.userRef.name)" = test-user ] || fail "webhook didn't set spec.userRef: $(cat $OUT)" + + # Claiming another owner: the webhook replaces spec.userRef with + # the caller, and then the name no longer matches. + code=$(call test-user POST "$(url test-user votes)" \ + '{"apiVersion":"feedback.miloapis.com/v1alpha1","kind":"Vote","metadata":{"name":"e2e-votes.other-user"},"spec":{"postRef":{"name":"e2e-votes"},"userRef":{"name":"other-user"}}}') + case $code in 403|422) echo "ok: vote for another user refused ($code)";; *) fail "vote for another user: HTTP $code: $(cat $OUT)";; esac + gone votes e2e-votes.other-user || fail "a vote for other-user was created by test-user" + + expect 422 "vote not named ." test-user POST "$(url test-user votes)" \ + '{"apiVersion":"feedback.miloapis.com/v1alpha1","kind":"Vote","metadata":{"name":"my-vote"},"spec":{"postRef":{"name":"e2e-votes"}}}' + expect 403 "vote outside a user context" test-user POST "$(url platform votes)" \ + '{"apiVersion":"feedback.miloapis.com/v1alpha1","kind":"Vote","metadata":{"name":"e2e-votes.test-user"},"spec":{"postRef":{"name":"e2e-votes"},"userRef":{"name":"test-user"}}}' + expect 403 "vote on a post that doesn't exist" test-user POST "$(url test-user votes)" \ + '{"apiVersion":"feedback.miloapis.com/v1alpha1","kind":"Vote","metadata":{"name":"e2e-votes-missing.test-user"},"spec":{"postRef":{"name":"e2e-votes-missing"}}}' + expect 409 "second vote on the same post" test-user POST "$(url test-user votes)" \ + '{"apiVersion":"feedback.miloapis.com/v1alpha1","kind":"Vote","metadata":{"name":"e2e-votes.test-user"},"spec":{"postRef":{"name":"e2e-votes"}}}' + cleanup: + - script: + shell: bash + content: | + . ../lib.sh + cleanup_posts e2e-votes + + - name: who-sees-what + try: + - script: + shell: bash + content: | + . ../lib.sh + vote other-user e2e-votes + + # Other tests run alongside; look at this test's post only. + expect 200 "test-user lists votes" test-user GET "$(url test-user votes)" + names=$(jqo '[.items[].metadata.name | select(startswith("e2e-votes."))] | join(" ")') + [ "$names" = e2e-votes.test-user ] || fail "test-user listed: $names" + expect 200 "test-user lists other-user's votes by field selector" test-user GET \ + "$(url test-user 'votes?fieldSelector=spec.userRef.name%3Dother-user')" + [ "$(jqo '[.items[] | select(.spec.userRef.name != "test-user")] | length')" = 0 ] || fail "test-user listed other-user's votes: $(jqo '[.items[].metadata.name]')" + expect 404 "test-user gets other-user's vote" test-user GET "$(url test-user votes/e2e-votes.other-user)" + expect 404 "test-user deletes other-user's vote" test-user DELETE "$(url test-user votes/e2e-votes.other-user)" + expect 200 "test-user gets own vote" test-user GET "$(url test-user votes/e2e-votes.test-user)" + + expect 200 "other-user lists votes" other-user GET "$(url other-user votes)" + names=$(jqo '[.items[].metadata.name | select(startswith("e2e-votes."))] | join(" ")') + [ "$names" = e2e-votes.other-user ] || fail "other-user listed: $names" + + - name: follows-and-totals + try: + - script: + shell: bash + content: | + . ../lib.sh + eventually 60 "post total is 2" is 2 jget post e2e-votes '{.status.votes.total}' + is 2 jget post e2e-votes '{.status.votes.datum}' || fail "datum votes: $(jget post e2e-votes '{.status.votes}')" + for u in test-user other-user; do + eventually 30 "$u follows the post" is voted jget subscription "e2e-votes.$u" '{.spec.reason}' + eventually 30 "$u's vote is marked subscribed" is true jget vote "e2e-votes.$u" '{.status.subscribed}' + done + expect 200 "test-user sees their follow" test-user GET "$(url test-user subscriptions/e2e-votes.test-user)" + + expect 200 "test-user deletes their vote" test-user DELETE "$(url test-user votes/e2e-votes.test-user)" + eventually 60 "post total drops to 1" is 1 jget post e2e-votes '{.status.votes.total}' + + - name: audit-names-the-post + description: | + The post-ref ValidatingAdmissionPolicy (config/milo/activity) names the + post in the audit events of vote and follow deletes, whose bodies + don't carry it. Milo's audit events reach the stand-in sink in + milo-system (config/overlays/e2e/audit-sink.yaml). + try: + - script: + shell: bash + timeout: 3m + content: | + . ../lib.sh + expect 200 "test-user unfollows" test-user DELETE "$(url test-user subscriptions/e2e-votes.test-user)" + audited() { # audited : the post-ref annotation on its delete event + infra -n milo-system logs deployment/vector-audit-log-processor | + jq -R -r --arg r "$1" --arg n "$2" 'fromjson? | .items[]? | select(.objectRef.resource == $r and .objectRef.name == $n and .verb == "delete") + | .annotations["post-ref.feedback.miloapis.com/post"] // "none"' | tail -1 + } + eventually 120 "the vote delete's audit event names the post" is e2e-votes audited votes e2e-votes.test-user + eventually 120 "the unfollow's audit event names the post" is e2e-votes audited subscriptions e2e-votes.test-user