Repository navigation
feat(ingest): receive Cloud Logging entries from GCP Pub/Sub push - #1320
JeremyFunk wants to merge 28 commits into
Confidence 4/10 · 2 issues to address
🔴 Confidence 4/10 · risky as written
The RS256 verification is pinned by tests, but the account name the new auth path requires has no producer in the repository, and the route serves no traffic yet.
quality 80/100 · 2 warnings · tests covered · risk high · 1/2 new units observable
Adds a Pub/Sub OIDC path to the GCP logpush receiver: ID tokens verified against Google's JWKS, with an audience and push-account check and a one-time setup report. The verification is sound; the account name it requires is not one the repository creates.
- Pub/Sub pushes authenticate by a Google-signed ID token, with the URL secret still accepted
GooglePushAuthverifies RS256 tokens against Google's JWKS, cached for an hour- The first signed push to an unreported connector writes
applied_logs_enabled = true fetch_gcp_connector_by_idlooks a connector up without its secret
Before merge
- Env var · Set
INGEST_GCP_ORG_MAX_IN_FLIGHTin every environment, or confirm its default ·apps/ingest/alchemy.run.ts:619 - Infra · Check every environment has what this config now expects ·
apps/ingest/alchemy.run.ts
Findings
🟠 Warning · F3 · gcp_push_service_account builds an account name the repo never creates
correctness · apps/ingest/src/main.rs:5170-5176
It derives maple-push-<19 hex>@<project>, while the function its own comment cites, gcpConnectorResourceNames (packages/domain/src/gcp.ts:10), derives maple-<24 hex>@<project>; nothing else in the repository names a maple-push-* account, and the setup script's subscription sets no push-auth account. A subscription signed as the account the setup creates sends a token resolve_connector_by_token never matches, so the push is answered 401 and Pub/Sub redelivers until the message's retention expires.
Derive the name from `gcpConnectorResourceNames` (or add the push account to that function) so the Rust and TypeScript names cannot drift apart.
🟠 Warning · F4 · Google's signing-key fetch has no Client span
observability · MAP-01 · apps/ingest/src/gcp_push_auth.rs:267-284
fetch_keys calls https://www.googleapis.com/oauth2/v3/certs through the plain HttpClient (a bare reqwest::Client) with no span, where other outbound calls in this crate carry one (autumn.rs:147, postgres_client_span at main.rs:6687). When the fetch fails, verify turns it into PushAuthError::KeysUnavailable and the push is answered 503 for Pub/Sub to redeliver, with nothing in Maple showing the failed call.
Wrap the request in an `otel.kind = "client"` span carrying `"peer.service" = "google-oauth2"`, `http.request.method` and `server.address`, the way `autumn.rs` wraps its calls.
Production impact
Production traffic of the changed files (last 7 days)
| File | Calls/day | Busiest operations |
|---|---|---|
apps/ingest/src/main.rs |
199k | maple 199k/day, 95.8% errPOST 786/day, 4.2% errPOST 150/day, 0.4% err |
apps/ingest/src/gcp_logging.rs |
770 | GET 618/day, 2.8% errGET 152/day, 0.0% err |
What was checked
verify_with_keysfixesalgto RS256 and slices exactly the signed segments (gcp_push_auth.rs:118,:129), soalg: noneand swapped claims failrecord_gcp_setup_reportCOALESCEs its inputs, sometrics: Nonekeeps the last reported value (main.rs:7113)- An unknown
kidrefetches Google's keys at most once a minute (gcp_push_auth.rs:238)
Observability coverage: 1 of 2 changes observable
| Change | Kind | Observable | Evidence |
|---|---|---|---|
Pub/Sub push authentication (resolve_connector_by_token) |
inbound auth inside the existing server span | yes | records maple.gcp.push_auth on the route span (main.rs:4883, 4990) |
Google signing-key fetch (fetch_keys) |
outbound HTTP | no | plain HttpClient request, no Client span or peer.service (gcp_push_auth.rs:267) |
8aa9b90 · Updated on every push. Reply "won't fix" to dismiss a finding, or mention @maple-review-bot to ask about one. Check ids refer to Maple's instrumentation audit.
Annotations
Check warning on line 5176 in apps/ingest/src/main.rs
maple-review-bot / Maple / review
correctness: `gcp_push_service_account` builds an account name the repo never creates
It derives `maple-push-<19 hex>@<project>`, while the function its own comment cites, `gcpConnectorResourceNames` (`packages/domain/src/gcp.ts:10`), derives `maple-<24 hex>@<project>`; nothing else in the repository names a `maple-push-*` account, and the setup script's subscription sets no push-auth account. A subscription signed as the account the setup creates sends a token `resolve_connector_by_token` never matches, so the push is answered 401 and Pub/Sub redelivers until the message's retention expires.
Check warning on line 284 in apps/ingest/src/gcp_push_auth.rs
maple-review-bot / Maple / review
observability · MAP-01: Google's signing-key fetch has no Client span
`fetch_keys` calls `https://www.googleapis.com/oauth2/v3/certs` through the plain `HttpClient` (a bare `reqwest::Client`) with no span, where other outbound calls in this crate carry one (`autumn.rs:147`, `postgres_client_span` at `main.rs:6687`). When the fetch fails, `verify` turns it into `PushAuthError::KeysUnavailable` and the push is answered 503 for Pub/Sub to redeliver, with nothing in Maple showing the failed call.