feat(run-environment): mint CircleCI OIDC tokens for upload auth - #482
feat(run-environment): mint CircleCI OIDC tokens for upload auth#482fargito wants to merge 4 commits into
Conversation
CircleCI jobs could only authenticate their uploads with a `CODSPEED_TOKEN` secret. Mint an OIDC token instead when no static token is set, so a job needs no secret. The token CircleCI exposes in `CIRCLE_OIDC_TOKEN_V2` cannot be used: its audience is the id of the CircleCI organization, while CodSpeed requires its own. Only the `circleci` CLI can request a custom audience, so the runner shells out to it, once per upload since a minted token expires after an hour. Two cases keep needing the static token, and both are refused before the benchmarks run rather than at upload time: pull requests opened from a fork, whose token names the fork, and jobs whose image does not ship the CLI. A pipeline triggered by a custom webhook gets a token that names no repository, which only the upload can tell, so the message shown when it is refused mentions it. Closes COD-3257
Greptile SummaryAdds native CircleCI OIDC authentication for benchmark uploads.
Confidence Score: 5/5The PR appears safe to merge. The previously reported compatibility-probe defect is fixed because missing or outdated CircleCI CLIs now fail during provider initialization, before benchmark execution, while preserving the CLI's diagnostic output.
|
| Filename | Overview |
|---|---|
| src/run_environment/circleci/oidc.rs | Adds custom-audience token minting with exit-status, UTF-8, empty-output, and stderr handling while keeping token contents out of logs. |
| src/run_environment/circleci/provider.rs | Replaces the inadequate CLI version probe with an actual preflight mint and refreshes authentication before every upload. |
| src/upload/uploader.rs | Adds CircleCI-specific guidance for OIDC authentication failures caused by custom-webhook pipelines. |
| Cargo.toml | Advances the runner package to the corresponding alpha release version. |
| Cargo.lock | Synchronizes the locked root-package version with Cargo.toml. |
Sequence Diagram
sequenceDiagram
participant Runner
participant Provider as CircleCI Provider
participant CLI as circleci CLI
participant Benchmarks
participant Backend as CodSpeed Backend
Runner->>Provider: check_oidc_configuration()
Provider->>CLI: "run oidc get (aud=codspeed.io)"
CLI-->>Provider: Preflight token
Provider-->>Runner: OIDC available
Runner->>Benchmarks: Execute benchmarks
Benchmarks-->>Runner: Completed runs
loop Each upload
Runner->>Provider: set_oidc_token()
Provider->>CLI: "run oidc get (aud=codspeed.io)"
CLI-->>Provider: Fresh token
Runner->>Backend: Upload with token
end
Reviews (2): Last reviewed commit: "fix(run-environment): decide CircleCI OI..." | Re-trigger Greptile
Performance comparison unavailableThe base and head of this comparison were measured with different runner settings, so their benchmark values are not directly comparable. What changed between base and head:
Re-run the base with the same settings to get a valid performance comparison. Comparing |
|
@greptileai review |
Spawning `circleci version` only proved that something answers to that name. A CLI too old for `run oidc get`, or a job not allowed to issue tokens, passed the check and failed at upload time, once the benchmarks had run for nothing. Mint a token and discard it instead. It answers the same question about the CLI, plus the ones only the CLI can answer, and it answers them before the benchmarks run. The upload mints its own anyway, as a token expires an hour after it is minted. Errors now carry what the CLI reported: only the first error of a chain is shown outside of debug logging, so wrapping a mint failure in `context` was hiding the reason it failed. Refs COD-3257
The messages spelled out which cases OIDC cannot cover, which belongs in the integration docs rather than in an error string, and left the CircleCI page unreferenced while the sibling providers link theirs. That page does not exist yet. It has to answer `#authentication` and `#oidc-recommended`, and carry what the messages no longer say: a pipeline triggered by a custom webhook gets a token naming no repository, and a pull request opened from a fork gets one naming the fork. Refs COD-3257
12d9f72 to
600b868
Compare
Reading `CIRCLE_PR_NUMBER` as "this build is a forked pull request" rests on a guarantee of CircleCI's, so quote it and point at where it is made. Refs COD-3257
Mint an OIDC token so CircleCI jobs need no
CODSPEED_TOKENsecret.The token CircleCI exposes in
CIRCLE_OIDC_TOKENandCIRCLE_OIDC_TOKEN_V2cannot be used: itsaudience is fixed to the id of the CircleCI organization, while CodSpeed requires its own. Only
the
circleciCLI can request a custom audience — CircleCI publishes no endpoint for it, unlikethe
ACTIONS_ID_TOKEN_REQUEST_URLGitHub Actions provides — so the runner shells out tocircleci run oidc get, once per upload, as a token expires an hour after it is minted.Whether a job can mint at all is settled by minting a token and throwing it away, before the
benchmarks run. A CLI can be installed and still predate
run oidc get, so probing for the binarywould pass and the mint would fail once the benchmarks had run for nothing.
Two cases keep needing a static token, and both are refused upfront: pull requests opened from a
fork, whose token names the fork, and jobs that cannot mint at all. A third is only visible when
the upload is refused: a pipeline triggered by a custom webhook gets a token naming no repository.
The error messages link
integrations/ci/circleci/configuration, anchors#authenticationand#oidc-recommended. That page does not exist yet, and owes the three cases above.Stacked on #473. Server-side support is not released yet, so a real CircleCI job cannot be
verified end to end.
Closes COD-3257