Skip to content

Add S3/CloudFront mirror fallback to installers#44

Merged
plutoless merged 21 commits into
mainfrom
feat/s3-installer-fallback
Jun 29, 2026
Merged

Add S3/CloudFront mirror fallback to installers#44
plutoless merged 21 commits into
mainfrom
feat/s3-installer-fallback

Conversation

@plutoless

Copy link
Copy Markdown
Collaborator

Summary

Adds a GitHub-first → S3/CloudFront mirror fallback to install.sh and install.ps1, so the CLI installs in regions where GitHub is blocked or rate-limited. A new mirror-to-s3 release job publishes binaries, a latest.json version pointer, and the installer scripts to dl.agora.io on every tag.

  • Auto-failover (GitHub-first): each fetch (version resolution, archive, checksums) tries GitHub with a fast-fail profile (short connect-timeout, no retry) and, on any failure, retries the same resource from the mirror with the existing retry profile. Covers transport failures and HTTP errors (incl. 403/429 rate limits).
  • AGORA_INSTALL_SOURCE (auto|github|s3) overrides the behavior; s3 skips GitHub entirely for blocked regions (curl … | AGORA_INSTALL_SOURCE=s3 sh).
  • Mirror version pointer: latest.json ({"tag_name":"vX","version":"X"}) is read by both installers' existing parsers; the release job writes it only for stable (non-prerelease) tags.
  • mirror-to-s3 job: downloads release artifacts, verifies SHA-256 before upload, uploads versioned binaries (immutable cache) + scripts + latest.json (short cache) to S3 via static AWS keys, then invalidates CloudFront for the mutable paths. Sigstore bundle is mirrored best-effort.
  • Security: SHA-256 verification is unchanged and applied identically to mirror-served archives; the GitHub auth header is never sent to the mirror. Trust model documented in ADR-0001.

Both installers behave equivalently (same source semantics, fallback order, and error messaging that names only the sources actually attempted).

Test Plan

  • scripts/test-installer-fallback.sh — unit-tests the source-selection/fallback matrix (10 assertions; auto/github/s3 × success/fail).
  • scripts/test-installer-messages.sh — existing installer suite still passes (13 assertions).
  • CI: new Unix + PowerShell fallback E2E steps install from a mock S3 with GitHub pointed at a dead port, and assert SHA-256 rejection on a tampered mirror archive (both OSes).
  • sh -n install.sh clean; both workflows valid YAML; go test ./... unaffected (no Go changed).
  • Requires AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY GitHub secrets + IAM policy (s3:PutObject on dl-agora-io/cli/*, cloudfront:CreateInvalidation) before the mirror-to-s3 job runs on a real tag (see RELEASING.md).

🤖 Generated with Claude Code

plutoless and others added 21 commits June 29, 2026 06:30
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
….ps1

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ssages

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…tall.sh)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… fallback E2E

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…n leak)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ing smoke tests

Widen the fixture HTTP server readiness window from 10s to 30s across all four
installer smoke-test steps (Unix + PowerShell, pre-existing + new fallback) to
fix intermittent macOS failures where the server was not ready in time. Add
early server-death detection to the new fallback steps.

Pin AGORA_INSTALL_SOURCE=github in the two pre-existing smoke tests so they only
exercise the fixture (RELEASES_DOWNLOAD_BASE_URL) path: under the new auto-mode
fallback, a transient fixture hiccup would otherwise fast-fail and divert to the
real dl.agora.io mirror. github mode also restores the retry profile.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…, and installer headers

Add a 'Restricted networks (GitHub blocked)' section to the README and a
matching troubleshooting entry, plus a one-line mirror note in the install.sh /
install.ps1 headers. Blocked-region users could not previously discover the
dl.agora.io mirror, since the README only referenced GitHub-hosted script URLs
(agoraio.github.io / raw.githubusercontent) that are unreachable in the same
regions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… flake)

The pre-existing 'Smoke test Unix installer' fixture used an ephemeral port plus
a heredoc python server that wrote its port to a file. On macOS that python
process intermittently hung before writing the port file, so the readiness
check never found a URL and the step timed out (server alive, port never
written). Replace it with the same fixed-port 'python3 -m http.server 18080
--directory' pattern the S3 fallback steps already use reliably across all three
OSes, removing the port-file write race entirely.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@plutoless plutoless merged commit 829accf into main Jun 29, 2026
7 checks passed
@plutoless plutoless deleted the feat/s3-installer-fallback branch June 29, 2026 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant