Skip to content

Publish a prebuilt manifest index to remove the API from the install path - #3

Merged
TecharyJames merged 1 commit into
BETAfrom
feat/prebuilt-manifest-index
Sep 19, 2026
Merged

TecharyJames merged 1 commit into
BETAfrom
feat/prebuilt-manifest-index

Conversation

@TecharyJames

@TecharyJames TecharyJames commented Sep 19, 2026 •

Copy link
Copy Markdown
Member

Stacked on #2. Diff is against its branch; base retargets to BETA when #2 merges.

Why

#2 makes the GitHub API allowance survivable. This removes it from the normal path.

Resolving a package costs two api.github.com calls against a 60/hour-per-source-IP allowance. Doing that work once in CI and publishing the answer means endpoints never spend that allowance.

How

A nightly Action resolves every package in Index/Catalog.json — in CI, where GITHUB_TOKEN gives 5,000/hour — and publishes a single Manifests.json. Endpoints read that file from raw.githubusercontent.com, which is CDN-backed and carries no API rate limit.

The index is force-pushed to an orphan manifest-index branch, so it holds exactly one commit: a nightly refresh never grows the repo and never touches code history.

Resolution order in Get-GitHubInstaller:

local per-package cache  ->  prebuilt index  ->  live API  ->  stale cache
        (no network)          (CDN, no limit)     (2 calls)     (no network)

Every layer is optional. A missing, stale, unreachable or malformed index returns null and falls through to the live API exactly as today.

Adding a package

One line in Index/Catalog.json. The seed list holds the three IDs that already carry argument overrides in the code (Dell.CommandUpdate, 8x8.Work, SublimeHQ.SublimeText.4) plus common business apps, and should be replaced with the actual deployment list.

Scheduling constraint

GitHub only runs scheduled workflows from the repository's default branch, currently main. The nightly will not fire from BETA. workflow_dispatch and the push trigger work from any branch, so the index can be built on demand immediately. For the nightly, the workflow file needs to exist on the default branch, or BETA needs to become the default branch.

Verification

Builder, against live manifests:

ok    7zip.7zip                          x64    v26.03
ok    Notepad++.Notepad++                x64    v8.9.8
Wrote Manifests.json with 2 package(s).

Client:

Case Result
Index present, package indexed returns 7zip.7zip v26.03, source=index
Unknown package ID null → live API
Indexed package, unindexed arch (arm64) null → live API
No index at all (branch not yet created) warns, returns null → live API

Notes

  • .gitattributes is deliberately narrow: *.yml -text only. A CRLF run: block reaches bash on the runner with a literal CR per line and breaks it. * text=auto is not declared, because this repo's blobs are committed CRLF and that would queue a repo-wide renormalisation diff.
  • Per-package failures are reported and skipped; the build fails only if every package fails, which also prevents publishing an empty index.
  • arm64 misses are expected and logged as skip, not warn — most packages have no arm64 installer.

…path

Follow-up to the caching work: takes the steady state to zero
api.github.com calls rather than merely fewer.

A nightly Action resolves every package in Index/Catalog.json once, in CI,
where GITHUB_TOKEN gives a 5000/hour allowance, and publishes the result as
a single Manifests.json. Endpoints read that one file from
raw.githubusercontent.com, which is CDN-backed and carries no API rate
limit, so an install resolves without spending any of the site's 60/hour
public-IP allowance.

The index is force-pushed to an orphan manifest-index branch. That keeps
exactly one commit on it, so a nightly refresh never grows the repository
and never touches code history.

Resolution order in Get-GitHubInstaller is now:
  local per-package cache -> prebuilt index -> live API -> stale cache

Every layer is optional. A missing, stale, unreachable or malformed index
returns null and falls through to the live API exactly as before, so this
cannot make installs worse than they are today.

Adding a package to the index is a one-line edit to Index/Catalog.json. The
seed list holds the three IDs that already carry argument overrides in the
code plus common business applications; edit freely.

Verified: builder resolves 7zip.7zip v26.03 and Notepad++.Notepad++ v8.9.8
against live manifests and emits a valid index; client returns the correct
entry on a hit, and null on an unknown ID, an unindexed architecture and a
completely absent index, degrading to the live API in each case.
@TecharyJames TecharyJames self-assigned this Sep 19, 2026
@TecharyJames
TecharyJames changed the base branch from fix/api-rate-limit-and-failure-semantics to BETA September 19, 2026 11:48
@TecharyJames
TecharyJames merged commit 5050f24 into BETA Sep 19, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant