build: make the compile stages cacheable across machines

[kmannislands]

Test PR to measure whether this unblocks registry-cache reuse for the buildkitd/buildctl compile. Context: EarthBuild/earthbuild#731.

Problem

Three Dockerfile stages mount the build context — buildkit-version, buildctl, buildkitd — and all three miss the layer cache on any machine that has not already built them. runc, the one compile stage that mounts from runc-src rather than the context, is the only one that reuses a shared cache. That correlation is exact.

The cause is .git. A git checkout is not byte-reproducible: .git/index carries mtimes, logs/HEAD carries timestamps, packed-refs ordering varies. Including it makes the context digest differ per machine, so every stage keyed on the context misses.

buildkit-version was the only stage that actually needed .git, for REVISION=$(git rev-parse HEAD).

Measured impact downstream

In EarthBuild/earthbuild CI every test job builds this image in a fresh buildkitd. Of ~136s spent before any test runs, ~96s is the buildctl + buildkitd compile — and adding a registry cache barely moved it:

	prelude	compile stages still running
no cache (baseline)	155s	3
registry cache, mode=min	151s	2
registry cache, --max-remote-cache	136s	2

runc started hitting; buildkit-version, buildctl and buildkitd never did, in any configuration.

Change

• Exclude .git from the build context.
• buildkit-version takes REVISION as a build arg instead of shelling out to git, so nothing needs .git any more.
• Plumb REVISION through the Earthfile +build target.
• Drop a dead assignment: VERSION=$(git describe ...) has been unused since build: bake version.Package as github.com/EarthBuild/buildkit #18 switched the ldflags line to ${RELEASE_VERSION}.

Notes for review

• .earthlyignore as well as .dockerignore. The Earthfile here is VERSION 0.6, and earthly only falls back to .dockerignore from 0.8 (the use-docker-ignore feature). .earthlyignore is honoured at every version, so it is the one that actually takes effect for FROM DOCKERFILE. .dockerignore is kept in sync for plain docker builds.
• REVISION defaults to unknown, so buildkitd --version loses its revision unless a caller passes one. The value is known statically at the call site (the pinned buildkit sha), which is a better source than a git command inside the build — but this is a real behaviour change and the main thing worth pushing back on.
• Expect a one-time cache invalidation, since the context digest changes.

🤖 Generated with Claude Code

[github-actions]

⚠️ Are we earthbuild yet?

Warning: "earthly" occurrences have increased by 4 (1.84%)

📈 Overall Progress

Branch	Total Count
main	217
This PR	221
Difference	+4 (1.84%)

---

Keep up the great work migrating from Earthly to Earthbuild! 🚀

💡 Tips for finding more occurrences

Run locally to see detailed breakdown:

./.github/scripts/count-earthly.sh

Note that the goal is not to reach 0.
There is anticipated to be at least some occurences of earthly in the source code due to backwards compatibility with config files and language constructs.

[kmannislands]

Measured result from EarthBuild/earthbuild#731 CI (Docker +test-no-qemu-group7, prelude = build start → first test target):

	prelude	stages still compiling
no cache	155s	buildkit-version, runc, buildkitd, buildctl
registry cache, mode=min	151s	buildkit-version, buildkitd, buildctl
registry cache, --max-remote-cache	136s	buildkit-version, buildkitd, buildctl
+ this PR	115s	buildkitd, buildctl

So this PR does what it claims for buildkit-version — that stage now caches, worth ~21s.

But an important caveat on the ignore-file part of this PR, which I got wrong. buildkit-version caches because its context mount was removed, not because the context became stable. buildctl and buildkitd still mount the context and still miss.

The reason is on the earthly side: for remote/git targets, ignore files are not applied at all — buildcontext/git.go:220 is literally // TODO: Apply excludes / .earthignore., and llb.KeepGitDir() (lines 276, 454) keeps .git in the source. earthbuild consumes this repo as a remote target (github.com/EarthBuild/buildkit:<sha>+build), so both .earthlyignore and .dockerignore are no-ops on that path. They would only take effect for a local checkout build.

Making buildctl/buildkitd cacheable therefore needs one of:

1. implementing that TODO in earthbuild, or
2. a runc-src-style stage here: COPY . /src + rm -rf .git, with the compile stages mounting from that stage instead of the context. The cheap COPY stage would still miss, but its output is content-stable, so the expensive compiles would hit.

Option 2 is self-contained to this repo. Happy to add it here if that is the preferred direction — flagging rather than assuming, since it changes the shape of the build.

Separately, the downstream cache work has hit a reliability problem (GHCR blob fetch timeouts failing builds under ~45 concurrent importers), so the overall approach is under review. This PR does not depend on that.