feat(desktop): prefer the secrets engine for Docker Hub authentication

[Benehiko]

Warning

Deprecation notice: Docker Desktop's /registry/token endpoint is being deprecated. It is retained here only as a fallback while the secrets-engine SDK becomes the primary source of the Docker Hub session, and will be removed in a future change.

What

docker-agent now reads the default Docker Hub session from the Docker secrets engine before it asks Docker Desktop's backend API for a token. This matches docker/gordon#2510 and docker/cloud#304.

When the Docker Desktop source of the environment chain needs DOCKER_TOKEN, it tries these in order:

1. Secrets engine (new): HubAuth().GetDefaultSession() over Docker Desktop's engine socket (api.DesktopSocketPath()). The SDK defaults to the standalone daemon socket, which Desktop doesn't serve; this is the same fix as in docker/cloud#304.
2. Docker Desktop /registry/token (unchanged).
3. A token minted from the access token docker login stored (unchanged).
4. A forced Docker Desktop refresh (unchanged).

Also:

• Staging: when DOCKER_AGENT_HUB_LOGIN_URL points the token exchange at a Docker Hub staging host (*-stage.docker.com), the engine's staging realms are read instead (dockerhub.Staging()).
• GetUserInfo now names the account of the token in use, read from its claims, so it matches the account requests are sent as. Docker Desktop's account is the fallback for a token without one, such as a personal access token.
• docker agent debug auth reports the new source as docker desktop secrets engine and, when the engine couldn't be read, says why with the SDK's hint (see below).

Other credentials (CI etc.)

• The engine never shadows an explicit credential. It is only reached through the docker-desktop source at the end of the environment chain. A DOCKER_TOKEN from the environment, run secrets, an env file or a credential helper still wins, and the engine isn't asked.
• Machines without the engine behave as before. On CI runners, Linux without Docker Desktop, or older Desktop releases, the dial fails straight away and the existing sources take over. A missing engine (the SDK's ErrSecretsEngineNotRunning) is logged at debug level only. Other failures (permission denied, a timeout, access denied, a session that can't be decoded) log a warning, only for the first failure in a run, so a long-running process doesn't repeat it after every cooldown.
• The engine can't hold up requests. Each lookup is capped at 5 s on its own context, so the caller's context is still intact for the fallbacks. Concurrent callers share one lookup, and a caller waits at most half its remaining time for it. After a failure the engine is skipped for 30 s. "Nobody signed in" (dockerhub.ErrNoSession) is a clean miss with no cooldown, so a new sign-in is picked up right away.
• Tokens are vetted like the others. An expiring engine token, or one Docker already rejected (InvalidateToken), is never served. It falls through so minting or the forced refresh can replace it. Engine tokens share the existing 30 s in-memory cache.
• Only built where the SDK is. The engine code builds on darwin, linux and windows. Elsewhere, js/wasm included, the engine step finds no token and the existing sources take over.

How a failure shows up

A failed lookup never fails a request: the next source is tried. Without --debug the CLI discards logs, so debug auth is where a user sees why the engine was skipped. It prints the SDK's ConnectError reason, socket and hint, falling back to the full error when there is no known reason. An engine that isn't running is only reported when no token was found, since machines without Docker Desktop don't have one.

With Docker Desktop quit and minting disabled:

$ DOCKER_AGENT_NO_TOKEN_EXCHANGE=1 docker agent debug auth
No token found. Is Docker Desktop running, or are you logged in with `docker login`?

Secrets engine: not running
Socket:         /Users/benehiko/Library/Caches/docker-secrets-engine/engine.sock
Hint:           Start Docker Desktop and retry: the secrets engine runs as part of it.

When another source supplied the token, other failures are still reported. For example, when the user can't open the socket:

$ docker agent debug auth
Token:      eyJhbGciOi...
Source:     docker desktop
...
Status:     ✅ Valid

Secrets engine: permission denied
Socket:         /Users/benehiko/Library/Caches/docker-secrets-engine/engine.sock
Hint:           Your user lacks permission to connect to "/Users/benehiko/Library/Caches/docker-secrets-engine/engine.sock". Grant it read and write access to the socket and retry.

The JSON output adds a secrets_engine object that also keeps the full error:

$ DOCKER_AGENT_NO_TOKEN_EXCHANGE=1 docker agent debug auth --json | jq
{
  "error": "no token found (is Docker Desktop running, or are you logged in with `docker login`?)",
  "secrets_engine": {
    "error": "retrieve default account metadata: secrets engine is not running at /Users/benehiko/Library/Caches/docker-secrets-engine/engine.sock: unavailable: dial unix /Users/benehiko/Library/Caches/docker-secrets-engine/engine.sock: connect: no such file or directory",
    "reason": "not running",
    "socket": "/Users/benehiko/Library/Caches/docker-secrets-engine/engine.sock",
    "hint": "Start Docker Desktop and retry: the secrets engine runs as part of it."
  }
}

With --debug, the debug log records the same failure, with the hint in a hint= field.

Changes

• go.mod/go.sum: github.com/docker/secrets-engine/client v0.2.0 and github.com/docker/secrets-engine/x v0.8.3. v0.2.0 is the first release with ErrSecretsEngineNotRunning (feat(client)!: report why the client could not connect to the engine secrets-engine#677), and it reports failed connections as a ConnectError carrying the reason, socket and a hint. The only other new module is connectrpc.com/connect, pulled in indirectly.
• pkg/desktop/secrets_engine.go (darwin, linux, windows): the secretsEngine type, which owns the SDK client and handles the lookup, its time cap, shared lookups, the cooldown and failure logging, plus desktop.SecretsEngineError() for debug auth.
• pkg/desktop/secrets_engine_other.go: the stub everywhere else.
• pkg/desktop/login.go: the engine step, the SourceSecretsEngine label, and GetUserInfo naming the account of the token in use.
• pkg/hubauth/exchange.go: Staging(), which picks the engine's staging realms.
• cmd/root/debug_auth.go: why the engine supplied no token, in the text and JSON output.
• Docs: guides/secrets (Docker Authentication), features/cli (debug auth sources and secrets engine fields), and configuration/overview (staging sessions for DOCKER_AGENT_HUB_LOGIN_URL).

Testing

• pkg/desktop/secrets_engine_test.go runs the real dockerhub accessor against a fake resolver.
  • Engine tests build their own secretsEngine with a fake client and run with t.Parallel(). They cover: the cooldown after a missing engine, access denied or a lookup failure; no cooldown when nobody is signed in; a client that can't be created; the 5 s budget and the half-deadline rule; a lookup that outlives its callers; a canceled caller; concurrent callers sharing one lookup; and recording the last lookup error.
  • Tests through GetTokenWithSource cover: engine preferred; cached; signed out, then signing in; a failing engine falling back; expiring token; expired token falling through to minting; rejected token; and GetUserInfo naming the engine token's account.
• pkg/desktop/secrets_engine_logging_test.go checks the log level for each kind of failure, that connection failures carry the SDK's hint, and that other failures warn once per run.
• cmd/root/debug_auth_test.go checks which failures debug auth reports, with their reason, socket and hint, and how they print.
• pkg/hubauth/exchange_test.go checks staging host detection.
• The existing pkg/desktop tests install a signed-out fake engine, so they never reach a developer's own Docker Desktop.
• go test -race passes for pkg/desktop, pkg/environment, pkg/hubauth, pkg/sandbox and cmd/root.
• golangci-lint reports no issues in this branch's changes; go run ./lint . and the tidy check pass.
• Builds pass for native, linux/arm64, windows/amd64, and js/wasm with and without docker_agent_wasm_cloud.
• Manually: debug auth with Docker Desktop quit, as shown above.

Open question

docker/cloud#304 counts engine failures as an OTel secrets_engine_auth_failure metric; docker/gordon#2510 doesn't. This PR logs (debug when no engine is running, a warning for the first other failure in a run) and shows the failure in debug auth. It sends no telemetry. Happy to add an event if we want that signal.

[dgageot]

Cache hits add no socket work. On a cache miss, an absent engine adds a failed dial; a usable engine session needs two sequential RPCs instead of the previous single token request; a stalled engine can add five seconds. The deadline interaction below is a concrete regression. Reproduced the deadline, concurrent-probe, and identity cases with temporary tests; desktop race tests pass. The existing socket-error classification comment already covers the logging issue.

[Benehiko]

Thanks both. I've pushed fixes for all the review threads, with replies inline:

• eeb402e: shared engine lookups. Callers keep at least half their deadline for the fallbacks, a stalled engine always starts the cooldown, and the docs wording is fixed.
• 07865e1: only a missing or stopped engine is routine; permission denied and similar failures warn (SDK follow-up: feat(client)!: report why the client could not connect to the engine secrets-engine#677).
• d0069b0: GetUserInfo names the account of the token in use, falling back to /registry/info.
• ba26fc8: staging sessions are read when DOCKER_AGENT_HUB_LOGIN_URL targets a staging host.

On the latency summary: agreed on the numbers. On a cache miss, a usable engine session takes two sequential RPCs where there was one request before. The 30s token cache keeps misses rare, so I've left that as is. A stalled engine is now bounded by half the caller's remaining time, and it starts the cooldown.