E2E on MiniMax: harness, selective workflow and concurrency control (infrastructure only) #5230

Maintainer setup — do this first

• Add the secret MINIMAX_API_KEY (an org secret shared with TalonT-Org/AutoSkillit, or a repository secret). Its value is a MiniMax API key — the same kind of key your local minimax provider profile already uses. Today AutoSkillit can read no model key of any kind.
• Create the sandbox repository TalonT-Org/autoskillit-e2e-sandbox: a small project with a seconds-long test suite, a Taskfile.yml with install-worktree and test-check, a .autoskillit/config.yaml, and a GitHub Actions workflow that runs on pull requests — the implementation recipe stops when a PR has no CI checks (escalate_stop_no_ci).
• Create a fine-grained personal access token limited to the sandbox repository and save it as a repository secret on TalonT-Org/AutoSkillit named E2E_SANDBOX_TOKEN (repository-level, so no other org repository can read it). Start with Contents, Pull requests and Issues: read and write; Actions and Commit statuses: read (the recipes wait on CI with wait_for_ci). This ticket confirms the minimal set.

Not needed: ANTHROPIC_API_KEY, CLAUDE_CODE_OAUTH_TOKEN, a Claude login, or OPENAI_API_KEY. MiniMax's Claude Code guide puts the MiniMax key into Claude Code's ANTHROPIC_AUTH_TOKEN variable (guide); Claude Code sends that variable as a Bearer token to ANTHROPIC_BASE_URL (Claude Code authentication). The variable's name is Claude Code's; the credential is MiniMax's. On 2026-10-03, in a fresh container with no Claude credentials file, no ANTHROPIC_API_KEY and no CLAUDE_CODE_OAUTH_TOKEN, Claude Code 2.1.280 completed requests on MiniMax — including with MiniMax configured only in ~/.claude/settings.json and with explicit claude-sonnet-5 / claude-opus-5-5 model ids.

Depends on: #5229 (the user image).

Why

No CI job runs a recipe end to end with a real model. conformance-probes.yml exists only on develop, so its weekly schedule never fires (scheduled workflows run from the default branch, main); its keys are not visible to the repository either. task test-smoke collects no tests. autoskillit fleet run — one-shot headless dispatch that prints a JSON envelope and exits 0/1/2/3 (source) — has no CI caller.

This ticket builds the harness and workflow only. Each end-to-end test is its own ticket because each is likely to find something already broken.

How a run reaches MiniMax (design constraint)

• AutoSkillit strips ANTHROPIC_API_KEY, ANTHROPIC_AUTH_TOKEN and ANTHROPIC_BASE_URL from the inherited environment of headless Claude Code launches (set, skill sessions, orchestrator), so container environment variables do not reach sessions.
• The fleet run orchestrator is never given a provider profile: provider_name/profile_name default to "" (food truck) and nothing in fleet/dispatch/ sets them.
• Therefore the harness configures MiniMax in the container's ~/.claude/settings.json env block, which Claude Code reads for every process; AutoSkillit never passes --setting-sources, --settings or CLAUDE_CONFIG_DIR.

Scope

1. Harness (e.g. scripts/e2e/), run inside the user image built for the commit under test:
   • Write ~/.claude/settings.json env: ANTHROPIC_BASE_URL=https://api.minimax.io/anthropic, ANTHROPIC_AUTH_TOKEN from MINIMAX_API_KEY, ANTHROPIC_MODEL and ANTHROPIC_DEFAULT_{SONNET,OPUS,HAIKU}_MODEL=MiniMax-M3, CLAUDE_CODE_MAX_CONCURRENT_SUBAGENTS=6. Not lower: some skills require their declared children to run together — four for implement, three to six for resolve-review (resolve-review) — and a spawn past the limit fails without retry.
   • Set a git author identity (git config --global user.name / user.email) for the commits the recipes make.
   • Write ~/.autoskillit/config.yaml: set quota_guard.enabled: false — with no Anthropic credential every recipe step is otherwise rejected as quota_authority_unknown (launch identity, rejection), and fleet run --disable-quota-guard only changes the fleet run process's in-memory config (flag); enable the fleet and fleet_headless_run features; route every step the defaults send to Codex to claude-code, recipe by recipe, since agent_backend.recipe_overrides has no wildcard (defaults):

     agent_backend:
       recipe_overrides:
         implementation: {run_arch_lenses: claude-code}
         implementation-groups: {run_arch_lenses: claude-code}
         remediation: {investigate: claude-code, run_arch_lenses: claude-code}

   • Run autoskillit install in the container before fleet run. The orchestrator launch passes a plugin directory with --plugin-dir, and the food-truck path raises when it has no consumed-artifact plugin binding (check); confirm whether install is strictly required and drop the step only if it is not.
   • Authenticate gh with E2E_SANDBOX_TOKEN; clone the sandbox.
   • Run autoskillit fleet run <recipe> -i … --disable-quota-guard --timeout-sec <n>.
   • Assert structure, never prose: the envelope's success field, the expected branch/PR state on the sandbox, a non-empty diff.
   • Upload ~/.local/share/autoskillit/logs/ and the envelope as artifacts; redact the key from all output.
2. Canary test (proves the wiring for one request): inside the image, claude -p "<fixed prompt>" through the settings file returns the expected string.
3. Workflow .github/workflows/e2e.yml:
   • Triggers: pull_request with types: [opened, synchronize, reopened, labeled] (the default types omit labeled, so adding the e2e label to an open PR would not start a run), same-repository branches only (fork PRs get no secrets), and workflow_dispatch.
   • Selector job: eligible when changed_files ≥ F or additions + deletions ≥ L (fields on the pull request object), or labelled e2e; then sampled with probability p, seeded from PR number + head SHA so re-runs decide the same way; then pick one or two tests from the catalog, preferring tests whose paths the PR touched. Start with F = 10, L = 300, p = 0.5.
   • Concurrency: one repository-wide group with queue: max, so exactly one E2E run executes and later ones wait instead of being cancelled. Pipeline steps run one at a time, so a run peaks at one step's fan-out plus the orchestrator and that step's session — up to about 8 during resolve-review. One run at a time keeps CI at or below about 8 of the ~12 concurrent sessions the shared MiniMax budget allows.
   • Required check: a final e2e-gate job with if: always() that succeeds when no test was selected or every selected test passed, and fails otherwise. Make only e2e-gate required, so an unselected PR never waits on a skipped or pending job.
   • On failure, file or update an issue the way scripts/post-probe-failure.sh does.
4. Test catalog file (name, recipe, ingredients, peak sessions, timeout, trigger paths). It ships with only the canary; each test is added by its own ticket.

Known gaps, tracked separately

• The fleet run orchestrator ignores providers.default_provider. The settings-file route works around it.
• conformance-probes.yml cannot run on schedule until it reaches main, and its ANTHROPIC_API_KEY / CLAUDE_CODE_OAUTH_TOKEN / OPENAI_API_KEY secrets are not configured. Unrelated to this harness, which needs none of them.

Acceptance criteria

• The canary passes in CI with only MINIMAX_API_KEY and E2E_SANDBOX_TOKEN configured; no workflow step references an Anthropic secret.
• The selector is deterministic for a given PR number and head SHA, and has unit tests for eligibility, sampling and the 1–2 test cap.
• Two simultaneous E2E dispatches run one after the other.
• e2e-gate reports success on a PR the selector skips, and failure (blocking merge) when a selected test fails.
• Adding the e2e label to an open PR starts a run.
• Session logs and the envelope are uploaded on success and failure; the MiniMax key never appears in logs or artifacts.