You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
E2E test: remediation recipe on a seeded sandbox bug #5234
Seed an open bug issue in TalonT-Org/autoskillit-e2e-sandbox with a reproducible failing behavior (and a sandbox test that currently passes despite the bug, so the run has to find the gap).
Everything else comes from #5230 (MINIMAX_API_KEY, E2E_SANDBOX_TOKEN). No Anthropic credential.
Depends on: #5230; #5233 (implementation) should pass first.
What it proves
The bundled remediation recipe completes on MiniMax from bug report to open PR: investigate, rectify, dry-walkthrough, implement, test, audit, merge, PR.
Cost and limits
The heaviest test. Steps run one at a time: 6 concurrent sessions during implement (orchestrator, session, four explorers), up to 8 during resolve-review (three to six groups). rectify explores with parallel subagents, one or more per aspect it lists, so its peak depends on the issue.
investigate's fan-out depends on the issue — its child count is data-dependent — so the harness ceiling CLAUDE_CODE_MAX_CONCURRENT_SUBAGENTS=6 is what bounds it, and a spawn past the ceiling fails without retry. Watch whether investigate degrades or fails at that ceiling; that is a likely first defect. The same applies to audit-impl, which starts one deviation evaluator per plan deviation.
The defaults route remediation.investigate to the Codex backend; the harness must route it to claude-code.
Selected rarely: only for the largest PRs or by the e2e label, never alongside another pipeline-class test.
Scope
Catalog entry with trigger paths covering the remediation recipe and the investigate, rectify and audit-impl skills.
Run with open_pr=true, auto_merge=false; the harness closes the PR and deletes the branch afterwards.
Assertions: envelope success; a PR open on the sandbox referencing the seeded bug; the diff changes behavior and adds or changes a test; sandbox tests green on the PR branch.
Each defect found gets its own bug issue; the catalog entry is marked expected-failure with the link until fixed.
Acceptance criteria
Completes in CI on MiniMax with no Anthropic credential.
Peak concurrent sessions measured from the session logs, including investigate's, and recorded in the catalog entry.
Activity