Repository navigation
make-plan output fails bind_plan_set with allocation_step_missing #5259
Description
make-plan output fails bind_plan_set with allocation_step_missing
Problem
PR #5255's implementation E2E commissioning run now passes planning admission and
executes MiniMax make-plan, but the returned plans fail the sealed plan-set
coverage gate. Replanning returns the same class of failure and exhausts the
recipe's replan budget before implementation begins.
Live reproduction
- Run: https://github.com/TalonT-Org/AutoSkillit/actions/runs/38073163382
- Tested commit:
021beadb687a96882fd717dabf36171732a97598, including merged fix: one output_dir boundary check for run_skill admission and recipe rules #5257. - Scenario:
e2e.yml,tests=implementation. - Seed: https://github.com/TalonT-Org/autoskillit-e2e-sandbox/issues/1
- Artifact:
e2e-implementation-38073163382-1.
out/result.json records passed=false, outcome=failed, and no expected
findings. The fleet envelope's diagnostic result identifies step=bind_plan_set,
reason=coverage_failed, and allocation_step_missing for P-1 through P-14.
The terminal reports replanning exhausted, followed by issue release and failure
registration.
Retained summaries for sessions 9146aa3f-49b7-48e9-a649-7f547a833176 and
ee512612-95c1-4c5e-b4f7-2540a6b5e001 show successful make-plan execution,
planning-auditor calls, and Write calls. This is distinct from the output-dir
admission defect fixed by #5257 / #5256.
Investigation starting points
core/planset/allocation.py:verify_allocation_evidencerequires allocation
references to name implementation headings recognized bySTEP_HEADING_RE.core/io/markdown_sections.py:STEP_HEADING_REaccepts### Step Nand
### Step N: descriptionheadings.- The
make-planskill's plan-structure templates show allocation references
such asStep N.M, but leave the implementation-steps section as prose
placeholders without an explicit matching heading example.
The downloaded artifact does not include the plan files, so a specific generated
heading mismatch is not yet proven. The observed failure is that all referenced
implementation steps are missing from the binder's parsed heading map, rather
than a proven missing frontmatter key.
Replanning evidence and open question
Both successful planning-session summaries encode plan_set_coverage_gaps
as "" inside the bound /skill_command input packet. The final bind
diagnostic's /l3_payload/diagnostic_result/coverage_gaps is also "".
That diagnostic is retained L3 envelope data parsed from stdout; this review
does not treat it as a freshly verified raw binder response. Capture the
actual replies and generated plans on reproduction.
implementation.yaml:434,498 passes that captured field back into make-plan;
the skill documents it as the failed-bind gap input. Thus that named input
carried no gap text on replanning. Check the first bind response and the
diagnostic-to-input handoff; whether an empty field is intentional for this
failure or defective is not established.
There are two retained successful planning executions: the initial plan
(ee512612-95c1-4c5e-b4f7-2540a6b5e001) and one replan
(9146aa3f-49b7-48e9-a649-7f547a833176). The recipe's max_iterations: 2
loop exhausts when the incremented count reaches 2; do not interpret the
terminal's loose iteration wording as two additional retries.
Expected outcome
Recover the actual plans and determine whether their headings/references
violate the parser contract, the producer guidance is insufficient, the parser
is at fault, or replanning lacks the needed feedback. Repair the responsible
path while preserving requirement coverage enforcement, and verify a real
emitted plan passes bind_plan_set before rerunning commissioning.
Refs #5233 and PR #5255. Classification: bug, recipe:remediation.
Reproduction prerequisites
Pinned source: 021beadb687a96882fd717dabf36171732a97598, AutoSkillit 0.10.1266,
Claude Code 2.1.280, and that commit's scripts/docker/Dockerfile toolchain.
The harness uses MiniMax-M3[1m] at https://api.minimax.io/anthropic, with
CLAUDE_CODE_MAX_CONCURRENT_SUBAGENTS=6; the observed native model was MiniMax-M3.
Host prerequisites are Git, authenticated gh, uv and task for the
worktree/probes, and Docker with a working Buildx builder for the local image.
Check docker buildx version before that build.
Live CI requires repository write access, Actions secrets MINIMAX_API_KEY and
E2E_SANDBOX_TOKEN, and repository variable E2E_IMPLEMENTATION_ISSUE_URL set to
https://github.com/TalonT-Org/autoskillit-e2e-sandbox/issues/1. The sandbox token
must support issue claim/release, branch push/delete, PR open/close and CI reads.
Keep the seed OPEN and unclaimed, and preserve the unchanged sandbox baseline.
Observed sandbox main: d6c143d2494488deeb6680d7d5cf2e55e3e01732.
The fixture asks for normalize_whitespace in sandbox/text.py, focused unittest
cases, task test-check, and an open PR targeting main. The harness's MiniMax
settings-file setup matters; a host provider profile alone is not an equivalent
fleet-launch setup. The earlier local planning-only probe hit a managed-join
scope error; that local result is separate from this CI failure.
Replay the exact CI event
gh run rerun 38073163382 -R TalonT-Org/AutoSkillit
gh run view 38073163382 -R TalonT-Org/AutoSkillit \
--json headSha,attempt,status,conclusion,urlGitHub reruns preserve the original GITHUB_SHA and GITHUB_REF; reruns are
available for 30 days. GitHub rerun documentation.
For a fresh dispatch, create a unique verification ref at the tested commit.
The following is for a Bash session with repository write access; execute the
GitHub ref creation and workflow dispatch as separate tool calls when using an
agent, retaining the displayed variable values between calls.
set -euo pipefail
REPRO_DIR="$PWD/.autoskillit/temp/investigate-allocation-38073163382"
mkdir -p "$REPRO_DIR"
REPRO_SHA=021beadb687a96882fd717dabf36171732a97598
SANDBOX_SHA=d6c143d2494488deeb6680d7d5cf2e55e3e01732
test "$(gh api -X GET repos/TalonT-Org/autoskillit-e2e-sandbox/git/ref/heads/main --jq .object.sha)" = "$SANDBOX_SHA"
VERIFY_REF="verify-issue-5259-$(date -u +%Y%m%d-%H%M%S)"
gh api -X POST repos/TalonT-Org/AutoSkillit/git/refs \
-f ref="refs/heads/$VERIFY_REF" -f sha="$REPRO_SHA" \
--jq .object.sha > "$REPRO_DIR/verify-ref.sha" 2> "$REPRO_DIR/verify-ref.err"
test "$(cat "$REPRO_DIR/verify-ref.sha")" = "$REPRO_SHA"Then dispatch that exact ref:
sleep 1
gh workflow run e2e.yml -R TalonT-Org/AutoSkillit \
--ref "$VERIFY_REF" -f tests=implementation
gh run list -R TalonT-Org/AutoSkillit --workflow e2e.yml \
--branch "$VERIFY_REF" --event workflow_dispatch --limit 1 \
--json databaseId,headSha,status,url > "$REPRO_DIR/dispatched-run.json" 2>&1Once the run appears, confirm its headSha is $REPRO_SHA before treating its
result as a reproduction of this source. If the sandbox baseline check fails,
restore an authorized fixture baseline or use a separately owned equivalent
sandbox; a changed fixture is not the recorded reproduction environment.
The E2E queue serializes scenarios. Model behavior varies: the other commissioning
attempt stopped at a different gate. Capture the actual first failure; identical
model output is not guaranteed.
Retrieve attempt 1 evidence
From an AutoSkillit checkout:
REPRO_DIR="$PWD/.autoskillit/temp/investigate-allocation-38073163382"
mkdir -p "$REPRO_DIR"
gh run download 38073163382 -R TalonT-Org/AutoSkillit \
--name e2e-implementation-38073163382-1 --dir "$REPRO_DIR/artifact"
gh run view 38073163382 -R TalonT-Org/AutoSkillit --attempt 1 --log-failed \
> "$REPRO_DIR/failed.log" 2>&1The artifact name pins attempt 1; use the new attempt suffix for reruns. The
workflow retains artifacts for 14 days, so download them while available.
Inspect out/result.json, out/envelope.json, out/ownership.json,
out/session-concurrency.json and the cited logs/sessions/<session_id>/summary.json.
Inspect the envelope's l3_payload and diagnostic_result; the outer fleet reason
can be less specific than the actual failure.
Verified small probe
Save the complete Python block below as $REPRO_DIR/plan_allocation_probe.py. Obtain the
pinned code and configured environment without initializing a worktree project:
REPRO_SHA=021beadb687a96882fd717dabf36171732a97598
git fetch https://github.com/TalonT-Org/AutoSkillit.git "$REPRO_SHA" \
> "$REPRO_DIR/fetch.log" 2>&1
git worktree add --detach "$REPRO_DIR/source" "$REPRO_SHA"
(cd "$REPRO_DIR/source" && task install-worktree) \
> "$REPRO_DIR/install.log" 2>&1Use task install-worktree, not autoskillit init, inside the worktree.
import json
from autoskillit.core import parse_part_allocation, verify_allocation_evidence
body = """## Implementation Steps
{step}
## Issue Requirement Allocation
| Requirement ID | Allocation | Implementation Step |
| --- | --- | --- |
| P-1 | owned | Step 1 |
"""
results = {}
for name, step in (
("numbered_list", "1. Implement normalization for P-1."),
("recognized_heading", "### Step 1: Implement normalization\n\nImplement P-1."),
):
markdown = body.format(step=step)
rows = parse_part_allocation(markdown, part_key="part-1")
errors = verify_allocation_evidence(markdown, rows)
results[name] = [[requirement, reason.value] for requirement, reason in errors]
assert results["numbered_list"] == [["P-1", "allocation_step_missing"]]
assert results["recognized_heading"] == []
print(json.dumps(results, indent=2))Run:
"$REPRO_DIR/source/.venv/bin/python" "$REPRO_DIR/plan_allocation_probe.py" \
> "$REPRO_DIR/probe.json" 2>&1Verified exit 0: numbered_list=[["P-1","allocation_step_missing"]],
recognized_heading=[]. This demonstrates the parser contract, not the exact
unknown format of the missing original generated plans. A required ### Step N
heading must match the allocation reference, and its body must mention that
requirement ID. A present step missing the ID fails separately with
allocation_step_unreferenced.
For the live failure, recover .autoskillit/temp/make-plan/*.md inside the retained
clone(s). Preserve initial and replanned files, allocation tables and headings,
the plan_set_coverage_gaps inputs and both bind_plan_set responses. The original
uploaded artifacts omitted the plan files; recover those before selecting a fix.
Local Docker reproduction and retained files
This mirrors the tested CI user-image build/harness invocation. Supply
MINIMAX_API_KEY and E2E_SANDBOX_TOKEN through the environment. It was not
separately rerun as part of preparing this issue; the small probe above was.
REPRO_DIR is the temp directory established above.
REPRO_SHA=021beadb687a96882fd717dabf36171732a97598
mkdir -p "$REPRO_DIR"/{build-context,e2e/out,e2e/data,e2e/upload}
git fetch https://github.com/TalonT-Org/AutoSkillit.git "$REPRO_SHA" \
> "$REPRO_DIR/fetch.log" 2>&1
git archive "$REPRO_SHA" | tar -x -C "$REPRO_DIR/build-context"
docker buildx build --load --target user --tag autoskillit-commission-repro:021beadb \
--file "$REPRO_DIR/build-context/scripts/docker/Dockerfile" \
"$REPRO_DIR/build-context" > "$REPRO_DIR/build.log" 2>&1
chmod 0777 "$REPRO_DIR/e2e/out" "$REPRO_DIR/e2e/data" "$REPRO_DIR/e2e/upload"
SANDBOX_SHA=d6c143d2494488deeb6680d7d5cf2e55e3e01732
test "$(gh api -X GET repos/TalonT-Org/autoskillit-e2e-sandbox/git/ref/heads/main --jq .object.sha)" = "$SANDBOX_SHA"
export E2E_IMPLEMENTATION_ISSUE_URL=https://github.com/TalonT-Org/autoskillit-e2e-sandbox/issues/1
REPRO_CONTAINER="autoskillit-commission-repro-$(date -u +%Y%m%d-%H%M%S)"
if docker run --name "$REPRO_CONTAINER" \
--env MINIMAX_API_KEY --env E2E_SANDBOX_TOKEN --env E2E_IMPLEMENTATION_ISSUE_URL \
--volume "$REPRO_DIR/build-context/scripts/e2e:/opt/e2e:ro" \
--volume "$REPRO_DIR/e2e/out:/artifacts" \
--volume "$REPRO_DIR/e2e/data:/home/autoskillit/.local/share/autoskillit" \
autoskillit-commission-repro:021beadb \
python3 /opt/e2e/e2e_harness.py run --test implementation --out /artifacts \
> "$REPRO_DIR/e2e/out/local-run.log" 2>&1; then
REPRO_EXIT=0
else
REPRO_EXIT=$?
fi
printf '%s\n' "$REPRO_EXIT" > "$REPRO_DIR/e2e/out/local-run.exit"The scenario budget is 7,200 seconds. The conditional records the expected
nonzero failure without aborting an errexit shell before file recovery. Retain the
container name and inspect the files before cleanup. The missing --rm is
intentional: the stopped container preserves workspace files omitted by CI upload.
Recover /workspace/sandbox for the lineage records, and
/workspace/autoskillit-runs for generated plans:
docker cp "$REPRO_CONTAINER:/workspace/sandbox" \
"$REPRO_DIR/retained-sandbox" > "$REPRO_DIR/recover-sandbox.log" 2>&1
docker cp "$REPRO_CONTAINER:/workspace/autoskillit-runs" \
"$REPRO_DIR/retained-workspaces" > "$REPRO_DIR/recover-plans.log" 2>&1The first directory is the harness source. /workspace/autoskillit-runs exists
only after clone succeeds: skip its recovery command for the reported pre-clone
failure in #5258. For the observed #5259 failure route, implementation.yaml
registers the clone with status: error, and the clone registry preserves error
clones. Thus a retained stopped container should contain its generated plans.
If they are nevertheless absent, record that unexpected cleanup and perform a
new run with failure-workspace preservation configured before launch.
docker cp cannot recover an already deleted clone; the summaries alone cannot
establish the exact original plan-format defect.
Before sharing new harness output, use the same artifact redactor as CI:
docker run --rm --env MINIMAX_API_KEY --env E2E_SANDBOX_TOKEN \
--volume "$REPRO_DIR/build-context/scripts/e2e:/opt/e2e:ro" \
--volume "$REPRO_DIR/e2e:/e2e" \
autoskillit-commission-repro:021beadb \
python3 /opt/e2e/e2e_harness.py redact --dest /e2e/upload \
--secret-env MINIMAX_API_KEY --secret-env E2E_SANDBOX_TOKEN \
/e2e/out /e2e/data/logs > "$REPRO_DIR/redact.log" 2>&1Keep raw recovered workspaces local; that redactor covers harness output/session
logs rather than arbitrary recovered checkout files. After investigation, verify
the seed remains OPEN/unclaimed and clean up each proven run-owned PR and branch.
At direct remote-ref inspection (snapshot saved 2026-10-10 19:26 UTC), these
fixture refs both pointed to d6c143d2494488deeb6680d7d5cf2e55e3e01732:
refs/heads/e2e-implementation-fixture-normalize-whitespace/1
refs/heads/e2e-implementation-fixture-normalize-whitespace/1-2
This is an inventory observation, not proof that this commissioning run created
both refs. Both ownership artifacts report branches: []. The seed was OPEN with
the fail label and without in-progress; it was not unlabeled. The second run's
peak/coverage report omitted its known planning children. These harness-evidence
gaps are recorded on PR #5255 and are separate from this issue's production cause.
Inspect current refs with gh api -X GET repos/TalonT-Org/autoskillit-e2e-sandbox/git/matching-refs/heads/.
Share only the redacted $REPRO_DIR/e2e/upload output. Fetch, install, build,
recovery and redactor logs outside that tree are raw and stay local. The local run
log is now under e2e/out, so the documented redaction includes it.
After recovering the files and after the Actions run is finished, remove the
stopped container you created:
docker rm "$REPRO_CONTAINER"If the fresh-ref method was used, also remove that investigator-owned verification
ref after the run completes. In an agent session, use the literal recorded ref
from the creation step when issuing this single mutation:
sleep 1
gh api -X DELETE "repos/TalonT-Org/AutoSkillit/git/refs/heads/$VERIFY_REF"If the sandbox harness left resources, supply RUN_OWNED_PR and RUN_OWNED_BRANCH
only after proving their identity from that run's PR, branch and head evidence.
Omit the PR command when that run created no PR; recheck the branch head against
the captured head before deleting it. Do not use the two inventory names above as
ownership proof. These are separate mutating commands:
gh pr close "$RUN_OWNED_PR" -R TalonT-Org/autoskillit-e2e-sandboxsleep 1
gh api -X DELETE "repos/TalonT-Org/autoskillit-e2e-sandbox/git/refs/heads/$RUN_OWNED_BRANCH"Source at the tested commit
- src/autoskillit/core/planset/allocation.py
- src/autoskillit/core/io/markdown_sections.py
- src/autoskillit/skills_extended/make-plan/SKILL.md
Investigation findings (2026-10-10)
Investigated on develop at 265f1e00e. The relevant code (planset, markdown_sections,
make-plan, rectify, the three plan-binding recipes, server/) is identical at the tested
commit 021beadb. Evidence: code and git history, the attempt 1 artifact of run
38073163382, and read-only probes that call the real verifier. No live rerun or Docker
reproduction was performed, and the plan files were not recovered. The findings were
re-checked by two adversarial review passes, each claim against its source.
Conclusion
Two defects compound. Both shipped together in #5102 (04d669217, 2026-09-19).
- Cause A: the binder enforces a plan-part format that the producer is never told.
Conformance is accidental. - Cause B: a failed bind gives the replan no feedback. The one permitted replan was
an uninformed retry of the same prompt.
Answers to the four possibilities in "Expected outcome":
| Possibility | Finding |
|---|---|
| Headings/references violate the parser contract | Yes, by construction. allocation_step_missing is emitted only when the cell equals no ### Step … heading key, and all 14 rows returned it. The exact variant used is unknown. |
| Producer guidance is insufficient | Yes. Primary cause. |
| Parser is at fault | No. It behaves as written and as tested. |
| Replanning lacks the needed feedback | Yes, on every failure path this run could take. |
Cause A: unstated producer contract
What bind_plan_set requires of each plan part (core/planset/allocation.py:19-67,
core/io/markdown_sections.py:10-55):
| Rule | Failure when broken |
|---|---|
Exactly one ## Issue Requirement Allocation section whose non-blank lines are only header, separator and rows |
ValueError, returned as coverage_failed |
The Implementation Step cell equals, as an exact string, the key of a heading matching ^###\s+(Step\s+[1-9][0-9]*(?:\.[1-9][0-9]*)*)(?::[^\n]*)?$ |
allocation_step_missing |
| The whole Requirement ID cell occurs as a substring of that step's block | allocation_step_unreferenced |
Allocation is owned or shared |
ValueError, returned as coverage_failed |
What make-plan tells the model (skills_extended/make-plan/SKILL.md:440-449, repeated at
:469-478): ## Implementation Steps is the free-form placeholder
{Ordered steps, each making one or more of the above tests pass}, and the table cell is
Step {N.M}. The file has no line starting with ###. It shows the section name, the
header and the owned | shared vocabulary, but not the step-heading form, the
single-key cell, the bare ID cell, the ID-in-step-text rule, or that the section may hold
only the table. rectify's template (skills_extended/rectify/SKILL.md:295-309) shows
### Step 1: and ### Step 2: headings with the same Step {N.M} cell, so a faithful
Step 1.1 reference fails there too.
No other channel delivers the contract: the bound invocation packet carries five inputs
only, the make-plan entry in recipe/skill_contracts.yaml constrains emitted output
tokens, and the plan-auditor agents do not mention headings or the allocation table.
Probe results against the current verifier:
| Plan shape | Result |
|---|---|
### Step 1: title, cell Step 1, bare ID in the step text |
pass |
### Step 1.1: title, cell Step 1.1 |
pass |
| Numbered-list steps | allocation_step_missing |
### Step 1: title with listed sub-steps, cell Step 1.1 (the most template-faithful shape) |
allocation_step_missing |
#### Step 1.1: title, ### Step 1 — title, ### Step 1.1 - title |
allocation_step_missing |
Cell Steps 1, 2, Step 1.1, Step 1.2, 1.1, or a backticked key |
allocation_step_missing |
ID cell P-1 (Add fn) or P-1: Add fn |
allocation_step_unreferenced |
| One prose sentence before the table | ValueError: table rows must be pipe-delimited |
Running the same verifier over the 62 plan files with an allocation section retained in a
development checkout (dated 2026-09-21 to 2026-10-10): none passes. 49 fail with
allocation_step_missing only, 2 with both reasons, 1 with allocation_step_unreferenced
only, 10 at table parse. Typical shapes are ### Step 1 — Title, cells such as
Steps 1, 3, 7, 8, annotated ID cells, and a sentence above the table. Those plans were
written from the same template but were never submitted to the gate, and the models that
wrote them are not recorded.
Cause B: empty replan feedback
coverage_failed is returned from three sites in server/_plan_set_materializer.py:
| Site | Trigger | coverage_gaps |
|---|---|---|
:365-366 |
allocation table parse error | "" |
:368-373 |
allocation evidence errors | "" |
:404-425 |
sealed aggregate coverage did not pass | built by _coverage_gaps (:69-82), which omits parts_without_obligations |
The first two go through _failure(reason, error) (:47-48), so the detail is only in
error. For an unenumerated or no-issue inventory the aggregate check can fail only on
parts_without_obligations (core/planset/coverage.py:29-34), so the third site is
empty as well. The sandbox seed issue extracts as unenumerated with zero requirements:
in this run no failure path could have produced gap text.
implementation.yaml:498 captures only result.coverage_gaps, :502-503 routes
coverage_failed to check_replan_iteration (max_iterations: '2', which is one
replan), and :434 passes the captured value to make-plan. The artifact confirms both
planning sessions received plan_set_coverage_gaps: "". This answers "Replanning
evidence and open question": the empty field is produced by the binder's failure path,
not lost in the hand-off.
Forwarding error alone would not be enough. make-plan describes the input as
"requirement gaps … allocate every listed requirement", which does not tell a model what
allocation_step_missing means. Cause B cannot be repaired without Cause A.
What the artifact shows about the failed plans
- The initial session
ee512612…emittednormalize_whitespace_plan_2026-10-10_175932.md
and the replan9146aa3f…emittednormalize_whitespace_plan_2026-10-10_175632.md:
two different files, one part each. No plan text is retained. - Both binds returned
coverage_failed; each was followed by thecheck_replan_iteration
call. - Telemetry records the two bind results at 1615 and 1701 bytes. The handler returns the
serialized result unwrapped: 559 bytes with an empty error, 982 for fourteen bare-ID
rows, 1604 for the annotated error string in the retained diagnostic. The observed
sizes fit annotated ID cells such asP-1 (description)and exclude bare ones. Those
plans would failallocation_step_unreferencedeven with correct headings. - The retained
diagnostic_resultis one bind result relayed by the orchestrating model.
Nothing attributes it to the first or the second bind.
Open risk: stale plan part on replan
The plan step captures plan_parts with capture_list (implementation.yaml:442-443),
which recipe/schema.py:200-202 defines as accumulating. bind_plan_set binds
context.plan_parts (:489). If the replan's part was appended to the failed first
part, the second bind re-verified the first plan, and no replan can pass in this loop.
The artifact does not settle it: two fully failing parts are excluded by result size,
but a replan-only bind and a stale-part bind with one more failing row both fit. The
orchestrator transcript was not retained.
Other paths that reach the same checks
salvage_plan(implementation.yaml:459-484) accepts any non-empty plan file after a
planning context limit and routes tobind_plan_set.renew_plan_set(implementation.yaml:576-593) and the renewal steps in
remediation.yamlre-run the checks afterdry-walkthroughedits. A failure there is
terminal.bind_group_parts(implementation-groups.yaml:303-323) has nocoverage_failed
route, so the same failure is terminal in grouped planning.remediation.yamlwires rectify and make-plan tobind_plan_setthe same way as
implementation.yaml.
Why tests did not catch it
- Every positive plan fixture is hand-written in the binder's shape
(### Step 1.1: …, bare ID in the text). - No test feeds the make-plan or rectify template through
parse_part_allocationand
verify_allocation_evidence. SKILL.md is checked by substring only. coverage_gapsis asserted only for anunassignedgap
(tests/server/test_plan_set_materializer.py:80, 121). The allocation branches and the
parts_without_obligationscase have no gap-text assertion.- Recipe tests check only that the replan loop is bounded.
- No end-to-end run exercised make-plan into
bind_plan_setwith a real model before
the Add seeded sandbox implementation E2E validation #5255 commissioning scenario, and the artifact upload omits plan files.
Recommended direction
- State the plan-part contract to the producer and bind that statement to the
parser. make-plan and rectify should show the exact step-heading form, one heading
key per row, a bare ID cell, the ID in the step's text, and a table-only section. Tie
it to the parser with a contract test that runs the template's worked example through
the real verifier.dry-walkthroughedits must keep the same form. - Make every
coverage_failedresult carry actionable feedback, at all three
sites, naming the row, the offending cell and the expected form. Guard that invariant
with a test, and update make-plan'splan_set_coverage_gapsdescription to cover
format failures. Decide what the bind sites without a replan route should do. - Confirm the replan binds only the replacement plan set, with a recipe-level test.
- Keep the gate strict. The step cell is stored verbatim in the content-addressed
authority and handed to dry-walkthrough. Accepting multi-step cells or other
separators is a separate, deliberate decision. - Verify with a real emitted plan before rerunning commissioning, and preserve plan
files and the orchestrator transcript in failure artifacts.
Files that move together: the three recipe YAML files and their tracked compiled .json
counterparts, recipe/skill_contracts.yaml, both skill templates, and the tests above.
Still unknown
- The text of the two MiniMax plans. This is not needed to choose the direction, since
the producer is never told the contract, but only a real run can show whether those
plans had a further defect. - What
context.plan_partsheld at the second bind. - Whether MiniMax-M3 follows the contract once it is stated.
- Whether any pipeline run has passed a sealed bind since fix: bind multipart plans to sealed issue coverage authority #5102. None was found in
local logs.
Live real-model bind verification (2026-10-11)
P-10 has now been performed successfully in the persistent Docker test container.
- AutoSkillit
0.10.1269, develop commit0bf84f34b8a0a032e2a3beff3ff198c64daa1907, containing merged PR Fix plan-part contract binding, replan feedback, and E2E evidence #5262. - MiniMax
MiniMax-M3[1m]ran the actualmake-planskill for the original reusable sandbox seed issue. Sandbox main remainedd6c143d2494488deeb6680d7d5cf2e55e3e01732, matching the recorded reproduction. - All three declared planning readers finished with success, and the join ledger reported complete.
- The first
bind_plan_setcall returnedsuccess: True,plan_set_state: sealed,coverage_status: pass, and emptycoverage_gaps. No replan was needed. - Generated plan:
/workspace/sandbox/.autoskillit/temp/make-plan/normalize-whitespace-fixture_plan_2026-10-11_181500.md. - Bound authority:
/workspace/sandbox/.autoskillit/temp/plan-set-authority/planset-8ae380e5ba203dc7b2c2f1e7/7f90d2d3104f5dcdf54d4c42.json; digestsha256:7f90d2d3104f5dcdf54d4c420b5abb08abcf6e7ea3520490a6c6ea19c6b16790. - Controller session:
7fbd4146-bfd5-4891-a379-d43b6fe00876; planning worker:2121101f-86bf-4343-b1ab-13e487159aa8. - Local evidence report:
/home/talon/projects/generic_automation_mcp/.autoskillit/temp/resolve-review/real-model-check/report_5259_live.md; the actual binder tool receipt, generated plan, authority JSON, and redacted transcripts are retained beside it.
Earlier local attempts were blocked by container registration drift: a stale settings-level package hook wrote an unresolved skill binding before the valid projected hook ran. The supported autoskillit install registration repair removed the stale copy; projected guards stayed active. A separate existing log-locator defect for cwd paths containing a dot was exposed by the temporary nested fixture; the final successful check used the documented /workspace/sandbox location.
This verifies real planning and sealed binding only. No implementation, sandbox test execution, generated PR, or full #5255 commissioning run was performed. Full commissioning retains its separate requirements.
Additional live verification — 2026-10-16, head20ee39237
Both fresh MiniMax-M3 implementation attempts passed make-plan and sealed plan binding at 20ee39237e8e89a8d21f403c3a825273fcac6bf7, which includes develop 75141403a04ca98bb4c7e6c3af9ce03eaf838f03 and the #5283 fix. This confirms the binding repair in the live implementation scenario; it does not claim full commissioning success.
In run38960061259, root session 5a5cdc35-9fdf-4480-92c2-78d5fbcb8592, paired bind_plan_set result at retained transcript record74 returns success=true, sealed, coverage_status=pass and empty coverage gaps. Plan: /workspace/autoskillit-runs/impl-20261016-201244-772303/.autoskillit/temp/make-plan/normalize_whitespace_plan_2026-10-16_202617.md. Authority: /workspace/autoskillit-runs/impl-20261016-201244-772303/.autoskillit/temp/plan-set-authority/planset-64a04c4d1b9e82df17a094df/031b15882fcbadeca16ea309.json; digest sha256:031b15882fcbadeca16ea309668bb1113aa3f2ebeba6ae4efb6b94238448bc95.
Run38960069882 independently passed planning/binding and failed later at dry-walkthrough zero-write adjudication. The first attempt failed later after divergent recipe progression/review. Neither result is a recurrence of this binding failure. PR5255 retains the commissioning acceptance record.
Activity
Fix plan-part contract binding, replan feedback, and E2E evidence (#5262
Fix remaining runtime-state visibility and failure reporting (#5263)