Skip to content

make-plan output fails bind_plan_set with allocation_step_missing #5259

Closed

Description

@Trecek

make-plan output fails bind_plan_set with allocation_step_missing

Problem

PR #5255's implementation E2E commissioning run now passes planning admission and
executes MiniMax make-plan, but the returned plans fail the sealed plan-set
coverage gate. Replanning returns the same class of failure and exhausts the
recipe's replan budget before implementation begins.

Live reproduction

out/result.json records passed=false, outcome=failed, and no expected
findings. The fleet envelope's diagnostic result identifies step=bind_plan_set,
reason=coverage_failed, and allocation_step_missing for P-1 through P-14.
The terminal reports replanning exhausted, followed by issue release and failure
registration.

Retained summaries for sessions 9146aa3f-49b7-48e9-a649-7f547a833176 and
ee512612-95c1-4c5e-b4f7-2540a6b5e001 show successful make-plan execution,
planning-auditor calls, and Write calls. This is distinct from the output-dir
admission defect fixed by #5257 / #5256.

Investigation starting points

  • core/planset/allocation.py:verify_allocation_evidence requires allocation
    references to name implementation headings recognized by STEP_HEADING_RE.
  • core/io/markdown_sections.py:STEP_HEADING_RE accepts ### Step N and
    ### Step N: description headings.
  • The make-plan skill's plan-structure templates show allocation references
    such as Step N.M, but leave the implementation-steps section as prose
    placeholders without an explicit matching heading example.

The downloaded artifact does not include the plan files, so a specific generated
heading mismatch is not yet proven. The observed failure is that all referenced
implementation steps are missing from the binder's parsed heading map, rather
than a proven missing frontmatter key.

Replanning evidence and open question

Both successful planning-session summaries encode plan_set_coverage_gaps
as "" inside the bound /skill_command input packet. The final bind
diagnostic's /l3_payload/diagnostic_result/coverage_gaps is also "".
That diagnostic is retained L3 envelope data parsed from stdout; this review
does not treat it as a freshly verified raw binder response. Capture the
actual replies and generated plans on reproduction.
implementation.yaml:434,498 passes that captured field back into make-plan;
the skill documents it as the failed-bind gap input. Thus that named input
carried no gap text on replanning. Check the first bind response and the
diagnostic-to-input handoff; whether an empty field is intentional for this
failure or defective is not established.

There are two retained successful planning executions: the initial plan
(ee512612-95c1-4c5e-b4f7-2540a6b5e001) and one replan
(9146aa3f-49b7-48e9-a649-7f547a833176). The recipe's max_iterations: 2
loop exhausts when the incremented count reaches 2; do not interpret the
terminal's loose iteration wording as two additional retries.

Expected outcome

Recover the actual plans and determine whether their headings/references
violate the parser contract, the producer guidance is insufficient, the parser
is at fault, or replanning lacks the needed feedback. Repair the responsible
path while preserving requirement coverage enforcement, and verify a real
emitted plan passes bind_plan_set before rerunning commissioning.

Refs #5233 and PR #5255. Classification: bug, recipe:remediation.

Reproduction prerequisites

Pinned source: 021beadb687a96882fd717dabf36171732a97598, AutoSkillit 0.10.1266,
Claude Code 2.1.280, and that commit's scripts/docker/Dockerfile toolchain.
The harness uses MiniMax-M3[1m] at https://api.minimax.io/anthropic, with
CLAUDE_CODE_MAX_CONCURRENT_SUBAGENTS=6; the observed native model was MiniMax-M3.

Host prerequisites are Git, authenticated gh, uv and task for the
worktree/probes, and Docker with a working Buildx builder for the local image.
Check docker buildx version before that build.

Live CI requires repository write access, Actions secrets MINIMAX_API_KEY and
E2E_SANDBOX_TOKEN, and repository variable E2E_IMPLEMENTATION_ISSUE_URL set to
https://github.com/TalonT-Org/autoskillit-e2e-sandbox/issues/1. The sandbox token
must support issue claim/release, branch push/delete, PR open/close and CI reads.
Keep the seed OPEN and unclaimed, and preserve the unchanged sandbox baseline.
Observed sandbox main: d6c143d2494488deeb6680d7d5cf2e55e3e01732.

The fixture asks for normalize_whitespace in sandbox/text.py, focused unittest
cases, task test-check, and an open PR targeting main. The harness's MiniMax
settings-file setup matters; a host provider profile alone is not an equivalent
fleet-launch setup. The earlier local planning-only probe hit a managed-join
scope error; that local result is separate from this CI failure.

Replay the exact CI event

gh run rerun 38073163382 -R TalonT-Org/AutoSkillit
gh run view 38073163382 -R TalonT-Org/AutoSkillit \
  --json headSha,attempt,status,conclusion,url

GitHub reruns preserve the original GITHUB_SHA and GITHUB_REF; reruns are
available for 30 days. GitHub rerun documentation.

For a fresh dispatch, create a unique verification ref at the tested commit.
The following is for a Bash session with repository write access; execute the
GitHub ref creation and workflow dispatch as separate tool calls when using an
agent, retaining the displayed variable values between calls.

set -euo pipefail
REPRO_DIR="$PWD/.autoskillit/temp/investigate-allocation-38073163382"
mkdir -p "$REPRO_DIR"
REPRO_SHA=021beadb687a96882fd717dabf36171732a97598
SANDBOX_SHA=d6c143d2494488deeb6680d7d5cf2e55e3e01732
test "$(gh api -X GET repos/TalonT-Org/autoskillit-e2e-sandbox/git/ref/heads/main --jq .object.sha)" = "$SANDBOX_SHA"
VERIFY_REF="verify-issue-5259-$(date -u +%Y%m%d-%H%M%S)"
gh api -X POST repos/TalonT-Org/AutoSkillit/git/refs \
  -f ref="refs/heads/$VERIFY_REF" -f sha="$REPRO_SHA" \
  --jq .object.sha > "$REPRO_DIR/verify-ref.sha" 2> "$REPRO_DIR/verify-ref.err"
test "$(cat "$REPRO_DIR/verify-ref.sha")" = "$REPRO_SHA"

Then dispatch that exact ref:

sleep 1
gh workflow run e2e.yml -R TalonT-Org/AutoSkillit \
  --ref "$VERIFY_REF" -f tests=implementation
gh run list -R TalonT-Org/AutoSkillit --workflow e2e.yml \
  --branch "$VERIFY_REF" --event workflow_dispatch --limit 1 \
  --json databaseId,headSha,status,url > "$REPRO_DIR/dispatched-run.json" 2>&1

Once the run appears, confirm its headSha is $REPRO_SHA before treating its
result as a reproduction of this source. If the sandbox baseline check fails,
restore an authorized fixture baseline or use a separately owned equivalent
sandbox; a changed fixture is not the recorded reproduction environment.

The E2E queue serializes scenarios. Model behavior varies: the other commissioning
attempt stopped at a different gate. Capture the actual first failure; identical
model output is not guaranteed.

Retrieve attempt 1 evidence

From an AutoSkillit checkout:

REPRO_DIR="$PWD/.autoskillit/temp/investigate-allocation-38073163382"
mkdir -p "$REPRO_DIR"
gh run download 38073163382 -R TalonT-Org/AutoSkillit \
  --name e2e-implementation-38073163382-1 --dir "$REPRO_DIR/artifact"
gh run view 38073163382 -R TalonT-Org/AutoSkillit --attempt 1 --log-failed \
  > "$REPRO_DIR/failed.log" 2>&1

The artifact name pins attempt 1; use the new attempt suffix for reruns. The
workflow retains artifacts for 14 days, so download them while available.
Inspect out/result.json, out/envelope.json, out/ownership.json,
out/session-concurrency.json and the cited logs/sessions/<session_id>/summary.json.
Inspect the envelope's l3_payload and diagnostic_result; the outer fleet reason
can be less specific than the actual failure.

Verified small probe

Save the complete Python block below as $REPRO_DIR/plan_allocation_probe.py. Obtain the
pinned code and configured environment without initializing a worktree project:

REPRO_SHA=021beadb687a96882fd717dabf36171732a97598
git fetch https://github.com/TalonT-Org/AutoSkillit.git "$REPRO_SHA" \
  > "$REPRO_DIR/fetch.log" 2>&1
git worktree add --detach "$REPRO_DIR/source" "$REPRO_SHA"
(cd "$REPRO_DIR/source" && task install-worktree) \
  > "$REPRO_DIR/install.log" 2>&1

Use task install-worktree, not autoskillit init, inside the worktree.

import json

from autoskillit.core import parse_part_allocation, verify_allocation_evidence

body = """## Implementation Steps
{step}

## Issue Requirement Allocation
| Requirement ID | Allocation | Implementation Step |
| --- | --- | --- |
| P-1 | owned | Step 1 |
"""
results = {}
for name, step in (
    ("numbered_list", "1. Implement normalization for P-1."),
    ("recognized_heading", "### Step 1: Implement normalization\n\nImplement P-1."),
):
    markdown = body.format(step=step)
    rows = parse_part_allocation(markdown, part_key="part-1")
    errors = verify_allocation_evidence(markdown, rows)
    results[name] = [[requirement, reason.value] for requirement, reason in errors]
assert results["numbered_list"] == [["P-1", "allocation_step_missing"]]
assert results["recognized_heading"] == []
print(json.dumps(results, indent=2))

Run:

"$REPRO_DIR/source/.venv/bin/python" "$REPRO_DIR/plan_allocation_probe.py" \
  > "$REPRO_DIR/probe.json" 2>&1

Verified exit 0: numbered_list=[["P-1","allocation_step_missing"]],
recognized_heading=[]. This demonstrates the parser contract, not the exact
unknown format of the missing original generated plans. A required ### Step N
heading must match the allocation reference, and its body must mention that
requirement ID. A present step missing the ID fails separately with
allocation_step_unreferenced.

For the live failure, recover .autoskillit/temp/make-plan/*.md inside the retained
clone(s). Preserve initial and replanned files, allocation tables and headings,
the plan_set_coverage_gaps inputs and both bind_plan_set responses. The original
uploaded artifacts omitted the plan files; recover those before selecting a fix.

Local Docker reproduction and retained files

This mirrors the tested CI user-image build/harness invocation. Supply
MINIMAX_API_KEY and E2E_SANDBOX_TOKEN through the environment. It was not
separately rerun as part of preparing this issue; the small probe above was.
REPRO_DIR is the temp directory established above.

REPRO_SHA=021beadb687a96882fd717dabf36171732a97598
mkdir -p "$REPRO_DIR"/{build-context,e2e/out,e2e/data,e2e/upload}
git fetch https://github.com/TalonT-Org/AutoSkillit.git "$REPRO_SHA" \
  > "$REPRO_DIR/fetch.log" 2>&1
git archive "$REPRO_SHA" | tar -x -C "$REPRO_DIR/build-context"
docker buildx build --load --target user --tag autoskillit-commission-repro:021beadb \
  --file "$REPRO_DIR/build-context/scripts/docker/Dockerfile" \
  "$REPRO_DIR/build-context" > "$REPRO_DIR/build.log" 2>&1
chmod 0777 "$REPRO_DIR/e2e/out" "$REPRO_DIR/e2e/data" "$REPRO_DIR/e2e/upload"
SANDBOX_SHA=d6c143d2494488deeb6680d7d5cf2e55e3e01732
test "$(gh api -X GET repos/TalonT-Org/autoskillit-e2e-sandbox/git/ref/heads/main --jq .object.sha)" = "$SANDBOX_SHA"
export E2E_IMPLEMENTATION_ISSUE_URL=https://github.com/TalonT-Org/autoskillit-e2e-sandbox/issues/1
REPRO_CONTAINER="autoskillit-commission-repro-$(date -u +%Y%m%d-%H%M%S)"
if docker run --name "$REPRO_CONTAINER" \
  --env MINIMAX_API_KEY --env E2E_SANDBOX_TOKEN --env E2E_IMPLEMENTATION_ISSUE_URL \
  --volume "$REPRO_DIR/build-context/scripts/e2e:/opt/e2e:ro" \
  --volume "$REPRO_DIR/e2e/out:/artifacts" \
  --volume "$REPRO_DIR/e2e/data:/home/autoskillit/.local/share/autoskillit" \
  autoskillit-commission-repro:021beadb \
  python3 /opt/e2e/e2e_harness.py run --test implementation --out /artifacts \
  > "$REPRO_DIR/e2e/out/local-run.log" 2>&1; then
  REPRO_EXIT=0
else
  REPRO_EXIT=$?
fi
printf '%s\n' "$REPRO_EXIT" > "$REPRO_DIR/e2e/out/local-run.exit"

The scenario budget is 7,200 seconds. The conditional records the expected
nonzero failure without aborting an errexit shell before file recovery. Retain the
container name and inspect the files before cleanup. The missing --rm is
intentional: the stopped container preserves workspace files omitted by CI upload.
Recover /workspace/sandbox for the lineage records, and
/workspace/autoskillit-runs for generated plans:

docker cp "$REPRO_CONTAINER:/workspace/sandbox" \
  "$REPRO_DIR/retained-sandbox" > "$REPRO_DIR/recover-sandbox.log" 2>&1
docker cp "$REPRO_CONTAINER:/workspace/autoskillit-runs" \
  "$REPRO_DIR/retained-workspaces" > "$REPRO_DIR/recover-plans.log" 2>&1

The first directory is the harness source. /workspace/autoskillit-runs exists
only after clone succeeds: skip its recovery command for the reported pre-clone
failure in #5258. For the observed #5259 failure route, implementation.yaml
registers the clone with status: error, and the clone registry preserves error
clones. Thus a retained stopped container should contain its generated plans.
If they are nevertheless absent, record that unexpected cleanup and perform a
new run with failure-workspace preservation configured before launch.
docker cp cannot recover an already deleted clone; the summaries alone cannot
establish the exact original plan-format defect.

Before sharing new harness output, use the same artifact redactor as CI:

docker run --rm --env MINIMAX_API_KEY --env E2E_SANDBOX_TOKEN \
  --volume "$REPRO_DIR/build-context/scripts/e2e:/opt/e2e:ro" \
  --volume "$REPRO_DIR/e2e:/e2e" \
  autoskillit-commission-repro:021beadb \
  python3 /opt/e2e/e2e_harness.py redact --dest /e2e/upload \
    --secret-env MINIMAX_API_KEY --secret-env E2E_SANDBOX_TOKEN \
    /e2e/out /e2e/data/logs > "$REPRO_DIR/redact.log" 2>&1

Keep raw recovered workspaces local; that redactor covers harness output/session
logs rather than arbitrary recovered checkout files. After investigation, verify
the seed remains OPEN/unclaimed and clean up each proven run-owned PR and branch.
At direct remote-ref inspection (snapshot saved 2026-10-10 19:26 UTC), these
fixture refs both pointed to d6c143d2494488deeb6680d7d5cf2e55e3e01732:

refs/heads/e2e-implementation-fixture-normalize-whitespace/1
refs/heads/e2e-implementation-fixture-normalize-whitespace/1-2

This is an inventory observation, not proof that this commissioning run created
both refs. Both ownership artifacts report branches: []. The seed was OPEN with
the fail label and without in-progress; it was not unlabeled. The second run's
peak/coverage report omitted its known planning children. These harness-evidence
gaps are recorded on PR #5255 and are separate from this issue's production cause.
Inspect current refs with gh api -X GET repos/TalonT-Org/autoskillit-e2e-sandbox/git/matching-refs/heads/.

Share only the redacted $REPRO_DIR/e2e/upload output. Fetch, install, build,
recovery and redactor logs outside that tree are raw and stay local. The local run
log is now under e2e/out, so the documented redaction includes it.

After recovering the files and after the Actions run is finished, remove the
stopped container you created:

docker rm "$REPRO_CONTAINER"

If the fresh-ref method was used, also remove that investigator-owned verification
ref after the run completes. In an agent session, use the literal recorded ref
from the creation step when issuing this single mutation:

sleep 1
gh api -X DELETE "repos/TalonT-Org/AutoSkillit/git/refs/heads/$VERIFY_REF"

If the sandbox harness left resources, supply RUN_OWNED_PR and RUN_OWNED_BRANCH
only after proving their identity from that run's PR, branch and head evidence.
Omit the PR command when that run created no PR; recheck the branch head against
the captured head before deleting it. Do not use the two inventory names above as
ownership proof. These are separate mutating commands:

gh pr close "$RUN_OWNED_PR" -R TalonT-Org/autoskillit-e2e-sandbox
sleep 1
gh api -X DELETE "repos/TalonT-Org/autoskillit-e2e-sandbox/git/refs/heads/$RUN_OWNED_BRANCH"

Source at the tested commit

Investigation findings (2026-10-10)

Investigated on develop at 265f1e00e. The relevant code (planset, markdown_sections,
make-plan, rectify, the three plan-binding recipes, server/) is identical at the tested
commit 021beadb. Evidence: code and git history, the attempt 1 artifact of run
38073163382, and read-only probes that call the real verifier. No live rerun or Docker
reproduction was performed, and the plan files were not recovered. The findings were
re-checked by two adversarial review passes, each claim against its source.

Conclusion

Two defects compound. Both shipped together in #5102 (04d669217, 2026-09-19).

  • Cause A: the binder enforces a plan-part format that the producer is never told.
    Conformance is accidental.
  • Cause B: a failed bind gives the replan no feedback. The one permitted replan was
    an uninformed retry of the same prompt.

Answers to the four possibilities in "Expected outcome":

Possibility Finding
Headings/references violate the parser contract Yes, by construction. allocation_step_missing is emitted only when the cell equals no ### Step … heading key, and all 14 rows returned it. The exact variant used is unknown.
Producer guidance is insufficient Yes. Primary cause.
Parser is at fault No. It behaves as written and as tested.
Replanning lacks the needed feedback Yes, on every failure path this run could take.

Cause A: unstated producer contract

What bind_plan_set requires of each plan part (core/planset/allocation.py:19-67,
core/io/markdown_sections.py:10-55):

Rule Failure when broken
Exactly one ## Issue Requirement Allocation section whose non-blank lines are only header, separator and rows ValueError, returned as coverage_failed
The Implementation Step cell equals, as an exact string, the key of a heading matching ^###\s+(Step\s+[1-9][0-9]*(?:\.[1-9][0-9]*)*)(?::[^\n]*)?$ allocation_step_missing
The whole Requirement ID cell occurs as a substring of that step's block allocation_step_unreferenced
Allocation is owned or shared ValueError, returned as coverage_failed

What make-plan tells the model (skills_extended/make-plan/SKILL.md:440-449, repeated at
:469-478): ## Implementation Steps is the free-form placeholder
{Ordered steps, each making one or more of the above tests pass}, and the table cell is
Step {N.M}. The file has no line starting with ###. It shows the section name, the
header and the owned | shared vocabulary, but not the step-heading form, the
single-key cell, the bare ID cell, the ID-in-step-text rule, or that the section may hold
only the table. rectify's template (skills_extended/rectify/SKILL.md:295-309) shows
### Step 1: and ### Step 2: headings with the same Step {N.M} cell, so a faithful
Step 1.1 reference fails there too.

No other channel delivers the contract: the bound invocation packet carries five inputs
only, the make-plan entry in recipe/skill_contracts.yaml constrains emitted output
tokens, and the plan-auditor agents do not mention headings or the allocation table.

Probe results against the current verifier:

Plan shape Result
### Step 1: title, cell Step 1, bare ID in the step text pass
### Step 1.1: title, cell Step 1.1 pass
Numbered-list steps allocation_step_missing
### Step 1: title with listed sub-steps, cell Step 1.1 (the most template-faithful shape) allocation_step_missing
#### Step 1.1: title, ### Step 1 — title, ### Step 1.1 - title allocation_step_missing
Cell Steps 1, 2, Step 1.1, Step 1.2, 1.1, or a backticked key allocation_step_missing
ID cell P-1 (Add fn) or P-1: Add fn allocation_step_unreferenced
One prose sentence before the table ValueError: table rows must be pipe-delimited

Running the same verifier over the 62 plan files with an allocation section retained in a
development checkout (dated 2026-09-21 to 2026-10-10): none passes. 49 fail with
allocation_step_missing only, 2 with both reasons, 1 with allocation_step_unreferenced
only, 10 at table parse. Typical shapes are ### Step 1 — Title, cells such as
Steps 1, 3, 7, 8, annotated ID cells, and a sentence above the table. Those plans were
written from the same template but were never submitted to the gate, and the models that
wrote them are not recorded.

Cause B: empty replan feedback

coverage_failed is returned from three sites in server/_plan_set_materializer.py:

Site Trigger coverage_gaps
:365-366 allocation table parse error ""
:368-373 allocation evidence errors ""
:404-425 sealed aggregate coverage did not pass built by _coverage_gaps (:69-82), which omits parts_without_obligations

The first two go through _failure(reason, error) (:47-48), so the detail is only in
error. For an unenumerated or no-issue inventory the aggregate check can fail only on
parts_without_obligations (core/planset/coverage.py:29-34), so the third site is
empty as well. The sandbox seed issue extracts as unenumerated with zero requirements:
in this run no failure path could have produced gap text.

implementation.yaml:498 captures only result.coverage_gaps, :502-503 routes
coverage_failed to check_replan_iteration (max_iterations: '2', which is one
replan), and :434 passes the captured value to make-plan. The artifact confirms both
planning sessions received plan_set_coverage_gaps: "". This answers "Replanning
evidence and open question": the empty field is produced by the binder's failure path,
not lost in the hand-off.

Forwarding error alone would not be enough. make-plan describes the input as
"requirement gaps … allocate every listed requirement", which does not tell a model what
allocation_step_missing means. Cause B cannot be repaired without Cause A.

What the artifact shows about the failed plans

  • The initial session ee512612… emitted normalize_whitespace_plan_2026-10-10_175932.md
    and the replan 9146aa3f… emitted normalize_whitespace_plan_2026-10-10_175632.md:
    two different files, one part each. No plan text is retained.
  • Both binds returned coverage_failed; each was followed by the check_replan_iteration
    call.
  • Telemetry records the two bind results at 1615 and 1701 bytes. The handler returns the
    serialized result unwrapped: 559 bytes with an empty error, 982 for fourteen bare-ID
    rows, 1604 for the annotated error string in the retained diagnostic. The observed
    sizes fit annotated ID cells such as P-1 (description) and exclude bare ones. Those
    plans would fail allocation_step_unreferenced even with correct headings.
  • The retained diagnostic_result is one bind result relayed by the orchestrating model.
    Nothing attributes it to the first or the second bind.

Open risk: stale plan part on replan

The plan step captures plan_parts with capture_list (implementation.yaml:442-443),
which recipe/schema.py:200-202 defines as accumulating. bind_plan_set binds
context.plan_parts (:489). If the replan's part was appended to the failed first
part, the second bind re-verified the first plan, and no replan can pass in this loop.
The artifact does not settle it: two fully failing parts are excluded by result size,
but a replan-only bind and a stale-part bind with one more failing row both fit. The
orchestrator transcript was not retained.

Other paths that reach the same checks

  • salvage_plan (implementation.yaml:459-484) accepts any non-empty plan file after a
    planning context limit and routes to bind_plan_set.
  • renew_plan_set (implementation.yaml:576-593) and the renewal steps in
    remediation.yaml re-run the checks after dry-walkthrough edits. A failure there is
    terminal.
  • bind_group_parts (implementation-groups.yaml:303-323) has no coverage_failed
    route, so the same failure is terminal in grouped planning.
  • remediation.yaml wires rectify and make-plan to bind_plan_set the same way as
    implementation.yaml.

Why tests did not catch it

  • Every positive plan fixture is hand-written in the binder's shape
    (### Step 1.1: …, bare ID in the text).
  • No test feeds the make-plan or rectify template through parse_part_allocation and
    verify_allocation_evidence. SKILL.md is checked by substring only.
  • coverage_gaps is asserted only for an unassigned gap
    (tests/server/test_plan_set_materializer.py:80, 121). The allocation branches and the
    parts_without_obligations case have no gap-text assertion.
  • Recipe tests check only that the replan loop is bounded.
  • No end-to-end run exercised make-plan into bind_plan_set with a real model before
    the Add seeded sandbox implementation E2E validation #5255 commissioning scenario, and the artifact upload omits plan files.

Recommended direction

  • State the plan-part contract to the producer and bind that statement to the
    parser.
    make-plan and rectify should show the exact step-heading form, one heading
    key per row, a bare ID cell, the ID in the step's text, and a table-only section. Tie
    it to the parser with a contract test that runs the template's worked example through
    the real verifier. dry-walkthrough edits must keep the same form.
  • Make every coverage_failed result carry actionable feedback, at all three
    sites, naming the row, the offending cell and the expected form. Guard that invariant
    with a test, and update make-plan's plan_set_coverage_gaps description to cover
    format failures. Decide what the bind sites without a replan route should do.
  • Confirm the replan binds only the replacement plan set, with a recipe-level test.
  • Keep the gate strict. The step cell is stored verbatim in the content-addressed
    authority and handed to dry-walkthrough. Accepting multi-step cells or other
    separators is a separate, deliberate decision.
  • Verify with a real emitted plan before rerunning commissioning, and preserve plan
    files and the orchestrator transcript in failure artifacts.

Files that move together: the three recipe YAML files and their tracked compiled .json
counterparts, recipe/skill_contracts.yaml, both skill templates, and the tests above.

Still unknown

  • The text of the two MiniMax plans. This is not needed to choose the direction, since
    the producer is never told the contract, but only a real run can show whether those
    plans had a further defect.
  • What context.plan_parts held at the second bind.
  • Whether MiniMax-M3 follows the contract once it is stated.
  • Whether any pipeline run has passed a sealed bind since fix: bind multipart plans to sealed issue coverage authority #5102. None was found in
    local logs.

Live real-model bind verification (2026-10-11)

P-10 has now been performed successfully in the persistent Docker test container.

  • AutoSkillit 0.10.1269, develop commit 0bf84f34b8a0a032e2a3beff3ff198c64daa1907, containing merged PR Fix plan-part contract binding, replan feedback, and E2E evidence #5262.
  • MiniMax MiniMax-M3[1m] ran the actual make-plan skill for the original reusable sandbox seed issue. Sandbox main remained d6c143d2494488deeb6680d7d5cf2e55e3e01732, matching the recorded reproduction.
  • All three declared planning readers finished with success, and the join ledger reported complete.
  • The first bind_plan_set call returned success: True, plan_set_state: sealed, coverage_status: pass, and empty coverage_gaps. No replan was needed.
  • Generated plan: /workspace/sandbox/.autoskillit/temp/make-plan/normalize-whitespace-fixture_plan_2026-10-11_181500.md.
  • Bound authority: /workspace/sandbox/.autoskillit/temp/plan-set-authority/planset-8ae380e5ba203dc7b2c2f1e7/7f90d2d3104f5dcdf54d4c42.json; digest sha256:7f90d2d3104f5dcdf54d4c420b5abb08abcf6e7ea3520490a6c6ea19c6b16790.
  • Controller session: 7fbd4146-bfd5-4891-a379-d43b6fe00876; planning worker: 2121101f-86bf-4343-b1ab-13e487159aa8.
  • Local evidence report: /home/talon/projects/generic_automation_mcp/.autoskillit/temp/resolve-review/real-model-check/report_5259_live.md; the actual binder tool receipt, generated plan, authority JSON, and redacted transcripts are retained beside it.

Earlier local attempts were blocked by container registration drift: a stale settings-level package hook wrote an unresolved skill binding before the valid projected hook ran. The supported autoskillit install registration repair removed the stale copy; projected guards stayed active. A separate existing log-locator defect for cwd paths containing a dot was exposed by the temporary nested fixture; the final successful check used the documented /workspace/sandbox location.

This verifies real planning and sealed binding only. No implementation, sandbox test execution, generated PR, or full #5255 commissioning run was performed. Full commissioning retains its separate requirements.

Additional live verification — 2026-10-16, head20ee39237

Both fresh MiniMax-M3 implementation attempts passed make-plan and sealed plan binding at 20ee39237e8e89a8d21f403c3a825273fcac6bf7, which includes develop 75141403a04ca98bb4c7e6c3af9ce03eaf838f03 and the #5283 fix. This confirms the binding repair in the live implementation scenario; it does not claim full commissioning success.

In run38960061259, root session 5a5cdc35-9fdf-4480-92c2-78d5fbcb8592, paired bind_plan_set result at retained transcript record74 returns success=true, sealed, coverage_status=pass and empty coverage gaps. Plan: /workspace/autoskillit-runs/impl-20261016-201244-772303/.autoskillit/temp/make-plan/normalize_whitespace_plan_2026-10-16_202617.md. Authority: /workspace/autoskillit-runs/impl-20261016-201244-772303/.autoskillit/temp/plan-set-authority/planset-64a04c4d1b9e82df17a094df/031b15882fcbadeca16ea309.json; digest sha256:031b15882fcbadeca16ea309668bb1113aa3f2ebeba6ae4efb6b94238448bc95.

Run38960069882 independently passed planning/binding and failed later at dry-walkthrough zero-write adjudication. The first attempt failed later after divergent recipe progression/review. Neither result is a recurrence of this binding failure. PR5255 retains the commissioning acceptance record.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugExisting behavior is brokenin-progressIssue is being actively worked onrecipe:remediationRoute: investigate/decompose before implementation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions