Skip to content

E2E fleet sessions run at a 200K context window despite the harness's 1M setup #5274

Closed

Description

@Trecek

Fix target and integration order — mandatory routing

Repository: TalonT-Org/AutoSkillit.
Fix PR base branch: develop, currently 381afb4e59cc23acf4ca9ace7a7644b75e6e7b14.
Scope: how E2E fleet sessions get their context window. The harness settings, the README text and the Claude Code version pin involved are identical on develop and on the #5255 branch, so this does not need the unmerged implementation E2E harness.

Open the remediation PR against develop. Once merged, the owner of parent PR #5255 integrates the develop commit.

Defect

The E2E harness is set up for a 1M-token context window, but fleet sessions run with a 200K window. The root therefore auto-compacted below 200k tokens in both retained implementation runs.

Evidence

Run 38431354139, attempt 1 at 93457f16ecc30249799c02084c20f41d9cd88455. Artifact e2e-implementation-38431354139-1.

What the harness configures (scripts/e2e/e2e_harness.py:26-27 and :128-132 on develop):

  • ANTHROPIC_MODEL and ANTHROPIC_DEFAULT_{SONNET,OPUS,HAIKU}_MODEL are MiniMax-M3[1m].
  • CLAUDE_CODE_AUTO_COMPACT_WINDOW is 1000000.
  • scripts/e2e/README.md:74-75 says the [1m] suffix "makes Claude Code assume a 1M-token window for the unrecognized model id; the compact window is capped at it".

What fleet sessions actually run as.

  • AutoSkillit passes an explicit model id, not the alias: CLAUDE_MODEL_ALIASES maps sonnet to claude-sonnet-5 (src/autoskillit/core/types/execution/_type_backend.py:348-349). The README states this too at :81 ("the explicit claude-* model ids AutoSkillit passes with --model").
  • logs/sessions.jsonl: all nine sessions have configured_model: "sonnet", provider_used: "anthropic", profile_name: "".
  • Claude Code's own output for two worker sessions (logs/sessions/fcaa0a53-…/raw_stdout.jsonl, logs/sessions/3a50360d-…/raw_stdout.jsonl): the init record has model: "claude-sonnet-5", and the result record has modelUsage: {"claude-sonnet-5": {"contextWindow": 200000, "maxOutputTokens": 64000}}. The assistant messages report MiniMax-M3.
  • So the [1m] suffix, which sits on the custom id MiniMax-M3[1m], only affects sessions started without --model. Fleet sessions are sized as claude-sonnet-5.

Consequence.

  • The root's raw output is not retained, but it was launched the same way and auto-compacted at preTokens 194567 (root transcript record 382). The root in run 38170521660 compacted at 166958.
  • The root's 64,000-token output at transcript record 376 matches the maxOutputTokens above.

What the Claude Code documentation says (https://code.claude.com/docs/en/model-config and /env-vars, retrieved 2026-10-13):

  • CLAUDE_CODE_AUTO_COMPACT_WINDOW: "The effective window is also capped at the model's context window."
  • "Fable 5.1, Fable 5, Sonnet 5 and later, Haiku 5.5, and Opus 4.7 and later run with the 1M window by default, with no [1m] suffix needed."
  • Behind a custom ANTHROPIC_BASE_URL, "Claude Code gives each model it recognizes the same context window the model has on the Anthropic API."
  • "To keep sessions at a 200K window, set CLAUDE_CODE_DISABLE_1M_CONTEXT=1".

By that documentation claude-sonnet-5 should have had a 1M window here. Nothing in this repository sets CLAUDE_CODE_DISABLE_1M_CONTEXT, CLAUDE_CODE_MAX_CONTEXT_TOKENS or autoCompactWindow. The image pins Claude Code 2.1.280 (scripts/docker/Dockerfile:15).

Not established

  • Why Claude Code 2.1.280 reported a 200K window for claude-sonnet-5 in this environment.
  • Whether AutoSkillit sessions on the first-party API are sized the same way. If they are, this is wider than E2E.
  • What context limit the MiniMax endpoint really enforces.

Required behaviour

  • E2E fleet sessions run with the window the harness documents. Assert it from modelUsage.*.contextWindow in a session result, not from configuration.
  • Correct the README so it says which sessions each setting covers.
  • First step: in the E2E image, run one claude -p --model claude-sonnet-5 --output-format json call with and without the MiniMax base URL and read modelUsage.*.contextWindow. That separates a Claude Code behaviour from a harness configuration gap.
  • Run pre-commit run --all-files and the configured task test-local-gate.

Coordination

Related to #5272, which requires that a fleet root does not auto-compact. With compaction blocked, a 200K window turns a compaction into a hard stop near 200k tokens, so the window still matters. Independent of #5270 and #5271; can be worked in parallel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugExisting behavior is brokenin-progressIssue is being actively worked onrecipe:remediationRoute: investigate/decompose before implementation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions