feat(core): use each model's real context window by wudilyy999 · Pull Request #476 · OpenCoworkAI/open-codesign

wudilyy999 · 2026-10-10T18:27:35Z

Summary

Every generation run treated the model as having a 200k-token context window. buildPiModel hard-coded contextWindow: 200000, and the context pack read a contextWindow that ModelRef never carries, so the history budget meant to follow the model was 12,000 characters for every model. This is the first step of the context-control proposal in #475.

Core now resolves the window with resolveContextWindow, in this order:

the provider's configured value;
pi-ai's catalog entry for the provider and model;
the smallest catalog window among entries with the same model id, since relays and gateways usually serve upstream ids (an overestimate overflows the request, an underestimate only trims history sooner);
the existing 200k default.

The catalog is searched with the id sent on the wire, so Gemini ids stored as models/gemini-… match their catalog entries.

The agent model and the context pack both use it. Providers gain an optional contextWindow (tokens) in config.toml, editable next to Reasoning depth in Settings → Models; leaving it empty keeps the catalog value. This matters for local runtimes such as Ollama and LM Studio and for relays the catalog does not know. Existing configs and unknown models behave as before.

Type of change

Linked issue

Refs #475, #394

Checklist

I checked the linked issue / relevant context before starting
pnpm lint && pnpm typecheck && pnpm test passes locally
Added/updated tests for the change
Added a changeset (pnpm changeset) if user-visible
Updated docs if behavior changed

Principles

Compatibility: the new field is optional; existing configs and models the catalog does not know keep the 200k default.
Upgradeability: windows come from pi-ai's catalog, so pi-ai upgrades bring new models' windows.
No bloat: no new dependencies.
Elegance: one resolver feeds both the agent model and the context pack; the unused contextWindowForContextPack is removed.

Screenshots / recordings (UI changes)

The new field next to Reasoning depth in a Settings → Models provider row. The two controls were rendered on their own in system Chrome with the app stylesheet (after saving 32768, and empty in Chinese); the full app was not run.

Checks

pnpm -r typecheck, pnpm lint (686 files), CI=1 pnpm test: 10 tasks passed; core 559 tests, shared 326, desktop 180 files / 2,629 tests (after rebasing on e331e69).
context-window.test.ts: configured value, exact catalog entry, smallest same-id window, a Gemini models/-prefixed id on Google's OpenAI-compatible endpoint (fails without the id normalization), default. These run against a mocked catalog, so pi-ai upgrades do not change the expectations; context-window.catalog.test.ts checks against the installed pi-ai that an unknown provider and model fall back to the default instead of throwing.
The configured window reaches the agent model (agent.test.ts), the context pack and the agent input (generate.workspace-rename.test.ts), the active model resolution (provider-settings.test.ts), and is stored, cleared, and validated through config:v1:update-provider (onboarding-ipc.test.ts). Each of these fails when its wiring is removed.
The settings input, rendered on its own in system Chrome: saving 32768, rejecting abc with the previous value restored, and clearing back to Auto. Not exercised in the running app.

Every generation run treated the model as having a 200k-token window: buildPiModel hard-coded contextWindow: 200000, and the context pack read a contextWindow that ModelRef never carries, so its history budget was the same for every model. Core now resolves the window with resolveContextWindow: the provider's configured value, then pi-ai's catalog entry for the provider and model, then the smallest catalog window for the same model id (relays usually serve upstream ids), then the 200k default. The agent model and the context pack both use it. Providers gain an optional contextWindow in config, editable next to Reasoning depth in Settings -> Models, for local runtimes and relays the catalog does not know.

github-actions

Review mode: initial

Findings

[Nit] ContextWindowInput can fire a second updateProvider call while the first is still in flight. setSaving(true) re-renders the input with disabled={saving}; disabling the currently focused element blurs it, which re-enters save(). At that moment draft still differs from the not-yet-refreshed saved prop, so the trimmed === saved early-return does not apply and a duplicate write + duplicate success toast can occur (apps/desktop/src/renderer/src/components/settings/primitives.tsx:651).
Suggested fix / next action: guard the entry, e.g. add if (saving) return; as the first line of save() (or track in-flight state in a ref instead of via disabled).
[Nit] resolveContextWindow casts the model id with model.modelId as never (packages/core/src/context-window.ts:24). Casting to never satisfies any parameter type and silently disables the model-id type check, so a future pi-ai signature change or a wrong ModelRef field would not be caught. Prefer a narrower cast (or String(model.modelId) if the API accepts a plain string) with a short comment on why the catalog lookup needs loosening.

Questions

Does pi-ai 0.72.1 getModel / getModels return undefined / [] (rather than throw) for a provider id that is not in the catalog? resolveContextWindow and context-window.test.ts both assume it degrades quietly for unknown providers (relays, local runtimes). If any of those helpers throws for an unknown provider, every custom/relay generation would fail at the resolver instead of falling through to the same-id or default branches.

Summary

Review mode: initial
The core change is directionally sound: a single resolveContextWindow(model, configured) resolver is threaded through both the agent model (packages/core/src/agent.ts:319) and the context pack (apps/desktop/src/main/ipc/generate.ts:1319), replacing the previous hard-coded 200k and the dead contextWindowForContextPack helper. The precedence (configured → exact catalog entry → smallest same-id → 200k default) is documented and the “underestimate trims history, overestimate overflows” rationale is stated in the JSDoc.
Config plumbing is consistent across layers: zod schema (packages/shared/src/config.ts:186), IPC parser with positive-integer/null validation (apps/desktop/src/main/onboarding/provider-parsers.ts:403), CRUD mutation (providers-crud.ts), ProviderRow/ActiveModelResolution, and the preload contract all carry the new optional field. null clears to undefined in the same tri-state style as tlsRejectUnauthorized, and an absent value is omitted from the GenerateInput passthrough so existing configs keep prior behavior.
The PR correctly uses Refs #475, #394 rather than a closing keyword, and a changeset is included — no linked-issue completion claim to validate.
No new dependencies and no license/install-size impact.

Testing

Not run (automation). Coverage is proportional to the change: context-window.test.ts (mocked catalog) exercises all four resolver branches, agent.test.ts asserts the configured window reaches the pi model, generate.workspace-rename.test.ts asserts both the context pack and the agent input, plus provider-settings.test.ts, onboarding-ipc.test.ts, and config.test.ts for wiring/persistence/validation. The remaining gap is the UI-only path (the PR notes the input was rendered standalone, not in the running app), which matches the nits above.

Open-CoDesign Bot

Copilot

🟡 Changes recommended

Stored Gemini model IDs can miss known catalog entries and receive the incorrect fallback window.

1 open finding

Normalize Gemini model IDs before catalog lookup · New

What changed in this PR

Adds model-aware context-window resolution across core generation, provider configuration, and Settings.

Changes:

Resolves context windows from provider overrides, pi-ai’s catalog, or a 200k fallback.
Adds persisted provider settings, validation, UI controls, and translations.
Adds coverage for resolution and end-to-end propagation.

File	Description
`.changeset/provider-context-window.md`	Records the user-visible feature.
`packages/shared/src/config.ts`	Adds the provider context-window schema.
`packages/shared/src/config.test.ts`	Tests schema validation.
`packages/i18n/src/locales/en.json`	Adds English strings.
`packages/i18n/src/locales/es.json`	Adds Spanish strings.
`packages/i18n/src/locales/pt-BR.json`	Adds Portuguese strings.
`packages/i18n/src/locales/zh-CN.json`	Adds Chinese strings.
`packages/core/src/index.ts`	Exports the resolver and generation input.
`packages/core/src/context-window.ts`	Implements catalog resolution.
`packages/core/src/context-window.test.ts`	Tests resolution precedence.
`packages/core/src/agent.ts`	Applies the resolved window to agent models.
`packages/core/src/agent.test.ts`	Tests agent propagation.
`apps/desktop/src/preload/index.ts`	Extends renderer IPC types.
`apps/desktop/src/main/provider-settings.ts`	Exposes configured windows.
`apps/desktop/src/main/provider-settings.test.ts`	Tests active-model propagation.
`apps/desktop/src/main/onboarding/providers-crud.ts`	Persists and clears overrides.
`apps/desktop/src/main/onboarding/provider-parsers.ts`	Validates update payloads.
`apps/desktop/src/main/onboarding-ipc.test.ts`	Tests IPC persistence and validation.
`apps/desktop/src/main/ipc/generate.ts`	Uses resolved windows during generation.
`apps/desktop/src/main/ipc/generate.test.ts`	Removes obsolete helper tests.
`apps/desktop/src/main/ipc/generate.workspace-rename.test.ts`	Tests generation wiring.
`apps/desktop/src/renderer/src/components/settings/primitives.tsx`	Adds the settings input.

🧠 Review effort: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Gemini's OpenAI-compatible model listing stores ids such as models/gemini-2.5-pro, while pi-ai's catalog ids have no models/ prefix, so those models missed the catalog and got the 200k default. resolveContextWindow now takes the base URL and normalizes the id with normalizeGeminiModelId, as the request does. The exact lookup searches getModels(provider), which removes the cast to never.

github-actions

Review mode: follow-up after new commits

Findings

No new findings introduced by the follow-up commits (f96b525..b776962). The follow-up stays within the PR's scope and completes Gemini id handling for the resolver.

Residual observations

[Nit, carried over] ContextWindowInput still has no in-flight guard. setSaving(true) re-renders the focused input with disabled={saving}; disabling a focused element blurs it and re-enters save() while draft already differs from the not-yet-refreshed saved prop, so the trimmed === saved early-return does not apply and a second updateProvider write plus a second success toast can fire (apps/desktop/src/renderer/src/components/settings/primitives.tsx:651). Not a merge blocker; a one-line if (saving) return; at the top of save() would close it.
Previously reported model.modelId as never nit is resolved: resolveContextWindow now looks up the catalog with getModels(...).find(...) and normalizes Gemini models/-prefixed ids via normalizeGeminiModelId (packages/core/src/context-window.ts:23-24), with a matching test (packages/core/src/context-window.test.ts:35).

Questions

Carried from the prior review and still unverified from the public repo: does pi-ai 0.72.1 getModels(provider) return [] (rather than throw) for a provider id absent from the catalog? The resolver calls getModels(model.provider as KnownProvider) directly as the first lookup (packages/core/src/context-window.ts:24), so a relay/local provider not in the catalog reaches this before the same-id and default branches. context-window.test.ts mocks @mariozechner/pi-ai, so it cannot confirm the real behavior. If you can confirm against the installed package, good; otherwise consider an existence check against getProviders() before the direct lookup.

Summary

Review mode: follow-up after new commits.
The diff since the last review is small and targeted: baseUrl/canonicalBase is now threaded into resolveContextWindow at both call sites — packages/core/src/agent.ts:319 and apps/desktop/src/main/ipc/generate.ts:1319 — so Gemini models/-prefixed ids resolve consistently for both the agent model and the context pack. context-window.test.ts and the providers mock in generate.workspace-rename.test.ts were updated to match.
Bundling, config, IPC, and preload contracts are unchanged from the reviewed revision; no new dependencies, no license or install-size impact, and no release/distribution files touched.
Only residual items are the unverified pi-ai getModels behavior for unknown providers and the still-present duplicate-save guard in the new UI input. On the available information neither is a merge blocker.
The UI input is still not exercised in the running app (the PR notes it was rendered standalone), so the field's end-to-end behavior is unverified.

Testing

Not run (automation). Coverage remains proportional: context-window.test.ts (configured → exact catalog → smallest same-id → default, plus the Gemini prefix case), agent.test.ts, generate.workspace-rename.test.ts (context pack + agent input), provider-settings.test.ts, onboarding-ipc.test.ts, and config.test.ts. Suggested addition: a test that calls the real (unmocked) resolveContextWindow with a custom/relay provider id to confirm getModels degrades quietly for unknown providers.

Open-CoDesign Bot

…atalog

github-actions

Review mode: follow-up after new commits

Findings

No new findings introduced by the follow-up commit (b776962..47d78bc). The only diff since the last review is the new packages/core/src/context-window.catalog.test.ts, which exercises the real (unmocked) resolveContextWindow and closes the prior review's open question about getModels behavior for unknown providers.

Residual observations

[Nit, carried over; not a merge blocker] ContextWindowInput.save() still has no in-flight guard (apps/desktop/src/renderer/src/components/settings/primitives.tsx:651). setSaving(true) re-renders the focused input with disabled={saving}; if disabling a focused element fires blur, save() re-enters before the refreshed saved prop lands, so trimmed === saved does not short-circuit and a second updateProvider write / success toast can fire. A one-line if (saving) return; at the top of save() closes it. Unchanged by this follow-up.
[Nit] packages/core/src/context-window.catalog.test.ts:7 asserts the installed pi-ai catalog does not know qwen3:32b. That is the property under test (real-catalog fallback), but it couples the test to the pinned catalog contents: a future pi-ai patch that adds qwen3:32b under any provider would fail this test for reasons unrelated to the resolver. Acceptable given the test's purpose; if you want it hermetic, use an id that cannot appear (e.g. a clearly synthetic one).

Summary

Review mode: follow-up after new commits.
The follow-up is a single new test file. It calls the real resolveContextWindow for a relay provider absent from the catalog (packages/core/src/context-window.catalog.test.ts) and confirms the resolver returns DEFAULT_CONTEXT_WINDOW rather than throwing, which is exactly the getModels(unknownProvider) question left open in the previous review (packages/core/src/context-window.ts:24). Run against the installed @mariozechner/pi-ai@0.72.1, the contract is now covered at the real boundary rather than only through mocks.
No changes to bundling, config/IPC contracts, dependency set, licenses, or install size; no release/distribution files touched. The rest of the PR is unchanged from the previously reviewed revision, and the earlier resolved item (catalog lookup by normalizeGeminiModelId instead of as never) still holds.
Remaining items are the two non-blocking nits above. Neither is a merge blocker; the PR is otherwise ready from this review's perspective.
The new settings field is still only exercised in a standalone render, not in the running app (per the PR description); end-to-end UI behavior remains unverified.

Testing

Not run (automation). New coverage: context-window.catalog.test.ts (real catalog, unknown provider → default). Existing coverage from earlier revisions (mocked resolver, agent model, generate IPC context pack/agent input, provider-settings, onboarding IPC store/clear/validate, config schema) is unchanged.

Open-CoDesign Bot

Leaving the window blurs the focused input without moving focus, so disabling it during the save can blur it again and repeat the write and the success toast. The catalog test now uses a synthetic model id, so a future catalog entry cannot change its result.

github-actions

Review mode: follow-up after new commits

Findings

No new findings. The diff since the last review (47d78bc..d5adbec) consists only of the two previously raised non-blocking items, and both are addressed:
- apps/desktop/src/renderer/src/components/settings/primitives.tsx:653 now guards save() with if (saving || !window.codesign?.config?.updateProvider) return;, closing the re-entrancy path where setSaving(true) → disabled={saving} could blur the focused input and fire a second updateProvider write plus a duplicate success toast.
- packages/core/src/context-window.catalog.test.ts:6 now uses clearly synthetic ids (codesign-test-relay / codesign-test-unlisted-model), so a future pi-ai catalog addition can no longer fail this test for reasons unrelated to the resolver.

Residual observations (non-blocking, unchanged by this diff)

The saving guard is a best-effort lock: the re-entrant blur would be handled by the props closure React has committed by dispatch time. If double-fire is ever observed on real blur behavior, a useRef lock removes the dependency on prop-commit timing. Not worth blocking on now.
The new "Context window" field is still only exercised in a standalone render, not in the running app (per the PR description). The persistence/validation paths are covered at the IPC and schema boundary (apps/desktop/src/main/onboarding-ipc.test.ts, packages/shared/src/config.test.ts), but no test or manual run covers the component's own draft/saved/saving state machine or the clear-back-to-Auto round trip in the real app.
The Gemini models/-prefix branch (packages/core/src/context-window.ts:26) is covered only by a mocked catalog (packages/core/src/context-window.test.ts); packages/core/src/context-window.catalog.test.ts exercises the real installed catalog but only the unknown-model fallback. If pi-ai's google entries ever carried the models/ prefix themselves, the normalized lookup would miss and return 200k — an underestimate, which is the safe direction by design, but silent.
The window is per provider, not per model, so one value applies to every model behind that provider. That matches the schema comment (packages/shared/src/config.ts:185) and the PR description; noted only so the granularity reads as a deliberate choice.

Summary

Review mode: follow-up after new commits.
No blockers, majors, minors, or nits introduced by the follow-up commit. Both items from the previous review are resolved, and the rest of the PR is unchanged from the previously reviewed revision (resolver in packages/core/src/context-window.ts, plumbing through packages/core/src/agent.ts, apps/desktop/src/main/provider-settings.ts, apps/desktop/src/main/ipc/generate.ts, config schema in packages/shared/src/config.ts, and the settings field in apps/desktop/src/renderer/src/components/settings/primitives.tsx).
The PR links with Refs #475, #394, i.e. partial work, so there is no completion claim to validate against those issues' acceptance criteria; the remaining proposal slices are correctly left open.
No new dependencies, no license or install-size impact, no release/distribution files touched, and the changeset (.changeset/provider-context-window.md) covers shared/core/desktop/i18n.
Remaining items are the residual observations above and are not merge blockers. From this review's perspective the PR is ready to merge; the main unverified surface is end-to-end behavior of the new settings field in the running app.

Testing

Not run (automation). Coverage present: context-window.test.ts (mocked catalog: configured > exact > smallest same-id > default, Gemini prefix), context-window.catalog.test.ts (real catalog, unknown provider/model → default), agent.test.ts (agent model gets configured window), generate.workspace-rename.test.ts (context pack budget and agent input receive 32,768), provider-settings.test.ts, onboarding-ipc.test.ts (store/clear/validate), config.test.ts (schema accept/reject). Suggested addition if you want the UI state machine covered: a renderer test for ContextWindowInput asserting invalid input restores the previous value and empty input sends contextWindow: null.

Open-CoDesign Bot

## Summary Second step of #475. A typical run already sends about 7.5k tokens before any chat history: the system prompt is about 13k characters and the 13 tool definitions about 16k. Local runtimes often default to a 4k–8k window (Ollama's default `num_ctx`), and an over-long request is truncated by the server rather than rejected, so the model works from a cut-off system prompt. Core now measures the system prompt, tool definitions, and the prompt (which carries the brief, memory, and attachments as text) in tokens the way pi estimates them, and keeps a quarter of the window, at most the 32,000 tokens a run requests, free for the reply. The request asks for no more output than that reserve, so input plus requested output fits the window: models with a window of 128k tokens or more keep 32,000, a 32k window requests 8,192. - If those parts do not fit the window resolved in #476, the run fails before the request with `GENERATION_CONTEXT_EXCEEDED`. The message names the tokens needed and the window, and suggests shortening the request or its attachments, setting the provider's context window, or using a model with a larger window; the diagnostic offers Settings → Models, where the context window is set. - Chat history keeps its newest messages that fit what remains, starting at a user message. The context pack's own history cap is unchanged. - The budget (window, reserve, part sizes, history kept and dropped) is logged with each run, for the visibility step that follows. ## Type of change - [ ] Bug fix - [x] New feature - [ ] Refactor (no behavior change) - [ ] Documentation - [ ] Build / CI / tooling - [ ] Breaking change ## Linked issue Refs #475, #394 ## Checklist - [x] I checked the linked issue / relevant context before starting - [x] `pnpm lint && pnpm typecheck && pnpm test` passes locally - [x] Added/updated tests for the change - [x] Added a changeset (`pnpm changeset`) if user-visible - [ ] Updated docs if behavior changed ## Principles - [x] Compatibility: with today's 200k default and the catalog windows, normal runs fit and keep their history; only runs that cannot fit now stop early. - [x] Upgradeability: estimates use pi's `estimateTokens`. - [x] No bloat: no new dependencies; 118 lines outside tests, strings, and the changeset. - [x] Elegance: one budget function decides what fits; the agent starts with the history it returns. ## Checks - `pnpm -r typecheck`, `pnpm lint`, `CI=1 pnpm test`: 10 tasks passed; core 568 tests, shared 327, desktop 180 files / 2,629 tests. - `context-budget.test.ts`: reserve in large and small windows, tool definitions counted by name, description and parameters (a tool alone can push a run past the window), oldest history dropped first, a leading assistant message dropped, prompt images counted, and the error when the fixed parts do not fit. - `agent.test.ts`: a 32,768-token window gives the agent model that window and an 8,192-token output limit; a 2,048-token window fails before the agent starts; with a 32,768-token window the agent starts with the newest history that fits, opening at a user message. - `diagnostics.test.ts`: the new code maps to a Settings → Models fix. - Each of the fail-fast check, the budgeted history, and the user-message start fails its tests when removed.

Copilot AI balanced review requested due to automatic review settings October 10, 2026 18:27

github-actions Bot added docs Documentation area:desktop apps/desktop (Electron shell, renderer) area:core packages/core (generation orchestration) labels Oct 10, 2026

Copilot started reviewing on behalf of wudilyy999 October 10, 2026 18:28 View session

github-actions Bot reviewed Oct 10, 2026

View reviewed changes

Copilot AI reviewed Oct 10, 2026

View reviewed changes

Comment thread packages/core/src/context-window.ts

github-actions Bot reviewed Oct 10, 2026

View reviewed changes

test(core): resolve an unknown provider against the installed pi-ai c…

47d78bc

…atalog

github-actions Bot reviewed Oct 10, 2026

View reviewed changes

Sun-sunshine06 merged commit 856dba8 into OpenCoworkAI:main Oct 11, 2026
6 checks passed

wudilyy999 mentioned this pull request Oct 11, 2026

feat(core): fit each run into the model's context window #478

Merged

9 of 15 tasks

Navigation Menu

Repository navigation

feat(core): use each model's real context window - #476

feat(core): use each model's real context window#476
Sun-sunshine06 merged 4 commits into
OpenCoworkAI:mainfrom
wudilyy999:feat/design-context-control

wudilyy999 commented Oct 10, 2026 •

edited

Loading

Uh oh!

github-actions Bot left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

github-actions Bot left a comment

Uh oh!

github-actions Bot left a comment

Uh oh!

github-actions Bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

wudilyy999 commented Oct 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Type of change

Linked issue

Checklist

Principles

Screenshots / recordings (UI changes)

Checks

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

wudilyy999 commented Oct 10, 2026 •

edited

Loading