chore(models): refresh stale model defaults (NodeBenchAI)

[HomenShum]

What this is

PR 1 of 3 for NodeBenchAI from the model-defaults spec. It moves no model id, except registering the one verified free lane. It fixes request shapes that would answer 400 on Claude 4.7+/5.x and Gemini 3.x, makes an unknown model id fail instead of silently becoming a paid default, and wires the migration eval so later receipts can name a candidate. PR 2 (model-refresh/nodebench-ids, the id moves) and PR 3 (release/nodebench-mcp-3.3.0) are deferred until this PR is merged.

Plain-language version: today, if a caller asks for a model name the code does not recognise, it quietly runs the expensive default model and bills for it. A pipeline run could do that without any spending check. After this PR an unknown name is rejected at the places listed under "Fail-closed" below (a name ending in -free goes to the one free model instead), and a request that starts on a free model stays on free models when it falls back. The places where that is not yet true are listed under "Where the fail-closed claim stops". Separately, several places send settings (temperature, a thinking budget) that the newer Claude and Gemini models refuse, so the first request after a model upgrade would fail; those are fixed first so PR 2 can be tested.

Live traffic note: this PR is where auto-free and getBestFreeModel() actually move from laguna-s-2.1-free to nemotron-3-super-free, so free-lane requests change model here, not in PR 2. The free lane has no live measurement in this repo (the pick is NodeVideo's benchmark), so this PR carries needs-live-eval. A free-lane receipt costs $0; laguna expires on 2026-10-31.

Request-shape fixes (Breaking parameters)

Where	Change
backend/convex/events.ts (3 raw Anthropic fetches)	temperature deleted; max_tokens 700/260/300 -> 2048 (thinking counts toward it); text read by block type
domains/product/extendedThinking.ts + workers/node/pipeline/extendedThinkingRunner.ts	thinking:{enabled,budget_tokens} -> {type:"adaptive"} + output_config.effort:"medium" for claude-sonnet-4-6 and claude-opus-4-7 only; every other id keeps main's manual {type:"enabled", budget_tokens} with no effort (round 4: Claude Haiku 4.5 supports only manual thinking and is not on the effort compatibility list; an allowance under the API minimum of 1,024 sends no thinking block); max_tokens = answer cap + remaining thinking allowance. That bounds thinking plus answer per checkpoint, not thinking alone (adaptive thinking has no separate cap), so a run spends at most thinkingBudgetTokens + MAX_OUTPUT_TOKENS on thinking, at checkpoint granularity. The run is charged from usage.output_tokens minus the answer (estimateThinkingTokens), because on Sonnet 5.x / Opus 4.7+ the thinking text is omitted by default and the old text-length estimate was always 0, so the budget would never run out once PR 2 moves the id; the body is built by buildCheckpointRequestBody; model is the wire id (claude-sonnet-4-6), not the stored dotted alias
shared/llm/modelCatalog.ts	explicit ANTHROPIC_WIRE_IDS table + anthropicWireId() (no regex/string surgery) + LEGACY_PASSTHROUGH_IDS (round 4: the 16 resolver-recognised ids main sent unchanged); getModelWithFailover returns wire ids for Anthropic; anthropicText() reads blocks by type. A test pins the table to the resolver's MODEL_SPECS (Convex cannot import it from shared/)
16 generateWithProvider copies (ai.ts, morningDigest, fastAgentChat, entityInsights, dealFlow, dailyBriefPersonalOverlay, dailyBriefWorker, modelComparison, paperDetails, repoScout, stackImpact, strategyMetrics, postDedupAction, claimVerificationAction, tags_actions, benchmark.ts)	content[0] -> anthropicText(content). Each copy's own function is now driven by providerCopies.requestShape.test.ts (15 cases; benchmark.ts by benchmark.requestShape.test.ts), so reverting one copy fails a test. Thirteen of them gained an export keyword as the test seam; they are not Convex functions
ai/models/modelRouter.ts, search/fusion/benchmark.ts, packages/mcp-local/.../llmTools.ts, researchWritingTools.ts	temperature not sent (modelRouter: kept for non-Anthropic/non-Google candidates). Tests drive the real handlers: modelRouter.requestShape.test.ts (route action, fake generateText), benchmark.requestShape.test.ts (judge fetch per provider), anthropicRequestShape.test.ts (mcp-local, fake SDK)
shared/search/triSearch.ts (rerank and condenseQuery), scripts/ui/judgeScratchnodeChatVideo.mjs, search/fusion/benchmark.ts (Gemini judge)	thinkingBudget:0 -> thinkingLevel:"low"; round 3: generationConfig.temperature (0, 0, 0.2, 0) removed too, because sampling is deprecated on Gemini 3.x (ignored today, a 400 later; spec s1.3). Absence is asserted through the real builders (triSearch.test.ts for both calls, benchmark.requestShape.test.ts, the new judgeScratchnodeChatVideo.requestShape.test.ts which runs the script against a fake @google/genai client). The benchmark test used to pin temperature === 0 for the Gemini judge; it now asserts the opposite and the OpenAI judge keeps its determinism pin. Gemini sampling at the ~30 other call sites this PR does not rewrite (workers/node/routes/search.ts, deepDiligence.ts, runDogfoodGeminiQa.mjs, ...) stays for PR 2 with their id moves
scripts/tool-gen/analyze-gap.mjs, generate-tool.mjs	each exports its own buildClaudeRequest (analyze-gap: claude-haiku-4-5-20251001, 2048; generate-tool: claude-sonnet-4-6, 4096), no sampling keys; text read by block type through claudeText (generate-tool imports it from analyze-gap). In round 0 this row said generate-tool used buildClaudeRequest; it did not (it built its body inline). It does now, and generate-tool.mjs has its own request-shape test

Fail-closed resolver (A.0.10) and free lane (Codex v3 N2)

• getLanguageModelSafe(input, opts?), normalizeModelInput, and now also normalizeNodeBenchRuntimeModel (the first resolver on the FastAgent public entry points createThread, createThreadForUserInternal, initiateAsyncStreaming, streamAsync, and runtimeRouting.ts for signed-in callers; an anonymous caller's model is replaced by the pinned lane before any resolver runs, see "Where the fail-closed claim stops" item 7) share one rule (resolveOrFailClosed): unknown free-looking input (-free, :free, free route alias) -> FREE_LANE_MODEL; any other unknown input throws UnknownModelError; opts.allowPaidFallback === true (exactly) -> DEFAULT_MODEL, counted by getPaidFallbackCount(). No call site passes the flag in this PR. Empty input still takes the profile default. A known free model keeps its cost class in every profile (it used to be replaced by paid kimi-k2.6 / gemini-3.1-flash-lite-preview / gemini-3.1-pro-preview); a known paid model the profile does not allow still takes the profile default.
• runtimeRouting.ts: resolveRequestedExecutorModel no longer swallows an unknown requested id (it throws or goes to the free lane); chooseAvailableRouteModels does not append paid candidates to a free primary. Round 2: the anonymous, context_rot_guard, tool_heavy_executor, structured_executor and large_context_executor branches never read requestedModel, so a user who picked the free lane and typed "patch this script" was routed to gemini-3-flash-preview. Every executor branch now starts from the explicitly requested free model (resolveRequestedFreeModel), and a request that names no free model keeps each branch's paid default unchanged.
• agentChatActions.continueThread rethrows UnknownModelError from the stored thread model instead of logging and continuing on DEFAULT_MODEL.
• shared/llm/modelCatalog.ts getModelWithFailover (reached from ai.ts and the other generateWithProvider copies): a free-looking unknown id (suffix -free/:free, or the free route aliases free, auto-free, nodebench:auto-free, kilo-auto/free, openrouter/free) -> the free lane on OpenRouter as the wire id nvidia/nemotron-3-super-120b-a12b:free, never the alias (round 1 returned the alias, which every caller hands straight to a provider; spec rule (b)); the free lane's own catalog id is converted the same way; any other unknown id throws (it used to become gpt-5.4-mini); a free catalog model whose provider is not configured throws instead of failing over to a paid provider. Round 2: a gpt-/claude-/gemini- prefix no longer makes an invented id "known": gpt-not-real with only Gemini configured used to fail over to paid gemini-3.1-flash-lite-preview. Round 3: round 2 still passed a prefix-shaped unknown id to its own provider when that provider was configured (gpt-not-real and gpt-5.5-pro returned {provider:"openai", isFallback:false} with OPENAI_API_KEY set). That was wrong: it is reachable from a public action (next bullets) and the spec requires an unknown id to throw. getModelWithFailover now throws for every id that is neither in the catalog nor a known wire id (ANTHROPIC_WIRE_IDS values; for OpenAI and Gemini the resolver sdkId equals the catalog name, pinned by anthropicWireIds.test.ts), before any provider-availability check. The old "operator-supplied wire id" justification does not hold: getLlmModel has no env override, and the one test that pinned claude-haiku-4-5-20251001 passing through was changed to assert a throw. Round 4 reverses that part: the resolver still recognises that id (and 15 others) and main sent them unchanged, so rejecting them broke callers that pass model to a public action; they pass through again (LEGACY_PASSTHROUGH_IDS, see "Round 4 review fixes"). A prefix-shaped id the resolver does not recognise still throws. FREE_LANE_CATALOG_ID and the new FREE_LANE_WIRE_ID are pinned to the resolver (MODEL_SPECS[FREE_LANE_MODEL].sdkId) by anthropicWireIds.test.ts (shared/ cannot import from backend/convex, so the wire string exists twice and the test is what keeps it one).
• ai/models/modelRouter.ts: the static fallback list for the free tier no longer appends TIER_MODELS.cheap. Round 2: route() now caps a tier: "free" request at free after the chain is built (a candidate must resolve to a free approved model). Before, requireVision: true put z-ai/glm-4.7-flash in the free chain and it was dispatched and reported isFree: false, actualTier: cheap. If no free model qualifies the call fails with All 0 models failed instead of billing. chainResolver itself is unchanged (tierFloor is still a floor for the other tiers), and pinnedModel is untouched (an explicit pin is what the caller asked for). Round 4 replaced the zero MODEL_PRICING row for nemotron-3-super-free with the general fix: calculateCost, isFree and the cache path now use the resolved model's cost class (isFreeAlias), see the next bullets (the paid ids' prices still land in PR 2).
• Fallback chain keeps a free request free. FREE_LANE_MODEL was missing from MODEL_PRIORITY_ORDER, so executeWithModelFallback({ startModel: "nemotron-3-super-free" }) restarted at index 0 and dispatched paid kimi-k2.6 while reporting isFree: false (reached from piRuntime's AI-SDK fallback on an auto-free run when the free lane answered 401/404/429/5xx). It is now the head of the free segment (resolver and web mirror; the drift test enforces the mirror), and executeWithModelFallback, getFallbackChain and getLanguageModelWithFallback throw for an unlisted FREE start model instead of restarting at the paid head (round 2: an approved paid id outside the order, such as glm-4.7-flash, deepseek-r1, gemini-2.5-*, claude-opus-4, is back to main's behaviour: it starts at kimi-k2.6; changing that is a caller-argument decision, see C item c). A test pins that every free id sits after every paid id, and that a chain started on FREE_LANE_MODEL or laguna-s-2.1-free never reaches a paid model, even when every free model fails. piRuntime's fallback result also stopped reporting estimatedUsd: 0 regardless of the model that answered; it is priced from the model used.
• Round 3, free requests stay free at the tier gate. selectTierEligibleRuntimeModel (used by streamAsync and by the post-stream fallback in fastAgentPanelStreaming) always appended the paid TIER_SAFE_RUNTIME_FALLBACKS and queued the paid suggestedModel from checkRequestAccess. On the free billing tier (no openrouter provider in tierLimits, shared/llm/modelCatalog.ts; the helper is exercised for the anonymous tier too, but an anonymous caller's model is replaced earlier, item 7 below) a user who picked auto-free was therefore answered by gpt-5.4-nano / gpt-5.4-mini, with no reservation on the anonymous path. The same request already turned paid one step earlier on origin/main, so this is not new, but it is an A.0.10 violation and round 2's "the A.0.10 exception" was wrong. Fix: the helper takes requireFree (default: the primary is free). When set it enqueues only free candidates: no tier-safe list, no paid suggestion, and the request fails with the tier's own reason. The post-stream fallback passes requireFree: isFreeModel(activeModel), so a failed free run cannot continue on a paid model either. Tests drive the real selection with the real isModelAllowedForTier / getBestModelForTier for the anonymous, free and pro tiers; a paid primary keeps its fallbacks. Consequence to know: a signed-in free-tier user who explicitly picks the free lane now gets the tier refusal instead of a paid answer. An anonymous user does not get that refusal: resolveFastAgentRequestedModel replaces the client's model with the pinned paid gemini-3.1-flash-lite-preview before any resolver or this helper runs (fastAgentPanelStreaming.ts createThread and initiateAsyncStreaming), so an anonymous free request runs on that paid model, and an unknown id is accepted silently there. That is the spec-kept bounded anonymous lane (item 7 below), not changed in this PR (an earlier version of this body said the opposite).
• Round 3, the two daily-brief public actions validate model. dailyBriefWorker.runNextTask and dailyBriefPersonalOverlay.runNextPersonalTask took model: v.string() and passed it through executeFeature -> generateWithProvider -> getModelWithFailover -> the provider SDK. They now call normalizeModelInput first, to validate only: an unknown id throws UnknownModelError before any internal action runs; a known or free-looking id is forwarded exactly as the caller wrote it (round 3 forwarded the resolver's alias, which turned claude-sonnet-4-5-20250929 into the retired claude-sonnet-4; fixed in round 4), and the catalog then sends a free-looking id to the free lane; an omitted model stays omitted (dailyBriefModelBoundary.test.ts, in the CI list). The internal actions keep trusting their callers (cron, other internals).
• Round 3, the fallback helpers resolve their start model. executeWithModelFallback, getFallbackChain and getLanguageModelWithFallback called isFreeModel on the raw start string, so retired-model-free, not-a-model and the LEGACY_ALIASES wire id nvidia/nemotron-3-super-120b-a12b:free died with Cannot read properties of undefined (reading 'pricing') before the callback ran (reachable from documents.pdfInsights.generatePDFInsights, which casts args.model). They now resolve through resolveOrFailClosed first: a free-looking id or the wire id runs on FREE_LANE_MODEL, any other unknown id throws UnknownModelError, an omitted model is still DEFAULT_MODEL. The "unlisted free model throws" and "paid unlisted keeps main's behaviour" rules are unchanged. generatePDFInsights catches the error and returns success: false with the message.
• Round 3, modelRouter.route records what it dispatched. It priced and classified the raw candidate string, so a pinned auto-free, the free-lane wire id or an unknown -free id was dispatched to the free lane but recorded costUsd > 0, isFree: false, actualTier: "premium". It now resolves the candidate once (normalizeModelInput) and uses the resolved alias for dispatch, pricing, isFree, actualTier, the cache entry and the modelRouterCalls row. Round 4 correction: an earlier version of this body said "I found no budget reader of these rows". That was wrong. modelRouterQueries.checkAgentBudget sums today's modelRouterCalls.costUsd into the agent's daily dollar budget, and route() calls it before every call that passes an agentId. The partial MODEL_PRICING table had rows for only 4 of the 19 approved free aliases, so the other 15 (laguna-s-2.1-free, laguna-xs-2.1-free, and 13 more) were priced at the $1/$5 default, recorded isFree: false / actualTier: "premium", and used the budget up: 7 laguna calls at 1,000 in / 100 out exhausted a $0.01 budget and call 8 was refused. That is pre-existing on main, but it is an A.0.10 cost-class defect, so it is fixed here (round 4 section).
• resolvePipelineModelSelection: resolvedModelId: resolved ?? DEFAULT_MODEL deleted. startPipelineRun and startPipelineRunInternal call assertKnownPipelineModel(args.modelId) before reservePipelineLaunchAdmission. parseModelKey (piRuntime.ts) only falls through to the raw parser when the resolver module fails to load; UnknownModelError propagates.
• FREE_LANE_MODEL = "nemotron-3-super-free" registered: APPROVED_MODELS, MODEL_SPECS (sdkId nvidia/nemotron-3-super-120b-a12b:free, zero price so isFreeModel is true), LEGACY_ALIASES for the wire id, web APPROVED_MODELS + MODEL_UI_INFO, and the zero-price row in shared/llm/modelCatalog.ts. auto-free and getBestFreeModel() return it (the laguna lanes expire on OpenRouter 2026-10-31). TIER_MODELS.free drops the three unapproved names that used to route to paid kimi-k2.6. The web auto-free literal is now pinned to the backend route resolver by the drift test (reverting getBestFreeModel to laguna used to fail only modelResolver.test.ts).
• apps/web/src/shared/llm/approvedModels.ts mirrors the resolver again (priority order, lanes, and the free-keeps-cost-class rule) and approvedModels.drift.test.ts keeps it that way.
• Provider-prefixed strings that wikiDreamingGraph.ts passes (google:models/gemini-3-flash-preview, google:models/gemini-3.1-pro-preview) were unknown and silently ran on the default; failing closed would break that job, so they now resolve to the Gemini ids they name (see discrepancies).

Where the fail-closed claim stops (stated, not hidden)

1. Decision needed (C item c), narrowed in round 2: a fallback helper (executeWithModelFallback, getFallbackChain, getLanguageModelWithFallback) called with an approved paid model that is not in MODEL_PRIORITY_ORDER (e.g. glm-4.7-flash, deepseek-r1, gemini-2.5-*, mistral-large, claude-opus-4) behaves as on main: it restarts at kimi-k2.6. Round 1 made all of them throw, which changed caller arguments outside A.0.10 (callers: pdfInsights with a user-supplied args.model, distiller, competingExplanations, diligenceCheckpointStructuring, piRuntime's AI-SDK fallback, where the public-research route is glm-4.7-flash). Only an unlisted free start model throws, because restarting it at the paid head is the A.0.10 violation. Pinned by modelResolver.test.ts ("keeps main's behaviour", 5 ids). Whether to list those ids in the order, or make the helpers refuse them, is yours.

2. Decision needed (C item a), narrowed in round 2: chainResolver.resolveChain still treats tierFloor as a floor ("this tier or above") for every tier. The A.0.10 gap is closed one layer up instead: route() filters a tier: "free" candidate list to free models (see the modelRouter.ts bullet), so a free request can no longer be dispatched to, or billed as, a paid model. The side effect is deliberate: a free request whose capability requirement (for example requireVision) only a paid model meets now fails instead of upgrading silently. If you want free to mean "free or better" for those callers, that is a product decision, not a fix.

3. Round 3: closed, not an exception. The billing-tier-safe fallbacks (TIER_SAFE_RUNTIME_FALLBACKS, paid) are now appended only for a paid primary; a free primary fails with the tier reason (see the round 3 bullet above). Round 2 called this path "the A.0.10 exception" because a reservation exists; that was false: there is no allowPaidFallback: true anywhere, the anonymous path holds no reservation, and a reservation alone is not the spec's exception. continueThread still falls back to DEFAULT_MODEL when the read of the thread fails (the lane is unknown then); only an unknown stored model is rethrown.

4. The web mirror's normalizeNodeBenchRuntimeModel only knows approved ids (it is a selector, it dispatches nothing); it keeps returning the profile default for other strings. Spec C.6 (fastAgentPanelStreaming.ts:1193) is unchanged: the site only receives ids already normalized by the resolvers above.

5. getModelWithFailover callers have no OpenRouter branch. All 16 generateWithProvider copies, fastAgentChat and ai.ts handle openai, anthropic and gemini; any other provider falls to new OpenAI().chat.completions.create({ model }). Since round 2 the free lane reaches that client as the wire id, not the alias, so it cannot silently run a paid model, but with OPENAI_BASE_URL unset the request goes to OpenAI and fails (404) rather than running on OpenRouter. The same is true today for kimi-k2.6 and every other OpenRouter model. Adding the OpenRouter branch is a change to all 18 call sites and belongs with PR 2's rows for those files; it is not done here. Other free catalog aliases (qwen3-coder-free, ...) still leave getModelWithFailover as aliases; PR 2 prunes them to the live :free list.

6. Round 3: closed. Round 2 listed unrecognised prefix-shaped ids (gpt-not-real with OPENAI_API_KEY set) as still reaching their own provider, justified as "operator-supplied wire ids (A.0.4)". That did not hold for the user-supplied model argument of the two daily-brief actions, and getLlmModel has no env override. getModelWithFailover now throws for them and both actions validate at the boundary (see the round 3 bullets). Unknown ids no longer reach a provider through this path.

7. Anonymous bounded lane (spec-kept exception; round 4 correction). For an anonymous caller shared/llm/fastAgentRuntimeContract.ts:19 (resolveFastAgentRequestedModel) replaces the client's model with the paid gemini-3.1-flash-lite-preview before normalizeNodeBenchRuntimeModel runs (fastAgentPanelStreaming.ts createThread and initiateAsyncStreaming). Consequences: an anonymous user who picks the free lane is answered by that paid model (this is the spec's bounded anonymous lane, bounded by the anonymous tier limits, and the spec moves only its id, in PR 2: row fastAgentRuntimeContract.ts:1), and an unknown model id is accepted silently on that path instead of throwing. This PR leaves it as is. The optional hardening (validate args.model before the override so an unknown id throws) is not done; it would change what an anonymous client sees and is a decision for you (Still open).

Eval harness (so the declared migration eval is real)

• scripts/eval-harness/runner.ts: --tier <prefix>, --seeds <N>; hard-coded claude-sonnet-4-6 / modelVersion removed (config.model = EVAL_MODEL, required when EVAL_BACKEND is not stub); createStubTelemetry replaced by invokeAgent; RunTelemetry.echoedModel added. The stub backend is still the default, so convex-mcp-eval-gate.yml's --list smoke is unchanged (verified: runner.ts --list prints the task list).
• agentInvoker.ts: the commented Messages call is real (model from options, max_tokens 8192, no sampling, tool_choice auto, blocks by type, bounded response read, timeout, response.model recorded as echoedModel). Round 2: only stop_reason: "end_turn" is a finished answer. max_tokens (on Claude 5.x adaptive thinking counts toward the cap, so a cap sized for 4.5 truncates), pause_turn, tool_use, refusal and a missing stop_reason are all errors, so migrationEvalCommand cannot exit 0 when every response is truncated. The response read is bounded for real: a content-length over the cap is refused before any byte is read, otherwise the stream is cancelled the moment it crosses 4,000,000 characters (round 1 called response.text() first, so a 6 MB body was buffered whole). An HTTP error, a response with no model, or a tool_use turn is likewise an error, never a clean run; taskSuccessRate stays 0 because acceptance criteria are not graded there. A tool_use block is a request, not an execution: no tool runs and no result is sent back, so requested tools go to a new RunTelemetry.requestedTools list and toolCalls (and with it toolCallCount and knowledgeReuseRate) counts only executed tools. The default bare mode used by migrationEvalCommand offers no tools and was not affected; the mcp_* modes were.
• runner.ts main(): with a real backend (EVAL_BACKEND not stub) it exits 1 when any run has an error, after writing every telemetry file and listing the failures. Before, ten HTTP 400 responses wrote ten error rows and exited 0, so the declared migrationEvalCommand could pass without one successful provider call. The stub smoke (--list, EVAL_BACKEND=stub) still exits 0.
• production-agent.json migrationEvalCommand -> EVAL_BACKEND=anthropic EVAL_MODEL=<candidate> npx tsx scripts/eval-harness/runner.ts --tier tier1 --seeds 3.
• scripts/run-comprehensive-eval.ts: minimax-m2.1 removed; --models must be a canonical approved id (resolveModelAlias(m) === m), so unknown ids and legacy aliases such as gpt-4o-mini (which resolves to gpt-5.4-mini and would disqualify every row only after the paid run) abort before any request; each row records resolvedModelId (returned by runPersonaEpisodeEval) and fallbackUsed, including error rows; the run exits 1 if any row has no resolved identity (a Convex deployment built before this PR), resolved to a different id, has no explicit fallbackUsed: false, or its retry switched models.

Test proof: fails before, passes after (A.0.6)

Round 4 review fixes (this push)

Plain-language version: the previous push made two things worse for people who name a model themselves. (1) A signed-in user who typed a model that is still live (for example claude-sonnet-4-5-20250929) had it silently swapped for an older, already-retired one, because the code forwarded the resolver's lossy shorthand instead of what they typed. (2) The shared catalog threw an error for 16 ids the repo still recognises and that main sent through. Both are fixed by validating at the door and then forwarding the caller's own string, and by listing the recognised ids the catalog must let through. Also fixed: Claude Haiku 4.5 research runs were sent a thinking setting that model rejects; free models were counted as paid spend against an agent's dollar budget; a research run for a made-up model was queued before being rejected; and four test files could not notice an assistant "prefill" being added to a request.

#	Defect	Fix	Proof
1	daily-brief actions forwarded the resolver alias (A.0.4: PR 1 moves no ids)	dailyBriefWorker.runNextTask / dailyBriefPersonalOverlay.runNextPersonalTask call normalizeModelInput only to validate and forward args unchanged	dailyBriefModelBoundary.test.ts: all 16 pass-through ids forwarded byte-for-byte and the catalog sends the same string to the provider; unknown ids still throw
2	getModelWithFailover threw for ids main passed through	LEGACY_PASSTHROUGH_IDS (16 ids: the 3 dated Claude snapshots, 10 gpt-*, 3 gemini-*) is part of "known"; anthropicWireId returns the Claude snapshots unchanged. Left rejecting on purpose: dotted claude-opus-4.6/4.5, claude-sonnet-4.5 (never wire ids, the provider 404s) and gpt-oss* (OpenRouter free aliases the gpt- prefix used to send to paid OpenAI)	anthropicWireIds.test.ts: pass-through for each id; a drift guard that no provider-prefixed LEGACY_ALIASES key is rejected; the throw assertion for claude-haiku-4-5-20251001 replaced (old expectation recorded in a comment); a made-up snapshot still throws
3	extended thinking sent adaptive + effort to Haiku 4.5	ADAPTIVE_THINKING_WIRE_IDS (claude-sonnet-4-6, claude-opus-4-7); all other ids keep main's {type:"enabled", budget_tokens} with no effort; under 1,024 tokens no thinking	extendedThinking.requestShape.test.ts drives advanceRun for claude-haiku-4-5, claude-haiku-4.5, claude-haiku-4-5-20251001; builder test covers the dated 4.5 snapshots. The provider's 400 for adaptive/effort on Haiku 4.5 is read from the migration guide, not observed
4	free aliases billed and classified as paid, using up checkAgentBudget	isFreeAlias (resolver cost class) drives calculateCost, isFree, actualTier, the cached path and the free-tier filter	modelRouter.requestShape.test.ts: every approved free alias (19) through 20 route() calls and the real checkAgentBudget over the logged rows: cost 0, isFree, actualTier: "free", budget never used; a paid control model does exhaust the budget. Against the previous modelRouter.ts: 15 of the 19 fail
5	PR-body claims about the anonymous lane	corrected above (item 7); no code change	n/a
6	disposition rows modelResolver.ts:143/:564/:710 said "defer"	marked partly done in PR 1 (table below)	n/a
7	requestExtendedRun queued a run for an unknown model	resolveCheckpointModel (the same function buildCheckpointRequestBody uses) runs before the insert	extendedThinking.requestShape.test.ts: claude-not-real, invented-free, kimi-k2.6, gpt-5.4 rejected with no insert and no scheduled job; claude-sonnet-4.6, claude-haiku-4-5, claude-haiku-4-5-20251001 accepted and stored as given
8	four request-shape suites could not see a prefill	each now asserts the outgoing conversation ends on a user turn (messages.at(-1).role === "user"; Gemini contents.at(-1).role)	I added a terminal assistant (or Gemini model) turn to each source in turn: benchmark.ts, llmTools.ts, researchWritingTools.ts, modelRouter.ts, judgeScratchnodeChatVideo.mjs: each suite failed (1, 1, 1, 5, 1 tests); all restored
9	CI	see Discrepancies item 1 and the CI paragraph below	n/a

Measured at the round 4 head: CI runtime-smoke list 47 files / 660 tests pass (619 at round 3); npx tsc --noEmit exit 0; npx tsc -p backend/convex --noEmit exit 0. npm run test:run exits 1 exactly as before: app-vitest 8 failed files / 197 passed (22 tests), mcp-local 21 failed files (160 tests, better-sqlite3 binding), convex-mcp 3, openclaw 1; none of the failing files is touched by this PR, and the counts equal the origin/main row in "Check output". No paid provider call was made.

CI on the round 3 head a9e09067 (read, not caused by this change): Typecheck, Runtime smoke, Build, Tier B vs preview URL pass, and those four are the only required contexts in main's branch protection (read through the API). eval-gate and delta-dogfood-gate failed inside npm ci || npm install with an npm edgesOut crash; no manifest changed and both passed on cabf937b, but that alone did not prove the crash is pre-existing. Round 4 push (d70d188a): both failed again at the same step. Reproduced outside this PR: npm install --no-audit --no-fund --ignore-scripts on a copy of packages/convex-mcp-nodebench/package.json (byte-identical to origin/main; the package has no lockfile, so every run resolves the registry afresh) fails locally with the same Cannot read properties of null (reading 'edgesOut') (npm 10.9.7). So it is an npm/registry-state failure that does not depend on this PR's diff; it cannot be fixed here without changing that package's manifest, which is out of scope. Pipeline quality benchmark fails because NODEBENCH_API_URL is unset (also red on main). The Branch name check fails on claude/model-refresh-nodebenchai (details under Discrepancies item 1).

Round 3 review fixes

New or extended: agentRuntimeTierFallback.test.ts (free primary on the anonymous, free and pro tiers through the real isModelAllowedForTier / getBestModelForTier; post-stream requireFree; a paid primary keeps its fallbacks), anthropicWireIds.test.ts (unknown prefix-shaped and unlisted ids throw with their provider configured; the former claude-haiku-4-5-20251001 pass-through now asserts a throw, reason in a comment), dailyBriefModelBoundary.test.ts (new, both public actions, in the CI list), modelResolver.test.ts (x-free, retired-model-free, not-a-model and the free-lane wire id through all three fallback helpers), modelRouter.requestShape.test.ts (pinned auto-free, wire id, invented-free: pricing, isFree, actualTier, cache, audit row), triSearch.test.ts and benchmark.requestShape.test.ts (no Gemini sampling keys; the old temperature === 0 assertion is replaced, not kept), judgeScratchnodeChatVideo.requestShape.test.ts (new, in the CI list).

Against the previous head cabf937b source (the ten changed source files stashed, the new tests kept): 8 of 8 files fail, 25 tests fail, 127 pass (the 127 are behaviours that were already right).

Measured at the round 3 head: CI runtime-smoke list 47 files / 619 tests pass (it was 45 / 592 at cabf937b); npx tsc --noEmit exit 0; npx tsc -p backend/convex --noEmit exit 0; node --test scripts/nodebench-packet-contracts.node.mjs 20/20. The earlier claim handed to review named fa669cca (37 files / 504 tests, 104/104); that is stale and is superseded by these numbers. Not clean, and not caused by this PR: npm run test:run and npx vitest run backend/convex shared workers scripts fail in files that fail identically on origin/main in a clean worktree (better-sqlite3 native binding missing for the whole mcp-local segment; convex-test cannot find _generated for fastAgentPanelStreaming.anonymousAccess.test.ts; ResizeObserver missing in seven apps/web UI files, 21 tests, same 21 on main; workers/node/commandBridge, sharedContextRoute, hyperloopRoute, deeptrace/evidenceTruth and others). No paid provider call was made.

Round 2 review fixes

New or extended: evalHarness.test.ts (end_turn only, bounded stream, content-length), modelResolver.test.ts (paid unlisted ids keep main's behaviour), runtimeRouting.test.ts (free request on all five executor branches), modelRouter.requestShape.test.ts (free cap with requireVision, free-lane pricing, outgoing model strings and top_p/top_k), anthropicWireIds.test.ts (wire id, free route aliases, prefix-shaped invented ids), triSearch.test.ts (condenseQuery), instagramClaimVerification.test.ts (new, added to the CI runtime-smoke list), pipelineModelBoundary.test.ts (rewritten, see below), mcp-local anthropicRequestShape.test.ts (polish model id).

Against the previous head 7096f29f source (the seven changed source files checked out from it, the new tests kept): 7 of 7 files fail, 18 tests fail, 82 pass (the 82 are behaviours that were already right). The boundary test is different: the old version still passed with assertKnownPipelineModel moved after reservePipelineLaunchAdmission, because the Convex mutation rolls the admission write back on throw. It now fills the owner's launch window first (control call: a valid launch gets "launch limit reached"), so admission-first would surface the limit error instead of Unknown model. Verified by swapping the two lines: 1 of 4 fails; reverted.

Round 1 review fixes

New or extended in this push: modelResolver.test.ts (fallback helpers, normalizeNodeBenchRuntimeModel), runtimeRouting.test.ts, approvedModels.drift.test.ts (pins every selector option to resolvePipelineModelSelection), anthropicWireIds.test.ts (getModelWithFailover fails closed), evalHarness.test.ts (runner exit status, requested vs executed tools, missing model), comprehensiveEvalModels.test.ts (canonical models, missing identity, error rows keep identity), extendedThinkingRunner.test.ts and extendedThinking.requestShape.test.ts (usage-based charge, run-level bound, through the real advanceRun), modelRouter.requestShape.test.ts, benchmark.requestShape.test.ts, providerCopies.requestShape.test.ts, agentChatActions.failClosed.test.ts, toolGenRequestShape.test.ts (generate-tool), triSearch.test.ts (model in the URL), mcp-local anthropicRequestShape.test.ts.

Against the previous head fa669cca (clean worktree, only these test files copied in, minus agentChatActions.failClosed.test.ts): 10 of 14 files fail, 48 tests fail, 114 pass (the 114 are behaviours that were already right; benchmark.requestShape, anthropicRequestShape and 3 of 4 modelRouter cases pass on the old source because they pin behaviour PR 1 already had). Mutation checks on the new source, each made a test fail and was then reverted: dropping FREE_LANE_MODEL from the priority order, removing the free-keeps-cost-class rule, restoring the paid push in modelRouter, reverting one generateWithProvider copy to content[0], disabling the runner exit code, loosening identityViolations, zeroing the thinking charge, removing the agentChatActions rethrow, restoring the gpt-5.4-mini default in getModelWithFailover, un-filtering the free primary's candidates, and pointing the web auto-free literal back at laguna.

Round 0 (original push)

New or extended tests (all in the CI runtime-smoke list): events.requestShape.test.ts, extendedThinkingRunner.test.ts, triSearch.test.ts, toolGenRequestShape.test.ts, evalHarness.test.ts, anthropicWireIds.test.ts, comprehensiveEvalModels.test.ts, pipelineModelBoundary.test.ts, approvedModels.drift.test.ts, plus the extended modelResolver.test.ts and pipelineModelRoutes.test.ts. Each request-shape test builds the request through the repo's own builder or exported function with a fake transport and asserts the forbidden keys (temperature, top_p, top_k, thinking.budget_tokens, thinkingBudget) are absent and the model string is the expected wire id.

Run against the pre-change source (a clean worktree at 3a716d8b with only the new/extended test files copied in, and export added to the three events.ts functions so the old behaviour could be exercised): 11 of 11 files fail, 51 tests fail, 53 pass (the 53 are unchanged behaviours). Examples: AssertionError: temperature must be absent (all three events.ts paths); expected { thinkingBudget: +0 } to deeply equal { thinkingLevel: 'low' }; expected { model: 'claude-sonnet-4.6', ... } to match { model: 'claude-sonnet-4-6' }; expected 'SCAFFOLD: Anthropic API call not yet wired' to match /HTTP 400/; unresolvedModels is not a function; parseModelKey is not a function; an unknown pipeline model id reached the workflow component instead of throwing. After the change: all 104 pass.

Check output

nodekit.yaml commands.check = npm run test:run. It does not pass, and it fails the same way on origin/main (so the failures are not from this change): the same test files fail in every segment on both trees, from the missing better-sqlite3 native binding in the packages/* segments and 8 UI test files in apps/web.

this branch (round 4)  app-vitest: Test Files 8 failed | 197 passed | 4 skipped (209)   Tests 22 failed | 1432 passed   (same failing files as origin/main; mcp-local 21 failed files / 160 tests, convex-mcp 3, openclaw 1, identical)
this branch (round 2)  app-vitest: Test Files 7 failed | 198 passed | 4 skipped (209)   Tests 21 failed | 1433 passed   (identical to round 1; mcp-local, convex-mcp and openclaw-mcp segments identical too)
this branch (round 1)  app-vitest: Test Files 7 failed | 198 passed | 4 skipped (209)   Tests 21 failed | 1433 passed
this branch (round 0)  app-vitest: Test Files 8 failed | 197 passed | 4 skipped (209)   Tests 23 failed | 1430 passed
origin/main            app-vitest: Test Files 8 failed | 196 passed | 4 skipped (208)   Tests 22 failed | 1426 passed
mcp-local-vitest:     21 failed | 24 passed | 7 skipped (52); 160 failed | 709 passed   (main: 21 failed | 23 passed (51); 160 failed | 707 passed)
convex-mcp-vitest:    3 failed (3); 9 failed | 8 passed | 63 skipped                     (both trees)
openclaw-mcp-vitest:  1 failed (1) "Could not locate the bindings file"                 (both trees)

The one extra app test failure on the branch (RichNotebookEditor.finder.test.tsx, "Cmd+F opens ...") failed once in the full branch run; run alone it passes on both trees (that file's "Escape closes the finder" case fails on both). The new approvedModels.drift.test.ts accounts for the +1 passing file.

Targeted gates, all on this branch:

npx tsc --noEmit --pretty false                          exit 0
npx tsc -p backend/convex --noEmit --pretty false        exit 0
CI runtime-smoke list (47 files, round 4)               Test Files 47 passed (47)   Tests 660 passed (660)
node --test scripts/nodebench-packet-contracts.node.mjs  tests 20, pass 20, fail 0
the 15 files in the round 1 proof above                  Test Files 15 passed (15)   Tests 165 passed (165)

No paid provider call was made. eslint over the changed files reports one error, prefer-const at events.ts:2768, which is not a line this PR touches.

Runtime-site dispositions (spec A.2.10: 129 runtime-role sites + 16 class E/R rows = 145; migrate 110, freeze 33, defer 2)

"Spec disposition" is the spec's. "This PR" says what PR 1 did at that site; every other migrate row is a deferred id move in PR 2, which starts when this PR is merged.

Site	Spec disposition	This PR (PR 1)
backend/convex/domains/agents/mcp_tools/models/modelResolver.ts:137	freeze	freeze: DEFAULT_MODEL kimi-k2.6 listed; evidence-gated (eval:capability, 187+244 history files keyed by id) -> Part B v2 with receipts naming kimi-k3
backend/convex/domains/agents/mcp_tools/models/modelResolver.ts:140	freeze	freeze: FALLBACK_MODEL gpt-5.4-mini active
backend/convex/domains/agents/mcp_tools/models/modelResolver.ts:143	migrate	partly done in PR 1: FREE_LANE_MODEL placed at the head of the free segment of MODEL_PRIORITY_ORDER (so a free start model never restarts at the paid head); the id updates (3.1-flash-lite-preview -> 3.5-flash-lite, claude 4.x -> 5.5, pruning) stay in PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/mcp_tools/models/modelResolver.ts:181	freeze	freeze: advisor kimi-k2.6 listed
backend/convex/domains/agents/mcp_tools/models/modelResolver.ts:183	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/mcp_tools/models/modelResolver.ts:191	freeze	freeze: background: gemini-3.1-pro-preview first (only Pro); gpt-5.4 active
backend/convex/domains/agents/mcp_tools/models/modelResolver.ts:270	migrate	PR 1: auto-free -> FREE_LANE_MODEL (via getBestFreeModel). auto-fast id moves in PR 2.
backend/convex/domains/agents/mcp_tools/models/modelResolver.ts:380	freeze	freeze: gpt-5.4 family active (gpt-5.4-nano retires 2027-04-01: advisory)
backend/convex/domains/agents/mcp_tools/models/modelResolver.ts:401	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/mcp_tools/models/modelResolver.ts:422	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/mcp_tools/models/modelResolver.ts:450	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/mcp_tools/models/modelResolver.ts:495	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/mcp_tools/models/modelResolver.ts:564	migrate	partly done in PR 1: the nemotron-3-super-free row added to MODEL_SPECS (the FREE_LANE_MODEL registration, zero price, isFreeModel true); pruning the free specs to the live :free list stays in PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/mcp_tools/models/modelResolver.ts:710	migrate	partly done in PR 1: LEGACY_ALIASES gained the nvidia/nemotron-3-super-120b-a12b:free wire-id row and the two google:models/* strings (wikiDreamingGraph.ts); fixing the two lossy maps (claude-sonnet-4-5-20250929, claude-opus-4-5-20251101) and adding the old->new rows stays in PR 2 (condition: PR 1 merged). Round 4: because the maps stay lossy in PR 1, no public path forwards a resolver alias to a provider
backend/convex/domains/agents/mcp_tools/models/modelResolver.ts:1234	migrate	PR 1: getBestFreeModel returns FREE_LANE_MODEL. Live auto-free traffic moves from laguna here; the PR carries needs-live-eval. MODEL_PRIORITY_ORDER now lists it at the head of the free segment (web mirror too).
backend/convex/domains/agents/mcp_tools/models/modelResolver.ts:908	migrate	PR 1: fail closed (getLanguageModelSafe, normalizeModelInput, normalizeNodeBenchRuntimeModel, allowPaidFallback, fallback helpers); see "Where the fail-closed claim stops".
shared/llm/modelCatalog.ts:146	migrate	defer: PR 2 (condition: PR 1 merged)
shared/llm/modelCatalog.ts:195	migrate	defer: PR 2 (condition: PR 1 merged)
shared/llm/modelCatalog.ts:265	freeze	freeze: DEFAULT_FALLBACK_MODEL gpt-5.4-mini active
shared/llm/modelCatalog.ts:512	migrate	defer: PR 2 (condition: PR 1 merged)
shared/llm/modelCatalog.ts:649	migrate	defer: PR 2 (condition: PR 1 merged)
shared/llm/modelCatalog.ts:696	migrate	defer: PR 2 (condition: PR 1 merged)
shared/llm/modelCatalog.ts:765	migrate	defer: PR 2 (condition: PR 1 merged)
apps/web/src/shared/llm/approvedModels.ts:9	migrate	PR 1: web mirror gains nemotron-3-super-free; approvedModels.drift.test.ts added.
apps/web/src/shared/llm/approvedModels.ts:478	freeze	freeze: DEFAULT kimi-k2.6 listed
apps/web/src/shared/llm/approvedModels.ts:481	migrate	PR 1: priority order and lanes aligned with the resolver; drift test enforces it.
apps/web/src/shared/llm/pipelineModelRoutes.ts:50	migrate	PR 1: auto-free -> FREE_LANE_MODEL. auto-fast id moves in PR 2.
apps/web/src/features/redesign/components/UniversalComposer.tsx:38	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/redesign/chatRuns.ts:257	migrate	defer: PR 2 (condition: PR 1 merged)
nodeagent.yaml:23	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/scratchnodeLiveCues.ts:116	migrate	defer: PR 2 (condition: PR 1 merged)
shared/search/triSearch.ts:58	migrate	PR 1: thinkingBudget:0 -> thinkingLevel:"low"; round 3: temperature also removed from both Gemini calls (rerank, condenseQuery). Id moves in PR 2.
shared/llm/router.ts:125	migrate	defer: PR 2 (condition: PR 1 merged)
shared/llm/router.ts:131	migrate	defer: PR 2 (condition: PR 1 merged)
shared/llm/router.ts:140	migrate	defer: PR 2 (condition: PR 1 merged)
shared/llm/router.ts:157	migrate	defer: PR 2 (condition: PR 1 merged)
shared/llm/fastAgentRuntimeContract.ts:1	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/events.ts:2511	migrate	PR 1: request shape (no temperature, max_tokens 2048, blocks by type). Id moves in PR 2.
backend/convex/events.ts:2610	migrate	PR 1: request shape. Id moves in PR 2.
backend/convex/events.ts:2927	migrate	PR 1: request shape. Id moves in PR 2.
backend/convex/domains/product/extendedThinking.ts:54	migrate	PR 1: adaptive thinking + output_config.effort; wire id from an explicit table. Id moves in PR 2.
backend/convex/domains/financialOperator/realExtractors.ts:136	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/adapters/types.ts:28	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/adapters/registerDefaultAdapters.ts:52	freeze	freeze: kimi-k2.6 / gpt-5.4 family / glm-4.7-flash active; no announced retirement (gpt-5.4-nano 2027-04-01 advisory)
backend/convex/domains/agents/adapters/anthropic/anthropicReasoningAdapter.ts:85	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/adapters/openai/openaiAgentsAdapter.ts:78	freeze	freeze: kimi-k2.6 / gpt-5.4 family / glm-4.7-flash active; no announced retirement (gpt-5.4-nano 2027-04-01 advisory)
backend/convex/domains/agents/adapters/vercel/vercelAiSdkAdapter.ts:65	freeze	freeze: kimi-k2.6 / gpt-5.4 family / glm-4.7-flash active; no announced retirement (gpt-5.4-nano 2027-04-01 advisory)
backend/convex/domains/agents/runtimeRouting.ts:44	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/runtimeTierFallback.ts:207	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/fastAgentPanelStreaming.ts:6019	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/digestAgent.ts:572	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/ai/models/modelRouter.ts:105	migrate	PR 1: dropped the three unapproved free names. Id moves in PR 2.
backend/convex/domains/ai/models/capabilityRegistry.ts:106	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/ai/models/autonomousModelResolver.ts:36	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/ai/models/freeModelDiscovery.ts:137	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/ai/models/livePerformanceEval.ts:465	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/evaluation/llmJudge.ts:49	freeze	freeze: judge lane kimi-k2.6 frozen for comparability; only the no-OpenRouter-key fallbacks move (claude-sonnet-4.6 -> 5.5, gemini-3-flash-preview -> 3.8, ...
backend/convex/domains/evaluation/agentRunJudge.ts:39	freeze	freeze: judge lane kimi-k2.6 frozen for comparability; only the no-OpenRouter-key fallbacks move (claude-sonnet-4.6 -> 5.5, gemini-3-flash-preview -> 3.8, ...
backend/convex/domains/evaluation/ultraLongChat/judge.ts:58	freeze	freeze: judge lane kimi-k2.6 frozen for comparability; only the no-OpenRouter-key fallbacks move (claude-sonnet-4.6 -> 5.5, gemini-3-flash-preview -> 3.8, ...
backend/convex/domains/evaluation/comprehensiveEval.ts:849	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/evaluation/dogfood/screenshotQa.ts:59	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/orchestrator/geminiVideoWrapper.ts:230	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/search/fusion/benchmark.ts:72	freeze	PR 1: temperature deleted from the Anthropic judge request; round 3: also from the Gemini judge request (generationConfig), the OpenAI judge keeps temperature: 0. freeze: judge kimi-k2.6 / DEFAULT
backend/convex/domains/search/rag.ts:340	freeze	freeze: kimi-k2.6 / gpt-5.4 family / glm-4.7-flash active; no announced retirement (gpt-5.4-nano 2027-04-01 advisory)
backend/convex/domains/search/ragEnhanced.ts:350	freeze	freeze: kimi-k2.6 / gpt-5.4 family / glm-4.7-flash active; no announced retirement (gpt-5.4-nano 2027-04-01 advisory)
backend/convex/domains/search/linkupClient.ts:11	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/tools/teachability/teachingAnalyzer.ts:117	freeze	freeze: kimi-k2.6 / gpt-5.4 family / glm-4.7-flash active; no announced retirement (gpt-5.4-nano 2027-04-01 advisory)
backend/convex/domains/knowledge/entityInsights.ts:601	freeze	freeze: kimi-k2.6 / gpt-5.4 family / glm-4.7-flash active; no announced retirement (gpt-5.4-nano 2027-04-01 advisory)
backend/convex/domains/research/dealFlow.ts:45	migrate	PR 1: anthropic branch sends the wire id, reads blocks by type (all 16 generateWithProvider copies). Id moves in PR 2.
backend/convex/domains/ai/ai.ts:79	freeze	freeze: kimi-k2.6 / gpt-5.4 family / glm-4.7-flash active; no announced retirement (gpt-5.4-nano 2027-04-01 advisory)
backend/convex/domains/research/modelComparison.ts:143	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/agentLoop.ts:248	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/emailAgent.ts:209	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/dueDiligence/ddOrchestrator.ts:59	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/traceOrchestrator.ts:51	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/blips/blipVerification.ts:51	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/research/narrative/newsroom/agents/analystAgent.ts:60	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/enrichment/llmEnrichment.ts:64	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/workflows/dailyLinkedInPost.ts:608	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/social/linkedinPrePostVerification.ts:35	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/mcp_tools/reasoningTool.ts:85	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/canonicalPlanner.ts:228	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/swarmOrchestrator.ts:125	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/arbitrage/tools/contradictionDetection.ts:65	freeze	freeze: kimi-k2.6 / gpt-5.4 family / glm-4.7-flash active; no announced retirement (gpt-5.4-nano 2027-04-01 advisory)
backend/convex/domains/verification/entailmentChecker.ts:124	freeze	freeze: kimi-k2.6 / gpt-5.4 family / glm-4.7-flash active; no announced retirement (gpt-5.4-nano 2027-04-01 advisory)
backend/convex/domains/documents/quickCapture/voiceMemos.ts:38	freeze	freeze: kimi-k2.6 / gpt-5.4 family / glm-4.7-flash active; no announced retirement (gpt-5.4-nano 2027-04-01 advisory)
backend/convex/domains/documents/pdfAnalysis.ts:71	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/actions/externalOrchestrator.ts:95	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/product/wikiDreamingGraph.ts:362	migrate	PR 1: provider-prefixed strings added to LEGACY_ALIASES so they resolve instead of failing closed. Id moves in PR 2.
backend/convex/tools/meta/dynamicPromptEnhancer.ts:76	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/lib/xaiClient.ts:221	freeze	freeze: xAI not inventoried in frontier.json
backend/convex/domains/ai/realtimeTranscription.ts:25	defer	defer: gpt-4o-transcribe retires 2027-02-26 -> gpt-transcribe (advisory, >90 d)
backend/convex/domains/search/embedRowOnUpdate.ts:65	freeze	freeze: text-embedding-3-small; 1536-dim columns schema.ts:1986,6979,7099; change = re-embed
backend/convex/domains/integrations/billing/rateLimiting.ts:24	migrate	defer: PR 2 (condition: PR 1 merged)
workers/node/agentHarness.ts:2425	migrate	defer: PR 2 (condition: PR 1 merged)
workers/node/agentHarness.ts:2552	freeze	freeze: gpt-5.4 family active; max_tokens + temperature:0 body is a precondition fix for any gpt-6 move (defer params)
workers/node/agentHarness.ts:2750	migrate	defer: PR 2 (condition: PR 1 merged)
workers/node/harnessRuntime.ts:189	migrate	defer: PR 2 (condition: PR 1 merged)
workers/node/nemoclaw/agentRunner.ts:79	migrate	defer: PR 2 (condition: PR 1 merged)
workers/node/claudeCodeBridge.ts:106	migrate	defer: PR 2 (condition: PR 1 merged)
workers/node/pipeline/searchPipeline.ts:924	migrate	defer: PR 2 (condition: PR 1 merged)
workers/node/routes/search.ts:2393	migrate	defer: PR 2 (condition: PR 1 merged)
workers/node/routes/session.ts:46	migrate	defer: PR 2 (condition: PR 1 merged)
packages/mcp-local/src/tools/llmTools.ts:35	migrate	PR 1: temperature deleted. The id moves in PR 2 (spec rows); PR 3 is only the release that ships it.
packages/mcp-local/src/tools/researchWritingTools.ts:36	migrate	PR 1: temperature deleted. The id moves in PR 2 (spec rows); PR 3 is only the release that ships it.
packages/mcp-local/src/tools/visionTools.ts:88	migrate	defer: PR 2 (condition: PR 1 merged)
packages/mcp-local/src/index.ts:1944	migrate	defer: PR 2 (condition: PR 1 merged)
packages/mcp-local/src/tools/webTools.ts:239	migrate	defer: PR 2 (condition: PR 1 merged)
packages/mcp-local/src/tools/entityLookupTools.ts:41	migrate	defer: PR 2 (condition: PR 1 merged)
packages/mcp-local/src/profiler/modelRouter.ts:25	migrate	defer: PR 2 (condition: PR 1 merged)
packages/mcp-local/src/tools/founderTools.ts:2141	migrate	defer: PR 2 (condition: PR 1 merged)
packages/eval-engine/src/judges/text-judge.ts:94	migrate	defer: PR 2 (condition: PR 1 merged)
services/ingestion-extract/extractors/claim_extractor.py:231	migrate	defer: PR 2 (condition: PR 1 merged)
docker-compose.yml:21	migrate	defer: PR 2 (condition: PR 1 merged)
scripts/tool-gen/analyze-gap.mjs:57	migrate	PR 1: request builder (no sampling, blocks by type). Id moves in PR 2.
scripts/eval-harness/agentInvoker.ts:29	migrate	PR 1: real Messages call restored (model from EVAL_MODEL). Default id moves in PR 2.
scripts/run-comprehensive-eval.ts:535	freeze	PR 1: minimax-m2.1 removed; unknown --models aborts; rows record resolvedModelId / fallbackUsed. freeze: judge kimi-k2.6 (comparability). SAME PR: remove 'minimax-m2.1' (not an approved id) from availableModels; every --models value must resolve or the...
package.json:137	freeze	freeze: eval:capability hard-codes --models kimi-k2.6 (the unchanged default). The PR 2 receipt is NOT this script: it is `npx tsx scripts/run-comprehensiv...
scripts/run-enterprise-investigation-eval.ts:156	migrate	defer: PR 2 (condition: PR 1 merged)
scripts/generate-live-digest.ts:35	migrate	defer: PR 2 (condition: PR 1 merged)
scripts/ui/runDogfoodGeminiQa.mjs:44	migrate	defer: PR 2 (condition: PR 1 merged)
scripts/qa/redesignAgentQa.ts:36	migrate	defer: PR 2 (condition: PR 1 merged)
scripts/ui/judgeScratchnodeChatVideo.mjs:22	migrate	PR 1: thinkingBudget:0 -> thinkingLevel:"low"; round 3: temperature: 0.2 removed. Id moves in PR 2.
scripts/ci-check-models.sh:20	freeze	freeze: dead guard (unwired; scans nonexistent dirs; denylist contains active ids); do not wire as-is
.github/workflows/ci.yml:102	freeze	PR 1: nineteen test files added to the runtime-smoke list (nine in round 0, seven in round 1, one in round 2, two in round 3: dailyBriefModelBoundary.test.ts, judgeScratchnodeChatVideo.requestShape.test.ts). The list went from 28 to 47 files; round 2 said sixteen and missed instagramClaimVerification.test.ts. freeze: test list; exact-id tests move to the resolver in the same PR
.github/workflows/convex-mcp-eval-gate.yml:90	defer	defer: runs --list only ($0) - not a receipt. Becomes the real nightly (runner.ts --tier tier1 --seeds $EVAL_SEEDS) only inside Environment `model-bum...
.github/workflows/daily-tool-gen.yml:52	migrate	defer: PR 2 (condition: PR 1 merged)
.github/workflows/attrition-qa.yml:111	freeze	freeze: active on 2026-10-10; no announced retirement
apps/web/src/features/editor/components/UnifiedEditor/useInlineFastAgent.ts:248	migrate	defer: PR 2 (condition: PR 1 merged)
shared/llm/modelCatalog.ts:63	migrate	PR 1: zero-price row for nemotron-3-super-free; anthropicWireId/anthropicText helpers. 5.5 / 3.8 rows in PR 2.
production-agent.json:114	migrate	PR 1: migrationEvalCommand -> real runner. pinnedModel moves in PR 2.
backend/convex/domains/agents/batchAPI.ts:514	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/ai/models/modelRouter.ts:434	migrate	PR 1: temperature not sent to Anthropic/Google candidates.
backend/convex/crons.ts:764	freeze	freeze: hourly free-model discovery + daily livePerformanceEval: existing automation that writes a DB table and never edits defaults; its seeds move with f...
backend/convex/domains/evaluation/evaluationPrompts.ts:23	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/domains/agents/mcp_tools/models/promptCaching.ts:148	migrate	defer: PR 2 (condition: PR 1 merged)
backend/convex/schema.ts:3775	freeze	freeze: persisted model strings (also :7542,:7593,:8025) read through LEGACY_ALIASES; rows never rewritten
workers/node/pricingScraper.ts:52	migrate	defer: PR 2 (condition: PR 1 merged)
packages/eval-engine/src/telemetry/cost-tracker.ts:23	migrate	defer: PR 2 (condition: PR 1 merged)
scripts/eval-harness/deeptrace/llm-proposer.ts:32	migrate	defer: PR 2 (condition: PR 1 merged)
scripts/pricing/anthropicApiPricing.ts:162	migrate	defer: PR 2 (condition: PR 1 merged)
packages/mcp-local/src/__tests__/gaiaCapabilityEval.test.ts:790	migrate	defer: PR 2 (condition: PR 1 merged)
packages/mcp-local/src/benchmarks/searchQualityEval.ts:281	migrate	defer: PR 2 (condition: PR 1 merged)
apps/web/src/features/financialOperator/components/FinancialOperatorOverlay.tsx:29	migrate	defer: PR 2 (condition: PR 1 merged)
apps/web/src/features/benchmarks/components/ModelLeaderboard.tsx:13	freeze	freeze: labels of recorded benchmark runs (also WorkbenchRunsTable.tsx:20-30, ModelEvalDashboard.tsx:95-102); historical display data

getLanguageModelSafe call sites (93 listed; 92 are calls, #34 is text inside a console.log; round 2 removed the former #86, a paid fallback)

Row #34 (glmFlashWithReasoning.ts:495) is a string printed by console.log as an example, not a call. All other rows are real calls.

allowPaidFallback: true is permitted only where the same function awaits reserveLlmRequestInternal. None of the 93 sites passes it in this PR. normalizeModelInput follows the same rule.

#	Call site	Argument	Decision
1	domains/agents/adapters/langgraph/langgraphAdapter.ts:186	model	fails closed (no reservation in this function)
2	domains/agents/adapters/vercel/vercelAiSdkAdapter.ts:84	model	fails closed (no reservation in this function)
3	domains/agents/agentLoop.ts:248	"qwen3-coder-free"	approved alias literal: resolves; unknown would throw
4	domains/agents/arbitrage/agent.ts:51	modelName	fails closed (no reservation in this function)
5	domains/agents/coordinator/agent.ts:40	modelName	fails closed (no reservation in this function)
6	domains/agents/core/coordinatorAgent.ts:1378	model	fails closed (no reservation in this function)
7	domains/agents/core/subagents/document_subagent/documentAgent.ts:18	modelName	fails closed (no reservation in this function)
8	domains/agents/core/subagents/document_subagent/documentAgentWithMetaTools.ts:47	model	fails closed (no reservation in this function)
9	domains/agents/core/subagents/document_subagent/documentAgentWithMetaTools.ts:144	model	fails closed (no reservation in this function)
10	domains/agents/core/subagents/dossier_subagent/dossierAgent.ts:35	model	fails closed (no reservation in this function)
11	domains/agents/core/subagents/entity_subagent/entityResearchAgent.ts:83	model	fails closed (no reservation in this function)
12	domains/agents/core/subagents/media_subagent/mediaAgent.ts:18	modelName	fails closed (no reservation in this function)
13	domains/agents/core/subagents/openbb_subagent/openbbAgent.ts:59	model	fails closed (no reservation in this function)
14	domains/agents/core/subagents/openbb_subagent/openbbAgent.ts:191	model	fails closed (no reservation in this function)
15	domains/agents/core/subagents/sec_subagent/secAgent.ts:19	modelName	fails closed (no reservation in this function)
16	domains/agents/dataAccess/agent.ts:35	modelName	fails closed (no reservation in this function)
17	domains/agents/dataAccess/agent.ts:113	modelName	fails closed (no reservation in this function)
18	domains/agents/deliberationToEvolution.ts:107	"gemini-3.1-flash-lite-preview"	approved alias literal: resolves; unknown would throw
19	domains/agents/digestAgent.ts:786	model	fails closed (no reservation in this function)
20	domains/agents/digestAgent.ts:1457	model	fails closed (no reservation in this function)
21	domains/agents/dueDiligence/branches/marketCompetitive.ts:305	"qwen3-coder-free"	approved alias literal: resolves; unknown would throw
22	domains/agents/dueDiligence/branches/teamDeepResearch.ts:368	"qwen3-coder-free"	approved alias literal: resolves; unknown would throw
23	domains/agents/dueDiligence/ddEnhancedOrchestrator.ts:1396	SYNTHESIS_MODEL	fails closed (no reservation in this function)
24	domains/agents/dueDiligence/ddEnhancedOrchestrator.ts:1489	SYNTHESIS_MODEL	fails closed (no reservation in this function)
25	domains/agents/dueDiligence/ddOrchestrator.ts:1157	SYNTHESIS_MODEL	fails closed (no reservation in this function)
26	domains/agents/dueDiligence/ddOrchestrator.ts:1262	SYNTHESIS_MODEL	fails closed (no reservation in this function)
27	domains/agents/dueDiligence/investorPlaybook/agenticPlaybook.ts:130	"qwen3-coder-free"	approved alias literal: resolves; unknown would throw
28	domains/agents/emailAgent.ts:210	normalizeModelInput(modelId)	fails closed (no reservation in this function)
29	domains/agents/emailAgent.ts:321	normalizeModelInput(modelId)	fails closed (no reservation in this function)
30	domains/agents/fastAgentPanelStreaming.ts:1193	modelInput	fails closed; allowPaidFallback only if Homen decides so (SPEC C.6). It awaits reserveLlmRequestInternal at :3462, the only site that could.
31	domains/agents/glmFlashWithReasoning.ts:49	"glm-4.7-flash"	approved alias literal: resolves; unknown would throw
32	domains/agents/glmFlashWithReasoning.ts:135	"glm-4.7-flash"	approved alias literal: resolves; unknown would throw
33	domains/agents/glmFlashWithReasoning.ts:198	"glm-4.7-flash"	approved alias literal: resolves; unknown would throw
34	domains/agents/glmFlashWithReasoning.ts:495	"glm-4.7-flash"	not a call: example text inside console.log; nothing to decide
35	domains/agents/selfEvolution.ts:224	"qwen3-coder-free"	approved alias literal: resolves; unknown would throw
36	domains/agents/testGlmFlash.ts:20	"glm-4.7-flash"	approved alias literal: resolves; unknown would throw
37	domains/agents/testGlmFlash.ts:55	"qwen3-coder-free"	approved alias literal: resolves; unknown would throw
38	domains/agents/testGlmFlash.ts:116	"glm-4.7-flash"	approved alias literal: resolves; unknown would throw
39	domains/agents/testGlmFlash.ts:187	modelInfo.name	fails closed (no reservation in this function)
40	domains/agents/testGlmFlashFix.ts:17	"glm-4.7-flash"	approved alias literal: resolves; unknown would throw
41	domains/agents/testGlmFlashFix.ts:125	modelInfo.name	fails closed (no reservation in this function)
42	domains/agents/testParallelOrchestrator.ts:45	"glm-4.7-flash"	approved alias literal: resolves; unknown would throw
43	domains/agents/testParallelOrchestrator.ts:118	"deepseek-v3.2"	approved alias literal: resolves; unknown would throw
44	domains/agents/testParallelOrchestrator.ts:194	"glm-4.7-flash"	approved alias literal: resolves; unknown would throw
45	domains/agents/traceOrchestrator.ts:303	TRACE_MODEL	fails closed (no reservation in this function)
46	domains/ai/models/modelRouter.ts:441	modelId	fails closed (no reservation in this function)
47	domains/blips/blipClaimExtraction.ts:140	"qwen3-coder-free"	approved alias literal: resolves; unknown would throw
48	domains/blips/blipGeneration.ts:193	"qwen3-coder-free"	approved alias literal: resolves; unknown would throw
49	domains/blips/blipPersonaLens.ts:262	"qwen3-coder-free"	approved alias literal: resolves; unknown would throw
50	domains/blips/blipVerification.ts:190	"qwen3-coder-free"	approved alias literal: resolves; unknown would throw
51	domains/enrichment/llmCompanyExtraction.ts:141	"qwen3-coder-free"	approved alias literal: resolves; unknown would throw
52	domains/enrichment/llmEnrichment.ts:64	"qwen3-coder-free"	approved alias literal: resolves; unknown would throw
53	domains/evaluation/agentRunJudge.ts:129	model	fails closed (no reservation in this function)
54	domains/evaluation/evidencePlanner.ts:176	"claude-haiku-4.5"	approved alias literal: resolves; unknown would throw
55	domains/evaluation/llmJudge.ts:453	model	fails closed (no reservation in this function)
56	domains/evaluation/llmJudge.ts:752	judgeModel	fails closed (no reservation in this function)
57	domains/evaluation/personaEpisodeEval.ts:1066	model	fails closed (no reservation in this function)
58	domains/evaluation/personaLiveEval.ts:209	String(arch.reasoningModel)	fails closed (no reservation in this function)
59	domains/evaluation/testAgentDirect.ts:20	model	fails closed (no reservation in this function)
60	domains/evaluation/testLlmJudge.ts:101	modelName	fails closed (no reservation in this function)
61	domains/evaluation/ultraLongChat/judge.ts:212	judgeModel	fails closed (no reservation in this function)
62	domains/integrations/email/dailyEmailReport.ts:97	normalizeModelInput(modelId)	fails closed (no reservation in this function)
63	domains/knowledge/adaptiveEntityEnrichment.ts:285	DEFAULT_MODEL	fails closed (no reservation in this function)
64	domains/missions/preExecutionGate.ts:145	"gemini-3.1-flash-lite-preview"	approved alias literal: resolves; unknown would throw
65	domains/operations/encounters/encounterCapture.ts:293	"qwen3-coder-free"	approved alias literal: resolves; unknown would throw
66	domains/proactive/actions/testDraftGenerator.ts:166	"claude-sonnet-4.6"	approved alias literal: resolves; unknown would throw
67	domains/proactive/actions/testDraftGenerator.ts:323	modelInfo.name	fails closed (no reservation in this function)
68	domains/proactive/actions/testSimpleDraft.ts:20	"gemini-3-flash-preview"	approved alias literal: resolves; unknown would throw
69	domains/product/diligenceLlmJudgeRuns.ts:392	modelName	fails closed (no reservation in this function)
70	domains/product/wikiDreamingGraph.ts:207	model	fails closed (no reservation in this function)
71	domains/research/financial/financialAnalystAgent.ts:50	"claude-sonnet-4.6"	approved alias literal: resolves; unknown would throw
72	domains/research/financial/financialAnalystAgent.ts:111	"claude-sonnet-4.6"	approved alias literal: resolves; unknown would throw
73	domains/research/financial/financialAnalystAgent.ts:241	"claude-sonnet-4.6"	approved alias literal: resolves; unknown would throw
74	domains/research/financial/interactiveDCFSession.ts:504	"claude-sonnet-4.6"	approved alias literal: resolves; unknown would throw
75	domains/research/narrative/newsroom/agents/analystAgent.ts:436	cfg.model	fails closed (no reservation in this function)
76	domains/research/narrative/newsroom/agents/curatorAgent.ts:188	cfg.model	fails closed (no reservation in this function)
77	domains/research/narrative/newsroom/agents/publisherAgent.ts:259	model	fails closed (no reservation in this function)
78	domains/research/narrative/newsroom/agents/publisherAgent.ts:490	model	fails closed (no reservation in this function)
79	domains/research/narrative/newsroom/agents/signalCollectorAgent.ts:215	cfg.model	fails closed (no reservation in this function)
80	domains/search/fusion/reranker.ts:51	DEFAULT_MODEL	fails closed (no reservation in this function)
81	domains/social/linkedinPrePostVerification.ts:98	FREE_MODELS[0]	fails closed (no reservation in this function)
82	domains/social/linkedinPrePostVerification.ts:161	FREE_MODELS[0]	fails closed (no reservation in this function)
83	domains/social/linkedinPrePostVerification.ts:299	FREE_MODELS[0]	fails closed (no reservation in this function)
84	domains/social/linkedinQualityJudge.ts:111	modelAlias	fails closed (no reservation in this function)
85	domains/verification/instagramClaimVerification.ts:151	modelAlias	free ids only (FREE_MODEL_PRIORITY); when none builds (OpenRouter not configured) getFreeModel now throws. Round 1 said "fails closed" here and missed that the loop's fallback built paid gemini-3-flash-preview (former row 86, deleted with the fallback); instagramClaimVerification.test.ts pins it
86	tools/calendar/recentEventSearch.ts:251	DEFAULT_MODEL	fails closed (no reservation in this function)
87	tools/meta/contextEnhancementActions.ts:113	"gemini-3-flash-preview"	approved alias literal: resolves; unknown would throw
88	tools/meta/dynamicPromptEnhancer.ts:76	"claude-haiku-4.5"	approved alias literal: resolves; unknown would throw
89	tools/meta/dynamicPromptEnhancer.ts:271	"gemini-3-flash-preview"	approved alias literal: resolves; unknown would throw
90	tools/sec/secCompanySearch.ts:171	DEFAULT_MODEL	fails closed (no reservation in this function)
91	workflows/agentProjectIdeaPost.ts:377	modelId	fails closed (no reservation in this function)
92	workflows/founderPostGenerator.ts:202	modelId	fails closed (no reservation in this function)
93	workflows/specializedLinkedInPosts.ts:264	"qwen3-coder-free"	approved alias literal: resolves; unknown would throw

Discrepancies between the spec text and the real code

1. Branch name: the spec says model-refresh/nodebench-params; this run was told claude/model-refresh-nodebenchai. The CI check "Branch name - /" fails on it. Whether it is required: .github/workflows/pr-style.yml labels the job "Required" in a comment, but main's branch protection (read through the API, round 4) lists only Typecheck, Runtime smoke, Build and Tier B vs preview URL, and all four pass on a9e09067; so the claim is "not a required status context", which a reviewer who cannot read branch protection cannot confirm from the workflow file. It would also fail on the spec's own model-refresh/... names, because the allowed types are feat, fix, chore, and so on; chore/model-refresh-nodebench-params would pass. Renaming the head branch would close this PR and drop its review history, so I left it; the spec's naming for PR 2 / PR 3 needs a decision (section C). The "Pipeline quality benchmark" check is red on main too (NODEBENCH_API_URL unconfigured), so it is not this PR's.
2. wikiDreamingGraph.ts:362 passes google:models/... strings that the spec does not mention; with fail-closed they would throw, so two LEGACY_ALIASES rows were added (PR 2 moves the ids).
3. getBestFreeModel returns the constant only; the spec's optional "or the freeModelDiscovery verified pick" is not implemented (no consumer needs it yet).
4. The Anthropic wire-id table lives in shared/llm/modelCatalog.ts (the code that builds the dotted-alias requests), not in modelResolver.ts, because shared/ cannot import from backend/convex; anthropicWireIds.test.ts fails if the two tables drift.
5. Two raw content?.[0]?.text reads remain (workers/node/nemoclaw/agentRunner.ts:460, packages/mcp-local/src/index.ts:1988). They were not in the spec's "x14" list and neither sends a Claude 5.x id yet. Blocking items, not notes: the agentRunner.ts fix (read text blocks by type, with a thinking-before-text test) must land in PR 2 before its ids move, and the index.ts fix in PR 2 (the mcp-local ids move in PR 2's rows; PR 3 is only the release; that site still names the retired claude-3-5-haiku-latest) before its ids move.
6. The commit trailer names Sonnet 5.5, the model that wrote this, not the Opus 5.5 named in the task text.

Still open (not done here, by rule)

• Needs a paid, keyed run (not made): EVAL_BACKEND=anthropic EVAL_MODEL=claude-sonnet-5-5 npx tsx scripts/eval-harness/runner.ts --tier tier1 --seeds 3. 10 tier1 tasks x 3 seeds = 30 Messages calls, <= 8192 output tokens each; upper bound about $2.6 at $2/$10 per M tokens. Not required for PR 2 merge (spec); the nightly stays --list until it runs in Environment model-bump-eval.
• Needs a paid, keyed run (not made), PR 2 receipt: npx tsx scripts/run-comprehensive-eval.ts --models claude-sonnet-5-5,claude-haiku-5-5,claude-opus-5-5,gemini-3.8-flash,gemini-3.5-flash-lite,nemotron-3-super-free --suite full --judge --judge-model kimi-k2.6 --metrics in Environment model-bump-eval. Not run; not part of this PR's gate. The free-lane receipt for nemotron-3-super-free alone costs $0 and is the live evidence needs-live-eval waits for.
• Spec/C decisions still open: (a) tier: "free" in chainResolver is a floor; route() now caps it at free (the cap is a behaviour change for requireVision-style callers, see the fail-closed section); (b) the branch-name convention for PR 2 / PR 3: the CI check Branch name - <type>/<short-slug> rejects the spec's model-refresh/... and release/... names, so I propose chore/model-refresh-nodebench-ids (PR 2) and chore/release-nodebench-mcp-3-3-0 (PR 3), to be confirmed before they are opened; (c) whether approved paid ids outside MODEL_PRIORITY_ORDER should be listed or refused (they behave as on main in this PR).
• Homen's decision (round 4): the anonymous bounded lane (item 7 above) accepts an unknown model silently; validating args.model before resolveFastAgentRequestedModel would make it throw. Not done: the spec keeps the pinned lane and the change alters what an anonymous client sees.
• Homen's decision (round 4): the head branch name. Renaming means a new PR and loses this review history; the Branch name check is not a required context. If you want it green, say so and the same commits can be opened as chore/model-refresh-nodebenchai.
• Homen's decision (SPEC C.6): whether fastAgentPanelStreaming.ts:1193 takes allowPaidFallback: true. Until then an unknown model there throws.
• PR 2 model-refresh/nodebench-ids and PR 3 release/nodebench-mcp-3.3.0: deferred; condition: this PR merged. The free lane carries needs-live-eval for PR 2.

Spec: model-defaults SPEC Part A §A.2 (NodeBenchAI)

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
nodebench-ai	Ready	Preview	Oct 15, 2026 11:51pm UTC

[github-actions]

PR size advisory

This PR adds 1618 lines of substantive change. CONTRIBUTING.md defines a soft limit of ~400 LOC.

If the PR is genuinely cohesive (e.g. an architecture map, a generated migration, a deletion of a dead module), no action is needed. Otherwise consider:

• splitting into 2-3 PRs along independent concerns
• pre-discussing the architecture change in a GitHub Discussion before merge

This is advisory — it does not block the merge.

[github-actions]

✅ Dogfood Visual QA Gate — capture integrity: PASSED

Public guest captures only. A ready entry or enabled composer is not a completed live answer or reopened saved report. This gate checks artifact files and observed metadata; it does not grade pixels, keyboard/tablet coverage, or production deployment.

Check	Status
Screenshots	7 captured (count met)
Walkthrough	10 chapters (count met)
Key Frames	10 extracted (count met)
Scribe Steps	9 how-to steps (count met)
Build	success

Unperformed actions:

• NOT_RUN — Interaction: Home to Chat: This guest capture does not submit live research. An email-backed account and a live answer have not been exercised.
• NOT_RUN — Interaction: Reports to Chat: The guest Reports entry shows Saved research instructions. No saved report is selected or opened in Chat.

Artifacts

Download the dogfood-evidence-ac11232 artifact from the Actions tab for full screenshots, frames, and walkthrough video.

---

Generated by Dogfood Capture Integrity Gate