Repository navigation
[Fix] Delegated work uses the wrong model when users request or confirm a model - #3439
Conversation
|
No new code issues found. All nine prior findings remain addressed. See task Reviewed the latest intent-classifier wording change in context with chronological human evidence, scoped proposal acceptance, capability selection, named-default precedence and both launch/review callers. It retains earlier human model references while explicitly including capability, cost and speed requests. Confidence thresholds, provenance boundaries, replay persistence and routing fallback are unchanged. Current-head Test, Lint, Type Check and Knip passed. App Docker build and JavaScript/TypeScript analysis were still running when checked. Repository validation suites were left to CI; this review did not independently rerun live-classifier evaluations. The PR description's semantic-evaluation results remain pinned to
Reviewed 2b55567 |
|
Fixed both accepted findings in 22346b2c: replay consent is recovered at the original persisted human boundary, and the catalog test includes the capability choice. Both threads are resolved; automated re-review reports no new issues. The original catalog failure was introduced by this PR: 1 failed/4 passed on 9ee75c3 versus 5/5 on develop 688ff09 with the same dependencies/environment. Replay regressions fail against the original PR service source and pass with the boundary fix. Full package validation: 2523 passed, one existing skip. Remote results on final head 22346b2: all 20 executed checks succeeded, including Test and Roomote review. No pending/failing checks. One unaffected worker-image build is intentionally skipped, not an executed pass. Fresh unchanged frozen real-classifier sets: main 132/136, supplement 46/46; one separately preserved transport-timeout recovery passed 1/1. Three below-threshold rule-only misses remain documented; thresholds were not changed. Original and new raw evidence are preserved in the originating task. The PR remains draft. No merge or auto-merge action. |
|
Pushed the semantic request repair and named-default correction in ee718f27766b9a69ac1f3b6b5f062b0c928df707. Canonical human provenance defines authority; the classifier interprets freely worded requests over all enabled model options, with no phrase-based eligible-ID veto. Owned proposal interpretation is separate, and replay/steer snapshots retain their durable boundaries. A confident named deployment-default choice now overrides routing without requiring non-default intent. The preceding grammar-based completion claim is superseded. Exact-head evidence: full cloud-agents suite2571passed,oneexisting skip. New144-case natural-phrasing set, frozen before scoring and rerun twice on this immutable head: must-switch72/72twice and negatives72/72twice. Develop688ff092:60/72must-switch and72/72negatives twice; prior22346b2c:72/72for both twice. Zero paired regressions in this set. Original13 safety regressions expanded across catalogs/claims:52/52twice. The separate older464 template set remains intact: develop341/464twice,prior410/464twice,current460/464twice. Every safety group passes; four earlier-choice continuation cases miss the unchanged0.5intent gate in each pass. All8failed outcomes and6paired outcomes worse than at least one pin are retained with exact inputs/scores, along with all intermediate failures. No threshold tuning, relabeling, removed cases or overwritten primary results. No final transport errors. Final-head Test, Lint, Type Check, Knip, app Docker build and CodeQL analyses passed. Automated re-review is clean on ee718f2, and all nine historical findings are resolved. Worker-image build intentionally skipped. Judgment caveat: successful remote wrapper jobs have incomplete evidence/inference coverage; the full local attempt was also incomplete. The bounded final-correction commit judgment passed6judgments, which does not replace full coverage. PR metadata refreshed and draft restored after automatic promotion. No merge/auto-merge, Slack post or production rollout. Browser proof is not applicable. Full evidence remains in the originating task. Finite classifier evaluation does not establish universal authorization correctness. |
daniel-lxs
left a comment
There was a problem hiding this comment.
Validated this wording-only change against develop and the previous head with live classifier comparisons covering explicit model requests, confirmation and continuation, conflicting saved routing, and negative provenance controls. No observed develop regression in the independent cases; original-choice fixtures now pass repeatedly. Model handoff was also verified through real delegated launches on the preceding head, whose launch logic is unchanged here. Confidence-based fallback limitations remain documented.
Related issue
Internal maintainer-requested model-routing repair; no linked GitHub issue.
Why this PR exists
Delegated work missed genuine human model requests and confirmations. A phrase-based eligibility gate then rejected freely worded requests despite high classifier intent. This revision replaces that gate while retaining canonical provenance and durable replay boundaries.
What changed
Both task launches and PR reviews use this resolver. Confirmation paths add one semantic proposal call; their maximum combined decision wait can therefore reach ten seconds. This is a classifier-backed interpretation boundary, not a universal authorization guarantee.
How it was tested
Exact pushed head:
ee718f27766b9a69ac1f3b6b5f062b0c928df707.688ff092132/144 twice, prior22346b2c144/144 twice, current 144/144 twice. Zero current failures or paired regressions in this set. One automatically redacted phrase from case authoring was preserved unchanged.typesafe/jev-1.13, observedtypesafe/jev-1.13-20260917, with every raw request/response and both stages preserved. No final transport errors or primary-result replacement retries.Browser proof is not applicable to this server-side change. No production rollout was performed. The PR remains draft.
Checklist
[Fix],[Feat],[Improve],[Refactor],[Docs], or[Chore]followed by a user-facing descriptionpnpm lintandpnpm check-typespass locallypnpm changesetThis is a larger single-boundary repair. Normal fast static gates passed; formatting-inclusive legacy commands and an interactive changeset were not run. Residual classifier misses and incomplete Judgment coverage remain explicit review caveats.