fix(planner): route query tasks through LLM planning (#419), and measure localization by ceilf6 · Pull Request #423 · ceilf6/FrontAgent

ceilf6 · 2026-08-01T12:10:57Z

Linked Issue Or Context

Fixes [Bug] query tasks never use LLM planning, so they can only read files the caller supplied #419, addresses [Bug] trigger-policy uses bare focus-dir names as scan roots, so navigation can scan nothing #420.
Follows the 2026-08-01 ablation (docs(eval): filesense ablation results — no measurable difference, and why #422), which returned 4/12 vs 4/12 — zero difference. This PR fixes the three causes of that zero that were mine to fix.

Why the previous run measured nothing

The report decomposed the zero by category. Three of the four causes were defects in my own experiment, not properties of filesense:

category	both arms	cause
query 0/4	identical	#419 — never reached LLM planning; nothing consumed navigation
create 2/3	identical	the task text named the target directory
bugfix 0/3	identical	codegen quality; localization already worked
refactor 2/2	identical	tasks easy enough that one extra exploration step sufficed

Changes

1. query tasks reach LLM planning (#419)

planner.ts:151 short-circuited every query task to rule-based planning, which emits read_file only for task.context.relevantFiles. Nothing pre-supplied → a plan that reads nothing → "no evidence available", in both arms. Meanwhile the plan prompt teaches a glob-discovery workflow and search_code / list_directory are both in ACTION_ENUM — none of it reachable, because the planner never asked.

Query now takes the same path as every other type, with rule-based as the fallback it already is elsewhere.

2. the tasks no longer say where to look

Before: "在共享工具库（放通用纯函数的那个目录）新增 truncate", "给 checkout 这个 feature 新增一个 hook".

After: "它不属于任何业务域，任何模块都该能用它——放到代码库中同类函数已经在的地方", "把它放到负责结算的那个业务域里".

All twelve rewritten to give behaviour, not location. The fixture already has 24 features carrying identical filenames (format.ts, Button.tsx, useToggle.ts); with the hints removed, finding the right one is the work — which is the thing under test.

3. a metric that is sensitive to localization

Pass/fail is a poor instrument here: both arms can grind to the answer with list_directory / search_code, reaching the same result at different cost, and cost is what navigation changes. Each task now declares targetDirs, and the harness counts exploration steps (read_file / list_directory / search_code) landing outside them as offTargetExplored.

targetDirs is declared by the task, never derived from the outcome — deriving it would be scoring against the answer key.

4. #420, corrected

Focus-dir names lifted from the task text were used as scan roots, ahead of the directories derived from the plan. I had the mechanism partly wrong and checked it: the engine skips a non-existent root with "Path does not exist: hooks" and carries on — it does not zero the scan. The real cost is the paths cap of 5: a name resolving to nothing spends a slot that a real directory could have used. Real directories now sort first, joined variants are derived (src/features/checkout + hooks → src/features/checkout/hooks), and bare names sort last.

Impact Scope

packages/core/src/planner.ts — the query branch.
packages/core/src/filesense/trigger-policy.ts — mergeFocusDirs ordering.
benchmarks/eval/{run-eval,report-filesense}.mjs, tasks-deep.json — metric and task set.
Tests: three existing query tests pin useLLM: false (they assert rule-based step shape, which used to be the only path); one new test pins that query now attempts LLM planning.

Behaviour change worth reviewer attention

Query tasks now cost an LLM planning call and can run multi-step plans, where they previously cost one call and read only what the caller named. That is the point — but it is a latency and token increase on a path that was previously cheap and useless.

GitNexus Impact Summary

Risk level: MEDIUM
Critical skeleton changes: yes — packages/core/src/planner.ts, with tests in the same package.
GitNexus impact: the query branch now falls through to the existing useLLM block; rule-based generation is unchanged and still reached on fallback and when useLLM: false. mergeFocusDirs is called only from decideFilesense; it returns the same shape and still honours the 5-path cap, so no consumer changes. Callers passing relevantFiles keep working — the LLM receives them as context and the rule-based fallback still uses them.
Verification: pnpm quality:precommit exit 0.

Verification

pnpm quality:precommit — exit 0.
planner.test.ts 32 pass, including the new [Bug] query tasks never use LLM planning, so they can only read files the caller supplied #419 contract test.
Engine behaviour on a non-existent scan root verified directly rather than assumed.

Checklist

I have linked an issue or explained why this PR stands alone.
I have kept the diff focused on the stated change.
I have run pnpm quality:precommit, or explained why it could not run.
I have run pnpm quality:local for critical skeleton changes, or explained why it could not run.
I have updated docs or tests when behavior, public APIs, or Harness contracts changed.
For critical skeleton changes, I have filled the GitNexus impact summary with concrete results.

🤖 Generated with Claude Code

…ure localization The 2026-08-01 ablation returned 4/12 vs 4/12 — zero difference. The report decomposed that zero, and three of its four causes were mine to fix: **1. query tasks never reached LLM planning (#419).** `planner.ts:151` short-circuited them to rule-based planning, which emits `read_file` only for `task.context.relevantFiles`. With nothing pre-supplied the plan reads nothing, so all four query tasks answered "no evidence available" in *both* arms and contributed zero discriminating power. They are now planned like every other task type, with rule-based as the fallback it already is elsewhere. **2. the tasks told the agent where to look.** "在共享工具库（放通用纯函数的那个目录）新增 truncate", "给 checkout 这个 feature 新增 hook" — the target directory was in the prompt, so navigation had nothing to contribute and both arms guessed correctly. All twelve are rewritten to give **behaviour, not location**: "它不属于任何业务域，任何模块都该能用它——放到代码库中同类函数已经在的地方". Finding the place is now the work, which is the thing under test. **3. pass/fail is the wrong instrument.** Both arms can grind to the answer with `list_directory` / `search_code`; they reach the same result at different cost, and the cost is what navigation actually changes. Each task now declares `targetDirs`, and the harness counts exploration steps landing **outside** them — `offTargetExplored`. Declared by the task, never derived from the outcome. Also #420: focus-dir names lifted from the task text (`hooks`, `components`) were used as scan roots ahead of the directories the plan derived. The engine skips a non-existent root with a warning rather than zeroing the scan — I had that part wrong — but `paths` is capped at 5, so a name that resolves to nothing costs a slot. Real directories now come first, joined variants are derived (`src/features/checkout` + `hooks`), and bare names sort last. The three query planner tests now pin `useLLM: false` explicitly: they assert the rule-based step shape, which used to be the only path and is now the fallback. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

github-actions

🛡️ ceilf6/repo-guard

代码评审报告: fix(planner): route query tasks through LLM planning (#419), and measure localization

风险等级: 高
处理建议: 请求修改
决策摘要: planner.ts 的 #419 修复本身正确且链路已验证（LLM 计划 → read_file → collectedContext.files → 回答证据），但本 PR 的第 3 项改动（定位精度指标）引用了一个从未初始化、也从未被写入的 eventDetails.explored，会在第一个任务上抛 TypeError 并中断整臂评测——主指标零可用；同时任务改写让 deep-query-structure 不再命中 decideFilesense 的任何触发词，反而缩小了消融面。

级联分析

变更符号: Planner.generatePlan（query 分支）、mergeFocusDirs、run-eval.mjs 记录构造、report-filesense.mjs 新表、tasks-deep.json 任务集与 targetDirs。
受影响流程: 所有未显式指定 type 的调用——run.ts:333/run.ts:516 与 agent.ts:444 均默认 'query'，故这条「默认路径」现在都走 LLM 规划；decideFilesense 的 query 分支；answer-generation.ts 的 query 回答证据；filesense 消融 harness 与报告。
变更集外调用方: mergeFocusDirs 仅被同文件 decideFilesense 调用（读文件确认，非仅文本搜索），返回形状与 5 条上限未变，无消费者需要改；generateQuerySteps 仍经 planner-skills.ts:111 作为回退可达；phase.repository-management 对 query 恒不注入（planner-skills.ts:183、planner-phase-helpers.ts:37）。
置信度: medium（未使用代码图谱，结论来自直接读文件 + 文本搜索；explored 缺失是读 run-eval.mjs:158-226,277-321 直接确认，非推断）

问题发现

[致命] eventDetails.explored 不存在，新指标一运行就崩溃
- 证据: run-eval.mjs:226 初始化为 { filesense: [], validationFailed: [], rollback: [] }，collectEventDetail（run-eval.mjs:158-221）只写 filesense / validationFailed / rollback，全仓搜索 explored 只有 311-313 三处消费。record 构造在 try/catch 之外（catch 于 271 行结束），故 eventDetails.explored.length 会以 Cannot read properties of undefined 直接终止整臂，一条记录都不落盘。pnpm quality:precommit 不跑评测，所以 exit 0 无法证伪这一条。
- 受影响调用方/流程: 两臂评测全部；report-filesense.mjs:169-171 的「本报告的主指标」永远没有输入。
- 最小可行修复: ① eventDetails 增加 explored: []；② 在 collectEventDetail 里处理 step_completed（types.ts:819，载荷含 step.action / step.params），对 read_file / list_directory / search_code 推入 { action, path }；③ 注意取路径要兼容 search_code 不用 path 的形参（directory / filePattern，见 planner.test.ts:46-64 的参数集），并在 offTargetExplored 的 filter 里对 path 为空做保护，否则 e.path.startsWith 会二次抛错；④ 路径需先规范成工作区相对路径再做前缀比较，绝对路径（/tmp/ws/src/app/...）与 targetDirs（src/app）永不匹配，会把 100% 探索都记成脱靶。
[高] 任务改写让 deep-query-structure 彻底不再触发导航，消融面反而变小
- 证据: trigger-policy.ts:96-101 的两条正则要求命中 where|which file|locate|find|入口|在哪|哪个文件|查找|寻找|定位 或 architecture|structure|overview|map|架构|结构|目录|模块|梳理|理解。旧文案「说明 src 下四个顶层目录各自的职责」命中 目录；新文案「这个代码库按什么原则分层？说明各层之间允许的依赖方向……」（tasks-deep.json:30）一个词都不命中 → skip('query task without repository-structure need')。deep-query-format-convention（:54）同样不命中，与 #419 里记录的 nav=0 状态一致。四条 query 里现在只有 route-source 与 shared-lib 会触发导航。
- 受影响调用方/流程: filesense 开臂在这两条任务上与关臂架构等价，脱靶数必然相同；而报告的解读注（report-filesense.mjs:173-174）会把这种相同归因为「任务设计」，实际是触发策略没覆盖——正是这份 harness 反复要避免的错误归因。
- 最小可行修复: 让每条任务记录 filesenseTriggered（events.filesense_navigated > 0 已有），新表按「仅触发子集」统计（与报告 :125-132 自述的口径一致），并对未触发任务显式列出；或把 structure 任务文案调回含触发词的表述。
[中] 新指标表用了全量口径，且平均值被四舍五入成整数
- 证据: report-filesense.mjs:169-171 用 arms.off / arms.full 全量；avg（:62-63）走 Math.round，sum/avg 的 ?? 0 还会把 targetDirs 缺失时的 null 当成 0。单任务探索步数是个位数，四舍五入后 2.4 与 2.6 都显示 2——恰好抹掉这个指标要检测的差异。
- 受影响调用方/流程: 报告主结论的可读数性。
- 最小可行修复: 新表改用 pairedTriggered 子集，并给该表用保留一位小数的均值（新增 avg1），null 走排除而非计 0。
[中] mergeFocusDirs 的改动没有任何覆盖：单测没加，benchmark 也不再触发它
- 证据: trigger-policy.test.ts 三个用例只用 toContain，对顺序、拼接变体、上限交互均无断言，所以改前改后都过（这也是 quality:precommit exit 0 的原因）。另一方面 taskMentionedFocusDirs 要求整词匹配 hooks，而改写后的 deep-create-checkout-hook（tasks-deep.json:122）写的是「既有 hook」，单数不匹配；通读改写后的 12 条文案，FOCUS_DIRS 里没有任何词还能命中。于是 #420 的修复既没单测、也不再被评测执行。
- 受影响调用方/流程: decideFilesense 的 prepare_create / prepare_refactor / validate_freshness 分支。
- 最小可行修复: 补一条 decideFilesense 单测，用 create 任务 + create_file(path: 'src/features/checkout/hooks/x.ts') 断言「真实目录在前」「含拼接变体」「裸名在后」。
[中] 拼接变体仍是未解析路径，且按 真实目录 × 提到的名字 做叉积，#420 的名额浪费被放大而非消除
- 证据: trigger-policy.ts:150-156 对每个真实目录 × 每个裸名生成一条路径，全部不做仓库解析。1 个真实目录 + 3 个命中名字 → 3 条未解析路径占掉 5 个名额中的 3 个（并把裸名整体挤出）。#420 给的方向是「先对仓库解析，或只当排序提示」，本 PR 只做了重排。
- 受影响调用方/流程: 所有 mergeFocusDirs 消费路径（仅 decideFilesense）。
- 最小可行修复: 每个真实目录最多派生 1 条变体，或用 task.context.workingDirectory + existsSync 过滤后再入列；若不想在纯模块里引 fs，就把裸名与变体降级为候选排序提示，不做扫描根。
[中] query 计划不再是结构性只读，且默认任务类型就是 query
- 证据: 旧分支保证 query 只能产出 read_file / search_code / 浏览器步骤；改后 query 与其他类型共用同一份计划 prompt，create_file / apply_patch 就在动作说明里（plan-generation.ts:124-125,258-259）。而默认类型是 'query'（agent.ts:444、run.ts:333、run.ts:516），所以「未指定类型的调用」现在都可能规划出写步骤，且 phase.repository-management 对 query 恒不注入（planner-skills.ts:183）。写操作仍受 security.ts:50 的 WRITE_TOOLS 审批闸约束，因此不是无防护，但 PR 描述只提到延迟与 token，没有覆盖这一维。
- 受影响调用方/流程: CLI / runtime-node 未传 type 的全部调用。
- 最小可行修复: 在 convertLLMPlanToSteps 之后对 task.type === 'query' 过滤掉写动作并记 warning，或在 PR 描述与文档中显式声明这一行为扩张。
[低] query 回答证据仍不包含 list_directory 结果，#419 的目标只兑现了一半
- 证据: answer-generation.ts:45-106 的证据只来自 search_code 命中、filesenseContext 与 collectedContext.files（read_file）。list_directory 结果进了 filesystem-facts-update.ts:115-141 的 facts，但不会进回答 prompt。
- 受影响调用方/流程: 走目录列举做发现的 query 计划（如结构类问题）仍可能答「无法确定」。
- 最小可行修复: 本 PR 不必处理，但应在验证预期中说明，并按 #415 同族另开 issue。
[低] deep-query-format-convention 丢掉了唯一有区分度的断言
- 证据: 检查项从 checkout: 改成 features/checkout（tasks-deep.json:59）。24 个 feature 的 format.ts 都含 toFixed，只有返回值前缀（checkout: vs catalog:）能证明读的是结算域那一份。
- 受影响调用方/流程: 该任务的通过率含义。
- 最小可行修复: 两条都留（features/checkout + checkout:），任务已要求给出输出格式，成本为零。

行级发现

[benchmarks/eval/run-eval.mjs:311] eventDetails.explored 从未初始化（:226）也从未被写入（collectEventDetail），此行在 try/catch 之外，第一个任务即抛 TypeError 中断整臂；需先在 :226 加 explored: [] 并在 collectEventDetail 里按 step_completed 采集 read_file/list_directory/search_code。
[benchmarks/eval/run-eval.mjs:313] 采集补齐后此处仍会炸：search_code 步骤不带 path（用 directory/filePattern），且路径可能是绝对路径——先做空值保护并规范成工作区相对路径，再与 targetDirs 比前缀。
[benchmarks/eval/tasks-deep.json:30] 新文案不含 trigger-policy.ts:96-101 任何触发词（旧文案靠「目录」命中），这条 query 在开臂将完全不触发导航；请补一个被识别的表述，或在报告里把未触发任务单列而非计入均值。
[benchmarks/eval/tasks-deep.json:59] 删掉 checkout: 后，检查失去区分具体 feature 的能力（24 份 format.ts 都含 toFixed）；建议保留该模式并新增路径模式。
[benchmarks/eval/report-filesense.mjs:169] 主指标表用全量口径且 avg 取整（:62-63），个位数计数被四舍五入后差异被抹平；改用 pairedTriggered 子集 + 一位小数。
[packages/core/src/filesense/trigger-policy.ts:154] 拼接变体按叉积生成且不做仓库解析，1 个真实目录 + 3 个命中名字就吃掉 3 个名额——#420 的名额浪费被放大；每个 base 至多派生 1 条，或用 workingDirectory 做 existsSync 过滤。
[packages/core/src/filesense/trigger-policy.ts:144] 注释③称裸名「仍保留」，但 realDirs 本身已按 limit 截断，真实目录满 5 条时裸名与变体会被全部丢弃；请修正注释或显式为提示保留一个名额。
[packages/core/src/planner.ts:151] 分支放开后 query 计划不再结构性只读，而默认任务类型即 'query'（agent.ts:444、run.ts:333），且 phase.repository-management 对 query 恒不注入；建议对 query 结果过滤写动作，或在描述/文档中显式声明这一行为扩张。
[packages/core/src/planner.test.ts:109] 断言只检查 generateObject 被调用过，不校验抛错后是否回落到规则步骤、fallbackReason 是否被记录；补一条对 result.plan.steps 与 fallbackReason 的断言，才能把「不短路」和「回退仍可用」一起钉住。

Karpathy 评审

假设: PR 假设「query 走 LLM 规划」等价于「query 任务变强」，但默认类型即 query（agent.ts:444）这一点未在描述中体现，写动作可达性也未讨论；另假设改写文案只是去掉位置提示，实际同时改变了 decideFilesense 的触发结果。
简洁性: planner.ts 的改动是纯删分支，方向正确，无多余抽象；mergeFocusDirs 新增的叉积派生是本 PR 唯一新增的复杂度，且未解析、未测试，收益弱于成本。
结构质量: 无文件跨界膨胀、无薄 wrapper、无重复 canonical helper；trigger-policy.ts 仍是纯模块，改动没有破坏边界。唯一退化点是 mergeFocusDirs 的注释与实际行为（上限截断）不一致。
变更范围: 四件事（planner 修复、任务集改写、新指标、focus-dir 排序）打包在一个 PR 里，彼此耦合弱；其中任务改写与 focus-dir 修复互相抵消（见发现 4），拆成两个 PR 会让 #420 的效果可单独观察。
验证: quality:precommit 覆盖不到 benchmarks 目录，因此 exit 0 与 32 个 planner 用例通过都无法证明本 PR 的主体改动可用；发现 1 是运行即崩，发现 4 是改动无覆盖。

缺失覆盖

run-eval.mjs 至少需要一次真实（或 dry-run）执行，证明记录能落盘、exploredCount 非恒零；仅靠 quality:precommit 无法覆盖 benchmarks。
decideFilesense 需要一条断言 paths 顺序与拼接变体的用例，钉住 #420 的修复。
planner 需要一条断言「LLM 抛错后 query 回退到规则步骤且 fallbackReason 有值」的用例。
建议补一条断言：query 任务的 LLM 计划中若出现 create_file/apply_patch，系统的处理方式是明确的（过滤或告警），而不是隐式放行。

github-actions · 2026-08-01T12:17:13Z

+    // 否则就是拿答案去评分。通过率对「找得准不准」太不敏感：两臂都能靠
+    // list_directory / search_code 慢慢摸出来，结果一样、代价不同，
+    // 而代价才是导航能力的直接体现。
+    exploredCount: eventDetails.explored.length,


eventDetails.explored 从未初始化（:226）也从未被写入（collectEventDetail），此行在 try/catch 之外，第一个任务即抛 TypeError 中断整臂；需先在 :226 加 explored: [] 并在 collectEventDetail 里按 step_completed 采集 read_file/list_directory/search_code。

github-actions · 2026-08-01T12:17:13Z

+    // 而代价才是导航能力的直接体现。
+    exploredCount: eventDetails.explored.length,
+    offTargetExplored: task.targetDirs
+      ? eventDetails.explored.filter((e) => !task.targetDirs.some((d) => e.path.startsWith(d)))


采集补齐后此处仍会炸：search_code 步骤不带 path（用 directory/filePattern），且路径可能是绝对路径——先做空值保护并规范成工作区相对路径，再与 targetDirs 比前缀。

github-actions · 2026-08-01T12:17:13Z

    "category": "query",
    "smoke": false,
-    "task": "说明 src 下四个顶层目录各自的职责，以及一个 feature 目录内部的分层约定。",
+    "task": "这个代码库按什么原则分层？说明各层之间允许的依赖方向，以及一个业务域内部是怎么组织的。",


新文案不含 trigger-policy.ts:96-101 任何触发词（旧文案靠「目录」命中），这条 query 在开臂将完全不触发导航；请补一个被识别的表述，或在报告里把未触发任务单列而非计入均值。

github-actions · 2026-08-01T12:17:13Z

      {
        "kind": "result_contains",
-        "pattern": "checkout:"
+        "pattern": "features/checkout"


删掉 checkout: 后，检查失去区分具体 feature 的能力（24 份 format.ts 都含 toFixed）；建议保留该模式并新增路径模式。

github-actions · 2026-08-01T12:17:13Z

+
+| 指标 | filesense 关 | filesense 开 |
+|---|---|---|
+| 平均探索步数 / 任务 | ${avg(arms.off, 'exploredCount')} | ${avg(arms.full, 'exploredCount')} |


主指标表用全量口径且 avg 取整（:62-63），个位数计数被四舍五入后差异被抹平；改用 pairedTriggered 子集 + 一位小数。

github-actions · 2026-08-01T12:17:13Z

+  for (const base of realDirs) {
+    if (base === '.') continue;
+    for (const name of mentioned) {
+      derived.push(`${base}/${name}`);


拼接变体按叉积生成且不做仓库解析，1 个真实目录 + 3 个命中名字就吃掉 3 个名额——#420 的名额浪费被放大；每个 base 至多派生 1 条，或用 workingDirectory 做 existsSync 过滤。

github-actions · 2026-08-01T12:17:13Z

+ * 现在：① 计划推导出的真实目录排前面——那是即将写入的位置，证据强度高于
+ * 从散文里捞到的名字；② 额外补上拼接变体（`src/features/checkout` + `hooks`
+ * → `src/features/checkout/hooks`），这才是裸名真正想表达的位置；
+ * ③ 裸名本身排最后，仍保留（在根下确有同名目录的仓库里它是对的）。


注释③称裸名「仍保留」，但 realDirs 本身已按 limit 截断，真实目录满 5 条时裸名与变体会被全部丢弃；请修正注释或显式为提示保留一个名额。

github-actions · 2026-08-01T12:17:13Z

-      steps = this.generateStepsForTask(task, context);
-    } else if (this.config.useLLM) {
-      // 使用 LLM 生成计划
+    if (this.config.useLLM) {


分支放开后 query 计划不再结构性只读，而默认任务类型即 'query'（agent.ts:444、run.ts:333），且 phase.repository-management 对 query 恒不注入；建议对 query 结果过滤写动作，或在描述/文档中显式声明这一行为扩张。

github-actions · 2026-08-01T12:17:13Z

+    );
+
+    // 被调用过就说明没有短路；抛错后回退到规则生成，与其他任务类型一致
+    expect(calledWith).toBeDefined();


断言只检查 generateObject 被调用过，不校验抛错后是否回落到规则步骤、fallbackReason 是否被记录；补一条对 result.plan.steps 与 fallbackReason 的断言，才能把「不短路」和「回退仍可用」一起钉住。

Copilot AI review requested due to automatic review settings August 1, 2026 12:10

Copilot AI reviewed Aug 1, 2026

ceilf6 merged commit 2c8e57a into develop Aug 1, 2026
6 of 7 checks passed

ceilf6 deleted the fix/query-llm-planning branch August 1, 2026 12:11

github-actions Bot requested changes Aug 1, 2026

View reviewed changes

ceilf6 mentioned this pull request Aug 1, 2026

fix(eval): actually capture exploration steps, across all their path shapes #424

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(planner): route query tasks through LLM planning (#419), and measure localization - #423

fix(planner): route query tasks through LLM planning (#419), and measure localization#423
ceilf6 merged 1 commit into
developfrom
fix/query-llm-planning

ceilf6 commented Aug 1, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

github-actions Bot left a comment

Uh oh!

github-actions Bot Aug 1, 2026

Uh oh!

github-actions Bot Aug 1, 2026

Uh oh!

github-actions Bot Aug 1, 2026

Uh oh!

github-actions Bot Aug 1, 2026

Uh oh!

github-actions Bot Aug 1, 2026

Uh oh!

github-actions Bot Aug 1, 2026

Uh oh!

github-actions Bot Aug 1, 2026

Uh oh!

github-actions Bot Aug 1, 2026

Uh oh!

github-actions Bot Aug 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

ceilf6 commented Aug 1, 2026

Linked Issue Or Context

Why the previous run measured nothing

Changes

1. query tasks reach LLM planning (#419)

2. the tasks no longer say where to look

3. a metric that is sensitive to localization

4. #420, corrected

Impact Scope

Behaviour change worth reviewer attention

GitNexus Impact Summary

Verification

Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

代码评审报告: fix(planner): route query tasks through LLM planning (#419), and measure localization

级联分析

问题发现

行级发现

Karpathy 评审

缺失覆盖

Uh oh!

github-actions Bot Aug 1, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Aug 1, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Aug 1, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Aug 1, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Aug 1, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Aug 1, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Aug 1, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Aug 1, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Aug 1, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants