[david6666666]

Refs #314

What changed

• Restore the performance line for comparable independent candidate measurements with unique round numbers. The chart clearly identifies non-cumulative results, preserves quality failures and rollbacks, and retains explicitly recorded cumulative charts.
• Add optional live observation with inferrsi dashboard --input <RUN_OR_CAMPAIGN> --live --port 8765: a read-only loopback server supplies recorded updates about every three seconds without a page reload. The page retains selections where possible, shows refresh failures and retains its last view. Offline export remains available.
• Address Dong1017's translation review: caller overlays apply only to explicitly marked narrative, while verdicts, quality and missing-evidence states use code-owned translations. Add regression coverage for hostile overlays in both languages.
• Bind story logs and knowledge captures to their campaign and capture IDs; check references against the captures actually named by each round. Withhold corrupt or mismatched records and keep authored narratives separate from technical stage events.
• Add an offline English/Chinese language switch, with a reviewed caller-supplied translation overlay for recorded narratives and access to the original knowledge context.
• Export a self-contained, offline optimization dashboard for existing runs and explicitly recorded campaigns. Candidate details bring together the change, code or configuration locations, benchmark and accuracy methods, redacted commands, and measured evidence.
• Show a caller-reviewed optimization trace with the assembled knowledge context, search queries and sources, declared search space, ordered observations, attempts, decisions, rationale, and success and failure summaries. The separate Selected optimization and Stage coverage sections have been removed.
• Add a bounded trace snapshot format and a CLI operation that inserts an explicitly reviewed context file without silently truncating it. Candidate, optional run and round identities, source file hashes, and explicit event references are checked; missing, redacted, corrupt, mismatched and ambiguous records remain visible as such.
• Add a campaign-level Optimization story after the two performance charts, so a reviewer can follow the whole optimization rather than one candidate at a time: the goal; each knowledge retrieval and why it was made; the declared search space with options marked tried, pruned or untried and the reason; every round's hypothesis, why this optimization was chosen, decision and next step; and success and failure summaries that cite evidence and name the failing stage. Views: Optimization process, Knowledge context, Search space, Successes and failures.
• Add inferrsi campaign capture-context. It runs the same collect operation the agent uses, with a context budget, and keeps the complete envelope: the request, what search selected, what was left out of the context and why, and every resource body with its SHA-256. The assembled text is re-derived when the dashboard loads, so an edited, missing or mismatched capture is withheld rather than shown. This replaces hand-supplied context text for campaigns; the per-candidate snapshot above is unchanged.
• Add inferrsi campaign log, which validates and stores a numbered, versioned campaign optimization log (schema_version: 2). Earlier versions are kept; the latest is shown only when its hash and campaign identity match. Validation errors name the offending field.
• Cross-check the narrative against evidence. The dashboard resolves each reference in the log against captures, rounds.jsonl and recorded runs and lists disagreements under Narrative vs evidence: knowledge cited but never retrieved, or selected by search but dropped by the context budget before it was included in the assembled context; a decision that differs from rounds.jsonl; an adopted candidate whose recorded outcome is not an improvement; a search option marked tried that no round tried; a success whose evidence did not improve; and a recorded failure, regression or rollback that no failure summary cites.
• Keep technical runner and agent events searchable, and allow a narrative step to open a linked event only when its source, line and source-file hash match. A candidate without a recorded run cannot read evidence from a placeholder directory.
• Retain the charts, diagnostic evidence, local review export, and conservative command filtering. The host agent owns context capture and search decisions; the dashboard does not intercept unrelated agents or independently verify caller narrative.

The issue's original offline scope is documented in the issue discussion. The reviewed context snapshot is an explicit addition requested in subsequent feedback; raw logs, environments and unreviewed agent requests remain outside the export.

Validation

• Current-head round trend: targeted Windows tests and Ruff passed; 125 targeted tests and Ruff passed on remote Linux. The Chinese ten-round live recording uses explicitly synthetic data.
• Live mode: tested incremental record updates, conditional responses, local-only binding, invalid Host rejection, source-file isolation, refresh failure and recovery, and offline CSP. Browser checks covered automatic updates, filter and language preservation, and failure/retry. The PR comment contains an accelerated, synthetic browser recording.
• Windows: 153 targeted dashboard, campaign, trace, plugin and CLI tests passed; ruff check src tests tools passed.
• Remote Linux at 806e71d: 123 targeted live dashboard, campaign, trace, story and language tests passed, and ruff check src tests tools passed. One known pre-existing nested-JSON error-message assertion was excluded; it also fails on the unchanged baseline.
• Optimization story: 198 targeted tests passed (dashboard, campaign, trace, plugin, CLI, collect and knowledge CLI, including 18 new tests in tests/test_optimization_story.py); ruff check src tests tools passed. Full suite on Windows: the only errors are in test_mlc_llm_repo_to_skill.py, test_mlx_swift_lm_repo_to_skill.py and test_anthropic_edit_agent.py, caused by Windows limits (environment variable longer than 32767 characters, GBK default encoding, no os.mkfifo). They also error on the unmodified PR head, though not with identical counts; they need a Linux CI run.
• Current-head browser checks covered all four story views, candidate links, context originals, both languages, and the position of both charts. A hostile translation overlay could not change FAIL, INCONCLUSIVE, quality, or missing-evidence labels. The screenshots use synthetic campaign data; the demo's knowledge capture is packaged-library collect output.
• Earlier independent review covered trace privacy and identity binding, path replacement, missing runs, event-hash navigation, old records and rendering. Browser checks covered real and explicitly synthetic snapshots, language switching, candidate selection, context originals, localized review export and responsive layout.
• The story's narrative is caller-authored. Capture hashes check internal consistency, not authenticated origin or model consumption; the dashboard checks references and recorded outcomes, not the soundness of the reasoning. Chinese support covers interface text; log prose appears as written unless a translation overlay is supplied.
• The new story code has had no independent review yet.
• Existing records cannot reconstruct a context that was never saved. Captured text must be reviewed before recording and sharing. Existing candidate records do not yet have a CLI operation to replace their trace reference. This is a snapshot, and a recorded validation command does not prove untested quality properties.

Consult the current-head GitHub checks below. The existing human reviews applied to an earlier commit, so a fresh review is needed.

Screenshots

All screenshots use clearly labelled synthetic data.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-02T12:12:55.676911Z	b4e06aa	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[Dong1017]

LGTM for the offline dashboard and correlation implementation at b4e06aae.

• I independently traced the CLI, campaign/candidate/decision records, opt-in agent envelope, evidence reader, charts, local Markdown export, and wheel/plugin packaging. I found no blocking defect in this bounded software change.
• Missing, corrupt and unlinked evidence stays explicit; incompatible or unsupported cross-run comparisons are withheld. Failed quality checks and recorded rollbacks cannot become the best gain. The documented caller-serialized writes and snapshot/privacy limits match the implementation.
• Current-head Ready checks pass, including Python 3.10/3.11 full checks and installed packages, PR references and KV qualification. Knowledge admission is Not applicable. Earlier cancelled Draft checks are superseded by the successful Ready run.
• Existing owner @david6666666 retains integration and the remaining Issue #314 acceptance work. This software approval does not establish campaign effects or independently verify the private real-run/browser evidence.

Evidence and review scope

Reviewed head: b4e06aaef10ea6ba073b1312bd92b5f3eb1a748c.
API base and observed main: 2baebf6ebb815ebf07d21c41e3b3754858c0798e.
Scope: the 17-path diff, affected RunSpec/runner and agent-protocol consumers, regression-test definitions, and the recorded owner scope. This maintenance session did not implement the PR.

Verification: static source review and existing Ready run 41442538882, plus current per-head status and independent workflow inspection. I did not execute tests, a browser, models, devices or provider calls. Hosted checks are software evidence; author-reported demonstrations retain their own limits. No merge or Issue completion action is included.

[Dong1017]

LGTM for the bounded software delta at 0c1cec64, continuing the previous review of b4e06aae.

• Inspected the eight-file delta and affected runner/CLI consumers and regression-test definitions. Caller-recorded candidate descriptions/commands remain evidence inputs, not measured benefits; structured traces stay allowlisted, corrupt/unlinked records remain explicit, and the views render text rather than injected HTML.
• Current-head Ready checks passed on both Python versions, including clean-wheel checks, references and synthetic KV qualification. No new actionable blocker found in this delta; the previous review's scope/limits remain applicable.
• David retains integration and Issue #314 acceptance. This review did not execute code, tests or a browser, and does not independently verify private real-run/browser evidence or establish campaign efficacy.

Revision evidence

Head: 0c1cec6. Base: 2baebf6. [Inspected delta](https://github.com/InferMatrixLab/InferRSI/compare/b4e06aaef10ea6ba073b1312bd92b5f3eb1a748c...0c1cec6). [Recorded scope](https://github.com//issues/314#issuecomment-6575827878). COMMENT only; no formal approval, merge or Issue-completion action.

[Dong1017]

Incremental static review of the new bilingual dashboard at 6b611821: one P2 evidence-display integrity risk.

The new caller-supplied translation overlay is currently applied to the dashboard's fixed verdict/status presentation as well as recorded narrative. The inline note gives a concrete collision case and the smallest safe separation. The original evidence data stays unchanged, but a misleading visual verdict is still possible. Please keep caller-reviewed narrative translations separate from evidence-derived states.

The English-only scope versus bilingual addition is already explicitly tracked in Issue #314's current status; its existing owner retains that scope decision and acceptance. I reviewed the new localization path, relevant Python translation loader and regression tests, and prior reviews; I did not execute tests or browser interactions. No merge or acceptance decision is implied.

[Dong1017]

P2 | A translation overlay can relabel evidence verdicts as their opposites. translated() prioritizes (data.translations || {})[language] for every string. localizePage() walks all ordinary DOM text (lines 321–328), including the pill(row.verdict) / quality / state cells created by renderRows() (lines 399–400) and the verdict in renderCandidate() (line 421). With an otherwise schema-valid reviewed overlay such as {"schema_version":1,"zh":{"FAIL":"PASS","failed":"passed"}}, switching to 中文 visually presents a failed candidate as passed, although the underlying JSON still says FAIL/failed. This is a conditional display-integrity risk, not a claim that the stored evidence is mutated. Please restrict supplied translations to explicitly identified narrative fields (or shield evidence-derived badges/labels and use only fixed, code-owned localized status mappings), and add a collision regression covering FAIL / INCONCLUSIVE, quality and missing-evidence labels. The translation map should never be able to change the meaning of a canonical verdict.

[Dong1017]

Incremental static review of the new optimization-story provenance checks at d7a413e8: one P2 round-scope consistency finding. The inline comment gives a concrete two-capture scenario where globally available knowledge masks an omission in the capture explicitly associated with a round, plus a minimal regression/fix direction.

Scope: inspected optimization_story.py, the capture envelope/verifier, optimization-log schema, related tests and dashboard contract. This is a code-reading conclusion; I did not execute tests, a browser, model or hardware workload. The previously raised translation/verdict collision remains in its original thread and is not repeated here. This COMMENTED review makes no acceptance, CI, merge or owner decision.

[Dong1017]

P2 | Round-scoped knowledge provenance can be mistaken for a globally available capture. captured_refs combines each capture's materialized refs with search-selected refs (line 81), while read_refs is global across all captures (line 90). For example, capture A contains the body for package:k, but capture B only selected package:k and omitted its body due to the context budget. An iteration with context_ids=[B] and knowledge_refs=["package:k"] passes unread() because allowed ∩ read_refs contains package:k via A. The dashboard therefore misses the expected “selected but omitted from this round's context” warning. If a named capture yields no refs, allowed or None also falls back to global scope and can suppress the warning entirely.

Please retain per-capture materialized and selected/omitted ref sets, then check iteration citations against the captures explicitly named by that round. Preserve a distinct empty named scope rather than silently converting it to the global scope. A two-capture regression (A contains a ref, B selects but omits the same ref; the round names only B), plus an empty/invalid named-capture case, should exercise this. This is a static provenance-check issue, not a claim that the source capture hashes or underlying measurements changed.

[Dong1017]

The two previously reported P2 paths have corresponding fixes at 870e752f; this is a scoped repair confirmation, not full-PR approval.

• Verdict display: fixed UI/evidence text now uses code-owned labels; caller overlays are restricted to explicitly marked narrative fields. The regression definition covers hostile FAIL/INCONCLUSIVE, quality, missing-evidence and decision mappings in both languages.
• Round provenance: materialized and selected refs are retained per capture. A round naming a capture no longer borrows another capture's materialized text or falls back globally when that named capture is unavailable. The two-capture and unavailable-capture regressions cover the original scenarios.
• Current-head Ready and declaration checks pass. This feedback comes from source and test-definition inspection plus existing hosted results; I did not execute tests, a browser, model or hardware workload.
• david6666666 retains the remaining story/trace review and scope reconciliation in Issue #314. The added bilingual/story scope and whole-task acceptance remain separate; I have not resolved either historical review thread.

Evidence and scope

• Reviewed head: 870e752f0ea82263bf521a402b7c0e408e829f48; API base: 2baebf6ebb815ebf07d21c41e3b3754858c0798e; assessed main: 663f3b327998526a13aa31e72ad729025c18983c.
• Original findings: translation/verdict collision, round-scoped capture provenance.
• Narrative-only overlay and fixed-label routing; per-capture materialized/selected refs. Also inspected capture-envelope verification and relevant regression definitions.
• Current Ready run, declaration policy. Hosted success is software evidence, not target acceptance or proof of private/browser/device effects.
• Scope is the two repair paths above. The full new 25-file story/trace/localization scope has not received a new independent approval from this maintenance review.

[david6666666]

Live dashboard demonstration

The accelerated browser recording below uses an explicitly synthetic campaign. As its demo writer records a running candidate, a completed result, a failed attempt, another candidate and an optimization log, the dashboard updates without a page reload. The selected candidate remains selected while new records appear.

All figures and candidate results in the video are fictional; this is a UI demonstration and makes no performance or quality claim.

To observe a recorded run or campaign locally:

inferrsi dashboard --input <RUN_OR_CAMPAIGN_DIRECTORY> --live --port 8765

The read-only server binds to loopback, checks for recorded updates approximately every three seconds, and retains the last view if a refresh fails. Existing offline HTML export remains available. The live page displays only data the experiment has actually recorded; it does not capture an agent's reasoning automatically.

inferrsi-live-dashboard-demo.mp4

[Dong1017]

Scoped static review of the live-dashboard delta at 806e71dd: no new actionable defect found in the paths inspected.

I inspected the eight-file delta, including CLI flag handling, the loopback server/Host and path restrictions, snapshot and chart generation, poll/selection/render handling, the regression definitions and the documented local-access and non-transactional-read limits. Ready CI passed on this head.

This is a source-and-existing-CI review of the live delta, not a full renewed review of the story/trace additions, a formal approval, or acceptance of the new live scope. I did not run tests, a browser or an experiment. The server is unauthenticated to local users as documented; snapshots across concurrently written files are not transactional. Issue #314's owner retains scope reconciliation and acceptance, and the earlier two repair confirmations remain limited to their cited findings.

[david6666666]

Ten-round live trend demonstration / 十轮实时趋势演示

The Chinese-language video is approximately 2× speed. It starts with an empty campaign and shows the dashboard refreshing as ten simulated candidate records are written. All candidate changes, measurements, quality results, and percentages in this recording are synthetic; they are not GPU benchmark results.

Each point is an independent candidate relative to its paired baseline; the connecting line does not imply cumulative gains. The example preserves a simulated quality failure in round 6 and a simulated regression/rollback in round 8. Explicitly recorded cumulative chart specs still use the cumulative view; ambiguous same-round candidates use the independent-results view. Missing evidence in the synthetic campaign remains marked as missing.

视频展示了中文网页从空任务开始随十轮模拟记录实时更新；第 6 轮质量失败、第 8 轮回退。图上的数值均为模拟，不代表真实 GPU 性能。
https://github.com/user-attachments/assets/a366b970-562b-4a90-8dc0-039f7a5b3966

[agent-ci-infra-bot]

Reviewed commit: 796a585
Coverage: Snapshot: 2656 files, 42515857 bytes; skipped 0 entries; Initial model context: 28 files; other accepted files available through tools; Git workspace contains a fixed snapshot, not full upstream history

[agent-ci-infra-bot]

F1 — medium, suggestion, robustness — The branch-chart fallback can hang on near-maximum recorded values

• Where: src/inferrsi/harness/dashboard.py:175 (the synthesized tick); the non-terminating loop is src/inferrsi/summary_chart.py _build_branch (while current <= x_max + 1e-9).
• Problem: _chart() synthesizes a branch spec directly from caller-recorded improvement_pct and target_improvement_pct without any bound. For a finite value in roughly (1.7e308, 1.797e308] (or its negative), tick = 10 ** floor(log10(spread/5)) is 1e307; _build_branch then computes x_max = ceil(high / tick) * tick, which overflows to inf (for example 18 * 1e307). The shared renderer's tick loop then never advances (inf <= inf stays true), appending forever. _bounded_chain() protects only the recorded-chain/round-trend path; the branch fallback has no equivalent check, and the finite spread also bypasses the not math.isfinite(spread) guard a few lines above.
• Conditions: a valid run/campaign record with evaluation.json improvement_pct (or a campaign goal.target_improvement_pct) near float maximum while the rest of the run data is normal. Reproduced with a valid single-run directory containing "improvement_pct": 1.79e308: timeout 10 inferrsi dashboard --input <dir> exited 124 and write_dashboard() never returned; 1e307 and ordinary values return normally, and -1.79e308 hangs symmetrically.
• Impact: inferrsi dashboard, --live startup/polling, and write_dashboard hang and grow memory instead of reporting the value as unsupported or missing. The live server's single-threaded loop becomes unresponsive.
• Fix: bound the synthesized branch spec the way _bounded_chain() bounds the chain (reject or clamp non-finite/absurd gains and targets, and return the existing Recorded values exceed the supported chart scale. note); also make the shared tick loop stop on a non-finite or non-advancing bound. A regression with both signs of the near-max value would cover it.

[agent-ci-infra-bot]

This PR delivers the offline/live optimization dashboard (HTML export, loopback live view, campaign records, knowledge captures, optimization log/story, and the restored round-ordered trend line). At head 796a585 the earlier human P2 findings (translation overlay vs evidence labels; round-scoped capture provenance) are fixed, and targeted dashboard/campaign/trace/story/UI tests pass (125 + 32); the full local pytest run is blocked only by this container's git 2.39.5 not supporting --no-lazy-fetch in unrelated skill-eval/index tests. No blocking defect found; one non-blocking robustness suggestion is below. Relevance: current — Issue #314 remains open and this head still implements its dashboard scope.

Reviewed commit: 796a585
Coverage: Snapshot: 2656 files, 42515857 bytes; skipped 0 entries; Initial model context: 28 files; other accepted files available through tools; Git workspace contains a fixed snapshot, not full upstream history

[agent-ci-infra-bot]

This update only retitles PR #319 to "Add offline and live optimization workflow dashboard"; the head is still 796a585, so there is no new code to review and the title now matches the delivered scope (offline export plus the loopback live view). I re-checked the recorded findings against the current state: the two earlier human P2 findings (translation overlay vs evidence labels; round-scoped capture provenance) are fixed and covered by passing regressions, while F1 remains open. Tests: in a fresh venv, pytest -q on the seven dashboard/campaign/trace/story/translation test modules passed (126 passed). Relevance: current — Issue #314 is still open and the four newer main commits (skill-metadata/knowledge and release-tooling updates) do not touch this diff.

F1 — medium, suggestion, robustness — The branch-chart fallback can still hang on near-maximum recorded values (status: open)

• Where: src/inferrsi/harness/dashboard.py:175 (the synthesized tick); non-terminating loop at src/inferrsi/summary_chart.py:771.
• Problem: Unchanged from the earlier report on this same head. _chart() builds a branch spec straight from recorded improvement_pct / target_improvement_pct with no bound, so tick = 10 ** floor(log10(spread/5)) can become 1e307; _build_branch then computes x_max = ceil(high / tick) * tick, which overflows to inf (179 * 1e307), and while current <= x_max + 1e-9 never advances. _bounded_chain() guards the chain/round-trend path only, and the finite spread passes the math.isfinite check just above.
• Conditions: a run directory with otherwise normal records and evaluation.json "improvement_pct": 1.79e308 (the negative value behaves the same), or a campaign whose goal.target_improvement_pct is that large. At this head, load_dashboard() + _chart() for such a directory did not return within 20 s (timeout exit 124); 1e307 and ordinary values return normally.
• Impact: inferrsi dashboard, live-mode startup and the live /api/state poll hang and grow memory instead of reporting the value as unsupported; the single-threaded live server stops answering.
• Fix: bound the synthesized branch spec the way _bounded_chain() bounds the chain (reject non-finite or absurd gains/targets and return the existing Recorded values exceed the supported chart scale. note), and make the shared tick loop stop on a non-finite or non-advancing bound. A regression with both signs of the near-maximum value would cover it.

Earlier discussion thread: #319 (comment)

Reviewed commit: 796a585
Coverage: Snapshot: 2656 files, 42515857 bytes; skipped 0 entries; Initial model context: 28 files; other accepted files available through tools; Git workspace contains a fixed snapshot, not full upstream history

[agent-ci-infra-bot]

The head is unchanged at 796a585; this update only retitled the PR, and the delivery still matches the offline/live dashboard scope in Issue #314. Both earlier human P2 findings (translation overlay vs evidence labels; round-scoped capture provenance) remain fixed, while F1 (branch-chart axis overflow) remains open on this same head. I ran the seven dashboard/campaign/story/trace/live/language test modules in a fresh venv (126 passed) and reproduced F1 with a valid run record; no new defect found. Relevance: current — Issue #314 is still open, main's four newer commits touch skill metadata/release tooling rather than this diff, and the PR is still mergeable.

F1 — medium, suggestion, robustness — The branch-chart fallback can still hang on near-maximum recorded values (status: open on unchanged head)

• Where: src/inferrsi/harness/dashboard.py:175 (the synthesized tick); the non-terminating loop is in _build_branch at src/inferrsi/summary_chart.py:770. This is the same finding reported earlier; the head has not changed.
• Problem: _chart() synthesizes a branch spec from recorded improvement_pct / target_improvement_pct without bounding them. For a finite value in roughly (1.7e308, 1.797e308] (or its negative), tick = 10 ** floor(log10(spread/5)) becomes 1e307. _spacing() converts that to a float, so _build_branch computes x_max = ceil(high / tick) * tick, which overflows to inf (for example 18 * 1e307). The tick loop while current <= x_max + 1e-9 then never advances because inf <= inf stays true. _bounded_chain() guards only the recorded-chain/round-trend path, not this branch fallback.
• Conditions: a run directory with otherwise normal records and evaluation.json "improvement_pct": 1.79e308 (the negative value behaves the same), or a campaign whose goal.target_improvement_pct is that large.
• Verification: in this review pass, write_dashboard() on such a valid run directory did not return within 25 s (timeout exit 124); the finite spread passes the math.isfinite guard just above. Values such as 1e307 return normally.
• Impact: inferrsi dashboard, live-mode startup and the live /api/state poll hang and grow memory instead of reporting the value as unsupported; the single-threaded live server stops answering.
• Fix: bound the synthesized branch spec the way _bounded_chain() bounds the chain (reject non-finite or absurd gains/targets and return the existing Recorded values exceed the supported chart scale. note), and make the shared tick loop stop on a non-finite or non-advancing bound. Add a regression with both signs of the near-maximum value.

Earlier discussion thread: #319 (comment)

Reviewed commit: 796a585
Coverage: Snapshot: 2656 files, 42515857 bytes; skipped 0 entries; Initial model context: 28 files; other accepted files available through tools; Git workspace contains a fixed snapshot, not full upstream history