feat(evalboard): show the harness (RunConfig) column on the runs tables

[bai-uipath]

Duplicated from UiPath/coder_eval.bak#494 (the pre-public-migration repo), replayed onto the new public repo.

What

Two evalboard changes supporting scheduled non-Claude evals:

• Harness column — the main runs table shows which agent produced each run as its vendor logo (Anthropic / OpenAI / Google), read from the run-level RunConfig that eval_runner stamps into run.json (falling back to the per-task agent_config.type for unstamped runs).
• Gemini pricing — added the Gemini model rates to the frontend pricing table so the Tokens↔USD toggle and per-message cost price Gemini runs instead of showing "—", and repointed the pricing-parity drift guard at the post-proxy-removal pricing.py path.

Why

With Codex and Gemini runs joining the board alongside Claude, the run list gave no at-a-glance signal of which agent produced a row, and Gemini token costs rendered as "—". The vendor logo makes each run's agent obvious; the pricing rows make Gemini costs real.

Notes

• The Harness column is internal-only (isInternal, see lib/edition.ts) and lives on the main runs table only — the ad-hoc table omits it.
• A missing harness defaults to claude-code (the nightly). The environment (alpha/prod) is read and carried on the run overview but intentionally not displayed yet.
• Maturity stays a Claude-only concern enforced at write time in coder_eval_uipath; the board only reflects run.json, so non-Claude runs carry no maturity rows.

Companion to UiPath/coder_eval_uipath#39 — the RunConfig work that stamps the harness this column reads and adds the scheduled non-Claude evals. The two land together.

Screenshot

[github-actions]

Claude finished @bai-uipath's task in 1m 22s —— View job

---

Code Review in Progress

Review Checklist:

• Read .github/code_review.md for review guidelines
• Read CLAUDE.md for project conventions
• Get full diff with git diff origin/main...HEAD
• Read each changed file in full for context
• Analyze cross-file consistency
• Check for missing functionality
• Provide design-level scrutiny
• Format review according to guidelines

[bai-uipath]

Already reviewed and approved in https://github.com/UiPath/coder_eval.bak/pull/494 - merging.

[uipreliga]

Review: coder_eval — pr:10 (evalboard Harness column) — 10 files (7 code/test, 3 images), all axes

Scope: pr:10 (evalboard Harness column) — 10 files (7 code/test, 3 images), all axes · branch bai/evalboard-harness-column · ba8d63e · 2026-07-09T18:18Z · workflow variant

Change class: simple — adds an internal-only evalboard "Harness" logo column plus an extractRunConfig harness-derivation helper (RunConfig stamp → most-common agent_config.type fallback) with unit tests; localized, well-tested display logic, no subtle correctness reasoning

coder_eval is in excellent shape — 9.9/10 overall with clean Type Safety, Security, Error Handling, API Surface, and Evaluation Harness axes (all 10/10) and no critical/high findings anywhere; the only real risk is a thin seam of untested evalboard fallback branches (notably the harness-label partial-stamp path), plus a handful of cosmetic KISS/YAGNI nits, none of which touch task scoring or final_status, so the bottom line is a ship-ready codebase with a small, low-stakes test-and-tidy backlog.

Summary

Axis	Score	🔴	🟠	🟡	🔵	Top Issue
1. Code Quality & Style	9.8 / 10	0	0	0	2	mostCommonAgentType implements a frequency vote for data the code itself calls single-harness
2. Type Safety	10 / 10	0	0	0	0	—
3. Test Health	9.5 / 10	0	0	1	0	Fallback/edge-branch behaviors lack unit tests (extractRunConfig partial-stamp, HarnessBadge fallbacks, mostCommonAgentType tie-break)
4. Security	10 / 10	0	0	0	0	—
5. Architecture & Design	9.9 / 10	0	0	0	1	RunOverview.environment is captured and threaded but never consumed anywhere
6. Error Handling & Resilience	10 / 10	0	0	0	0	—
7. API Surface & Maintainability	10 / 10	0	0	0	0	—
8. Evaluation Harness Quality	10 / 10	0	0	0	0	—

Overall Score: 9.9 / 10 · Weakest Axis: Test Health at 9.5 / 10
Totals: 🔴 0 · 🟠 0 · 🟡 1 · 🔵 3 across 8 axes.

Blockers

None.

Non-blocking, but please consider before merge

1. [Axis 3] Fallback/edge-branch behaviors lack unit tests (extractRunConfig partial-stamp, HarnessBadge fallbacks, mostCommonAgentType tie-break) (evalboard/lib/runs.ts:843) — The three new tests cover only (1) a full run_config stamp, (2) no run_config at all, and (3) legacy null. The deliberately-coded partial-stamp branch is never exercised: at runs.ts:843-846 harness: typeof r.harness === "string" && r.harness ? r.harness : mostCommonAgentType(data.task_results ?? []) falls back to the per-task vote when a run_config OBJECT is present but its harness is empty/missing — the comment at 839-842 states this exists specifically to avoid mislabeling a real codex/antigravity run as the claude-code default. This is the branch that actually changes the harness label, yet no test constructs {environment_info:{run_config:{environment:"prod"}}, task_results:[{agent_config:{type:"codex"}}]} to assert harness==="codex" AND environment==="prod". The rc-present-but-environment-absent → null path (line 847-848) is likewise unexercised. Add a test that trips this partial-stamp branch.

Nits

1. [Axis 1] mostCommonAgentType implements a frequency vote for data the code itself calls single-harness (evalboard/lib/runs.ts:860) — mostCommonAgentType builds a Map<string,number> frequency table and an argmax scan (lines 860-876) to pick the run's harness, yet its own doc-comment states "A run is single-harness in practice" (line 858). For single-harness data a first-non-null pick — rows.find(r => typeof r.agent_config?.type === "string" && r.agent_config.type)?.agent_config?.type ?? null — is equivalent and ~12 fewer lines. The vote machinery exceeds what the goal needs (KISS / mild over-engineering). It is correct and readable, so this is informational, not a blocker; consider simplifying to first-non-null.
2. [Axis 1] Empty-state colSpan hardcodes the column count as a second value to keep in sync (evalboard/app/page.tsx:409) — colSpan={isInternal ? 6 : 5} (line 409) hardcodes the header column count (Run, [Harness], Pass rate, Tasks, Cost, Duration = 5 base + 1 gated). Adding/removing a <th> in the thead above requires remembering to bump these literals, or the empty-state row misaligns. Minor maintainability nit; could derive from a shared column-count constant, but acceptable as-is for a small static table.
3. [Axis 5] RunOverview.environment is captured and threaded but never consumed anywhere (evalboard/lib/runs.ts:802) — extractRunConfig (runs.ts:847-848) derives environment and readRunOverview spreads it onto RunOverview (environment?: string | null;, runs.ts:802), but no UI, listing row, or other code path reads RunOverview.environment — a repo-wide grep finds it only in runs.ts and runs.test.ts, never in overview.ts's RunListingRow/AdhocRunRow or app/. The field's own comment concedes it is 'intentionally NOT surfaced in the UI yet.' This is a benign YAGNI/dead-field nit, not an architecture defect: the value is a free by-product of parsing run_config (which already yields harness), it is one optional field, and it is test-covered. Either drop it until a consumer needs it, or leave as documented forward-plumbing — it does not couple layers or create a second source of truth.

What's Missing

Tests:

• 🟡 New branches shipped without tests: extractRunConfig's partial-stamp fallback (run_config object present but harness empty/missing -> mostCommonAgentType vote while environment still taken from the stamp, runs.ts:843-848), the rc-present-but-environment-absent null path, HarnessBadge's null-default and unknown-harness raw-text fallbacks (harness-badge.tsx:21-24), and mostCommonAgentType's tie-break/empty-rows path (runs.ts:860). The three added extractRunConfig tests only cover full-stamp, no-run_config, and legacy-null; none constructs {environment_info:{run_config:{environment:"prod"}}, task_results:[{agent_config:{type:"codex"}}]} to assert harness==="codex" AND environment==="prod" -- the branch that actually changes the label. (trigger: evalboard/lib/runs.ts) (restates: Axis 3: Fallback/edge-branch behaviors lack unit tests (extractRunConfig partial-stamp, HarnessBadge fallbacks, mostCommonAgentType tie-break))

Display & mapping dicts:

• 🔵 HARNESS_LOGO (harness-badge.tsx:8-18) maps only claude-code / codex / antigravity, but AgentKind (enums.py:107-111) also defines "none" (agentless/system task) and "unknown". A run whose agent_config.type is "none" or "unknown" -- the value mostCommonAgentType returns and feeds straight into the badge -- falls through to the raw-id <span> instead of a logo. Deliberate safe fallback and an internal-only column, so low, but the display dict is not exhaustive over the AgentKind universe; confirm "none"/"unknown" rendering as raw text is intended. _(trigger: evalboard/app/components/harness-badge.tsx)

Parallel paths:

• 🔵 The Harness column was added to the main run-listing table (page.tsx:334-338, 372-378) but not to the parallel ad-hoc runs table (page.tsx:464-486); buildAdhocRows (overview.ts:727) leaves the inherited RunListingRow.harness unset by design. Ad-hoc runs can also be codex/antigravity, so an internal viewer sees harness in one table and not the other. The omission is documented as deliberate ("Harness is a main-table-only (internal) column; ad-hoc rows omit it"), so this is a confirm-intent note, not a divergence bug. (trigger: evalboard/app/page.tsx)

Daily/nightly:

• 🔵 runs.ts:835 now consumes a new cross-repo contract field -- environment_info.run_config.{harness,environment} -- stamped by the external eval_runner (coder-eval-uipath) pipeline, a producer this repo no longer contains. The PR does not state whether the current nightly's eval_runner actually stamps run_config yet; until it does, every nightly run falls through to the mostCommonAgentType(agent_config.type) vote (which IS persisted in task_results, so it degrades gracefully to the correct harness, with environment left null and unused). Blast radius should be stated explicitly: today the Harness column is fed by the fallback vote, not the stamp. (trigger: evalboard/lib/runs.ts)

Downstream consumers:

• 🔵 extractRunConfig derives environment and readRunOverview threads it onto RunOverview.environment (runs.ts:802, 847-848), but no downstream consumer reads it -- overview.ts's RunListingRow/AdhocRunRow don't carry it and no page/listing renders it. A new field plumbed through the read path with zero consumer; either drop it until needed or leave as documented forward-plumbing. (trigger: evalboard/lib/runs.ts) (restates: Axis 5: RunOverview.environment is captured and threaded but never consumed anywhere)

Harness & Lint Improvements

Static checks (lint / type):

• (none)

Harness improvements (not statically reachable):

• Add a branch-coverage threshold gate to evalboard: turn on coverage.thresholds (e.g. branches/lines/functions >= 80) in evalboard/vitest.config.ts and run vitest run --coverage in the evalboard verify/CI step (the @vitest/coverage-v8 dep is already installed but never invoked). A deliberately-coded fallback branch then fails the build until a test exercises it, instead of silently shipping untested. Why not static: None of the four available static-check kinds (ruff/pyright/ce-lint/bandit-codeql) apply — these files are TypeScript. More fundamentally, branch coverage is a runtime property: no static rule can tell whether the extractRunConfig partial-stamp branch (runs.ts:843-848), the HarnessBadge fallbacks (harness-badge.tsx:21), or the mostCommonAgentType tie-break (runs.ts:870) are meaningfully exercised — that requires executing the tests with instrumentation. Prevents: A3-medium finding: the deliberately-coded fallback/edge branches (extractRunConfig partial-stamp where run_config is present but harness is empty so it falls to the per-task vote, plus HarnessBadge fallbacks and mostCommonAgentType tie-break) shipped with the three new tests covering only the happy/absent paths and never the branch that actually flips the harness label.
• Wire a dead-export / unused-symbol gate (knip or ts-prune, configured for the evalboard workspace) into evalboard verify/CI. It flags a symbol that is written and threaded but never read across the app — e.g. RunOverview.environment (runs.ts:802), derived in extractRunConfig and spread onto the interface yet consumed by no UI/listing/report path. Why not static: The four provided static-check kinds are Python-core gates and don't see evalboard TS at all; and tsc's own noUnusedLocals/noUnusedParameters do NOT flag an interface field that is assigned but never read (nor an unread export), so catching 'threaded-but-never-consumed' forward-plumbing needs a dedicated JS-ecosystem dead-code tool run as a CI gate rather than a compiler flag. Prevents: A5/A7-low finding: the dead RunOverview.environment field — a free by-product of parsing run_config that is captured, typed, and spread but has zero consumers (grep finds it only in runs.ts + runs.test.ts).

Top 5 Priority Actions

1. Add a unit test for the partial-stamp branch in extractRunConfig (evalboard/lib/runs.ts:843-848) using input {environment_info:{run_config:{environment:"prod"}}, task_results:[{agent_config:{type:"codex"}}]} asserting harness==="codex" AND environment==="prod" — this is the branch that actually flips the displayed harness label away from the claude-code default and is currently unexercised (no task score/final_status is affected, but it is the highest-leverage gap).
2. Cover the remaining untested fallback edges in the same test-coverage theme — HarnessBadge fallback rendering (evalboard/app/_components/harness-badge.tsx:21) and the mostCommonAgentType tie-break (evalboard/lib/runs.ts:870), plus the rc-present-but-environment-absent null path at runs.ts:848.
3. Resolve the dead RunOverview.environment field (evalboard/lib/runs.ts:802) — either drop it until a consumer exists or leave it as documented forward-plumbing, since it is captured and threaded but read nowhere outside runs.ts/runs.test.ts.
4. Simplify mostCommonAgentType (evalboard/lib/runs.ts:860-876) from a frequency-vote argmax to a first-non-null pick, matching its own "single-harness in practice" doc-comment and shedding ~12 lines.
5. Derive the empty-state colSpan from a shared column-count constant instead of the hardcoded colSpan={isInternal ? 6 : 5} (evalboard/app/page.tsx:409) so adding/removing a can't misalign the empty row.

---

Stats: 0 🔴 · 0 🟠 · 1 🟡 · 3 🔵 across 8 axes reviewed.