Conversation
|
🦞👀 Pull request received. I will update this pull request when review starts. ClawSweeper review completeClawSweeper finished reviewing this revision. The review result is being finalized. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 09cf7edb0f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Codex review: blocked before merge. Reviewed September 21, 2026, 6:54 AM ET / 10:54 UTC (Revision 66). ClawSweeper reviewWhat this changesReplace Grok context-occupancy totals with completed-turn usage, display recorded spend with labeled price estimates, and import eligible OpenCodex OAuth usage into Usage & Spend. Merge readiness⛔ Blocked before merge - 3 items remain This remains useful work absent from main and the latest release. Earlier correctness findings are addressed, and the runtime evidence is sufficient; the revised default dollar-display policy remains an owner decision. Priority: P2 Review scores
Verification
How this fits togetherCodexBar turns local provider logs into usage and spending summaries for its menu and dashboard. This change processes Grok completed turns and optional OpenCodex attempt records while retaining cost provenance and incomplete-history information. flowchart TD
A[Grok CLI logs] --> B[Bounded completed-turn scanner]
C[Recorded spend and price catalog] --> B
B --> D[Daily usage and cost provenance]
E[Optional OpenCodex logs] --> F[Reported OAuth attempt filter]
F --> D
D --> G[Window and coverage projection]
G --> H[Menu and spend dashboard]
Decision needed
Why: The revised accounting evidence supports recorded-first costs, but the owner has not approved this default-display change and the changes-requested review remains outstanding. Before merge
Agent review detailsSecurityNone. Review metrics
Merge-risk optionsMaintainer options:
Technical reviewBest possible solution: Use completed-turn accounting with recorded-first costs, visible estimate and coverage disclosures, and explicitly owner-approved upgrade defaults. Do we have a high-confidence way to reproduce the issue? Yes: main visibly sums context occupancy rather than completed-turn consumption, corroborated by independent real-session measurements; this review did not execute a reproduction. Is this the best way to solve the issue? Yes for the accounting approach: bounded completed-turn parsing and recorded-cost precedence fit the observed data, while the default dollar-display choice still requires owner approval. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning medium; reviewed against d8d0f3394989. LabelsLabel justifications:
EvidenceWhat I checked:
Likely related people:
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
HistoryReview history (65 earlier review cycles; latest 8 shown)
|
08360b5 to
e3cd3b9
Compare
|
Both automated findings are addressed, plus the review's other checklist items. The inline comments were left against P1 — Preserve the Grok fallback on repeated probe failuresFixed in if provider == .grok {
if self.tokenSnapshotPublicationForCurrentProviderConfig(for: provider) == nil {
Task { @MainActor [weak self] in
await self?.scanAndPublishGrokLocalTokenSnapshot(...)
}
}
} else if Self.tokenCostRequiresProviderSnapshot(provider) {
self.clearTokenSnapshot(for: provider)
}Regression coverage is in P2 — Refresh pricing before scanning Grok sessionsCorrect, and thank you — this was a genuine gap and not one the local tests would have surfaced. Fixed in Note the inline comment still points at Coverage: Real-session evidence
The same corpus on Those figures were cross-checked against an independent reimplementation of the pricing formula over the same logs; the two agree to the cent. Merge risk / branch stateRebased onto current One thing deliberately left undone: no |
|
Addressed both current findings in
Validation on the exact pushed head:
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
Final validation for
The PR body now has the final validation results and the immutable landed-producer source audit. The owner display-policy decision and older changes-requested review remain unresolved. The review environment's inability to retrieve the later producer carry is also recorded; the supplied source audit does not expand the pinned executable capture into a claim about a released producer version. |
|
Merged latest main The conflict resolution preserves upstream's checked numeric aggregation, parser-revision migrations, and Grok terminal-billing work gate while retaining completed-turn accounting, recorded-versus-estimated disclosure, custom-price precedence, and per-attempt OAuth attribution. The generated parser hash is Validation on this head:
The PR body records the current results and distinguishes them from the historical native-corpus proof. The owner display-policy decision remains open. @clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
Addressed the newly reported native overflow finding in The native parser now validates NSNumber conversions, preserves unknown counts, and uses checked addition across model/day/window totals. Later valid records and cache rereads cannot clear an unknown aggregate. Explicit valid totals retain precedence, valid neighboring token classes and recorded spend remain available, and incomplete token accounting does not establish full coverage. The added production-scanner regression suite covers per-record and cross-record overflow, multiple days, nested models, malformed number types, explicit totals, snapshot projection, and cache reuse. Validation:
The PR body has the current results. The new CI run is pending; the owner dollar-display decision remains open. @clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
Addressed the remaining menu-projection overflow finding in A checked, complete-count helper now covers the remote-backed Grok window projection, window request totals, comparison summaries, and menu/Widget fallback totals. Unknown daily values are retained as unknown instead of being dropped into a partial sum. This complements the native scanner repair in the preceding commit. The new end-to-end regression writes native completed-turn logs and exercises both remote-backed and fallback live consumers. It covers individually representable Int.max/1 totals across days and an already-unknown day followed by a valid day. The menu, Widget, and dashboard keep the aggregate unavailable; narrowing to the unaffected one-day window restores its known count. All four scenarios passed. Current-head validation: The owner's default dollar-display decision remains open. @clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
|
Final author-side validation for
The PR body now has the final local validation and exact CI limitation. The default-dollar display policy and the owner's outstanding changes-requested review remain maintainer decisions. @clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
Resolve the 136-commit drift since the last rebase. Provider-aware pricing on main replaced the Codex-only lookup this branch extended, so the xAI-specific behaviour is re-expressed on top of it instead of alongside it: - codexModelsDevPricingTargets keeps main's resolver-backed form, widened to xAI routes with the branch's grok `-build` alias fallback; the duplicate copy in CostUsagePricing+CodexResolver is dropped and only the route union remains. - The OpenCodex aggregator uses main's providerCostUSD path, with the branch's standalone-xAI gate kept ahead of it: raw xAI rows without Grok OAuth provenance stay token-only unless explicit custom prices exist. - OpenCodexUsageLog adopts main's TestProcessSafety-based path lookup. - The menu-card live-consumer projection moves with main's relocation of the card model into UsageStore+MenuCardModel; the override-card test uses the store API. - Window summaries keep the branch's complete token sum and main's CheckedSum request total; ProviderRow keeps both incompleteRequestCount and the Grok cost disclaimer. - Parser hash regenerated; main's 6d48baf0ed980828 joins the compatible predecessor list. Gatekeeper anchors re-pointed at the merged sources, and the merged openai route cluster and xAI alias fallback carry design markers. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Main regenerated its parser hash again after the previous merge. Keep main's predecessor entry for 6d48baf0ed980828, add its new 6a4df886696f4ab5 as the current-main predecessor, and regenerate this branch's hash. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
@steipete requesting re-review on Status of the four P1s and the P2 from your review:
All review threads are resolved. On this head: |
Only the gatekeeper allowlist conflicted; its anchors are re-pointed at the merged sources and the Grok refresh cluster entry is kept. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Chiming in as an affected Grok user on the outstanding default-display question, plus a source that resolves the last bit of uncertainty on the divisor. On On the default-display question: as a user currently seeing Happy to independently verify against my own |
|
Ran the PR's own `GrokWindowProvenanceProofTests` live-corpus test (`CODEXBAR_LIVE_GROK_CATALOG_PROOF=1`) against my real `~/.grok/sessions` (547M+ tokens across 30 days, larger/messier than the corpora referenced in the PR description). Good news — the core fix works. The scanner correctly produced real recorded spend:
But the test itself failed — 6 assertion failures, all downstream of the scanner. The raw snapshot (
So on my account, the live-consumer projection ( |
A real corpus mixes priced days with days the scanner could not price at all: turns the CLI attributed to no model and recorded no ticks for. The scanner declares those as unpriced requests, but two consumers treated such a day as making the whole window unknown, so the dashboard row showed no total, no disclosure, and no provenance while the scanner's snapshot was correct. - The generic window narrowing publishes a window cost only when every retained day carries one. Both Grok projections now sum the days that were priced through one shared helper; a Grok day without a cost is a disclosed gap, not an unknown total. - The dashboard's cost-consistency rule only knew Codex's and Cursor's ways of declaring a row unpriceable. A Grok day whose requests all went unpriced is now an explicit gap as well, so the recorded total survives and the coverage counts disclose the unpriced requests. The live proof now prints the coverage, active-day, unpriced-day and unpriced- request figures for each window, and asserts the documented no-total contract when a bounded scan cannot establish coverage, so a failing run says which gap it hit instead of only that the row was empty. Also cite xAI's cost-tracking documentation for the tick divisor. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The gate that keeps raw xAI rows token-only without Grok OAuth provenance keyed on the model namespace, so a router-hosted `xai/...` row billed by OpenRouter lost the catalog price its recorded provider entitles it to under steipete#3676. It now follows the resolved billing route: the `xai` provider, or the legacy OpenAI transport label carrying an xAI route. Router-hosted xAI rows keep their recorded-provider price; directly billed rows stay token-only as before. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Seven files conflicted. Main's steipete#3353 moved the per-provider dashboard rows into SpendDashboardProviderBreakdown, so the Grok cost disclosure moves with them: on the source row when a provider lists its sources, under the provider header when its single native source is folded into it. Main relocated resetTokenUsageState into UsageStore+TokenCost; the Grok scan cancellation moves with it, and the live-consumer selection this branch adds keeps main's localized no-data message beside it. Parser hash regenerated to 611a82b6eb2b5ffc; main's 03e43d1217789d16 and the previous PR head b004d0cf7d471304 join the compatible predecessor list, the exact-list assertion, and the SQLite adoption arguments. Gatekeeper anchors re-pointed at the merged sources; the shared dashboard construct now carries both main's and this branch's Grok references. Window narrowing moves into CostUsageModels+Narrowing.swift to keep CostUsageModels.swift within the file length limit. Release notes moved under 0.63.1 — Unreleased. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
@Chipagosfinest thank you — that run found a real defect, and I could reproduce the dashboard half of it from your numbers alone. Fixed on exact head What brokeYour scanner output was right; the two consumers downstream of it were not, and they failed on a corpus shape my own proofs never had: a window that mixes priced days with days the scanner could not price at all. That happens when a turn is attributed to no model and carries no
The regression writes one recorded day and one unpriced day, projects the 30-day window through One thing your run can still tell me
The docs.x.ai citation for the |
|
Both findings from the 2026-09-19 review are fixed on exact head [P1] Scope the xAI pricing exception to the billing providerYou were right: the gate keyed on the model namespace, so a router-hosted [P3] Move the Grok entries into the current unreleased sectionBoth entries now sit under Live-corpus defect (not in your review, found by a user running the PR's own proof)A window mixing priced days with days the scanner could not price at all — turns attributed to no model with no Gate on this head
The owner decision is unchanged: whether existing Grok users should receive CLI-recorded spend with disclosed list-price fallback by default. Your recommendation to approve it after the pricing and release-note findings were fixed is noted; both are. @clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
steipete#3805 added a regression that keeps local Grok tokens through the RPC-to-proxy fallback, with a fixture that carries only `signals.json` and expects its context-window occupancy to surface as 42 tokens. This branch deliberately stops counting occupancy — `signals.json` is session metadata only, and local history comes from completed turns — so on the merged tree the fallback kept no history and the test failed. The fixture now records one completed 42-token turn beside the same `signals.json`, which keeps the test's subject (the fallback retains local history) and demonstrates the branch's contract (occupancy is not counted). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
The red CI runs the test merge with The fixture now records one completed 42-token turn beside the same Also merged the three Gate on this head
|
|
Thanks @olddonkey. At fetched head |


Problem and resulting behavior
Grok's local fallback reads ending context occupancy from
signals.json, which is not completed-turn usage, and publishes no local cost. This PR reads completed turns from bounded native CLI logs and carries recorded-versus-estimated cost through the menu and Usage & Spend.session/updateand_x.ai/session/updaterecords inupdates.jsonl, with daily, model, and request breakdowns.usage.costUsdTicks / 1e10as the authoritative turn total. Count it once. Show nested model dollars only when all nested ticks exist and reconcile to that total; otherwise retain model tokens with unknown model dollars. Missing recorded cost falls back to disclosed public xAI list prices.Grok CLI-recorded spend, list price where unrecorded · not a bill.OpenCodex integration
With the existing Include OpenCodex usage logs switch enabled (off by default), Usage & Spend also includes reported physical Grok OAuth attempts. The producer contract originally proposed in OpenCodex #3642 has now landed in
devthrough the attributed carry #3762, merged asf00f2bcaea251ebe7ad4de9e38337b4be0ccee47. It writesattempts[].credentialSourcefrom the resolved upstream transport. Compatibility is verified against the pinned producer commit in the evidence below; availability in a released OpenCodex version is not claimed.The landed carry's immutable source contract has additionally been inspected directly, including resolved-adapter stamping, persistence normalization, and OAuth/API-key regression assertions. The executable capture remains pinned to its documented producer revision; this source audit does not claim a new capture or released-version coverage.
Only
provider: "xai",credentialSource: "grok-oauth"attempts with an upstream send and reported token usage qualify. API-key traffic, historic rows, locally answered requests, unknown sources, and estimated/unreported token counts stay excluded. Current configuration and top-level credential metadata cannot retroactively classify usage.Combo requests contribute each qualifying attempt's own token counts, never the parent aggregate. Duplicate request IDs are resolved before grouping; duplicate attempt ordinals are rejected. SQLite cache schema 3 retains attempt metadata and rebuilds older derived caches from the log. OpenCodex dollars use list prices and remain estimates, including when combined with native CLI-recorded spend. Missing token classes and unknown prices retain tokens without inventing a dollar value. The Grok menu continues to use native CLI logs.
Producer-to-dashboard evidence and review fixes
The committed evidence report and raw ledger fixture contain output from OpenCodex's unmodified production handlers and durable usage writer at
146ed679c9633e5d68726217fcadc8e0b107339b. Two localhost HTTP requests exercised OAuth 401 replay through Responses and native Chat API-key dispatch. Upstream responses and credentials used isolated fixtures; this is production-path capture evidence, not live vendor authentication or billing evidence. The capture helper rejects unexpected external fetches and is reproducible against that pinned checkout.CodexBar imports the exact captured bytes through the production disk loader, with no injected entries or loader closure, and reopens the persisted cache:
Standalone reports retain explicit xAI custom-price estimates from either the caller overlay or the application overlay, including known zero. Raw xAI records without an explicit price remain token-only, and the subscription fan-out still excludes API-key and historic records.
opencodeandopencode-freeretain their existing catalog and custom prices.Application overlays first match the original model name before the provider-qualified catalog name. Bare keys retain precedence when both keys exist; incomplete rates stay unknown, cached-input accounting is preserved, and caller-supplied custom pricing still takes precedence over the application overlay. New regressions cover these cases through the application-overlay parameter and the standalone disk/cache loader. Coverage includes known-zero overrides, incomplete rates, cache accounting, and both snapshot and application overlays.
Both async native Grok scan entry points propagate the executor's cancellation callback through discovery, JSONL reads, and aggregation. An in-flight regression cancels after parsing begins, observes
CancellationError, and confirms the next queued scan runs within one second. Cancelled partial parses are uncacheable and cannot establish complete history. The earlierc3919a224serial proof stopped at 4,874/40,000 decoded records and released the queue after 0.001864583 seconds. The evidence report also retains the initial measurement.Native numeric safety
Native token conversion rejects booleans, fractions, negative values, and out-of-range numbers without trusting NSNumber's clamping integer bridge. An absent token class retains the established zero default; a malformed count remains unknown. Explicit valid totals retain precedence, so decoding never adds input and output unnecessarily.
Checked addition carries unknown or overflowed values through model, day, and window aggregation, including after later valid records and cached-log rereads. Valid neighboring token classes and CLI-recorded spend remain available. Incomplete token accounting does not establish full history coverage; estimated pricing requires representable inputs. Production-scanner regressions cover Int.max plus one within a turn, sums across turns and days, nested models, invalid JSON number types, explicit totals, snapshots, and cache reuse.
The same checked, complete-count aggregation now covers remote-backed Grok menu projection, rolling-window requests, comparison summaries, and menu/Widget fallback totals. Unknown daily counts cannot be dropped into a plausible partial sum. An end-to-end scanner regression covers remote-backed and fallback projections for cross-day overflow and an unknown day followed by a valid day; menu, Widget, and dashboard totals stay unavailable, while narrowing to a valid one-day window restores its count.
Bounds and compatibility
Native scans run on the dedicated executor with limits of 64 MiB / 20,000 turns per file, 1 MiB per record, 256 sessions / 256 MiB / 100,000 turns per scan, and 4,096 discovery entries. The process cache retains at most 64 files or 50,000 turns. Cancellation and truncated history cannot publish complete coverage.
Merged main
8b254dbec11ddd5c5547878d9640e4e965306c71, retaining its checked token aggregation, daily spend ledger, native parser-revision migration, and Grok terminal-billing-failure work avoidance. The local-summary injection seam now invokes the PR's pricing-aware 365-day scanner only after the upstream billing/identity gate permits a snapshot. Failed billing still refreshes local spend through the app's existing independent fallback path.OpenCodex token counts retain upstream's safe numeric conversion and truncation policy. Attempt ordinals and send counts remain exact integers: booleans, fractional counts, and out-of-range values cannot qualify an OAuth attempt. Aggregation retains overflow as unavailable while keeping valid neighboring token classes and Grok estimate coverage. The upstream cursor parser version invalidates legacy numeric caches; schema 3 still preserves request-time attempt provenance.
Regenerated the native parser hash from the merged source:
a8559238a5fc0480. Current-main hashf5fdba377006d7beand prior-PR hashc3a879df4eff7187join the compatible predecessors. Stored rows and checkpoints are retained, while upstream's per-file parser revisions schedule bounded reparsing where needed. SQLite adoption tests cover the current-main and prior-PR hashes, and upstream whitespace/subagent migration tests remain in the validation set. The architecture gate keeps its exact provider-reference checks at their updated source locations. Grok release notes are under0.59.1 — Unreleased; all published sections match main.Merge with main (2026-09-19)
Head:
eb1647cc8ed68e39f5ed90eb8730ed3d182ee67d. Three merge commits bring the branch ontomainat0f8489dd8(the third,eb1647cc8, only re-anchors the gatekeeper allowlist after three more main commits). The first two: the first (e7b2b2203, main at88aab2ba6, 136 commits since the previous rebase) resolves eleven conflicting files; the second (32a73cd90, 25 further commits) only re-resolves the parser-hash family after main regenerated its hash again.Main's provider-aware pricing replaced the Codex-only lookup this branch had extended, so the xAI behaviour is re-expressed on top of it rather than alongside it:
codexModelsDevPricingTargetskeeps main's resolver-backed form, widened to xAI routes with this branch's grok-buildalias fallback. The duplicate copy inCostUsagePricing+CodexResolver.swiftis dropped; only the route union remains there.providerCostUSD, with this branch's standalone-xAI gate kept ahead of it: raw xAI rows without Grok OAuth provenance stay token-only unless explicit custom prices exist.OpenCodexUsageLogadopts main'sTestProcessSafetypath lookup; the branch's private test-detection helper is gone.UsageStore+MenuCardModel.swift; the override-card regression now uses the store API.CheckedSumrequest total.ProviderRowkeeps bothincompleteRequestCountand the Grok cost disclaimer.b004d0cf7d471304; main's6d48baf0ed980828and6a4df886696f4ab5are in the compatible predecessor list and the exact-list assertion. Gatekeeper anchors were re-pointed at the merged sources, and the merged openai route cluster and the xAI alias fallback carry design markers.Validation:
e7b2b2203:make checkclean;make test114 groups, 11,456 tests, 0 failures, 0 retries (792 s).32a73cd90:make check0 violations in 2,431 files with the parser-hash check clean; focused Grok, OpenCodex, CostUsage, gatekeeper, spend-dashboard and menu-card suites pass, with the concurrency and bounded-progress suites confirmed green in isolation.make teston this head: 116 groups, 11,577 tests, 0 failures, 0 retries (702 s).Merge with main (2026-09-21)
Head:
75945a360f90abe5abac262c6c80d4cf65ec2326, onmainatd8d0f3394(46 commits since the previous merge). Seven files conflicted:PreferencesSpendDashboardPane.swift: main's feat(spend): group usage details by provider #3353 moved the per-provider rows intoSpendDashboardProviderBreakdown, so the Grok cost disclosure moved with them — on the source row when a provider lists its sources, and under the provider header when its single native source is folded into it.UsageStore.swift/UsageStore+TokenCost.swift: main relocatedresetTokenUsageStateinto the token-cost extension; the Grok scan cancellation moved with it. The live-consumer selection this branch adds and main's localized no-data message both remain.611a82b6eb2b5ffc; main's03e43d1217789d16and the previous PR headb004d0cf7d471304join the compatible predecessor list, the exact-list assertion, and the SQLite adoption arguments. Gatekeeper anchors re-pointed at the merged sources.0.63.1 — Unreleased; published sections are identical tomain.Live-corpus window defect (2026-09-21)
A user ran this branch's own live proof on a 547M-token corpus and got a correct scanner snapshot but an empty dashboard row. The shape my corpora never had: a window mixing priced days with days the scanner could not price at all — turns attributed to no model with no
costUsdTicks, which the scanner declares as unpriced requests. Two consumers treated such a day as making the window unknown:The regression fails on the reported assertions with either fix reverted. The live proof now prints the coverage, active-day, unpriced-day and unpriced-request figures per window and asserts the documented no-total contract for a bounded scan, so a failing run says which shape it hit.
The standalone xAI gate in the OpenCodex aggregator now follows the resolved billing route rather than the model namespace, so router-hosted
xai/...rows keep their recorded-provider price (#3676) while directly billed rows stay token-only without Grok OAuth provenance.Validation
Verified on exact head
75945a360f90abe5abac262c6c80d4cf65ec2326, onmainatd8d0f3394:make check: passed — SwiftFormat clean, SwiftLint 0 violations in 2530 files, provider/package/documentation gates green.make test: passed — 1299/1299 selections, 119/119 groups successful on the first pass, 0 failed groups, 0 retries, 0 timeouts (785.8 s).GeminiStatusProbe…GrokCostUsagePricing, 8 suites,--no-parallel) passes on this head;GrokAccountContextTests8/8.git diff --checkclean; 0 behindmain.#3805 (merged into
mainduring the previous run) added a fallback regression whose fixture expectedsignals.jsonoccupancy to count as tokens; it is re-expressed on a completed turn here, since this branch counts completed turns only. Earlier heads' validation is recorded in the PR comments; the historical native-corpus proof and the pinned OpenCodex producer capture are unchanged and remain supplemental.Maintainer decision
Please revisit the 2026-08-21 Grok cost ruling before merge. It preferred public-card pricing based partly on my incorrect
1e9divisor; #3345 established1e10, and the corrected measurement explains the difference. This branch uses recorded spend with public-card fallback.Whether existing Grok users should receive this disclosed dollar surface by default remains an owner decision. The accounting corrections and source labeling do not override that decision. Maintainer approval is still required.