Skip to content

Fix Unicode CLI crash, Codex round counting and control timing - #331

Merged
dipeshbabu merged 1 commit into
mainfrom
codex/runner-rounds-and-control-time
Nov 17, 2026
Merged

dipeshbabu merged 1 commit into
mainfrom
codex/runner-rounds-and-control-time

Conversation

@dipeshbabu

Copy link
Copy Markdown
Owner

Bugs found by end-to-end testing

  1. The CLI crashed on Unicode when output was redirected on Windows. Piped or redirected output used cp1252, so a task ID such as 检查-✓ raised UnicodeEncodeError partway through improve capture. The command's real result was replaced by a traceback. A root Typer callback now reconfigures redirected stdout and stderr to UTF-8 (errors="replace"). Interactive consoles are already UTF-8 and are left unchanged. The new regression test forces PYTHONIOENCODING=cp1252, so it also runs on Linux CI. It fails without the fix and passes with it.
  2. The Codex runner over-counted model calls. A message such as "Checking…" followed by a tool call in the same model response became two model spans. That inflated the step counts repetition_detour reports ("2 more model steps" when it was 1). The runner now records one span per model round: from the turn start, or the end of the last running tool, until the next tool starts or the final message. Items within a round are listed in codex_items.
  3. Interleaved controls skipped the freeze-time check. finish already required the candidate capture to come after the frozen session, but not the control. It now checks both.
  4. Aggregate report headlines were misleading. Markdown and HTML said "N of N planned attempts passed" even when the aggregate decision failed. They now say how many pairs passed correctness and completion checks, matching the CLI.

End-to-end Codex validation (24 interleaved runs plus a 12-run baseline)

Exercised the aggregate decision, the interleaved control, repetition_detour and the fixed runner together, with 4 repetitions per task:

  • Baseline: 12/12 correct. One detour flagged (+20,035 tokens). Another extra round that added only about 480 tokens was correctly not flagged.
  • Candidate ("request all reads in one step"): 24/24 correct and no detours. But it was slower in 12/12 pairs (median −34.5%), so the aggregate decision rejected it. Its first round spent longer planning (2.42 vs 2.08 rounds on average). Documented in docs/CODEX.md.

Other checks

  • Full suite: 2,789 passed and 109 skipped. One Streamlit dashboard test hit its 60 s AppTest timeout while Codex was running in parallel; it passes alone in 4.6 s.
  • New and updated tests (runner rounds, stale control, Unicode on a legacy code page): test_codex_exec_runner.py 10 passed, test_improvement.py passing.
  • CLI scenarios run by hand: aggregate pass, wrong candidate, invalid --decision, failing control runner.
  • TypeScript SDK: 25/25 tests, type checks and npm audit are clean.
  • mkdocs build --strict and pre-commit are clean.

🤖 Generated with Claude Code

- Write UTF-8 when CLI output is redirected on a legacy code page, so
  Unicode task IDs no longer raise UnicodeEncodeError mid-command.
- Reject an interleaved control captured before the session was frozen.
- Count pairs that passed correctness checks in aggregate report headlines.
- Record one Codex model span per model round instead of splitting a
  message and its tool call into two model calls.
- Document a second Codex run that exercised the aggregate decision,
  interleaved control and repetition_detour finding together.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 17, 2026

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@github-actions

Copy link
Copy Markdown

AgentLoop replay self-test

This required repository check evaluates generated synthetic demo traces. It tests the replay and CI machinery; it does not show that this PR improves application performance.

AgentLoop CI Report

  • Status: passed
  • Replay: Replay passed: latency improvement 69.67%, cost improvement 78.82%, retry delta -1.
  • Merge recommendation: synthetic self-test only; assess application traces before merge
  • Findings: 1 total, 0 high severity, 1 patchable

Performance Gates

Gate Status Detail
latency_regression pass 0.00% regression <= 0.00% allowed
cost_regression pass 0.00% regression <= 0.00% allowed
latency_improvement pass 69.67% improvement >= 20.00% required
cost_improvement pass 78.82% improvement >= 5.00% required
retry_non_increase pass 0 candidate retries <= 1 baseline retries

Trace inputs

This comparison includes synthetic data. It does not establish application performance gains from a pull request.

Side Run ID Path Synthetic marker Source
baseline run_a2a9514434cf4cbe runs/research_agent_baseline.json yes agentloop_demo
candidate run_d70444563a314e65 runs/research_agent_optimized.json yes agentloop_demo

Metric Impact

Metric Impact
Latency improvement 69.67%
Cost improvement 78.82%
Retry delta -1
Candidate schema validity not provided
Candidate quality score not provided

Top Findings

Finding Severity Type Rewrite
al_cache_context_7ecd0b97bf28: Cache repeated prompt/context prefix medium cache_context Move stable instructions into cached prefixes, summaries, or framework-level memory instead of resending full text.

@dipeshbabu
dipeshbabu merged commit f1e7f66 into main Nov 17, 2026
25 checks passed
@dipeshbabu
dipeshbabu deleted the codex/runner-rounds-and-control-time branch November 17, 2026 07:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant