Skip to content

fix: align detection parity across TS, Rust and Python - #7

Merged
danjdewhurst merged 2 commits into
mainfrom
fix/detection-parity-edge-cases
Sep 12, 2026
Merged

danjdewhurst merged 2 commits into
mainfrom
fix/detection-parity-edge-cases

Conversation

@danjdewhurst

@danjdewhurst danjdewhurst commented Sep 12, 2026 •

Copy link
Copy Markdown
Contributor

Aligns detection and CLI behaviour across the TypeScript, Rust and Python implementations, and addresses the review feedback on this PR.

Behaviour changes in this PR

  • Stdio hints are truly opt-in (TS). Removed the process.stdout.isTTY / process.stdin.isTTY fallbacks. Deno reports isTTY === false for piped stdio, so detectAgent() and the CLI were emitting stdio:*-not-tty hints unasked, contrary to SPEC and unlike Rust/Python.
  • Portable whitespace set. SPEC now defines whitespace as ASCII space, \t, \n, \v, \f, \r. All three implementations trim exactly that set for AI_AGENT, AGENTHINT_AGENT, heuristic env values, parent process names and init <name>. Native trims disagreed: JS trims U+FEFF, Rust/Python trim U+0085, Python also trims U+001C–U+001F.
  • Whitespace-only heuristic values ignored in all three implementations, including the cowork classifier.
  • Python .exe stripping is case-insensitive (Codex.EXE → codex).
  • install.sh truthy check is case-insensitive via tr.
  • Python doctor now prints the lowercase human "no agent" hint, matching TS/Rust.
  • TS --help no longer prints an extra trailing blank line.
  • Removed code comments that only restated the spec; stdio docs clarified.

Tie-break (earliest rule wins), sorted prefix signals, leading-only init and the cowork classifier signal already landed on main in 87fd407. This PR only locks them in with shared fixtures.

Tests

  • New shared cases in fixtures/detection-cases.json cover ASCII vs non-ASCII whitespace, the cowork classifier, case-insensitive truthy overrides, and .EXE parent names (fixtures now accept parentProcessName).
  • New shared cases in fixtures/cli-cases.json cover --help, doctor with no agent, and whitespace-only init names.
  • Every implementation runs these fixtures, so the per-language duplicate tests were removed.
  • New TS test: stdio hints are not read from process without explicit options.
  • Confirmed the new fixtures fail on the previous commit and pass now.
  • npm run check passes: 26 node, 15+3 rust, 7 python, plus fmt/clippy/biome. sh -n install.sh passes.

Review items not changed

  • LC_ALL=C for tr in install.sh: the reported failure did not reproduce with GNU tr 9.11 under C, UTF-8, ISO8859-1 or Turkish locales, and no truthy value contains I.
  • lowered as a global in install.sh: matches the rest of the script, where all helpers use globals; local is not POSIX.
  • Non-string env values throwing in TS: Python behaves the same, and both APIs are typed as strings.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TYKH18k7c9ZUzJcWB3vMfq

@coderabbitai

coderabbitai Bot commented Sep 12, 2026 •

Copy link
Copy Markdown

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 10588bb9-f9f2-46a9-8546-ea3f54dd729c

📥 Commits

Reviewing files that changed from the base of the PR and between 87fd407 and 41640b7.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (12)
  • SPEC.md
  • crates/agenthint/src/lib.rs
  • crates/agenthint/src/main.rs
  • crates/agenthint/tests/cli.rs
  • docs/signals.md
  • fixtures/detection-cases.json
  • install.sh
  • python/agenthint/__init__.py
  • src/index.ts
  • test/cli.test.mjs
  • test/detect.test.mjs
  • test/python_test.py

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

📜 Recent review details
🧰 Additional context used
📓 Path-based instructions (4)
Rust source lives in `crates/agenthint/src/`

📄 CodeRabbit inference engine (AGENTS.md)

Files:

  • crates/agenthint/src/main.rs
  • crates/agenthint/src/lib.rs
If editing `install.sh`, run `sh -n install.sh` and keep it POSIX `sh` compatible

📄 CodeRabbit inference engine (AGENTS.md)

Files:

  • install.sh
TypeScript source lives in `src/` Keep detection results explainable: include `confidence` and `signals` Prefer explicit `AI_AGENT` support over heuristics Never print environment variable values that may contain secrets; signal names are e...

📄 CodeRabbit inference engine (AGENTS.md)

Files:

  • src/index.ts
Prefix shell commands with `rtk` unless the command genuinely needs raw shell behavior

📄 CodeRabbit inference engine (AGENTS.md)

Files:

  • install.sh
🪛 ast-grep (0.45.3)
test/python_test.py

[info] 99-99: Do not hardcode temporary file or directory names
Context: "/tmp/codex"
Note: [CWE-377] Insecure Temporary File.

(hardcoded-tmp-file)

src/index.ts

[warning] Importing child_process exposes a command-execution surface; ensure any command/argument built from input is validated, and prefer execFile/spawn with an argument array over exec.
Context: import { execFileSync } from "node:child_process";
Note: [CWE-78] Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').

(detect-child-process-typescript)

🪛 Ruff (0.16.4)
test/python_test.py

[error] 100-100: Probable insecure usage of temporary file or directory: "/tmp/codex"

(S108)

🔇 Additional comments (12)
crates/agenthint/src/main.rs (1)

17-18: LGTM!

crates/agenthint/tests/cli.rs (1)

75-90: LGTM!

test/cli.test.mjs (1)

108-111: LGTM!

Also applies to: 119-120

SPEC.md (1)

42-45: LGTM!

Also applies to: 91-95

docs/signals.md (1)

9-11: LGTM!

Also applies to: 48-50, 78-79

fixtures/detection-cases.json (1)

56-56: LGTM!

test/python_test.py (1)

49-57: LGTM!

Also applies to: 59-66, 68-76, 78-86, 88-96, 98-103, 140-144

crates/agenthint/src/lib.rs (1)

81-91: LGTM!

Also applies to: 303-321, 435-438, 446-446, 455-456, 838-889

install.sh (1)

76-78: LGTM!

python/agenthint/__init__.py (1)

72-73: LGTM!

Also applies to: 174-178, 229-229, 233-233, 261-261

src/index.ts (1)

50-56: LGTM!

Also applies to: 241-243, 248-250, 296-298

test/detect.test.mjs (1)

221-231: LGTM!

Also applies to: 233-241, 243-252, 254-263, 265-274, 276-284


📝 Summary

Summary by CodeRabbit

  • Bug Fixes

    • Environment values containing only whitespace are now ignored.
    • Detection ties now prefer the earliest matching rule.
    • Detection signals are reported in a consistent sorted order.
    • Claude Code cowork mode now reports its associated environment signal.
    • Truthy override values are recognised regardless of letter casing.
    • Parent process names ending in .exe are handled case-insensitively.
  • CLI

    • The init command is valid only when it is the first argument; otherwise, usage is rejected.
  • Documentation

    • Clarified environment signal, override, sorting, and opt-in stdio hint behaviour.

Walkthrough

The change updates environment signal matching across Rust, TypeScript, Python, and shell code. It adds cowork signal reporting, whitespace filtering, deterministic ordering, case-insensitive overrides, and tie-breaking. It also rejects non-leading init CLI arguments and adds coverage.

Changes

Detection and CLI behaviour

Layer / File(s) Summary
Detection matching and normalisation
SPEC.md, crates/agenthint/src/*, install.sh, python/agenthint/*, src/index.ts
Detection now ignores whitespace-only values, reports the cowork classifier signal, sorts prefix signals, preserves the earliest rule on confidence ties, accepts case-insensitive truthy overrides, and normalises .EXE process names case-insensitively.
Detection documentation and validation
docs/signals.md, fixtures/detection-cases.json, test/detect.test.mjs, test/python_test.py, crates/agenthint/src/lib.rs
Documentation, fixtures, and tests cover the updated matching rules and returned signals across implementations.
Leading init command validation
crates/agenthint/src/main.rs, crates/agenthint/tests/cli.rs, test/cli.test.mjs, test/python_test.py
The CLI accepts init only as the first argument. Non-leading init arguments return exit code 2 with an invalid-usage message.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 41640

The detection and CLI behavior updates are covered by matching implementation changes, documentation, fixtures, and regression tests across supported languages.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 20.93% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 43 functions across 9 files. (3 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarises the main change: aligning detection behaviour across the TypeScript, Rust and Python implementations.
Description check ✅ Passed The description directly explains the cross-language behaviour changes, review findings, tests, fixtures, and validation results.
Full details: Docstring Coverage

Explanation

Docstring coverage is 20.93% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 43 functions across 9 files. (3 skipped: 3 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
⚔️ Resolve merge conflicts 💡
  • Resolve merge conflict in branch fix/detection-parity-edge-cases

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

F1: prefer earliest rule on confidence ties in Rust
F2: sort prefix signals in TS and Python to match Rust
F3: document stdio hints as opt-in library signals only
F4: fix Python .exe stripping to be case-insensitive
F5: make install.sh truthy check case-insensitive
F6: fix Rust CLI non-leading init error path
F7: ignore whitespace-only heuristic env values
F8: include cowork classifier signal alongside trigger
@danjdewhurst
danjdewhurst force-pushed the fix/detection-parity-edge-cases branch from 41640b7 to df9618d Compare September 12, 2026 20:37
@danjdewhurst

Copy link
Copy Markdown
Contributor Author

Code review

🤖 This review was written by Claude Opus 5 (claude-opus-5) in Claude Code.

Build, lint and all three test suites pass on this branch: TS (31 tests), Rust (20 unit + 3 CLI, plus fmt and clippy) and Python (13 tests).

Bug

src/index.ts:256: the stdio TTY hints can still turn on without being asked for (medium)

This PR adds a SPEC rule: the stdio:stdout-not-tty and stdio:stdin-not-tty hints need an explicit stdoutIsTTY: false or stdinIsTTY: false option, and the CLI must never report piped output as a signal. But ttyHints still falls back to process.stdout.isTTY / process.stdin.isTTY. The new comment assumes those are undefined when stdio is piped. That's true on Node and Bun, but Deno returns false, so the hints turn on by themselves.

Reproduction (clean env):

deno run -A dist/cli.js --json </dev/null | cat
# => confidence: 0.2, signals: ["stdio:stdout-not-tty", "stdio:stdin-not-tty"]

Calling detectAgent() with no options under Deno gives the same result. Rust and Python only emit these hints when the caller passes the option, so TS now differs from the other two.

Suggested fix: drop the ?? process.stdout.isTTY / ?? process.stdin.isTTY fallbacks so the hints are truly opt-in.

Minor (not bugs)

  • Redundant tests: test/detect.test.mjs and test/python_test.py each add a "sorts prefix signals" test. In crates/agenthint/src/lib.rs, the new prefers_earliest_rule_on_confidence_ties repeats the existing equal_confidence_ties_prefer_the_earlier_rule, and sits next to the existing prefix_signals_are_sorted. Harmless, but could be removed.
  • Whitespace handling differs between languages: a value made only of  is ignored by TS (trim()) but counts as a signal in Rust and Python. Very unlikely to matter; noting it only for parity.

@danjdewhurst

Copy link
Copy Markdown
Contributor Author

Review — PR #7 fix: align detection parity across TS, Rust and Python

Note: I am Muse Spark (muse-spark-1.3-contributor), reviewed via OpenCode. This is an independent review, not from CodeRabbit.

Verdict: approve with minor nits. Merge risk minimal. mergeStateStatus: CLEAN, mergeable: MERGEABLE. I ran npm run check on df9618d — 31 node, 20+3 rust, 13 python tests all pass; sh -n install.sh passes.

What this PR actually changes

The description lists F1–F8, but most of F1/F2/F6/F8 already landed in parent 87fd407. This diff (main...pr-7, 10 files, +235/−15) contains 3 real behavior fixes + docs/comments + regression tests:

  • F7 real ✅ — whitespace-only heuristic values ignored in all three impls:
    • src/index.ts:235-246 (.trim() !== "")
    • crates/agenthint/src/lib.rs:435-455 (!value.trim().is_empty())
    • python/agenthint/__init__.py:231-236 (and value.strip())
    • AI_AGENT whitespace was already handled, so scoping to heuristics matches SPEC.md:40-42. Correct.
  • F4 real ✅ — python/agenthint/__init__.py:261-264: .name.lower().removesuffix(".exe") fixes Codex.EXE miss. TS (src/index.ts:223-225 /\.exe$/i) and Rust (lib.rs lower-then-strip) were already case-insensitive. requires-python >=3.10 so removesuffix is safe.
  • F5 real ✅ — install.sh:76-82: tr '[:upper:]' '[:lower:]' then match 1|true|yes|on now covers True/TrUe/YES/On etc., matching TS/Rust/Python toLowerCase()/lower(). POSIX sh compatible, sh -n passes.
  • F3 docs ✅ — SPEC.md:42,46,52-54,104, docs/signals.md:9-11,48,52,82-83 + opt-in stdio comments in all three impls. Accurate, no secret values printed (signal names only).
  • F1/F2/F6/F8 tests only — tie-break-first-wins, prefix .sort()/sorted(), leading-only init, cowork env:CLAUDE_CODE_IS_COWORK were already correct in main (cli.ts:13, main.rs:14, cli.py:22, isTruthy case-insensitive everywhere). New tests lock behavior, no impl change needed. Good, but description overclaims “now prefers/sorted/included” for this commit.

CLI contract preserved: foo init bar → exit 2 invalid usage: foo init bar in all three CLIs (tests in test/cli.test.mjs:108-120, crates/agenthint/tests/cli.rs, test/python_test.py:140-144). Exit codes 0/1/2 intact.

Nits (non-blocking)

  1. Redundant tests: prefers_earliest_rule_on_confidence_ties duplicates equal_confidence_ties_prefer_the_earlier_rule (lib.rs:774-779 vs :853-859); same for prefix_signals_are_sorted vs new sort test, detects_cowork_only_with_claude vs includes_cowork_classifier_signal, and equivalents in test/detect.test.mjs:221-284 / test/python_test.py:49-103. Consider merging to keep suite lean.
  2. No shared-fixture coverage for new edges: whitespace/Codex.EXE/mixed-case truthy only have per-lang unit tests. Adding cases to fixtures/detection-cases.json would enforce parity via the existing matches_shared_*_fixtures tests in all impls.
  3. python/agenthint/__init__.py:231-232 double lookup: env.get(name) and env.get(name).strip() calls get twice. v = env.get(name); return [... if v and v.strip()] is clearer. Same pattern in :236.
  4. install.sh:77 global leak: lowered=... without local pollutes global scope under set -eu. Fine for POSIX sh, but unset lowered after or case "$(printf ... | tr ...)" in avoids it. Pre-existing set -u + zero-arg is_truthy failure also remains, but out of scope.
  5. TS assumes string env values (src/index.ts:237,243 .trim()): safe for NodeJS.ProcessEnv, but a plain-object caller passing a non-string would throw where Python/Rust coerce/filter safely. Low risk.

No security concerns: no env values echoed, execFileSync("ps", [...]) uses arg array, /proc→ps fallback unchanged.

@danjdewhurst

Copy link
Copy Markdown
Contributor Author

Review — PR #7 fix: align detection parity across TS, Rust and Python

🤖 This review was written by DeepSeek V4.1 Flash (opencode-go/deepseek-v4.1-flash) via OpenCode.

mise exec -- npm run check passes on df9618d (31 Node, 20+3 Rust, 13 Python) and sh -n install.sh passes. I also ran an independent parity harness against the built CLIs: 56 env cases diffing isAgent/agent/confidence/signals, plus 23 argument cases diffing exit code, stdout and stderr. The changes in this diff behave as the tests claim; the gaps below are what the harness surfaced.

1. "Whitespace-only" still means different things per language (in scope for F7)

SPEC.md:42 now says whitespace-only heuristic values are ignored, but String.prototype.trim() follows ECMA-262 while Rust trim() / Python strip() follow Unicode White_Space, and the sets differ. Probing AI_AGENT=<single char> and diffing the three CLIs:

code point TS Rust Python
U+FEFF (BOM) ignored detected: agent=U+FEFF, 0.98 detected: same
U+0085 (NEL) detected: agent=U+0085, 0.98 ignored ignored
all others tested* agree agree agree

* TAB, VT, FF, SPACE, NBSP, OGHAM SPACE, EN QUAD, FIG SPACE, LS, PS, NNBSP, MMSP, IDEO all ignored by all three. ZWSP (U+200B) and U+180E are treated as normal characters by all three, so the earlier note about ZWSP does not reproduce — the JS-only whitespace is U+FEFF, and U+0085 diverges in the opposite direction.

Heuristic example: CODEX_HOME=\uFEFF → TS reports no agent; Rust/Python report codex at 0.92. Cowork example: CLAUDE_CODE=1 + CLAUDE_CODE_IS_COWORK=\uFEFF → TS reports claude-code with one signal; Rust/Python report cowork with the classifier signal. This is exactly the kind of edge F7 is trying to pin down, so either fix the set in SPEC (for example ASCII \t\n\v\f\r) and implement it explicitly, or add shared fixtures so the difference is deliberate.

2. Python doctor capitalizes the "no agent" hint (minor, pre-existing)

AGENTHINT_DISABLE=1 python3 -m agenthint.cli doctor | grep hint
hint: Agents should set AI_AGENT=<agent-name> before invoking tools.

AGENTHINT_DISABLE=1 node dist/cli.js doctor | grep hint
hint: agents should set AI_AGENT=<agent-name> before invoking tools.

python/agenthint/__init__.py:114 reuses the JSON hint from :284 verbatim; src/doctor.ts:40 and format_doctor in crates/agenthint/src/lib.rs hardcode the lowercase human variant. doctor --json agrees across all three (capital A), so only the human output differs. fixtures/cli-cases.json has no "doctor, no agent" case, which is why nothing catches it.

3. Nit: --help output differs by one trailing blank line (pre-existing)

TS emits ...Show this help\n\n because printHelp console.logs a template literal that already ends in a newline; Rust and Python emit a single \n.

Confirming prior review notes

  • Deno TTY fallback: independently reproduced on deno 2.9.6 with env -i PATH="$PATH" deno run -A dist/cli.js --json → confidence: 0.2, signals: ["stdio:stdout-not-tty","stdio:stdin-not-tty"]. Node and Bun emit no signals. src/index.ts:256 still falls back to process.stdout.isTTY, so the new SPEC.md:104 sentence and docs/signals.md:82-83 overstate current TS behavior. Dropping the ?? process.stdout.isTTY / ?? process.stdin.isTTY fallbacks would make the docs true and align TS with Rust/Python.
  • Test duplication: the new ties, sorted-prefix and cowork tests restate fixture cases already run by all three suites (fixtures/detection-cases.json:51,67,83). The genuinely new edges (whitespace-only, .EXE, mixed-case truthy) have no shared fixture and are only covered per language. Adding those three to detection-cases.json would enforce parity rather than document it, and the duplicate unit tests could then go.

Verified correct

  • F4: name.lower().removesuffix(".exe") matches TS \.exe$/i and Rust lower-then-strip for ASCII names.
  • F5: tr '[:upper:]' '[:lower:]' is POSIX and matches the library lower() behavior.
  • F6: foo init bar → exit 2, invalid usage: foo init bar in all three CLIs (verified).
  • F8: cowork promotion and the classifier signal are identical in all three, including CLAUDECODE as the triggering signal.
  • F7 for the common ASCII cases: all three treat empty and \t\n\r values as absent.
  • Exit codes 0/1/2 preserved.

No blocking bugs in the diff itself. The SPEC/TS mismatch from the Deno fallback is the only item I would want resolved or explicitly accepted before merge.

- TS: drop the process.stdout/stdin.isTTY fallbacks so stdio hints are
  truly opt-in (Deno reports isTTY=false for piped stdio).
- Define a portable ASCII whitespace set in SPEC and trim with it in all
  three implementations; native trims disagreed on U+FEFF, U+0085 and
  U+001C-U+001F.
- Python doctor: lowercase the human "no agent" hint to match TS/Rust.
- TS --help: remove the extra trailing blank line.
- Move whitespace, cowork, truthy and .EXE edge cases into shared
  fixtures (with parentProcessName support) and drop duplicate unit tests.
- Add CLI fixtures for --help, doctor with no agent and whitespace-only
  init names.
- Remove comments that restated the spec.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TYKH18k7c9ZUzJcWB3vMfq
@danjdewhurst
danjdewhurst merged commit 6c61594 into main Sep 12, 2026
2 checks passed
@danjdewhurst
danjdewhurst deleted the fix/detection-parity-edge-cases branch September 12, 2026 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant