Skip to content

feat(scan): add unified max-time investigation budgets - #1674

Open
mldangelo-oai wants to merge 5 commits into
mainfrom
mdangelo/codex/standard-time-budget-eval
Open

mldangelo-oai wants to merge 5 commits into
mainfrom
mdangelo/codex/standard-time-budget-eval

Conversation

@mldangelo-oai

@mldangelo-oai mldangelo-oai commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Add one minutes-based investigation setting across the CLI, SDK, and plugin without changing default scan behavior. This is the runtime foundation; the production-path comparison is isolated in #1679.

Changes

  • Add scan --max-time <minutes>, SDK maxTimeMinutes, and project scan.max_time_minutes. Standard remains unset by default; Deep retains its existing 5760-minute default and maximum.
  • Standard uses an advisory allowance starting after setup. One rendered policy and absolute deadline follow source-review workers and reconnects; separate custom validation, post-scan work, and finalization are outside it. Direct plugin use requires an explicit user request.
  • Preserve Deep's existing enforced discovery ceiling and hour-based storage; display its time-limit advice in minutes. Deprecate, but retain, the existing hour aliases; apply source precedence before checking conflicting aliases. Reruns start a new allowance; Deep resume keeps its original deadline.
  • Validate effective settings at execution boundaries, including actual bulk row modes. Keep behavioral coverage for timing, cancellation, cost limits, old saved recipes, precedence, and separate validation without duplicating prompt snapshots.

Testing

  • Runtime-only plugin/SDK builds and types pass; 429 focused SDK, CLI, and configuration tests pass after the rebase.
  • The CI follow-up uses numeric placeholders for invalid-minute test cases so Bun emits distinct test identities. All 29 Deep configuration tests pass, and the CI inventory collector accepts every identity. The existing summary regression also covers fractional-minute diagnostics without floating-point noise.
  • All required portable plugin checks pass: Ruff lint/format, build:ci, source compatibility, and its nine tests. Prettier, Markdown checks, and git diff --check pass.
  • SDK tests verify minute-to-hour resolution and serialized configuration. The plugin subprocess harness verifies two distinct canonical budgets through 12 fresh/resumed discovery/reducer launches, plus persisted budget and original start time when reopening a scan with changed configuration. These tests retain the plugin's standalone dependency boundary; plugin typecheck and targeted formatting pass.
  • Three fresh native reviews and an independent verifier report no findings. Required CI remains the merge gate; no live-model effectiveness claim is made.

Risk and rollout

Opt-in only for Standard: no new hard timeout, minimum duration, automatic continuation, or spending guarantee. Deep's coordinator and existing stopping conditions are unchanged. Invalid or unrepresentable deadlines fail before scan setup. Existing cost controls, cancellation, credential protections, permissions, and authorized scope remain intact.

The new setting does not establish improved detection or cost efficiency. No live-model effectiveness claim, run artifacts, or scan findings are included.

Public disclosure review

  • No customer, partner, prospect, or user identities, data, or identifying details are included.
  • No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
  • I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

@github-actions github-actions Bot added the skip-release-notes Omit internal changes from generated release notes label Oct 16, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-17T03:26:29.287031Z a36b4c2 New commits
🔒 Security Review ✅ Completed 2026-10-17T03:26:55.855912Z a36b4c2 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1fb684c535

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread evals/standard-time-budget/experiment.mts Outdated
Comment thread evals/standard-time-budget/grade.mts Outdated
@mldangelo-oai mldangelo-oai changed the title test(evals): prototype time-aware Standard scan prompts feat(scan): add unified max-time investigation budgets Oct 16, 2026
@mldangelo-oai
mldangelo-oai force-pushed the mdangelo/codex/standard-time-budget-eval branch from 1fb684c to f8d6983 Compare October 16, 2026 19:09
@github-actions github-actions Bot added enhancement New feature or request and removed skip-release-notes Omit internal changes from generated release notes labels Oct 16, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f8d6983364

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/src/cli.ts Outdated
@mldangelo-oai
mldangelo-oai force-pushed the mdangelo/codex/standard-time-budget-eval branch from f8d6983 to d176dcd Compare October 16, 2026 20:05

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2b877d5efe

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/src/deep-config.ts
@mldangelo-oai
mldangelo-oai force-pushed the mdangelo/codex/standard-time-budget-eval branch from 2b877d5 to 12abd7d Compare October 17, 2026 02:43

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 12abd7dc57

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/src/cli.ts Outdated

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants