Skip to content

fix(ci): avoid NumPy source builds and duplicate HTTPX coverage - #3573

Merged
apcha-oai merged 2 commits into
mainfrom
apcha/sdk-221-python-ci-performance
Aug 3, 2026
Merged

apcha-oai merged 2 commits into
mainfrom
apcha/sdk-221-python-ci-performance

Conversation

@apcha-oai

@apcha-oai apcha-oai commented Aug 3, 2026 •

Copy link
Copy Markdown
Contributor

Root cause

  • Rye caching was disabled, and the lockfiles pinned a NumPy version without Python 3.14 wheels, forcing slow source builds.
  • The HTTPX 2 job reran the ordinary HTTPX suite already covered by the main test matrix.
  • Separately, uv.lock lacked the Release Please marker needed to keep its SDK version current.

Fix

  • Restore Python-specific Rye environment caching and select prebuilt NumPy wheels for each supported interpreter.
  • Run only the HTTPX 2 suite in the HTTPX 2 job.
  • Add the missing Release Please annotation and register uv.lock for release updates.

Results

  • HTTPX 2: 10m42s → 2m12s
  • Python 3.14: 8m37s → 1m53s

Validated with lint, SDK builds, lockfile checks, and both HTTPX/HTTPX 2 suites under Pydantic 1 and 2.

Fixes SDK-221.

@apcha-oai
apcha-oai marked this pull request as ready for review August 3, 2026 21:04
@apcha-oai
apcha-oai requested a review from a team as a code owner August 3, 2026 21:04
@openai-sdks

openai-sdks Bot commented Aug 3, 2026 •

Copy link
Copy Markdown
Contributor

OkTest Summary

✅ 237/237 SDK tests passed in 12.669s for Python SDK PR #3573.

Test results — 42 files
Test Result Time
tests/chat-completions-complex-body.test.ts ✅ Passed 192ms
tests/chat-completions-create.test.ts ✅ Passed 270ms
tests/chat-completions-stream.test.ts ✅ Passed 150ms
tests/files-content-binary.test.ts ✅ Passed 269ms
tests/files-create-multipart.test.ts ✅ Passed 254ms
tests/files-list-pagination.test.ts ✅ Passed 197ms
tests/initialize-config.test.ts ✅ Passed 157ms
tests/instance-isolation.test.ts ✅ Passed 291ms
tests/models-list.test.ts ✅ Passed 250ms
tests/responses-background-lifecycle.test.ts ✅ Passed 283ms
tests/responses-body-method-errors.test.ts ✅ Passed 458ms
tests/responses-cancel-timeout.test.ts ✅ Passed 233ms
tests/responses-cancel.test.ts ✅ Passed 300ms
tests/responses-compact-retries.test.ts ✅ Passed 366ms
tests/responses-compact.test.ts ✅ Passed 355ms
tests/responses-create-advanced-stream.test.ts ✅ Passed 142ms
tests/responses-create-advanced.test.ts ✅ Passed 225ms
tests/responses-create-disconnect.test.ts ✅ Passed 1.275s
tests/responses-create-errors.test.ts ✅ Passed 227ms
tests/responses-create-malformed-api-responses.test.ts ✅ Passed 269ms
tests/responses-create-retries.test.ts ✅ Passed 401ms
tests/responses-create-stream-failures.test.ts ✅ Passed 150ms
tests/responses-create-stream-timeout.test.ts ✅ Passed 226ms
tests/responses-create-stream-wire.test.ts ✅ Passed 4.011s
tests/responses-create-stream.test.ts ✅ Passed 316ms
tests/responses-create-terminal-states.test.ts ✅ Passed 301ms
tests/responses-create-timeout.test.ts ✅ Passed 222ms
tests/responses-create.test.ts ✅ Passed 296ms
tests/responses-delete.test.ts ✅ Passed 261ms
tests/responses-input-items-errors.test.ts ✅ Passed 376ms
tests/responses-input-items-list.test.ts ✅ Passed 388ms
tests/responses-input-items-options.test.ts ✅ Passed 375ms
tests/responses-input-tokens-count-timeout.test.ts ✅ Passed 286ms
tests/responses-input-tokens-count.test.ts ✅ Passed 265ms
tests/responses-malformed-inputs.test.ts ✅ Passed 3.136s
tests/responses-not-found-errors.test.ts ✅ Passed 325ms
tests/responses-parse.test.ts ✅ Passed 454ms
tests/responses-retrieve-retries.test.ts ✅ Passed 430ms
tests/responses-retrieve.test.ts ✅ Passed 323ms
tests/responses-stored-method-errors.test.ts ✅ Passed 832ms
tests/retry-behavior.test.ts ✅ Passed 3.314s
tests/sdk-error-shape.test.ts ✅ Passed 421ms

View OkTest run #30853078980

SDK merge (03da934387f6) · head (73f5e4a7661f) · base (dd1202d5dacf) · OkTest (91635c6a2723)

@jbeckwith-oai jbeckwith-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the exact head (73f5e4a). The per-interpreter universal NumPy locks, explicit Rye virtualenv cache keys, HTTPX2-only test path, and Release Please uv.lock wiring are consistent and keep the workflow direct. I also regenerated both Rye lockfiles, ran the Python support-policy check and full lint, and verified the exact-head CI matrix is green.

@apcha-oai
apcha-oai merged commit b58332f into main Aug 3, 2026
31 checks passed
@apcha-oai
apcha-oai deleted the apcha/sdk-221-python-ci-performance branch August 3, 2026 21:11
@github-actions github-actions Bot mentioned this pull request Aug 3, 2026

@HAYDEN-OAI HAYDEN-OAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head 73f5e4a7661f2d4ee448af22e3f0c0336435d0f8. No substantive issues found: interpreter-specific Rye cache keys and invalidation, universal NumPy wheel selections, HTTPX/HTTPX2 and Pydantic v1/v2 coverage, required-check names, fork/release behavior, workflow permissions, and CodeQL coverage are consistent. Confirmed HTTPX2 improved from 10m42s to 2m12s and Python 3.14 from 8m37s to 1m53s. Release automation also propagated the version into uv.lock in #3575.

pull Bot pushed a commit to gitupdates/openai-python that referenced this pull request Aug 3, 2026
Automated Release PR
---


##
[2.53.0](openai/openai-python@v2.52.1...v2.53.0)
(2026-08-03)


### Features

* **api:** Add gpt-5.5 and tool name/namespace to Responses types
([openai#3569](openai#3569))
([dd1202d](openai@dd1202d))


### Bug Fixes

* **ci:** avoid NumPy source builds and duplicate HTTPX coverage
([openai#3573](openai#3573))
([b58332f](openai@b58332f))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Justin Beckwith <jbeckwith@openai.com>
yonasb added a commit to starslingdev/skills that referenced this pull request Oct 21, 2026
…#137)

* ci-speedup: four catalog patterns for slow test jobs (OPT85 to OPT88)

Adds four catalog entries, each backed by measured changes OpenAI
engineers made in their public repos (every figure re-read from the PR
body or post it quotes):

- OPT85 Tests Run in Many Processes Instead of One (structural, risk
  MEDIUM): a subprocess per test case, or a test binary per file.
  openai/openai-python#3830, openai/codex#2633. A new pattern, not a
  widening of OPT78: OPT78 is vitest's per-file module registry and its
  scope note keeps it vitest-only; OPT78 gains a one-line pointer.
- OPT86 CI Compiles a Dependency That Ships Prebuilt (data-driven, risk
  LOW). openai/openai-python#3573 (saving not split between its changes,
  said so), openai/whisper CPU-only torch index, openai-python
  UV_NO_BUILD.
- OPT87 Tests Wait on a Slow Dependency They Do Not Verify (structural,
  risk HIGH): stub at the boundary, never mock what the test verifies,
  keep a real-dependency run. OpenAI Core Experimentation post,
  openai/evals feed6c75.
- OPT88 Test Concurrency Is a Guess, Not a Measurement (data-driven,
  risk MEDIUM). openai/codex#31352, openai/openai-agents-python#2563
  (local measurement, said so).

No detector code: all four are catalog-only, like OPT25 and OPT74. The
structural pair lands in catalog_structural_patterns_without_detector;
the data-driven pair, like every detector: manual entry, is never
emitted. Counts 82 -> 86 (75 hygiene + 11 structural) in SKILL.md,
ARCHITECTURE.md, evals.json; README 70+ -> 80+. SKILL.md line count
unchanged (515).

Red-first exemption: no behavior to regress. The change is catalog and
doc text plus one scan.py comment; no detector, renderer or verifier
code changed. The existing count, breakdown and anchor guards cover the
new totals (scoped run: 257 passed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci-speedup: OPT85 to OPT88 say they are applied by hand, and their fix advice no longer breaks installs

Review fixes for the four new catalog entries.

- SKILL.md, ARCHITECTURE.md, structural-track.md, MAINTAINERS.md and the
  scan.py return-key comment now say OPT74, OPT85 and OPT87 have no router
  and are never emitted, instead of listing them as routed.
- Each new entry says no audit phase applies it; a reader applies it by hand.
- OPT86: --only-binary / --no-build-package scoped to the package that
  compiled, with the wide form only once every dependency ships a wheel;
  the PyTorch CPU index goes in its own install command; detection and the
  sizing ceiling cover the oversized-download case; UV_NO_BUILD is set
  "at workflow level", matching the source.
- OPT85: the test-list guardrail maps renamed Rust tests; random order is
  required where the runner can shuffle, with a Rust alternative.
- OPT87: recorded stub data must have credentials scrubbed; blog quote
  apostrophe now matches the source.
- codex #2633 is described as a developer-machine time measurement.

Text corrections are exempt from red-first (no behavior to regress). The
two strengthened guards in test_scan_detectors.py were shown failing on a
mutated copy (OPT85 added to structural_detected; SKILL.md split 76/10):
  AssertionError: OPT85 has no router and must be reported as uncovered
  AssertionError: SKILL.md claims 10 structural patterns but the catalog has 11 `class: structural` entries
and pass on this tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci-speedup: OPT85 to OPT88 detection steps match what real before and after CI logs show

Wording fixes from checking each entry's detection steps against OpenAI's
real before/after runs.

- OPT85: the test-category step is a gate only; a helper the parametrized
  test calls counts as inside the test; cases are counted from the
  collect-only listing; when the slow run has no slowest-tests report,
  corroborate against a known-good run and by timing one case with and
  without the process call.
- OPT86: the compile span runs to the closing `Built <pkg>==<version>`
  line; a `Building <project> @ file://` line is the project itself; only
  the compile span counts toward the ceiling on a leg slow for other reasons.
- OPT87: the post names no repository and links no CI runs; the 2023 evals
  run history is no longer retrievable.
- OPT88: concurrency can be set in the tool's own config files (such as
  `local_test_jobs` in `.bazelrc`); measure the core count of an
  undocumented custom runner; say so when nothing forced the setting.

Text-only corrections, exempt from red-first (no behavior to regress).
Headings and anchors unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

This branch was previously deployed

1 inactive deployment
ci — 73f5e4a7 Deployed Aug 3, 2026 by apcha-oai via examples #5345
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants