Skip to content

build(deps): set runtime floors to the oldest tested versions so pip stops resolving 0.15.0 - #1092

Open
pengfei-threemoonslab wants to merge 3 commits into
mainfrom
claude/issue-1028-runtime-floors
Open

pengfei-threemoonslab wants to merge 3 commits into
mainfrom
claude/issue-1028-runtime-floors

Conversation

@pengfei-threemoonslab

@pengfei-threemoonslab pengfei-threemoonslab commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Closes #1028. Rank 9 of the engineering-review order in #1060.

The runtime floors had tracked the newest upstream release, so pip install agents-shipgate beside an agent project's ordinary pins quietly installed 0.15.0 (cryptography==46.0.7, packaging==24.2) or failed or fell back further (CrewAI 1.15, pydantic<2.13). Reproduced before the change: with cryptography==46.0.7, pip resolves agents-shipgate 0.15.0; with crewai==1.15.27, it resolves 0.11.0 after five minutes of backtracking.

  • Floors (pyproject.toml): cryptography>=44.0.3, packaging>=24.2, pydantic>=2.12.5,<3, pyyaml>=6.0.2,<7, ruamel.yaml>=0.18.10,<1, typer>=0.20.0,<1. The cryptography<51 and packaging<27 caps are dropped. Each floor was confirmed by the full suite at that exact combination, not only by the CI subset (see Verification).
    • typer is 0.20.0, not the issue's 0.16.0. The root command loader passes app.suggest_commands, which Typer added in 0.20.0; on 0.16.0–0.19.x agents-shipgate --help exits 1 with AttributeError: 'Typer' object has no attribute 'suggest_commands', and every grouped subcommand fails the same way. The verifier's subset did not reach it.
  • scripts/dependency_floors.py (new): coinstall runs one pip install --dry-run --report per scenario (cryptography==46.0.7, packaging==24.2, crewai==1.15.27) against a metadata-only wheel of this tree and fails unless pip keeps both this version and the pin; compile runs uv pip compile --resolution lowest-direct --generate-hashes over the runtime requirements plus the test tools at their constraints/dev.txt versions, and fails unless every runtime dependency lands exactly on its floor (naming whatever raised it); installed re-checks the interpreter.
  • CI dependency-floors job (new, runs on every PR): installs the pinned uv and the scripts' own packaging in a separate hash-checked venv, runs coinstall, compiles the lowest-direct closure, installs it --require-hashes and pip checks it, runs the dependency-heavy tests and regenerate_goldens.py --check there, and runs ./shipgate doctor to confirm no dependency is below its floor.
  • Dependabot: versioning-strategy: increase-if-necessary on the pip block. See "Deviation" below for why not the literal lockfile-only.
  • doctor --json: two new mismatches[] codes from the existing _mismatches engine (no new block, no field changes): dependency_below_declared_floor (error, with a reinstall command) and, for a source checkout, dependency_differs_from_lock (warning). Declarations come from the running build (the checkout's pyproject.toml, or the installed wheel's Requires-Dist); versions from importlib.metadata, read at call time so tests patch that module. The module stays standard-library only (the launcher imports it to explain import failures), so the version comparison is a conservative PEP 440 release-segment comparison that never reports what it cannot read.
  • CONTRIBUTING § Changing dependencies now says when a floor may move (the code needs a newer API, or a version at/above it is known to break Agents Shipgate, including a reachable vulnerability) and that a Dependabot PR or lock refresh is never a reason; the old "raise any floor it changes" instruction is gone.
  • Docs: docs/diagnostics.md, AGENTS.md (+ regenerated llms-full.txt), CHANGELOG ### Packaging.

Deviation from the acceptance wording: increase-if-necessary, not lockfile-only

The issue offers either; the AC says "a lockfile-only strategy". I read dependabot-core's pip updater before choosing (python/lib/dependabot/python/update_checker.rb, update_checker/requirements_updater.rb, updater/.../create_security_update_pull_request.rb):

  • With lockfile-only, requirements_unlocked_or_can_be? is false, so the update runs with no requirement unlocked. Dependabot reads constraints/*.txt as requirement files (the :requirements resolver; they have no pip-compile header), whose == pins are the requirement. Nothing can move: weekly lock PRs stop, and a security update ends as update_not_possible.
  • With increase-if-necessary, a pyproject.toml range that already admits the new release is left unchanged (update_pyproject_version_if_needed), while a == pin in constraints/ is rewritten. That is "updates move locks, not floors" — the intent of the AC.
  • Unset (today), the strategy was library? ? widen : increase, where library? compares PyPI's summary for agents-shipgate with pyproject.toml's description. Any description edit between releases switched Dependabot to raising floors, which is how Build(deps-dev): Update mcp requirement from <3,>=2.1.1 to >=2.2.0,<3 #705 moved the mcp floor.

tests/test_dependency_floors.py::test_dependabot_moves_the_locks_and_never_a_floor pins the setting. Side effect worth knowing: the harness's claude-agent-sdk>=0.2.136,<0.3 floor-bump PRs (always closed: #1012, #982, #957…) stop; a release outside <0.3 still opens one.

Type

  • CLI or GitHub Action behavior (doctor --json gains two mismatches[].code values)
  • Check or risk-model change
  • Input adapter change
  • Report, schema, or SARIF output
  • Documentation only

Protected paths

This touches protected paths and needs owner review before merge:

Acceptance criteria

  • A scripted pip install --dry-run --report check resolves the current release with cryptography==46.0.7, with packaging==24.2, and alongside crewai==1.15.27. python scripts/dependency_floors.py coinstall prints OK for all three on this tree (exit 0, run with pip 26.0.1 and pip 25.0.1). On origin/main's pyproject.toml the same command fails all three with ResolutionImpossible (exit 1). Tests: tests/test_dependency_floors.py::test_pip_installs_this_release_beside_the_pins_agent_projects_carry runs the real check (slow, so nightly and in release verification; it skips when pypi.org is unreachable). The offline test_the_floors_admit_each_scenarios_direct_pins[*] and test_the_floors_admit_crewais_pydantic_range fail on the old floors.
  • A lowest-direct CI job runs on PRs and passes. dependency-floors in ci.yml. I ran each step locally in fresh venvs. compile put all six at exactly their floors (uv 0.13.1); the hashed install and pip check passed; the 14 test files passed (75 s on 10 cores, about 290 s of test time); regenerate_goldens.py --check changed 0 of 24 artifacts; doctor reported only dependency_differs_from_lock. In CI (run 39145905447, head f88fd4e) the job passed in 1m44s: three OK dry runs in 19 s, six floors resolved exactly, pip check clean, 698 passed and 2 skipped in 55 s, and 0 of 24 goldens changed.
  • dependabot.yml uses a lockfile-only strategy and CONTRIBUTING states when floors may move. increase-if-necessary, so updates move only the locks; see the deviation above. test_dependabot_moves_the_locks_and_never_a_floor fails on the old config. CONTRIBUTING § Changing dependencies → Runtime floors.
  • doctor --json reports a mismatch when a dependency is below its floor (unit test with patched importlib.metadata). tests/test_environment.py::test_doctor_reads_installed_versions_through_importlib_metadata patches importlib.metadata.version and runs environment_report on this checkout, and test_a_built_distribution_is_judged_by_its_own_requires_dist also patches importlib.metadata.requires. With the one line that adds the dependency mismatches reverted, 11 of the new tests fail. On the maintainer's launcher venv, ./shipgate doctor --json used to report mismatches: []. It now reports dependency_differs_from_lock for cryptography 46.0.7, packaging 26.2, pydantic 2.13.4 and typer 0.27.0. Against the old floors, the first two would be dependency_below_declared_floor.
  • scripts/verify_dependency_lock.py and the executable_pin parity claims still pass. verify_dependency_lock.py exit 0. tests/test_distribution_surface_parity.py passes. The constraints/dev.txt recompile changed only its declaration block: no pin moved, no marker was dropped, and no name[extra]== pin appeared.

Owner decisions

  1. The cryptography floor now admits releases with advisories. Every release before 50.0.0 has at least one (PyPI/OSV), and Add trusted authorization for exact coding-agent pushes #281 had raised the floor "past vulnerable wheels". I read each advisory that applies to 44.0.3–49.x: PKCS#7 decryption (GHSA-g6cj), X.509 path building (GHSA-jwv3), DNS name constraints (GHSA-m2h6, GHSA-m959), non-contiguous buffers (GHSA-p423), SECT-curve subgroups (GHSA-r6ph), and the bundled-OpenSSL advisory of 2026-06-09 (GHSA-537c: PKCS7/CMS/QUIC/OCSP/AES-OCB…). None reaches the one call Agents Shipgate makes, Ed25519PublicKey.from_public_bytes(...).verify(...) on bytes. A fresh install still resolves the newest release. CONTRIBUTING now states the rule: a vulnerability moves a floor only when it is reachable. If you would rather not admit those releases, the lowest floor the acceptance criteria allow is 46.0.7.
  2. Should dependency-floors become a required check? I recommend not yet. It resolves from the live index on purpose (lowest-direct means "the newest of everything else"), so a third-party release can turn it red on an unrelated PR. Leave it advisory for a few weeks of green runs. It runs as one job with the usual timeout-minutes: 15, alongside the shards, and does not touch their budget.
  3. The slow co-installation test now runs in release verification, which uses -m "not perf", so the release gate reaches PyPI for that one test. It skips when the index is unreachable. If you want the release gate fully offline, deselect it there.
  4. Caps kept: pydantic<3, pyyaml<7, ruamel.yaml<1, typer<1, where the next major is the dependency's own declared break. Caps dropped: cryptography<51, packaging<27. Both publish majors routinely, and either cap would repeat the 0.15.0 fallback on their next release. tests/test_dependency_floors.py::CAPPED records the reasons.
  5. Old releases remain the fallback. pip can still choose 0.x whenever a future constraint excludes every 1.x release. Yanking the pre-1.0 releases on PyPI would make pip refuse instead of falling back. That is your call, not part of this PR.
  6. What is not proven: floors are proven on Python 3.12 only (the supported range is decide: supported Python and OS range: cap requires-python, or test 3.13/3.14 and Windows #1058). Mixed combinations, such as one dependency at its floor with the others newest, or an older transitive dependency such as click 8.0 under typer 0.20, are proven by neither this job nor the locked suite.

Authorship

Verification

Locked environment: a fresh venv from constraints/dev.txt and constraints/build-backend.txt with --require-hashes, plus pip install -e . --no-deps --no-build-isolation, on Python 3.12.13.

Command Exit
python -m ruff check . 0
python scripts/verify_dependency_lock.py 0
python scripts/generate_schemas.py --check 0
python scripts/regenerate_goldens.py --check 0 (24 artifacts, 0 changed)
pytest tests/test_dependency_floors.py tests/test_environment.py tests/test_packaging.py tests/test_distribution_surface_parity.py tests/test_action_metadata.py tests/test_slow_tests_workflow.py tests/test_shard_partition.py tests/test_public_surface_contract.py tests/test_workflow_step_action_references.py tests/test_actions_token_scope.py tests/test_release_pipeline.py tests/test_agent_instructions_renderers.py 0
pytest -n auto -m "not perf" --ignore=tests/test_adapter_static_only.py 1: 23,295 passed, 2 failed. Both failures are the known local-only test_check_unmodelled_host_config_keys.py[local_settings_enabled_plugins-*] global-gitignore noise. All six CI shards pass.
pytest tests/test_adapter_static_only.py 0
pytest tests/test_latency_budget.py -m perf 0

At the floors (every runtime dependency at exactly its floor, the rest of the dev closure installed): pytest -n auto -m "not perf" --ignore=tests/test_adapter_static_only.py gave 23,247 passed and 6 failed. None of the six is a floor defect:

  • 3 × tests/test_schema_roundtrip.py: pydantic 2.12.5 renders CapabilityDiffRow without the additionalProperties: false that 2.14 adds (build(deps): refresh dev, build-backend and publish locks; uv 0.13.1, pydantic 2.14.1 #1088). That is the generator of the committed docs, not what the engine emits or validates. The floors job leaves generate_schemas.py --check out and says why.
  • 2 × test_check_unmodelled_host_config_keys.py[local_settings_*]: the known local-only global-gitignore noise.
  • 1 × test_release_advisory_pipeline.py::test_an_exercised_wheel_is_bound_by_bytes_and_never_called_qualified: a wall-clock flake under load. Its _wheel helper writes zip entries with the current time, so two wheels built in different seconds differ. It passes alone. This is pre-existing and unrelated to this PR.

An earlier full run at the issue's suggested typer==0.16.0 failed 80 tests by 26% before I stopped it. That is what moved the floor to 0.20.0.

coinstall took 46 s with a warm pip cache and 15 min on a slow local connection with a cold cache. In CI it took 19 s, so the job keeps the usual 15-minute cap.

Release-readiness notes

  • No user-code import added to default scan paths
  • No network access added to default scan paths (the network is used only by scripts/dependency_floors.py in CI and by one slow test that skips when PyPI is unreachable)
  • New or changed check IDs are documented in docs/checks.md (none changed)
  • Report/schema changes are additive or documented in STABILITY.md (no report or schema change; two additive doctor mismatch codes, documented in docs/diagnostics.md)

🤖 Generated with Claude Code

… reports a dependency below its floor

The runtime floors had tracked the newest upstream release, so pip install
agents-shipgate beside an agent project's ordinary pins quietly installed an
old release: 0.15.0 with cryptography==46.0.7 or packaging==24.2, and 0.11.0
beside crewai==1.15.27 (pydantic<2.13).

- pyproject.toml: cryptography>=44.0.3, packaging>=24.2, pydantic>=2.12.5,<3,
  pyyaml>=6.0.2,<7, ruamel.yaml>=0.18.10,<1, typer>=0.20.0,<1. The
  cryptography and packaging caps are dropped. typer is 0.20.0, not 0.16.0:
  the root command loader reads Typer.suggest_commands, added in 0.20.0. The
  full suite passes at exactly these floors (schema-generator and wall-clock
  failures aside, see the PR).
- scripts/dependency_floors.py: `coinstall` (pip --dry-run --report against a
  metadata-only wheel of this tree, per scenario), `compile` (uv pip compile
  --resolution lowest-direct, failing unless every runtime dependency lands
  exactly on its floor) and `installed`.
- ci.yml: a dependency-floors job runs all three, installs the lowest-direct
  closure with hashes, and runs the dependency-heavy tests, every sample
  golden and doctor there.
- dependabot.yml: versioning-strategy increase-if-necessary, so updates move
  the locks and never a floor. lockfile-only would stop Dependabot proposing
  the constraints/*.txt updates at all, because it reads them as requirement
  files.
- CONTRIBUTING.md says when a floor may move.
- doctor --json: mismatches[] gains dependency_below_declared_floor (error)
  and, for a source checkout, dependency_differs_from_lock (warning), read
  from importlib.metadata against the running build's declarations.
- constraints/dev.txt: recompiled with uv 0.13.1; only its declaration block
  changed.

Closes #1028
…15-minute cap

The first run took 1m44s: 19 s for the three pip dry runs and 55 s for the
698 tests. The 20-minute cap had been sized for a cold download cache that
GitHub's runners do not have.
@pengfei-threemoonslab

Copy link
Copy Markdown
Contributor Author

🤖 Engineering review — implementer self-review

I reviewed git diff origin/main...HEAD as an adversarial reviewer before opening this. What I found and fixed, then what is left for the owner.

Found and fixed before opening

  1. The issue's suggested typer floor (0.16.0) does not work. The root command loader passes app.suggest_commands to typer.main.get_group_from_info; that attribute first exists in Typer 0.20.0. On 0.16.0 agents-shipgate --help exits 1 with an AttributeError, and the first full-suite run at the issue's floors failed 80 tests by 26% before I stopped it. The floor is 0.20.0, and the full suite passes there with every other dependency at its floor.
  2. lockfile-only would have silenced Dependabot for the locks. dependabot-core reads constraints/*.txt as requirement files, so with lockfile-only an == pin can never move and security updates for them end as update_not_possible. I used increase-if-necessary, which leaves an admitting pyproject.toml range alone and still rewrites == pins. This deviates from the AC's wording but meets its intent; the PR body cites the source lines.
  3. Epoch versions inverted the stdlib comparison. The first draft of _below read 1!0.1 as release 1, so it would have called an epoch-1 install "below" a 2.0 floor. _release now refuses any epoch, and the PEP 440 parametrized test has that case with a floor that would expose the inversion.
  4. The co-installation check could have checked a different metadata than the wheel ships. coinstall resolves a metadata-only wheel so the job needs no build backend, whose own packaging would move a floor under test. tests/test_packaging.py::test_the_floors_check_resolves_the_metadata_the_wheel_ships now compares its name, version, Requires-Python and unconditional Requires-Dist with the real built wheel's. Its first run caught a real difference: the stand-in wrote ruamel.yaml where the backend writes ruamel-yaml. A resolver treats those as one name, so the test compares names in canonical form.
  5. A floor that something else raised would have been "tested" at the wrong version. compile fails unless each runtime dependency resolves exactly to its floor and names the requester that raised it, and installed re-checks the interpreter the tests run in.
  6. The job's cap was sized for a network I did not have. Locally the CrewAI dry run took 15 minutes on a slow connection with a cold cache, so the first draft gave the job 20 minutes. Its first CI run took 1m44s (19 s for the dry runs), so it is back to the usual 15 minutes, and the workflow comment states the measured times.
  7. Each test reverted to its old behaviour fails. With origin/main's floors, 4 offline floor tests fail and coinstall fails all three scenarios. With the one line adding the dependency mismatches reverted, 11 of the new doctor tests fail. With the epoch guard removed, its case fails. With the old dependabot.yml, the strategy test fails. Each file was restored afterwards and checked byte for byte.
  8. The committed JSON schemas are not a floor test. build(deps): refresh dev, build-backend and publish locks; uv 0.13.1, pydantic 2.14.1 #1088 moved them with pydantic 2.14, so generate_schemas.py --check would fail under 2.12.5 without saying anything about the engine. The job leaves it out, and the workflow comment says why.

Left for the owner

  • Protected paths. pyproject.toml, constraints/dev.txt, .github/dependabot.yml, .github/workflows/ci.yml and CONTRIBUTING.md all change here, so this needs owner review before merge.
  • The cryptography floor admits releases with advisories. None of them reaches raw Ed25519 verification, and the PR body lists each one. This reverses Add trusted authorization for exact coding-agent pushes #281's "past vulnerable wheels" floor on purpose. 46.0.7 is the alternative if you disagree.
  • increase-if-necessary instead of the literal lockfile-only. I did this deliberately; the PR body cites the dependabot-core source. It also ends the harness's always-closed claude-agent-sdk floor-bump PRs.
  • Whether dependency-floors should be required. I recommend advisory first, because it resolves from the live index by design.
  • The slow co-installation test reaches PyPI in release verification. It skips when offline. Deselect it there if you want that gate fully offline.
  • Not proven: Python 3.13/3.14 at the floors (decide: supported Python and OS range: cap requires-python, or test 3.13/3.14 and Windows #1058), and mixed combinations such as one dependency at its floor with the rest newest, or an old transitive click under typer 0.20.
  • Out of scope but found: tests/test_release_advisory_pipeline.py::_wheel writes zip entries with the wall-clock time, so test_an_exercised_wheel_is_bound_by_bytes_and_never_called_qualified fails whenever its two wheels straddle a second boundary under load. I saw it once in a loaded local run; it passes alone. Fixing it needs only zipfile.ZipInfo with a fixed date_time.
  • Fallback releases. Yanking the pre-1.0 releases would make pip refuse rather than silently fall back the next time a constraint excludes every 1.x release.

@pengfei-threemoonslab

Copy link
Copy Markdown
Contributor Author

🤖 Engineering review — independent review, cycle 1

Reviewed head: 9f37a37db6. No P0–P2 code findings. The remaining questions are owner decisions on protected paths, listed below.

What I checked

  • Floors. pyproject.toml floors are the versions the new dependency-floors job resolves with uv pip compile --resolution lowest-direct, and it fails unless every runtime dependency lands exactly on its floor. typer 0.20.0, rather than the issue's 0.16.0, is justified: cli/main.py reads app.suggest_commands, which Typer 0.20.0 introduced. constraints/dev.txt changed only in its declarations block, with no pin moved.
  • doctor. _dependency_mismatches extends the existing _mismatches engine. environment_report/probe_environment are called only from cli/_register_doctor.py, so the new reads of pyproject.toml and the lock never reach report.json, verifier.json or any identity, and a machine's lock drift cannot affect determinism. _below treats an unreadable version as not below, so it never sends a working environment to reinstall. It handles the one suffix that sorts below an equal release (pre/dev) and leaves .postN alone.
  • Dependabot. increase-if-necessary instead of lockfile-only is reasoned in the file and in the PR body. With lockfile-only, Dependabot would treat constraints/*.txt as requirement files, so lock and security updates would stop.
  • CI. The new job sets timeout-minutes: 15. Its tools venv installs from release-publish.txt and build-backend.txt with --require-hashes, and the floor closure installs with --require-hashes. Hosted CI is green on this head.

Owner decisions (protected paths)

  1. cryptography>=44.0.3 admits releases that carry advisories. This reverses Add trusted authorization for exact coding-agent pushes #281's floor, and 46.0.7 is the alternative.
  2. Whether dependency-floors becomes a required check. I recommend advisory first, since it resolves from the live index.
  3. Dropping the cryptography<51 and packaging<27 caps.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

packaging: set runtime floors to the oldest tested versions so pip stops resolving 0.15.0

1 participant