Skip to content

Bump pyupgrade and mypy pre-commit hooks for Python 3.14 - #759

Merged
puddly merged 1 commit into
devfrom
zigpy-bot/precommit-py314-hook-bumps
Sep 28, 2026
Merged

puddly merged 1 commit into
devfrom
zigpy-bot/precommit-py314-hook-bumps

Conversation

@zigpy-review-bot

Copy link
Copy Markdown
Collaborator

Problem

pre-commit run fails on Python 3.14 — two of the eight hooks crash before checking anything:

pyupgrade ... Failed   AttributeError: module 'ast' has no attribute 'Str'
mypy ....... Failed   The typed_ast package is not installed.

Both have the same root cause: Python 3.14 removed the long-deprecated ast.Str, ast.Num, ast.Bytes and ast.NameConstant aliases (deprecated since 3.8, still present in 3.13).

  • pyupgrade v3.3.1 evaluates LITERAL_TYPES = (ast.Str, ast.Num, ast.Bytes) in _plugins/identity_equality.py, so it dies while importing its plugins.
  • mypy v1.3.0's message is misleading — installing typed-ast does not help. mypy/fastparse.py wraps a large from ast import (..., NameConstant, Num, Str, ...) in a bare except ImportError: that assumes a missing typed_ast is the only possible cause. The actual ImportError is the four removed names.

The other six hooks (autoflake, black, flake8, isort, codespell, ruff) already pass on 3.14.

CI never caught this because the shared pre-commit job runs on Python 3.11 — see the note below.

Changes

  • pyupgrade v3.3.1 → v3.21.2. The first release unaffected by the ast.Str removal is v3.4.0; v3.3.2 still crashes.
  • mirrors-mypy v1.3.0 → v2.3.1. The first unaffected mypy is v1.5.0; v1.4.1 still prints the typed_ast message. The hook's args are identical between v1.3.0 and v2.3.1 (--ignore-missing-imports --scripts-are-modules), so this is not a strictness change — and [tool.mypy] ignore_errors = true in pyproject.toml means the hook only ever surfaced parse errors anyway.
  • --py38-plus → --py311-plus, matching requires-python = ">=3.11".

The target bump produces a single kind of rewrite across six files: AsyncGenerator[X, None] → AsyncGenerator[X], dropping the redundant send/return type parameters. It is annotation-only and behaviour-preserving. Note that current pyupgrade emits this rewrite under --py38-plus as well — it is not gated on the target version — so it is not avoidable by keeping the old argument.

Verification

  • pre-commit run --all-files on Python 3.14: all 8 hooks pass, where 2 failed before. A second pass is clean, so the rewrites are idempotent.
  • pre-commit run --all-files also passes on Python 3.11, 3.12 and 3.13 with the new pins, so the CI pre-commit job stays green.
  • Full test suite: 465 passed.
  • Single-argument subscripting of collections.abc.AsyncGenerator — the form all six files import — is valid at runtime on 3.11, the floor. (The typing.AsyncGenerator alias is not: it raises TypeError: Too few arguments on 3.11. Nothing here uses it, but that is why the distinction matters.) Every affected module also has from __future__ import annotations, so these annotations are never evaluated at import time regardless, and nothing in bellows or zigpy introspects them via typing.get_type_hints.
  • The single-argument style is already established in the codebase: bellows/ezsp/__init__.py has -> Generator[asyncio.Future] and -> Generator[None] on dev today, from the same collections.abc import.

Note for zigpy/workflows

Worth considering separately, in another repo: the shared pre-commit job in zigpy/workflows runs on PYTHON_VERSION_DEFAULT, currently 3.11, while the test matrix already spans "3.11", "3.12", "3.13", "3.14". That gap is exactly why this breakage never turned a PR red — it only bites contributors whose local default interpreter is 3.14. Bumping the pre-commit job's Python, or running it across the matrix, would surface this class of problem in CI rather than on contributors' machines. Not changed here.

Other zigpy repos with the same broken pins

Surveying every non-archived repo in the org, two others still pin affected versions:

  • zigpy/zigpy-deconz — pyupgrade v3.3.1, the identical crash. It has no mypy hook.
  • zigpy/zha-websocket-server — pyupgrade v2.31.1 and mirrors-mypy v0.902, both crash. Last pushed in October 2024, so likely dormant.

Already on unaffected versions: zigpy-xbee, zigpy-zigate, zigpy-ota, zigpy-znp and zha-device-handlers. zha, zigpy, zigpy-cli and the remainder pin neither hook.

Python 3.14 removed the long-deprecated `ast.Str`, `ast.Num`, `ast.Bytes` and
`ast.NameConstant` aliases, which breaks both pinned hooks:

- `pyupgrade` v3.3.1 fails with `AttributeError: module 'ast' has no attribute
  'Str'` from `_plugins/identity_equality.py`.
- `mypy` v1.3.0 reports `The typed_ast package is not installed.`, which is
  misleading -- `mypy/fastparse.py` wraps its `from ast import (..., NameConstant,
  Num, Str, ...)` in a bare `except ImportError` that assumes a missing
  `typed_ast` is the only possible cause. Installing `typed-ast` does not help.

Bump to `pyupgrade` v3.21.2 (first unaffected release: v3.4.0) and `mirrors-mypy`
v2.3.1 (first unaffected mypy: v1.5.0); both run clean on 3.14. The `mypy` hook
args are unchanged between v1.3.0 and v2.3.1.

Also switch `--py38-plus` to `--py311-plus` to match `requires-python = ">=3.11"`.
The resulting rewrite drops the redundant `None` send/return parameters from
`AsyncGenerator[...]` annotations.
@puddly
puddly merged commit b1bbab8 into dev Sep 28, 2026
20 checks passed
@puddly
puddly deleted the zigpy-bot/precommit-py314-hook-bumps branch September 28, 2026 00:31
@codecov

codecov Bot commented Sep 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.55%. Comparing base (15ccb49) to head (28c5381).
⚠️ Report is 1 commits behind head on dev.

Additional details and impacted files
@@           Coverage Diff           @@
##              dev     #759   +/-   ##
=======================================
  Coverage   99.55%   99.55%           
=======================================
  Files          64       64           
  Lines        4284     4284           
=======================================
  Hits         4265     4265           
  Misses         19       19           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@TheJulianJES

TheJulianJES commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

(Just testing zigpy-review-bot self-authored reviews via intermediate team review request, as GitHub disallows requesting review for self-authored PR. Previously, you had to write @zigpy-review-bot Review for this. So ignore review below):

(Note this is generally not needed for zigpy-review-bot-authored PRs though, as it does internal review rounds.)

@TheJulianJES
TheJulianJES requested a review from a team September 28, 2026 01:23

@zigpy-review-bot zigpy-review-bot left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review (for the record — bot-authored PR, already merged as b1bbab8)

No findings. This is a pre-commit tooling bump plus the annotation-only rewrite it produces, and every load-bearing claim in the description checks out at 28c5381:

  • Hook config. pyupgrade v3.3.1 → v3.21.2, --py38-plus → --py311-plus (matches requires-python = ">=3.11"), mirrors-mypy v1.3.0 → v2.3.1. The mypy hook sets no args of its own, and the mirror's default args are ["--ignore-missing-imports", "--scripts-are-modules"] at both tags (checked in .pre-commit-hooks.yaml at each ref), so the bump does not make the hook stricter. With [tool.mypy] ignore_errors = true in pyproject.toml it still only reports parse errors.
  • Code rewrite. All six touched modules (protocol.py, v4, v5, v7, v13, v14) import AsyncGenerator from collections.abc and have from __future__ import annotations. The one-argument form is valid at runtime on the 3.11 floor (collections.abc.AsyncGenerator[int] evaluates fine there), and the annotations aren't evaluated at import anyway. No Generator[..., None] / AsyncGenerator[..., None] forms remain in bellows/ or tests/, so the tree is consistent.
  • CI. Everything is green at the head: pre-commit, and tests on 3.11, 3.12, 3.13 and 3.14.

Nothing to follow up on in this repo. The zigpy/workflows note (the pre-commit job runs on 3.11 while tests run up to 3.14) and the two other org repos with the same broken pins (zigpy-deconz, zha-websocket-server) are the only open threads, and the description already covers them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants