Skip to content

Fix pex3 venv create --link-python for --pex-repository. - #3286

Merged
jsirois merged 4 commits into
pex-tool:mainfrom
jsirois:venv-create-pex-repository-link-python
Sep 18, 2026
Merged

jsirois merged 4 commits into
pex-tool:mainfrom
jsirois:venv-create-pex-repository-link-python

Conversation

@jsirois

@jsirois jsirois commented Sep 18, 2026

Copy link
Copy Markdown
Member

pex3 venv create --link-python for a foreign platform fails when the venv is installed from a --pex-repository:

A distribution for psutil could not be resolved for <local python>.
Found 1 distribution for psutil that does not apply:
1.) The wheel tags for psutil 5.9.5 are cp36-abi3-manylinux_2_12_x86_64, ... which do not match
    the supported tags of <local python>: cp313-cp313-manylinux_2_39_aarch64 ...

The distributions themselves install fine — it is populating the venv's sources that fails. _populate_first_party re-resolves the repository PEX to build the pex-repl script's activated_dists, and that resolve runs against a local interpreter, which a PEX built for the foreign platform alone holds nothing for. _install_from_pex has already resolved the distributions for the target, so this passes those down instead; the PEX_TOOLS=1 ... venv path keeps re-resolving, where the PEX and the interpreter always agree.

This is the combination Pants uses — it always invokes pex3 venv create with --pex-repository — so #3279 is unreachable from Pants without it.

Repro and verification, on a linux aarch64 host:

pex psutil==5.9.5 --platform linux_x86_64-cp-313-cp313 -o psutil.pex
pex3 venv create -d venv --pex-repository psutil.pex --platform linux_x86_64-cp-313-cp313 \
    --python-path $(command -v python3.13) --link-python /opt/python/bin/bob

Before: the error above. After: bin/python, bin/python3 and bin/python3.13 all link to /opt/python/bin/bob, and lib/python3.13/site-packages/psutil/_psutil_linux.abi3.so is ELF 64-bit LSB shared object, x86-64.

test_foreign_target_link_python_from_pex_repository covers it. I could not run it locally — the py310 fixture's CPython 3.10.7 build fails in my environment (ensurepip exits non-zero) — so CI is its first run; the commands above are the same shape by hand. uv run dev-cmd format lint typecheck is green.

Written with Claude Code, as was #3279.

🤖 Generated with Claude Code

https://claude.ai/code/session_012imZdyz1fhVXRNKPjbhSDw

Jason Barnett and others added 4 commits September 17, 2026 22:56
Populating a venv's sources re-resolved the repository PEX, and that resolve
runs against a local interpreter. For a venv laid out for a foreign platform
the PEX need hold no distribution that interpreter can use, so a PEX built for
the foreign platform alone failed with e.g.:

    A distribution for psutil could not be resolved for <local python>.

`_install_from_pex` has already resolved the distributions for the target, so
pass those down to the venv's `pex-repl` script instead of re-resolving. The
`PEX_TOOLS=1 ... venv` path keeps re-resolving, where the PEX and the
interpreter always agree.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012imZdyz1fhVXRNKPjbhSDw
The resulting yolo build using the local interpreter netted native wheels
not compatible with the foreign platform.
@jsirois

jsirois commented Sep 18, 2026

Copy link
Copy Markdown
Member Author

Supercedes #3285.

@jsirois
jsirois merged commit ab31461 into pex-tool:main Sep 18, 2026
94 of 108 checks passed
@jsirois
jsirois deleted the venv-create-pex-repository-link-python branch September 18, 2026 13:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant