Repository navigation
Build libpython one compiler at a time on macOS Intel - #3
Merged
Merged
Conversation
The macos-x86_64 job fails repeatedly with the cross compiler dying
inside CPython:
Python/symtable.c:1798 internal compiler error: Segmentation fault
Objects/descrobject.o internal compiler error: Segmentation fault
Python/Python-ast.c:9836 internal compiler error: Segmentation fault
A different translation unit each run, always one of the larger
generated ones, and only on that runner -- Linux x86_64 and arm64 pass
every time. setup_cpython.sh drives the archive with -j$(nproc), which
inside the 6 GB colima VM means three concurrent cc1 processes on
CPython's heaviest files.
PYTHONOS_BUILD_JOBS caps that, and the macOS Intel matrix entry sets it
to 1. Anything unset, empty, non-numeric or non-positive falls back to
nproc, so no other host changes behaviour and a typo cannot silently
serialise every build.
This is the second attempt at this failure -- 6f9c698 moved the VM from
VZ to QEMU and it kept crashing. If serialising the compiler does not
settle it either, the next thing to rule out is the QEMU backend itself
rather than the build.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
With libpython building one compiler at a time the cross build now
completes on that runner -- no compiler crash, 59 smoke tests green --
and the job gets as far as test-gui-x86_64, where compositor.start()
times out at 60 seconds.
That runner boots the guest under TCG inside a QEMU-backed colima VM, so
it is slower than the Linux hosts at every stage. PYTHONOS_GUI_COMMAND_-
TIMEOUT already existed for exactly this; set it to 240 there. It is a
deadline rather than a sleep, so a command that finishes quickly still
returns quickly and the Linux jobs are unaffected.
The two timeout readers now fall back to their defaults on anything
unusable. A matrix key that is only set for one entry reaches the others
as an empty string, and bare float("") raises -- the readers would have
crashed the test rather than using the default they appear to promise.
Empty, non-numeric and non-positive all fall back, so a typo cannot turn
a timeout into zero either.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The failure
macos-x86_64has failed four consecutive runs with the cross compiler dying inside CPython:A different translation unit each run, always one of the larger generated ones, and only on that runner —
x86_64andarm64pass every time.mainis red from it right now, so it is not specific to any one PR.tools/setup_cpython.shdrives the archive withmake -j$(nproc). Inside the 6 GB colima VM that is three concurrentcc1processes on CPython's heaviest files.The change
PYTHONOS_BUILD_JOBScaps the job count; the macOS Intel matrix entry sets it to1. It threads through$(DOCKER_RUN)into the build container.Every other host is unaffected — the variable is empty elsewhere and the script falls back to
nproc. Validated each input:PYTHONOS_BUILD_JOBS""-j$(nproc)1/3-j1/-j3abc/0/-2-j$(nproc)So a typo in the workflow cannot silently serialise every build, and it cannot produce
make -j0either.tests/ci_gate_test.pypasses (39 checks).Honest caveat
This is the second attempt at this failure. 6f9c698 moved the VM from VZ to QEMU and it kept crashing. A
SIGSEGVis not an OOM kill — the OOM killer sendsSIGKILL— so memory pressure is a plausible cause but not a proven one; a software-emulated QEMU backend miscompiling is another.Reducing parallelism is the cheaper hypothesis to test and is worth doing on its own merits regardless. If the job still crashes after this, the next thing to rule out is the QEMU backend rather than the build.
The cost is wall-clock on that one runner.
🤖 Generated with Claude Code