build-container: widen sigpending ulimit for Rosetta signal storms - #175
Merged
Merged
Conversation
Rosetta-for-Linux aborts the translated process when rt_tgsigqueueinfo returns EAGAIN (pend_signal: 11). .NET GC suspensions across UBT/UHT's hundreds of threads burst past the RAM-scaled per-UID default (~96k on a 24GB VM); the bursts drain in milliseconds, so a 1M bucket absorbs them. Applied to build and prewarm runs; a cap not a reservation, harmless on native hosts. Co-authored-by: Cursor <cursoragent@cursor.com>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
JPDuchesne
added a commit
that referenced
this pull request
Sep 16, 2026
Agent-isolation and emulated-Mac fixes: install-deps failure isolation with typed errors (#170), brew escalation to the prefix owner + AgentBootstrap sudoers edge (#171), BuildKit secret files in a verified per-uid dir under the data root — colima-visible and substitution-proof (#173), build-watcher guest-load liveness signal + 15m stall window (#174), sigpending ulimit for Rosetta signal-queue aborts (#175), and brew trust before tap in docker-install-build-deps (tap-rot fix).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Emulated (Apple-silicon) prewarms and builds die intermittently with:
exit 133, hotspot UHT/UBT. Rosetta-for-Linux forwards every cross-thread signal through
rt_tgsigqueueinfoand aborts the whole process when the kernel answers EAGAIN — no backoff, because retrying inside its own signal-forwarding path risks deadlock. The queue fills because a .NET GC suspension pings every runtime thread (UHT runs hundreds), each ping becoming a queued signal under emulation, whileRLIMIT_SIGPENDINGis one per-UID bucket shared by every process in the container (RAM-scaled default, ~96k on a 24GB VM). The crashed children also leave UBT sleeping on unreaped zombies — the deadlock the build-watcher reclaims — so this attacks that at the root too.Fix
--ulimit sigpending=1000000on every build and prewarmdocker run(sharedSIGPENDING_ULIMITconstant carrying the rationale). The bursts are transient — queues drain in milliseconds once a suspension completes — so ~10x headroom absorbs them.Safety
The limit is a cap, not a reservation: ~80 bytes of kernel memory per actually-queued signal, worst case ~80MB, nothing at rest. Its purpose is DoS protection against unprivileged users on shared machines, which defends nothing in a single-purpose build VM. Harmless on native hosts.
Context: snappy's
docs/macos-emulated-builds.mdcatalogues the full emulation failure-mode family this belongs to.Made with Cursor