Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/pgw1137.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- **pgw#1137 (DESIGN-RULINGS §4.30, and pgw#1127 §7 item 6): an AOT mint on the user's OWN machine is polite — declared, never toggled.** Since pgw#1127 S1 `cozy serve` drives the real `mint_delegate` → `mint_process` → `mint_child` chain, so cozy-local now compiles on a desktop using every clamp `aot_compile_pool` has — and all of those are written for a serving pod with a co-resident TENANT, not for a machine with a co-resident HUMAN. pgw#1127 §6 recorded the evidence: **zero `os.nice` calls anywhere in `src/`**. A new `compile_posture.CompilePosture` carries the one fact that decides this, and it is **declared by the entry that knows** (`local_serve`, the only site in the tree that says `USER_MACHINE`) rather than sniffed off any of the three near-miss proxies — `local_cell_store.trust_class()` and `publisher is None` are facts about the SINK, and a rented community-cloud pod matches both while having nobody sitting at it; `worker_goals` says what a pod was bought to do, and cozy-local was not bought. It is not an env var either (§1.17: an env may carry a VALUE, not a DECISION): it rides `MintRequest` as a typed field beside `vram_cap_bytes`, and the mint child publishes it before anything sizes a pool. **CPU:** the child drops itself to `nice 19` — by the CHILD, on itself, never through a `preexec_fn`, for the fork-vs-`posix_spawn`/gRPC-`pthread_atfork` hazard `arm_parent_death_signal` already writes down; and because every production mint has been weight-free (and therefore serial, in that very process) since pgw#1080, nicing the pool's spawns would have niced a path that does not currently run. Nice is inherited, so one call covers the serial compile, the entry children when pgw#1111 restores them, inductor's compile workers and every `cc1plus` under them. `entry_workers` additionally halves its core budget and halves its ceiling (8 → 4): priority is what preserves interactivity, but it does nothing for K concurrent `cc1plus`, K inductor caches and K working sets in the page cache. **Memory:** the host-RAM reserve doubles to 8 GiB, because `MemAvailable` counts the user's page cache as free and a mint that OOMs a desktop is worse than a slow one. **Honesty:** a `cozy serve` compile now says what it is doing, that it happens once, where the cell will be kept, what it is taking (nice level, worker cap, cores) and that Ctrl-C is safe — then prints a throttled progress line every 10 s. Before this the only signals were an activity addressed to a hub cozy-local does not have and a `logger.info` inside a subprocess. **Interruptibility:** the mint child arms `PR_SET_PDEATHSIG` on a user machine — it is spawned `start_new_session=True`, so a terminal's Ctrl-C or SIGHUP never reached it and a closed terminal left a full-speed compile tree running with nobody to reap it; `aot_resume`'s cross-attempt bank already lives outside the per-attempt workdir, so nothing it finished is lost. **The serving pod is untouched**: `FLEET` is the struct default everywhere, and a pod's width is proven arithmetically identical against an independent re-implementation of the pre-issue formula over six pod shapes. Every politeness claim is individually mutation-proven RED. Rejected with reasons in `compile_posture`'s docstring: an interactivity probe (cross-platform mess, and wrong for the developer running a build in a terminal — `nice` already IS the yield), `ionice` (no stdlib wrapper, a no-op on the schedulers desktops actually ship), and a third posture for community cloud.
50 changes: 42 additions & 8 deletions src/gen_worker/aot_compile_pool.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,9 @@
from . import aot_shape_hints

from . import aot_compile_spans, aot_device_lock, aot_resume, env_seal
from . import mint_budget, worker_goals
from . import compile_posture, mint_budget, worker_goals
from .compile_posture import (
USER_MACHINE_RSS_RESERVE_BYTES, CompilePosture)
from .worker_goals import WorkerGoals
from .postmortem import cpu_quota_cores
import hashlib
Expand Down Expand Up @@ -481,6 +483,11 @@ class PoolWidth:
#: exact case Paul's ruling requires to work.
serve_goal: bool = True
mint_goal: bool = False
#: §4.30 / pgw#1137: whose MACHINE this is. Distinct from the goals above,
#: which say what the pod was bought to do — a K held down for a human at
#: a keyboard and a K held down for a tenant are different decisions and a
#: reader must be able to tell them apart. ``FLEET`` on every pod.
posture: CompilePosture = compile_posture.FLEET

@property
def underwidth(self) -> int:
Expand Down Expand Up @@ -510,6 +517,7 @@ def facts(self) -> Dict[str, Any]:
"serve_goal": bool(self.serve_goal),
"mint_goal": bool(self.mint_goal),
"width_reason": self.reason,
**self.posture.facts(),
}
for block in (self.cpu, self.memory, self.device):
if block is not None:
Expand Down Expand Up @@ -716,6 +724,7 @@ def entry_workers(
limit: int = 0,
device_lock: Optional[bool] = None,
goals: Optional[WorkerGoals] = None,
posture: Optional[CompilePosture] = None,
) -> PoolWidth:
"""How many entries this pod may compile at once.

Expand Down Expand Up @@ -746,10 +755,22 @@ def entry_workers(
the constraint that actually bound. K is the mint's only multiplicative
lever — two mints of one cell differed 5-vs-3 with nothing recorded to
say why — so an unexplained K is a defect in itself.

§4.30 / pgw#1137: two of the three bounds are ALSO posture-aware. The
``goals`` above answer *"is there a tenant"*; ``posture`` answers *"is
there a human"*, and on a user's own desktop the CPU budget is halved and
the host-RAM reserve is doubled (:mod:`gen_worker.compile_posture` holds
the derivation for both). The DEVICE bound is untouched — a desktop GPU is
not shared with anyone the way its CPU and RAM are, and narrowing a bound
that already refuses to guess would only make first mints serial. The
default posture is ``FLEET``, so a pod's width is byte-identical to what it
was before this parameter existed.
"""
entries = max(0, int(entries))
if goals is None:
goals = worker_goals.current()
if posture is None:
posture = compile_posture.current()
# pgw#930 (§1.17): THREE of this policy's terms are tenant reserves, and a
# pod with no SERVE goal has no tenant. `SERVING_HEADROOM_CPUS` keeps cores
# for an eager forward and a heartbeat; `DEVICE_RESERVE_BYTES` keeps VRAM
Expand All @@ -770,7 +791,8 @@ def entry_workers(
reserve = goals.tenant_reserve_applies()
cpu_headroom = SERVING_HEADROOM_CPUS if reserve else 0
device_reserve = DEVICE_RESERVE_BYTES if reserve else 0
rss_reserve = ENTRY_RSS_RESERVE_BYTES if reserve else FORGE_RSS_RESERVE_BYTES
rss_reserve = posture.rss_reserve_bytes(
ENTRY_RSS_RESERVE_BYTES if reserve else FORGE_RSS_RESERVE_BYTES)
locked = aot_device_lock.supported() if device_lock is None \
else bool(device_lock)
if entries <= 1:
Expand All @@ -787,7 +809,7 @@ def entry_workers(
per_entry_rss_basis="not-read", per_entry_device_basis="not-read",
device_lock=locked, binding="entries", ceiling=1,
limit=max(0, int(limit)),
serve_goal=goals.serve, mint_goal=goals.mint,
serve_goal=goals.serve, mint_goal=goals.mint, posture=posture,
reason=(
f"{entries} entr{'y' if entries == 1 else 'ies'}: serial "
f"(no cpu/memory/device bound was read — the entry count "
Expand All @@ -798,7 +820,7 @@ def entry_workers(
else:
cpu = cpu_facts()
vcpus = cpu.vcpus
budget = vcpus - cpu_headroom
budget = posture.cpu_budget_cores(vcpus, headroom=cpu_headroom)
cpu_workers = max(1, budget // CPUS_PER_ENTRY_WORKER)

if available_bytes >= 0:
Expand Down Expand Up @@ -853,8 +875,11 @@ def entry_workers(

# A caller cap NARROWS. `limit` above MAX_ENTRY_WORKERS is a caller
# asking for more than the ceiling allows, and the ceiling wins.
ceiling = min(MAX_ENTRY_WORKERS, int(limit)) if limit > 0 \
else MAX_ENTRY_WORKERS
# ...and so does the POSTURE (§4.30): a user's machine caps at half the
# fleet ceiling, and a caller cap below that still wins. Both narrow;
# neither can widen.
ceiling = posture.entry_ceiling(
min(MAX_ENTRY_WORKERS, int(limit)) if limit > 0 else MAX_ENTRY_WORKERS)
# pgw#877: ONE definition of each basis. It was written twice — once for
# the row, once for the reason string — which is how the reason could
# have drifted from the field it explains.
Expand Down Expand Up @@ -884,7 +909,7 @@ def _width(
per_entry_device_basis=device_basis,
per_entry_rss_basis=rss_basis,
limit=max(0, int(limit)),
serve_goal=goals.serve, mint_goal=goals.mint)
serve_goal=goals.serve, mint_goal=goals.mint, posture=posture)

workers = max(
1, min(cpu_workers, mem_workers, device_workers, ceiling, entries))
Expand All @@ -900,8 +925,13 @@ def _width(
(cpu_workers, "cpu"), (mem_workers, "host-memory"),
(device_workers, "vram"), (ceiling, "ceiling"),
(entries, "entries"))[1]
polite = (
" [§4.30 user-machine: half the cores, "
f"{USER_MACHINE_RSS_RESERVE_BYTES // 1024**3} GiB RAM left alone, "
f"ceiling {ceiling}, nice {posture.nice_level()}]"
if posture.user_machine else "")
reason = (
f"K={workers} ({binding}-bound, goals="
f"K={workers} ({binding}-bound{polite}, goals="
f"{'serve+mint' if goals.serve and goals.mint else 'serve' if goals.serve else 'mint' if goals.mint else 'none'}): "
f"{vcpus} vCPU ({cpu.basis}) -> "
f"{cpu_workers}, {avail / 1024**3:.1f} GiB RAM ({memory.basis}) -> "
Expand Down Expand Up @@ -2213,6 +2243,10 @@ def _rewiden(self, *, trigger: str = "measured") -> None:
# not whatever a later `install()` published.
goals=WorkerGoals(
serve=base.serve_goal, mint=base.mint_goal),
# ...and the same rule for the posture: a mid-mint widen must
# not quietly stop being polite on a machine that was polite
# when the pool was built.
posture=base.posture,
)
except Exception: # noqa: BLE001 — a re-derivation never fails a mint
logger.debug("aot-pool: width re-derivation failed", exc_info=True)
Expand Down
Loading
Loading