diff --git a/changelog.d/pgw1142.md b/changelog.d/pgw1142.md new file mode 100644 index 00000000..45dce041 --- /dev/null +++ b/changelog.d/pgw1142.md @@ -0,0 +1 @@ +- **pgw#1142 (DESIGN-RULINGS §4.32 item 4): the operator can turn compiled serving OFF on a live worker — and back on.** Paul, verbatim: *"The consumer should have the ability to turn off compile entirely, and serve-eager only, if the compile is broken or they just don't care. I.e., they can send some sort of command to the worker to serve eager rather than serve compiled. This is useful for the hub and cozy-local."* Half the vocabulary already existed and is REUSED rather than duplicated: `eager_only` is a Plan `ExecutionSpec` backend (`STEADY_BACKEND_EAGER_ONLY` → `arm_ordered`: *"eager_only arms nothing, by order"*), and pgw#714's `operator_eager_pin` is the hub-resolved lane pinned in config. Both are decided BEFORE a pod serves and neither can be taken back, which is exactly what "the compile is broken" — a thing you discover mid-flight — needs. There was no env toggle to migrate (`GEN_WORKER_FORCE_EAGER` and friends do not exist, grep-verified); this is the first explicit mechanism, and it is a **command over the existing scheduler control channel** (`ServePosture` on `SchedulerMessage`, th#1808), never an env: §1.17 — an env may carry a VALUE, not a DECISION. **Scope, decided and recorded:** WHOLE-WORKER (the operator's question is "is compile broken on this pod"; per-function needs a second addressing vocabulary for a case nobody has had), RUNTIME (a boot-only switch means recycling the pod to use it, which is the cost it exists to avoid), and REVERSIBLE — which is what fixes the enforcement point. The order is read AT THE CALL in `aot_serve`'s wrapper, so an armed artifact is never unwrapped and releasing the order resumes compiled serving on the very next request with no re-arm, no re-materialize and no re-mint; unwrapping would have produced the same first half and a permanently eager pod afterwards. **One posture, one gate:** `compile_cache.arming_block` — the ONE precondition authority — answers the order first, so adoption, JIT intake, cold compile and every self-mint are suppressed by one check rather than by a check per call site; `fleet_cells.enable_compiled` short-circuits BEFORE the policy so no cell is resolved, downloaded or minted for a gate that has already decided; `arm_ordered` obeys the order over the hub's own Plan by arming nothing and serving (never `OrderedArmError` — killing the function is the opposite of "serve eager instead"); and the executor's boot path skips boot-adopt so nothing is fetched at all. **It composes with §4.31's sticky de-arm without being mistakable for it:** same eager posture, two triggers — the de-arm is automatic, per-cell, evidence, irreversible for the boot; this is operator-explicit, whole-worker, policy, reversible — and releasing the order deliberately does NOT resurrect a cell that de-armed itself for cause. **Telemetry says which:** a new `EagerPhase.OPERATOR_EAGER_ONLY` (`operator_eager_only`, pinned in the wire-contract test), aliased into `serving_mode` and `boot_adopt`'s gate reasons, plus a `serve_posture` activity kind carrying `eager_only_engaged`/`eager_only_released` with the operator's own words. `serving_mode.resolve` reports a suppressed request as `eager` rather than `aot_cell` — the cell is still armed, so naive classification reads exactly the wire lie pgw#1082/#1093 spent two pods closing — and it is not counted as a fallback, because nothing fell back. **cozy-local:** `gen-worker serve --eager-only` / `run --eager-only` for the boot, and a `{"posture": {...}}` control frame on the serve socket for a warm serve (cl#49 drives both). Named `--eager-only`, never `--eager`, which means eager LOADING and is a false friend this lane nearly inherited. diff --git a/docs/local-dev.md b/docs/local-dev.md index 6978a3c7..ecdf74c5 100644 --- a/docs/local-dev.md +++ b/docs/local-dev.md @@ -258,6 +258,23 @@ uses): If launching `serve` in the background, pass `--no-stdin` so it doesn't consume the parent shell's stdin. + **Turning compiled serving off** (DESIGN-RULINGS §4.32 item 4). Start with + `--eager-only` and the serve arms no compiled cell and mints none — for a + cell that turns out to be broken, or simply to skip a mint you do not want + to spend your machine's minutes on. On an already-running serve, send the + posture control frame; it is reversible, and the reverse costs nothing + because the artifact is never unwrapped, only bypassed: + + ```bash + $ printf '{"posture":{"eager_only":true,"reason":"the cell crashes"}}\n' \ + | nc -U .gen-worker.sock + {"ok":true,"eager_only":true,"changed":true,"posture":"compiled serving suppressed by cozy-local-cli"} + ``` + + (`cozy serve --eager-only` and `cozy eager-only / [on|off]` are + the cozy-local spellings of the same two calls.) Not to be confused with + `--eager`, which decides when `setup()` runs, not how forwards execute. + - **stdin/stdout NDJSON (default, single process):** pipe a batch of newline-delimited JSON requests in; get one NDJSON response line each. Logs go to stderr. The process exits when stdin closes. diff --git a/proto/PROTO_DIGEST b/proto/PROTO_DIGEST index 8c087e76..cb833f85 100644 --- a/proto/PROTO_DIGEST +++ b/proto/PROTO_DIGEST @@ -3,4 +3,4 @@ # BOTH repos commit this same digest, so a one-sided edit fails scripts/proto-drift-check.sh # offline, in the repo where it happened. Regenerate with: # sha256sum internal/orchestrator/grpc/proto/worker_scheduler.proto -b0c9964eb10d7b333ed69ebc99fd88b91ae6ad957a358621192d36a85dbd5e10 +c173533e248d44a4917080f80d1d32582886c5fb0a5027fe59d6d75ae8b2cb19 diff --git a/proto/worker_scheduler.proto b/proto/worker_scheduler.proto index 34815ca5..3e4f9e54 100644 --- a/proto/worker_scheduler.proto +++ b/proto/worker_scheduler.proto @@ -114,6 +114,8 @@ message SchedulerMessage { // behaves exactly as before. At the coordinated cutover RunJob is deleted // and field 2 + the name "run_job" become `reserved` (§1.27(f)). RunAttempt run_attempt = 7; + // §4.32 item 4 / th#1808 / pgw#1142: the operator's eager-only command. + ServePosture serve_posture = 8; } } @@ -1782,3 +1784,38 @@ message TokenRefresh { string token = 1; // replacement worker JWT int64 expires_at_unix = 2; // exp claim of the replacement (observability) } + +// --------------------------------------------------------------------------- +// Serve posture (DESIGN-RULINGS §4.32 item 4) +// --------------------------------------------------------------------------- + +// Orchestrator -> worker: the OPERATOR's eager-only command. Paul, §4.32 item +// 4: *"The consumer should have the ability to turn off compile entirely, and +// serve-eager only, if the compile is broken or they just don't care."* +// +// It is a COMMAND, not config and not an env var (§1.17): whole-worker, +// runtime, and REVERSIBLE in both directions on a live stream — "the compile is +// broken" is discovered while serving, and so is "it was not the compile". +// +// Distinct from the two eager postures that already exist, deliberately: +// * `ExecutionSpec`'s `STEADY_BACKEND_EAGER_ONLY` is PLAN-carried — chosen +// per dispatch, before the pod is serving, and reports +// `hub_ordered_eager`; +// * §4.31's sticky de-arm is AUTOMATIC and per-cell — a cell-attributable +// failure, evidence, irreversible for the boot. +// This one is operator-explicit, whole-worker and reversible, and it reports +// its own token (`operator_eager_only`) so a suppressed pod's telemetry is +// never mistakable for a broken adopt path. +// +// No reply message; the worker emits a typed activity event on each transition. +// Additive proto3 oneof field: a worker that predates it ignores it. +message ServePosture { + // true = serve eager only: arm nothing, mint nothing, and answer every + // request from the eager forward even where a cell is already armed + // (the arm is NOT unwrapped, which is what makes false reversible). + // false = release the suppression; already-armed cells serve compiled again + // on the next request, and later arming decisions run normally. + bool eager_only = 1; + string reason = 2; // operator's words, carried into the worker's event + string actor = 3; // who ordered it (admin principal / "cozy-local") +} diff --git a/src/gen_worker/activity.py b/src/gen_worker/activity.py index d23c2453..aeed491c 100644 --- a/src/gen_worker/activity.py +++ b/src/gen_worker/activity.py @@ -87,6 +87,14 @@ # ``detail`` naming the identifiers (ref/function/label/request) + the # exception. Fail-soft BEHAVIOR is unchanged — these are confessions. KIND_SERVE_DEGRADE = "serve_degrade" +# pgw#1142 / §4.32 item 4: the OPERATOR's eager-only order changed state on this +# worker. Two phases, both transitions and neither a degradation: +# `eager_only_engaged` (compiled serving suppressed — armed cells stay armed and +# are not called) and `eager_only_released`. Deliberately its own kind rather +# than a `serve_degrade` phase: every other member of that kind is something +# that went wrong, and an operator exercising a supported control must not land +# in a defect population the fleet reads as breakage. +KIND_SERVE_POSTURE = "serve_posture" KIND_LORA_HYGIENE = "lora_hygiene" # pgw#794: the serve-side adapter-fidelity gate. `phase=refused` is a # fail-CLOSED decision (the request also gets a typed error); `phase=degraded` diff --git a/src/gen_worker/aot_serve.py b/src/gen_worker/aot_serve.py index 4e586e80..84a941d0 100644 --- a/src/gen_worker/aot_serve.py +++ b/src/gen_worker/aot_serve.py @@ -95,6 +95,7 @@ from . import cell_key as cell_key_mod from . import host_isa from .cell_adopt import AdoptOutcome +from . import serve_posture from . import shape_growth from .compile_cache import ( AdoptError, @@ -2452,6 +2453,19 @@ def aot_forward(*args: Any, **kwargs: Any) -> Any: artifact_lora, eager_lora = adapter_call_kwargs(module, runner) eager_kwargs = {**kwargs, **eager_lora} kwargs = {**kwargs, **artifact_lora} + if serve_posture.eager_only(): + # pgw#1142 / §4.32 item 4: an operator ordered eager. This is THE + # reversibility seam — the artifact is not unwrapped and `state` + # is not touched, so releasing the order resumes compiled serving + # on the very next call with no re-arm, no re-materialize and no + # re-mint. Unwrapping here would have been the same posture for + # one boot and a lie afterwards. + # + # Ordered BEFORE the `failed` check only for cost; the two are + # independent and stay independent — releasing the order never + # resurrects a cell de-armed for cause (§4.31), because that + # de-arm is evidence and this is policy. + return original(*args, **eager_kwargs) if state["failed"]: return original(*args, **eager_kwargs) try: diff --git a/src/gen_worker/boot_adopt.py b/src/gen_worker/boot_adopt.py index 36dbb75d..82884d23 100644 --- a/src/gen_worker/boot_adopt.py +++ b/src/gen_worker/boot_adopt.py @@ -99,8 +99,19 @@ # boots without asking. Correct, and previously indistinguishable from a # boot-adopt that asked and was refused. "eager_only", + # pgw#1142 / §4.32 item 4: an OPERATOR ordered this worker eager-only, so + # the boot did not ask for a cell it is forbidden to call. Distinct from + # `eager_only` above because that one is a property of the pod that no + # command can lift, and this one is a decision that can be taken back — + # the same pod, re-asked after the order is released, adopts normally. + "operator_eager_only", ) +#: The reason token for :data:`GATE_REASONS`' operator entry. Named so the +#: executor references the vocabulary instead of re-typing the literal — the +#: drift channel `EagerPhase` was created to close, one module over. +OPERATOR_EAGER_ONLY = "operator_eager_only" + #: Step 1.5 (pgw#1127 S2) — THIS MACHINE's own store, addressed by the DERIVED #: key, before the hub is asked at all. §4.28: *"local cell, local repo-CAS, #: reused across its own boots — never uploaded, never requested."* diff --git a/src/gen_worker/cell_adopt.py b/src/gen_worker/cell_adopt.py index 31f06a3d..c681066d 100644 --- a/src/gen_worker/cell_adopt.py +++ b/src/gen_worker/cell_adopt.py @@ -166,6 +166,15 @@ class EagerPhase(StrEnum): #: `state_blocked_idle`, two replacements bought. ADOPTED_CELL_REFUSED = "adopted_cell_refused" + #: pgw#1142 / §4.32 item 4: an OPERATOR ordered this worker to serve eager + #: only, over the scheduler's control channel or the cozy-local CLI. It is + #: neither a defect nor a degradation — it is the answer, and it is + #: reversible — so it must never be counted with the failure classes above + #: or with `hub_ordered_eager` (which is one PLAN's backend, not a standing + #: order about this pod). A worker holding this token has cells it could be + #: serving from, still armed, deliberately not called. + OPERATOR_EAGER_ONLY = "operator_eager_only" + #: `_fail_closed`'s default, for a caller that has not classified its exit. #: A new decline landing here rather than on its own member is the #: regression pgw#824 exists to catch. diff --git a/src/gen_worker/cli/run.py b/src/gen_worker/cli/run.py index 734f8fe4..f5325cec 100644 --- a/src/gen_worker/cli/run.py +++ b/src/gen_worker/cli/run.py @@ -34,6 +34,7 @@ from .. import local_serve from ..api.errors import CanceledError +from .. import serve_posture from ..models import provision from .local_context import build_local_context from ..discovery.project import load_project_config @@ -103,6 +104,14 @@ def add_subparser(sub: argparse._SubParsersAction[Any]) -> None: "missing model weights from tensorhub / huggingface." ), ) + p.add_argument( + "--eager-only", dest="eager_only", action="store_true", + help=( + "Serve EAGER ONLY: never arm a compiled cell and never mint one " + "(DESIGN-RULINGS §4.32). One invocation is where a cold machine " + "would otherwise spend minutes minting." + ), + ) p.add_argument( "--device", dest="device", default=None, help="Override the torch device (e.g. 'cuda:0', 'cpu').", @@ -1068,6 +1077,11 @@ async def _drain() -> int: # -------------------------------------------------------------------------- def _handle_run(args: argparse.Namespace) -> int: + # pgw#1142 / §4.32 item 4: before anything can arm or mint. `run` does its + # setup() per invocation, so the order has to stand before the first one. + if bool(getattr(args, "eager_only", False)): + serve_posture.apply_command( + True, actor="cozy-local-cli", reason="--eager-only") try: return _run_inner(args) except _UsageError as e: diff --git a/src/gen_worker/cli/serve.py b/src/gen_worker/cli/serve.py index 462c248e..70e936c2 100644 --- a/src/gen_worker/cli/serve.py +++ b/src/gen_worker/cli/serve.py @@ -50,6 +50,7 @@ import msgspec from ..api.errors import CanceledError +from .. import serve_posture from ..discovery.names import slugify_name from ..models import memory from ..models.residency import Residency, Tier @@ -162,6 +163,17 @@ def add_subparser(sub: argparse._SubParsersAction[Any]) -> None: "fail-fast / pre-warming; default is lazy per-function loading." ), ) + p.add_argument( + "--eager-only", dest="eager_only", action="store_true", + help=( + "Serve EAGER ONLY: never arm a compiled cell and never mint one " + "(DESIGN-RULINGS §4.32). For a broken compile, or a machine whose " + "owner does not want minutes of it spent compiling. Reversible " + "while the serve runs — send {\"posture\":{\"eager_only\":false}} " + "on the socket. NOTE: unrelated to --eager, which is about when " + "setup() runs, not about how forwards execute." + ), + ) p.add_argument( "--vram-budget", dest="vram_budget", type=float, default=0.0, metavar="GB", @@ -682,6 +694,8 @@ def _parse_frame(line: bytes) -> Dict[str, Any]: Kinds: - ``{"kind":"request","function","payload","request_id"}`` - ``{"kind":"cancel","request_id"}`` (control frame: ``{"cancel":{...}}``) + - ``{"kind":"posture","eager_only","reason"}`` + (control frame: ``{"posture":{"eager_only":true,"reason":"..."}}``) - ``{"kind":"error","message"}`` """ try: @@ -690,6 +704,22 @@ def _parse_frame(line: bytes) -> Dict[str, Any]: return {"kind": "error", "message": f"request is not valid JSON: {e}"} if not isinstance(obj, dict): return {"kind": "error", "message": "request must be a JSON object"} + if "posture" in obj: + # pgw#1142 / §4.32 item 4: the cozy-local half of the eager-only + # command. It rides the control-frame shape `cancel` already + # established rather than opening a second channel — one socket, one + # protocol, and `cozy` learns one more frame instead of a new client. + p = obj.get("posture") or {} + if not isinstance(p, dict) or not isinstance(p.get("eager_only"), bool): + return { + "kind": "error", + "message": "posture.eager_only (boolean) is required", + } + return { + "kind": "posture", + "eager_only": bool(p["eager_only"]), + "reason": str(p.get("reason") or ""), + } if "cancel" in obj: c = obj.get("cancel") or {} rid = c.get("request_id") if isinstance(c, dict) else None @@ -736,6 +766,18 @@ def _emit_stdout(s: str) -> None: found = endpoint.interrupt_request(frame.get("request_id")) _write_response_line(_emit_stdout, {"ok": True, "canceled": found}) continue + if frame["kind"] == "posture": + changed = serve_posture.apply_command( + bool(frame["eager_only"]), + actor="cozy-local-cli", + reason=str(frame.get("reason") or ""), + ) + current = serve_posture.order() + _write_response_line(_emit_stdout, { + "ok": True, "eager_only": current.active, "changed": changed, + "posture": current.describe(), + }) + continue rid = frame.get("request_id") if frame.get("stream"): terminal = endpoint.dispatch( @@ -875,6 +917,27 @@ def _send(env: Dict[str, Any]) -> None: except OSError: pass return + if frame["kind"] == "posture": + # pgw#1142 / §4.32 item 4, the RUNTIME half: a warm serve holding a + # broken cell can be told to stop using it without being restarted, + # and told to use it again afterwards. The reply reports the posture + # that now stands, so a client never has to infer it. + changed = serve_posture.apply_command( + bool(frame["eager_only"]), + actor="cozy-local-cli", + reason=str(frame.get("reason") or ""), + ) + current = serve_posture.order() + try: + _send({ + "ok": True, + "eager_only": current.active, + "changed": changed, + "posture": current.describe(), + }) + except OSError: + pass + return if frame["kind"] == "error": try: _send(_error_envelope("usage", frame["message"])) @@ -988,6 +1051,14 @@ def _serve_inner(args: argparse.Namespace) -> int: candidates, getattr(args, "functions", None), ) + # pgw#1142 / §4.32 item 4: the order is installed BEFORE setup(), because + # setup() is where a cozy-local serve arms and (on a miss) mints. A flag + # that only took effect afterwards would still have spent the compile the + # operator asked us not to spend. + if bool(getattr(args, "eager_only", False)): + serve_posture.apply_command( + True, actor="cozy-local-cli", reason="--eager-only") + # 2. Boot the endpoint — setup() once per class, hold instances warm. endpoint = _Endpoint( offline=bool(args.offline), diff --git a/src/gen_worker/compile_cache.py b/src/gen_worker/compile_cache.py index 968ddaba..ccc63b31 100644 --- a/src/gen_worker/compile_cache.py +++ b/src/gen_worker/compile_cache.py @@ -73,7 +73,7 @@ from . import ( cell_key, dist_records, env_seal, guard_closure, hot_swap, - settings_authority, + serve_posture, settings_authority, ) from .api.errors import RetryableError from .models import w8a8_lora @@ -2328,6 +2328,18 @@ def arming_block( Deliberately side-effect free — :func:`apply` still owns the arming mutations; this only names. """ + # pgw#1142 / §4.32 item 4, and it is FIRST because it is the cheapest and + # the most authoritative: an operator has said this worker serves eager. + # Routing the command through the one precondition authority is what makes + # it suppress adoption, JIT intake, cold compile and every self-mint + # without a check per call site. Unlike every other reason here it is not + # deterministic for the life of the process — it can be released — which is + # sound for the callers that classify: a mint refused under the order is + # refused because the operator does not want one, and if that changes the + # next arming pass mints normally. + ordered_eager = serve_posture.block() + if ordered_eager: + return ordered_eager if _PROCESS_COMPILES_DISABLED: return f"process compiles are disabled: {_PROCESS_COMPILES_DISABLED}" if operator_eager_pin(pipeline): diff --git a/src/gen_worker/executor.py b/src/gen_worker/executor.py index 2050afa0..7d4a6dee 100644 --- a/src/gen_worker/executor.py +++ b/src/gen_worker/executor.py @@ -53,6 +53,7 @@ from . import mint_budget from . import settings_authority from . import progress as progress_mod +from . import serve_posture from . import serving_mode as serving_mode_mod from . import warmup from . import worker_credential @@ -6887,7 +6888,22 @@ async def _setup_locked_inner( component_paths: Dict[str, Dict[str, str]] = { slot: dict(res.component_paths) for slot, res in resolved_slots.items() if res.component_paths} - eager_only = self._eager_only_reason() + topology_eager = self._eager_only_reason() + # pgw#1142 / §4.32 item 4. The order joins the topology reason for + # every "do not go looking for a cell" decision below — this is the + # gate that runs BEFORE the hub round trip and the materialize, so an + # operator who says "stop compiling" is obeyed at the first boot phase + # rather than at the arm, having paid for a download in between. + # + # It deliberately does NOT join the REFUSAL two blocks down. The two + # are different in kind: a degree>1 topology CANNOT run the named cell + # (the collectives would hang), so a spec naming one is unsatisfiable + # and must fail typed; an operator order is a decision that the pod + # serve eager, and `arm_ordered` obeys it by arming nothing and + # serving — killing the function instead would be the opposite of what + # was asked for, and it would not be reversible. + ordered_eager = serve_posture.block() + eager_only = topology_eager or ordered_eager if eager_only and spec.compile is not None: logger.info("%s: %s", spec.name, eager_only) # pgw#904: the ONLY source of a pre-materialized artifact is a Plan's @@ -6895,10 +6911,10 @@ async def _setup_locked_inner( # digest-verified it). The connected snapshot scan that used to run # here is deleted — the hub no longer attaches cells to snapshots, and # a worker that could pick one would be a second resolver. - if arm is not None and arm.backend == "aot_cell" and eager_only: + if arm is not None and arm.backend == "aot_cell" and topology_eager: raise compile_cache.CompiledExecutionLaneUnavailableError( f"the spec names an exact cell but this pod cannot arm one: " - f"{eager_only}") + f"{topology_eager}") compile_selection = arm.selection if arm is not None else None compile_artifact = compile_selection.path if compile_selection else None # §4.27 steps 1-3 (pgw#1089/pgw#1090): with no Plan-named artifact, this @@ -6947,7 +6963,13 @@ async def _setup_locked_inner( # so it says so, rather than being the ninth way to look like a pod # that quietly self-minted. boot_adopt.refused( - "eager_only", eager_only, + # pgw#1142: WHICH eager, named. A pod that never asked because + # its topology forbids arming and a pod that never asked + # because an operator said so are two different boots, and the + # second one can be undone. + boot_adopt.OPERATOR_EAGER_ONLY if not topology_eager + else "eager_only", + eager_only, family=str(getattr(spec.compile, "family", "") or ""), function=str(spec.name or "")) # pgw#947: the serving-kernel lane comes from the CELL, and it has to diff --git a/src/gen_worker/fleet_cells.py b/src/gen_worker/fleet_cells.py index 803c708f..e2645014 100644 --- a/src/gen_worker/fleet_cells.py +++ b/src/gen_worker/fleet_cells.py @@ -69,6 +69,7 @@ from . import aot_identity, aot_serve, artifact_meta, cell_key, env_seal from . import boot_phases as boot_mod from . import local_cell_store +from . import serve_posture from . import compile_cache as cc from .cell_adopt import AdoptOutcome, CellAdoption, EagerPhase from .models.chunk_cas import sha256_file @@ -1216,6 +1217,16 @@ def enable_compiled( frame that made it, which is why the successful ones were narrated and the measured ones never sent. """ + if serve_posture.eager_only(): + # pgw#1142 / §4.32 item 4. `arming_block` would refuse every arm below + # anyway, but the policy would first resolve, download and materialize + # a cell to hand to a gate that has already decided — and on a MISS it + # would open a self-mint whose child re-derives the refusal minutes and + # a full compile later. The order is answered where it costs nothing, + # and it is answered with a token, not a bare False. + logger.info("fleet-cells: %s", serve_posture.block()) + return ArmOutcome( + armed=False, eager_reason=EagerPhase.OPERATOR_EAGER_ONLY) adoptions: List[CellAdoption] = [] outcome = _arming_policy( pipe, cfg, cache_dir, artifact, @@ -1265,6 +1276,18 @@ def arm_ordered( ``stage_artifact`` via ``expected``. ``dynamo`` arms JIT intake. ``eager_only`` arms nothing, by order. """ + if serve_posture.eager_only(): + # pgw#1142 / §4.32 item 4: the operator's order outranks the hub's + # Plan, and this is the only place in the ORDERED path where that is + # true. It is not a refusal — `OrderedArmError` would fail the attempt + # typed, and the operator asked for eager service, not for the function + # to go down — so the order is obeyed the way `eager_only` below is + # obeyed: arm nothing, serve eager, say which of the two eager orders + # it was. + logger.info("arm-ordered: %s", serve_posture.block()) + return ArmOutcome( + armed=False, eager_reason=EagerPhase.OPERATOR_EAGER_ONLY) + bucket = int(getattr(cfg, "lora_bucket", 0) or 0) if bucket and not cc.has_compile_target(pipe, cfg): bucket = 0 # bare component slot (gw#627): branchless-eager, not an error diff --git a/src/gen_worker/lifecycle.py b/src/gen_worker/lifecycle.py index cc8198bf..2cc7c462 100644 --- a/src/gen_worker/lifecycle.py +++ b/src/gen_worker/lifecycle.py @@ -16,6 +16,7 @@ from . import boot_phases as boot_mod from . import content_credentials from . import receipts +from . import serve_posture from . import mint_goal as mint_goal_mod from . import progress as progress_mod from . import worker_goals @@ -1052,6 +1053,18 @@ async def on_message(self, msg: pb.SchedulerMessage) -> None: "ignoring retired ModelOp (pgw#1032): hot compile-cache " "adoption is gone; a cell arrives only as a Plan's exact " "Arm.artifact (pgw#904) — the hub never pushes one") + elif which == "serve_posture": + # pgw#1142 / §4.32 item 4: the operator's eager-only command. + # Applied SYNCHRONOUSLY and unconditionally — it touches one + # process-global order and never blocks, so there is no state in + # which a worker is "too busy" to stop using a broken cell, and + # in-flight requests pick it up at their next compiled call. + # Idempotent, so the hub replaying it on reconnect is free. + serve_posture.apply_command( + bool(msg.serve_posture.eager_only), + actor=str(msg.serve_posture.actor or ""), + reason=str(msg.serve_posture.reason or ""), + ) elif which == "drain": # The hub asked, explicitly, for this budget on this drain — an # operator budget on a command, which pgw#887 keeps. diff --git a/src/gen_worker/pb/worker_scheduler_pb2.py b/src/gen_worker/pb/worker_scheduler_pb2.py index 06b6425d..84d1a2ea 100644 --- a/src/gen_worker/pb/worker_scheduler_pb2.py +++ b/src/gen_worker/pb/worker_scheduler_pb2.py @@ -24,7 +24,7 @@ -DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n\x16worker_scheduler.proto\x12\x11\x63ozy.scheduler.v1\"\xf7\x05\n\rWorkerMessage\x12)\n\x05hello\x18\x01 \x01(\x0b\x32\x18.cozy.scheduler.v1.HelloH\x00\x12\x34\n\x0bstate_delta\x18\x02 \x01(\x0b\x32\x1d.cozy.scheduler.v1.StateDeltaH\x00\x12\x36\n\x0cjob_accepted\x18\x03 \x01(\x0b\x32\x1e.cozy.scheduler.v1.JobAcceptedH\x00\x12\x32\n\njob_result\x18\x04 \x01(\x0b\x32\x1c.cozy.scheduler.v1.JobResultH\x00\x12\x36\n\x0cjob_progress\x18\x05 \x01(\x0b\x32\x1e.cozy.scheduler.v1.JobProgressH\x00\x12\x34\n\x0bmodel_event\x18\x06 \x01(\x0b\x32\x1d.cozy.scheduler.v1.ModelEventH\x00\x12:\n\x0e\x66n_unavailable\x18\x07 \x01(\x0b\x32 .cozy.scheduler.v1.FnUnavailableH\x00\x12\x34\n\x0b\x66n_degraded\x18\x08 \x01(\x0b\x32\x1d.cozy.scheduler.v1.FnDegradedH\x00\x12<\n\x0f\x61\x63tivity_update\x18\t \x01(\x0b\x32!.cozy.scheduler.v1.ActivityUpdateH\x00\x12\x44\n\x13hardware_unsuitable\x18\n \x01(\x0b\x32%.cozy.scheduler.v1.HardwareUnsuitableH\x00\x12\x36\n\x0cgoal_receipt\x18\x0b \x01(\x0b\x32\x1e.cozy.scheduler.v1.GoalReceiptH\x00\x12\x42\n\x12lifecycle_snapshot\x18\x0c \x01(\x0b\x32$.cozy.scheduler.v1.LifecycleSnapshotH\x00\x12\x32\n\nboot_phase\x18\r \x01(\x0b\x32\x1c.cozy.scheduler.v1.BootPhaseH\x00\x42\x05\n\x03msg\"\xf8\x02\n\x10SchedulerMessage\x12\x30\n\thello_ack\x18\x01 \x01(\x0b\x32\x1b.cozy.scheduler.v1.HelloAckH\x00\x12,\n\x07run_job\x18\x02 \x01(\x0b\x32\x19.cozy.scheduler.v1.RunJobH\x00\x12\x32\n\ncancel_job\x18\x03 \x01(\x0b\x32\x1c.cozy.scheduler.v1.CancelJobH\x00\x12.\n\x08model_op\x18\x04 \x01(\x0b\x32\x1a.cozy.scheduler.v1.ModelOpH\x00\x12)\n\x05\x64rain\x18\x05 \x01(\x0b\x32\x18.cozy.scheduler.v1.DrainH\x00\x12\x38\n\rtoken_refresh\x18\x06 \x01(\x0b\x32\x1f.cozy.scheduler.v1.TokenRefreshH\x00\x12\x34\n\x0brun_attempt\x18\x07 \x01(\x0b\x32\x1d.cozy.scheduler.v1.RunAttemptH\x00\x42\x05\n\x03msg\"\xb3\x03\n\x05Hello\x12<\n\x10protocol_version\x18\x01 \x01(\x0e\x32\".cozy.scheduler.v1.ProtocolVersion\x12\x11\n\tworker_id\x18\x02 \x01(\t\x12\x12\n\nrelease_id\x18\x03 \x01(\t\x12\x35\n\tresources\x18\x04 \x01(\x0b\x32\".cozy.scheduler.v1.WorkerResources\x12,\n\x05state\x18\x05 \x01(\x0b\x32\x1d.cozy.scheduler.v1.StateDelta\x12\x31\n\x06models\x18\x06 \x03(\x0b\x32!.cozy.scheduler.v1.ModelResidency\x12\x31\n\tin_flight\x18\x07 \x03(\x0b\x32\x1e.cozy.scheduler.v1.InFlightJob\x12\x1d\n\x15heartbeat_interval_ms\x18\x08 \x01(\x04\x12\x19\n\x11worker_session_id\x18\t \x01(\t\x12@\n\x12lifecycle_snapshot\x18\n \x01(\x0b\x32$.cozy.scheduler.v1.LifecycleSnapshot\"\xcb\x02\n\x0fWorkerResources\x12\x11\n\tgpu_count\x18\x01 \x01(\x05\x12\x18\n\x10vram_total_bytes\x18\x02 \x01(\x03\x12\x10\n\x08gpu_name\x18\x03 \x01(\t\x12\x0e\n\x06gpu_sm\x18\x04 \x01(\t\x12\x16\n\x0einstalled_libs\x18\x05 \x03(\t\x12\x14\n\x0cimage_digest\x18\x06 \x01(\t\x12\x12\n\ngit_commit\x18\x07 \x01(\t\x12\x13\n\x0binstance_id\x18\x08 \x01(\t\x12\x32\n\x0bhost_canary\x18\t \x01(\x0b\x32\x1d.cozy.scheduler.v1.HostCanary\x12\x15\n\rtorch_version\x18\n \x01(\t\x12\x1a\n\x12gen_worker_version\x18\x0b \x01(\t\x12\x13\n\x0bworker_mode\x18\x0c \x01(\t\x12\x16\n\x0e\x64river_version\x18\r \x01(\t\"\xa2\x02\n\x12HardwareUnsuitable\x12\x11\n\tworker_id\x18\x01 \x01(\t\x12\x12\n\nrelease_id\x18\x02 \x01(\t\x12\x14\n\x0creason_class\x18\x03 \x01(\t\x12\x0e\n\x06\x64\x65tail\x18\x04 \x01(\t\x12\x16\n\x0e\x64river_version\x18\x05 \x01(\t\x12\x10\n\x08gpu_name\x18\x06 \x01(\t\x12\x15\n\rtorch_version\x18\x07 \x01(\t\x12\x1a\n\x12torch_cuda_version\x18\x08 \x01(\t\x12\x1a\n\x12gen_worker_version\x18\t \x01(\t\x12\x14\n\x0cimage_digest\x18\n \x01(\t\x12\x13\n\x0binstance_id\x18\x0b \x01(\t\x12\x1b\n\x13reported_at_unix_ms\x18\x0c \x01(\x03\"\x88\x02\n\nHostCanary\x12\x13\n\x0bmemcpy_gbps\x18\x01 \x01(\x01\x12\x10\n\x08\x64\x32h_gbps\x18\x02 \x01(\x01\x12\x17\n\x0fpinned_alloc_ok\x18\x03 \x01(\x08\x12\x17\n\x0f\x63pu_single_mbps\x18\x04 \x01(\x01\x12\x16\n\x0e\x63pu_multi_mbps\x18\x05 \x01(\x01\x12\r\n\x05vcpus\x18\x06 \x01(\x05\x12\x14\n\x0cram_total_gb\x18\x07 \x01(\x01\x12\x13\n\x0b\x64uration_ms\x18\x08 \x01(\x03\x12\x14\n\x0cinterconnect\x18\t \x01(\t\x12\x11\n\tpeer_gbps\x18\n \x01(\x01\x12\x13\n\x0bpeer_access\x18\x0b \x01(\x08\x12\x11\n\ttopo_link\x18\x0c \x01(\t\"\x98\x01\n\x0eModelResidency\x12\x0b\n\x03ref\x18\x01 \x01(\t\x12.\n\x04tier\x18\x02 \x01(\x0e\x32 .cozy.scheduler.v1.ResidencyTier\x12\x12\n\nvram_bytes\x18\x03 \x01(\x03\x12\x17\n\x0fsnapshot_digest\x18\x04 \x01(\t\x12\x1c\n\x14residency_generation\x18\x05 \x01(\x04\"2\n\x0bInFlightJob\x12\x12\n\nrequest_id\x18\x01 \x01(\t\x12\x0f\n\x07\x61ttempt\x18\x02 \x01(\x03\"\x9f\x02\n\x08HelloAck\x12<\n\x10protocol_version\x18\x01 \x01(\x0e\x32\".cozy.scheduler.v1.ProtocolVersion\x12\x15\n\rfile_base_url\x18\x02 \x01(\t\x12\x37\n\x0bresolutions\x18\x03 \x03(\x0b\x32\".cozy.scheduler.v1.ModelResolution\x12>\n\x11\x64\x65sired_residency\x18\x04 \x01(\x0b\x32#.cozy.scheduler.v1.DesiredResidency\x12\x45\n\x15\x64\x65sired_state_command\x18\x05 \x01(\x0b\x32&.cozy.scheduler.v1.DesiredStateCommand\"\xaf\x02\n\x10\x44\x65siredResidency\x12\x12\n\ngeneration\x18\x01 \x01(\x04\x12\x11\n\tdisk_refs\x18\x02 \x03(\t\x12/\n\x03hot\x18\x03 \x03(\x0b\x32\".cozy.scheduler.v1.DesiredInstance\x12\x45\n\tsnapshots\x18\x04 \x03(\x0b\x32\x32.cozy.scheduler.v1.DesiredResidency.SnapshotsEntry\x12\x12\n\nrelease_id\x18\x05 \x01(\t\x12\x19\n\x11\x63onfig_generation\x18\x06 \x01(\x04\x1aM\n\x0eSnapshotsEntry\x12\x0b\n\x03key\x18\x01 \x01(\t\x12*\n\x05value\x18\x02 \x01(\x0b\x32\x1b.cozy.scheduler.v1.Snapshot:\x02\x38\x01\"\x99\x03\n\x13\x44\x65siredStateCommand\x12\x19\n\x11worker_session_id\x18\x01 \x01(\t\x12\x13\n\x0b\x63ommand_seq\x18\x02 \x01(\x04\x12\x0f\n\x07goal_id\x18\x03 \x01(\t\x12\x12\n\nrelease_id\x18\x04 \x01(\t\x12\x19\n\x11\x63onfig_generation\x18\x05 \x01(\x04\x12\x15\n\rconfig_digest\x18\x06 \x01(\x0c\x12\x19\n\x11issued_at_unix_ms\x18\x07 \x01(\x03\x12\x19\n\x11\x61\x63\x63\x65pt_by_unix_ms\x18\x08 \x01(\x03\x12\x1f\n\x17\x66irst_action_by_unix_ms\x18\t \x01(\x03\x12\x31\n\x07intents\x18\n \x03(\x0b\x32 .cozy.scheduler.v1.DesiredIntent\x12\x11\n\tmandatory\x18\x0b \x01(\x08\x12\x42\n\x16\x63hanged_config_classes\x18\x0c \x01(\x0b\x32\".cozy.scheduler.v1.ConfigClassMask\x12\x1a\n\x12parameter_snapshot\x18\r \x01(\x0c\"\x95\x03\n\rDesiredIntent\x12\x11\n\tintent_id\x18\x01 \x01(\t\x12\x32\n\x04kind\x18\x02 \x01(\x0e\x32$.cozy.scheduler.v1.DesiredIntentKind\x12\x34\n\x05\x63\x61use\x18\x03 \x01(\x0e\x32%.cozy.scheduler.v1.DesiredIntentCause\x12\x15\n\rfunction_name\x18\x04 \x01(\t\x12\x0b\n\x03ref\x18\x05 \x01(\t\x12\x17\n\x0fsnapshot_digest\x18\x06 \x01(\x0c\x12\x36\n\x0c\x64\x65sired_tier\x18\x07 \x01(\x0e\x32 .cozy.scheduler.v1.ResidencyTier\x12\x16\n\x0e\x62inding_digest\x18\x08 \x01(\x0c\x12\x18\n\x10parent_intent_id\x18\t \x01(\t\x12;\n\x10waiting_requests\x18\n \x03(\x0b\x32!.cozy.scheduler.v1.RequestAttempt\x12\x10\n\x08priority\x18\x0b \x01(\x05\x12\x11\n\tmandatory\x18\x0c \x01(\x08\"5\n\x0eRequestAttempt\x12\x12\n\nrequest_id\x18\x01 \x01(\t\x12\x0f\n\x07\x61ttempt\x18\x02 \x01(\x03\"\xd0\x02\n\x0bGoalReceipt\x12\x19\n\x11worker_session_id\x18\x01 \x01(\t\x12\x13\n\x0b\x63ommand_seq\x18\x02 \x01(\x04\x12\x0f\n\x07goal_id\x18\x03 \x01(\t\x12\x12\n\nrelease_id\x18\x04 \x01(\t\x12\x34\n\x06status\x18\x05 \x01(\x0e\x32$.cozy.scheduler.v1.GoalReceiptStatus\x12\x39\n\nerror_code\x18\x06 \x01(\x0e\x32%.cozy.scheduler.v1.LifecycleErrorCode\x12\x36\n\nrejections\x18\x07 \x03(\x0b\x32\".cozy.scheduler.v1.IntentRejection\x12\x0e\n\x06\x64\x65tail\x18\x08 \x01(\t\x12\x1b\n\x13received_at_unix_ms\x18\t \x01(\x03\x12\x16\n\x0e\x63ommand_digest\x18\n \x01(\x0c\"o\n\x0fIntentRejection\x12\x11\n\tintent_id\x18\x01 \x01(\t\x12\x39\n\nerror_code\x18\x02 \x01(\x0e\x32%.cozy.scheduler.v1.LifecycleErrorCode\x12\x0e\n\x06\x64\x65tail\x18\x03 \x01(\t\"R\n\x11LifecycleProgress\x12\x0c\n\x04\x64one\x18\x01 \x01(\x01\x12\r\n\x05total\x18\x02 \x01(\x01\x12\x0c\n\x04unit\x18\x03 \x01(\t\x12\x12\n\nrate_per_s\x18\x04 \x01(\x01\"\x98\x05\n\x0bIntentState\x12\x19\n\x11worker_session_id\x18\x01 \x01(\t\x12\x11\n\tstate_seq\x18\x02 \x01(\x04\x12\x0f\n\x07goal_id\x18\x03 \x01(\t\x12\x11\n\tintent_id\x18\x04 \x01(\t\x12\x12\n\nrelease_id\x18\x05 \x01(\t\x12\x19\n\x11\x63onfig_generation\x18\x06 \x01(\x04\x12\x38\n\x06status\x18\x07 \x01(\x0e\x32(.cozy.scheduler.v1.LifecycleIntentStatus\x12\x36\n\x05stage\x18\x08 \x01(\x0e\x32\'.cozy.scheduler.v1.LifecycleIntentStage\x12\x36\n\x06reason\x18\t \x01(\x0e\x32&.cozy.scheduler.v1.LifecycleWaitReason\x12\x15\n\rsince_unix_ms\x18\n \x01(\x03\x12\x1a\n\x12updated_at_unix_ms\x18\x0b \x01(\x03\x12\x1d\n\x15next_retry_at_unix_ms\x18\x0c \x01(\x03\x12\x1b\n\x13\x64\x65\x61\x64line_at_unix_ms\x18\r \x01(\x03\x12\x19\n\x11\x62locker_intent_id\x18\x0e \x01(\t\x12:\n\x0f\x62locker_request\x18\x0f \x01(\x0b\x32!.cozy.scheduler.v1.RequestAttempt\x12\x36\n\x08progress\x18\x10 \x01(\x0b\x32$.cozy.scheduler.v1.LifecycleProgress\x12\x39\n\nerror_code\x18\x11 \x01(\x0e\x32%.cozy.scheduler.v1.LifecycleErrorCode\x12\x0e\n\x06\x64\x65tail\x18\x12 \x01(\t\x12\x15\n\ractual_digest\x18\x13 \x01(\x0c\"\x83\x01\n\rModelIdentity\x12\x0b\n\x03ref\x18\x01 \x01(\t\x12\x17\n\x0fsnapshot_digest\x18\x02 \x01(\x0c\x12.\n\x04tier\x18\x03 \x01(\x0e\x32 .cozy.scheduler.v1.ResidencyTier\x12\x1c\n\x14residency_generation\x18\x04 \x01(\x04\"\xaa\x02\n\x12\x46unctionCapability\x12\x15\n\rfunction_name\x18\x01 \x01(\t\x12\x12\n\nrelease_id\x18\x02 \x01(\t\x12\x19\n\x11\x63onfig_generation\x18\x03 \x01(\x04\x12\x16\n\x0e\x62inding_digest\x18\x04 \x01(\x0c\x12\x0c\n\x04lane\x18\x05 \x01(\t\x12\x30\n\x06models\x18\x06 \x03(\x0b\x32 .cozy.scheduler.v1.ModelIdentity\x12%\n\x1d\x63ompile_target_incarnation_id\x18\x07 \x01(\t\x12\x39\n\x05state\x18\x08 \x01(\x0e\x32*.cozy.scheduler.v1.FunctionCapabilityState\x12\x14\n\x0cserving_tier\x18\t \x01(\t\"E\n\x0f\x43onfigClassMask\x12\x12\n\nparameters\x18\x01 \x01(\x08\x12\x10\n\x08\x62indings\x18\x02 \x01(\x08\x12\x0c\n\x04\x62oot\x18\x03 \x01(\x08\"\xf3\x02\n\x11\x43onfigApplication\x12\x12\n\nrelease_id\x18\x01 \x01(\t\x12\x19\n\x11target_generation\x18\x02 \x01(\x04\x12\x1b\n\x13received_generation\x18\x03 \x01(\x04\x12%\n\x1dparameter_snapshot_generation\x18\x04 \x01(\x04\x12 \n\x18\x62inding_ready_generation\x18\x05 \x01(\x04\x12\x17\n\x0f\x62oot_generation\x18\x06 \x01(\x04\x12\x38\n\x05state\x18\x07 \x01(\x0e\x32).cozy.scheduler.v1.ConfigApplicationState\x12;\n\x0fpending_classes\x18\x08 \x01(\x0b\x32\".cozy.scheduler.v1.ConfigClassMask\x12\x39\n\nerror_code\x18\t \x01(\x0e\x32%.cozy.scheduler.v1.LifecycleErrorCode\"\x89\x02\n\x0f\x44rainProjection\x12\x0f\n\x07goal_id\x18\x01 \x01(\t\x12\x11\n\tintent_id\x18\x02 \x01(\t\x12\x37\n\x06status\x18\x03 \x01(\x0e\x32\'.cozy.scheduler.v1.DrainLifecycleStatus\x12\x15\n\rsince_unix_ms\x18\x04 \x01(\x03\x12\x1a\n\x12updated_at_unix_ms\x18\x05 \x01(\x03\x12\x1b\n\x13\x64\x65\x61\x64line_at_unix_ms\x18\x06 \x01(\x03\x12\x39\n\nerror_code\x18\x07 \x01(\x0e\x32%.cozy.scheduler.v1.LifecycleErrorCode\x12\x0e\n\x06\x64\x65tail\x18\x08 \x01(\t\"\x8f\x03\n\x11LifecycleSnapshot\x12\x19\n\x11worker_session_id\x18\x01 \x01(\t\x12\x11\n\tstate_seq\x18\x02 \x01(\x04\x12/\n\x07intents\x18\x03 \x03(\x0b\x32\x1e.cozy.scheduler.v1.IntentState\x12;\n\x0c\x63\x61pabilities\x18\x04 \x03(\x0b\x32%.cozy.scheduler.v1.FunctionCapability\x12@\n\x12\x63onfig_application\x18\x05 \x01(\x0b\x32$.cozy.scheduler.v1.ConfigApplication\x12\x35\n\rgoal_receipts\x18\x06 \x03(\x0b\x32\x1e.cozy.scheduler.v1.GoalReceipt\x12\x31\n\x05\x64rain\x18\x07 \x01(\x0b\x32\".cozy.scheduler.v1.DrainProjection\x12\x14\n\x0c\x66ull_replace\x18\x08 \x01(\x08\x12\x1c\n\x14generated_at_unix_ms\x18\t \x01(\x03\"Y\n\x0f\x44\x65siredInstance\x12\x15\n\rfunction_name\x18\x01 \x01(\t\x12/\n\x06models\x18\x02 \x03(\x0b\x32\x1f.cozy.scheduler.v1.ModelBinding\"e\n\x0fModelResolution\x12\x0b\n\x03ref\x18\x01 \x01(\t\x12\x14\n\x0cresolved_ref\x18\x02 \x01(\t\x12\x0c\n\x04\x63\x61st\x18\x03 \x01(\t\x12\x0c\n\x04lane\x18\x04 \x01(\t\x12\x13\n\x0blane_pinned\x18\x05 \x01(\x08\"\x98\x03\n\nStateDelta\x12-\n\x05phase\x18\x01 \x01(\x0e\x32\x1e.cozy.scheduler.v1.WorkerPhase\x12\x1b\n\x13\x61vailable_functions\x18\x02 \x03(\t\x12\x19\n\x11loading_functions\x18\x03 \x03(\t\x12\x17\n\x0f\x66ree_vram_bytes\x18\x04 \x01(\x03\x12\x17\n\x0f\x66inalizing_jobs\x18\x05 \x01(\x05\x12%\n\x1dobserved_residency_generation\x18\x06 \x01(\x04\x12\x39\n\x0f\x63ompile_targets\x18\x07 \x03(\x0b\x32 .cozy.scheduler.v1.CompileTarget\x12\x33\n\x0c\x63\x65ll_lookups\x18\x08 \x03(\x0b\x32\x1d.cozy.scheduler.v1.CellLookup\x12\x36\n\ndisk_usage\x18\t \x01(\x0b\x32\".cozy.scheduler.v1.DiskUsageReport\x12\"\n\x1aobserved_config_generation\x18\n \x01(\x04\"\xac\x01\n\x10StorageTierUsage\x12,\n\x04tier\x18\x01 \x01(\x0e\x32\x1e.cozy.scheduler.v1.StorageTier\x12\x12\n\nmount_path\x18\x02 \x01(\t\x12\x13\n\x0btotal_bytes\x18\x03 \x01(\x03\x12\x12\n\nfree_bytes\x18\x04 \x01(\x03\x12\x12\n\nused_bytes\x18\x05 \x01(\x03\x12\x19\n\x11reclaimable_bytes\x18\x06 \x01(\x03\"b\n\x0f\x44iskUsageReport\x12\x32\n\x05tiers\x18\x01 \x03(\x0b\x32#.cozy.scheduler.v1.StorageTierUsage\x12\x1b\n\x13\x63\x61pacity_generation\x18\x02 \x01(\x04\".\n\nCellLookup\x12\x0e\n\x06\x66\x61mily\x18\x01 \x01(\t\x12\x10\n\x08\x63\x65ll_key\x18\x02 \x01(\t\"\xcc\x03\n\rCompileTarget\x12\x16\n\x0eincarnation_id\x18\x01 \x01(\t\x12\x0e\n\x06\x66\x61mily\x18\x02 \x01(\t\x12\x1c\n\x14pipeline_weight_lane\x18\x03 \x01(\t\x12\x13\n\x0blora_bucket\x18\x04 \x01(\x05\x12\x17\n\x0f\x63ontract_digest\x18\x05 \x01(\t\x12\x1a\n\x12\x61\x63tive_compile_ref\x18\x06 \x01(\t\x12&\n\x1e\x61\x63tive_compile_snapshot_digest\x18\x07 \x01(\t\x12\x16\n\x0e\x66unction_names\x18\x08 \x03(\t\x12?\n\x0emodel_bindings\x18\t \x03(\x0b\x32\'.cozy.scheduler.v1.CompileTargetBinding\x12\x1a\n\x12requested_cell_key\x18\n \x01(\t\x12T\n\x13requested_cell_axes\x18\x0b \x03(\x0b\x32\x37.cozy.scheduler.v1.CompileTarget.RequestedCellAxesEntry\x1a\x38\n\x16RequestedCellAxesEntry\x12\x0b\n\x03key\x18\x01 \x01(\t\x12\r\n\x05value\x18\x02 \x01(\t:\x02\x38\x01\"J\n\x14\x43ompileTargetBinding\x12\x0c\n\x04slot\x18\x01 \x01(\t\x12\x0b\n\x03ref\x18\x02 \x01(\t\x12\x17\n\x0fsnapshot_digest\x18\x03 \x01(\t\"\x8c\x05\n\x06RunJob\x12\x12\n\nrequest_id\x18\x01 \x01(\t\x12\x0f\n\x07\x61ttempt\x18\x02 \x01(\x03\x12\x15\n\rfunction_name\x18\x03 \x01(\t\x12\x15\n\rinput_payload\x18\x04 \x01(\x0c\x12\x12\n\ntimeout_ms\x18\x05 \x01(\x03\x12\x0b\n\x03org\x18\x06 \x01(\t\x12\x12\n\ninvoker_id\x18\x07 \x01(\t\x12\x18\n\x10\x63\x61pability_token\x18\x08 \x01(\t\x12\x32\n\x0boutput_mode\x18\t \x01(\x0e\x32\x1d.cozy.scheduler.v1.OutputMode\x12\x33\n\x07\x63ompute\x18\n \x01(\x0b\x32\".cozy.scheduler.v1.ResolvedCompute\x12/\n\x06models\x18\x0b \x03(\x0b\x32\x1f.cozy.scheduler.v1.ModelBinding\x12;\n\tsnapshots\x18\x0c \x03(\x0b\x32(.cozy.scheduler.v1.RunJob.SnapshotsEntry\x12\x45\n\x10required_compile\x18\r \x01(\x0b\x32+.cozy.scheduler.v1.RequiredCompileExecution\x12\x0c\n\x04lane\x18\x0e \x01(\t\x12\x33\n\x0cinput_assets\x18\x0f \x03(\x0b\x32\x1d.cozy.scheduler.v1.InputAsset\x12\x19\n\x11\x63onfig_generation\x18\x10 \x01(\x04\x12\x15\n\rconfig_params\x18\x11 \x01(\x0c\x1aM\n\x0eSnapshotsEntry\x12\x0b\n\x03key\x18\x01 \x01(\t\x12*\n\x05value\x18\x02 \x01(\x0b\x32\x1b.cozy.scheduler.v1.Snapshot:\x02\x38\x01\"w\n\nInputAsset\x12\x10\n\x08\x61sset_id\x18\x01 \x01(\t\x12\x12\n\nsource_ref\x18\x02 \x01(\t\x12\x0e\n\x06\x62lake3\x18\x03 \x01(\t\x12\x12\n\nsize_bytes\x18\x04 \x01(\x03\x12\x0c\n\x04kind\x18\x05 \x01(\t\x12\x11\n\tmime_type\x18\x06 \x01(\t\"\x82\x01\n\x18RequiredCompileExecution\x12\x1d\n\x15target_incarnation_id\x18\x01 \x01(\t\x12\x10\n\x08\x63\x65ll_ref\x18\x02 \x01(\t\x12\x1c\n\x14\x63\x65ll_snapshot_digest\x18\x03 \x01(\t\x12\x17\n\x0f\x63ontract_digest\x18\x04 \x01(\t\"9\n\x0fResolvedCompute\x12\x13\n\x0b\x61\x63\x63\x65lerator\x18\x01 \x01(\t\x12\x11\n\tgpu_index\x18\x02 \x01(\x05\"\xac\x02\n\x0cModelBinding\x12\x0c\n\x04slot\x18\x01 \x01(\t\x12\x0b\n\x03ref\x18\x02 \x01(\t\x12-\n\x05loras\x18\x03 \x03(\x0b\x32\x1e.cozy.scheduler.v1.LoraOverlay\x12\x1a\n\x12inference_defaults\x18\x04 \x01(\t\x12\x43\n\ncomponents\x18\x05 \x03(\x0b\x32/.cozy.scheduler.v1.ModelBinding.ComponentsEntry\x12\x11\n\tobjective\x18\x06 \x01(\t\x12\x11\n\tdistilled\x18\x07 \x01(\x08\x12\x18\n\x10\x64istilled_status\x18\x08 \x01(\t\x1a\x31\n\x0f\x43omponentsEntry\x12\x0b\n\x03key\x18\x01 \x01(\t\x12\r\n\x05value\x18\x02 \x01(\t:\x02\x38\x01\"F\n\x0bLoraOverlay\x12\x0b\n\x03ref\x18\x01 \x01(\t\x12\x0e\n\x06weight\x18\x02 \x01(\x01\x12\x1a\n\x12inference_defaults\x18\x03 \x01(\t\"J\n\x08Snapshot\x12\x0e\n\x06\x64igest\x18\x01 \x01(\t\x12.\n\x05\x66iles\x18\x02 \x03(\x0b\x32\x1f.cozy.scheduler.v1.SnapshotFile\"\xa4\x01\n\x0cSnapshotFile\x12\x0c\n\x04path\x18\x01 \x01(\t\x12\x12\n\nsize_bytes\x18\x02 \x01(\x03\x12\x0e\n\x06\x62lake3\x18\x03 \x01(\t\x12\x0b\n\x03url\x18\x04 \x01(\t\x12\x0e\n\x06\x64igest\x18\x05 \x01(\t\x12\x18\n\x10\x63hunk_size_bytes\x18\x06 \x01(\x03\x12+\n\x06\x63hunks\x18\x07 \x03(\x0b\x32\x1b.cozy.scheduler.v1.ChunkRef\"4\n\x08\x43hunkRef\x12\x0e\n\x06sha256\x18\x01 \x01(\t\x12\x0b\n\x03url\x18\x02 \x01(\t\x12\x0b\n\x03len\x18\x03 \x01(\x03\"\xb3\x01\n\nRunAttempt\x12-\n\x07\x61ttempt\x18\x01 \x01(\x0b\x32\x1c.cozy.scheduler.v1.AttemptId\x12.\n\x04spec\x18\x02 \x01(\x0b\x32 .cozy.scheduler.v1.ExecutionSpec\x12/\n\x05grant\x18\x03 \x01(\x0b\x32 .cozy.scheduler.v1.DeliveryGrant\x12\x15\n\rinput_payload\x18\x04 \x01(\x0c\"0\n\tAttemptId\x12\x12\n\nrequest_id\x18\x01 \x01(\t\x12\x0f\n\x07\x61ttempt\x18\x02 \x01(\x03\"\xc4\x05\n\rExecutionSpec\x12\x0e\n\x06\x64igest\x18\x01 \x01(\t\x12\x14\n\x0cspec_version\x18\x02 \x01(\r\x12\x33\n\x07release\x18\x03 \x01(\x0b\x32\".cozy.scheduler.v1.EndpointRelease\x12\x15\n\rfunction_name\x18\x04 \x01(\t\x12\x38\n\x0enumerical_lane\x18\x05 \x01(\x0b\x32 .cozy.scheduler.v1.ExecutionLane\x12#\n\x03\x61rm\x18\x06 \x01(\x0b\x32\x16.cozy.scheduler.v1.Arm\x12-\n\x08topology\x18\x07 \x01(\x0b\x32\x1b.cozy.scheduler.v1.Topology\x12\x38\n\ncomponents\x18\x08 \x01(\x0b\x32$.cozy.scheduler.v1.ComponentManifest\x12\x33\n\x08\x61\x64\x61pters\x18\t \x03(\x0b\x32!.cozy.scheduler.v1.AdapterBinding\x12\x31\n\x06\x63onfig\x18\n \x01(\x0b\x32!.cozy.scheduler.v1.ConfigSnapshot\x12\x38\n\tcontracts\x18\x0b \x01(\x0b\x32%.cozy.scheduler.v1.ContractIdentities\x12\x36\n\x0b\x65xploration\x18\x0c \x01(\x0b\x32!.cozy.scheduler.v1.ExplorationRef\x12\x33\n\x0b\x61ttribution\x18\r \x01(\x0b\x32\x1e.cozy.scheduler.v1.Attribution\x12\x32\n\x0boutput_mode\x18\x0e \x01(\x0e\x32\x1d.cozy.scheduler.v1.OutputMode\x12\x36\n\x0cinput_assets\x18\x0f \x03(\x0b\x32 .cozy.scheduler.v1.InputAssetRef\"s\n\x0f\x45ndpointRelease\x12\x0b\n\x03org\x18\x01 \x01(\t\x12\x10\n\x08\x65ndpoint\x18\x02 \x01(\t\x12\x12\n\nrelease_id\x18\x03 \x01(\t\x12\x14\n\x0cimage_digest\x18\x04 \x01(\t\x12\x17\n\x0f\x63ode_closure_id\x18\x05 \x01(\t\".\n\x0b\x41ttribution\x12\x0b\n\x03org\x18\x01 \x01(\t\x12\x12\n\ninvoker_id\x18\x02 \x01(\t\"R\n\rExecutionLane\x12.\n\x07weights\x18\x01 \x01(\x0e\x32\x1d.cozy.scheduler.v1.WeightLane\x12\x11\n\tmandatory\x18\x02 \x01(\x08\"\xd7\x01\n\x03\x41rm\x12\x1d\n\x15graph_contract_digest\x18\x01 \x01(\t\x12*\n\x05shape\x18\x02 \x01(\x0e\x32\x1b.cozy.scheduler.v1.ArmShape\x12\x1b\n\x13\x61\x64\x61pter_rank_bucket\x18\x03 \x01(\x05\x12\x31\n\x07\x62\x61\x63kend\x18\x04 \x01(\x0e\x32 .cozy.scheduler.v1.SteadyBackend\x12\x35\n\x08\x61rtifact\x18\x05 \x01(\x0b\x32#.cozy.scheduler.v1.ArtifactIdentity\"\x98\x01\n\x10\x41rtifactIdentity\x12\x10\n\x08\x63\x65ll_ref\x18\x01 \x01(\t\x12\x16\n\x0e\x63ontent_digest\x18\x02 \x01(\t\x12\x10\n\x08\x63\x65ll_key\x18\x03 \x01(\t\x12\x15\n\rpublisher_org\x18\x04 \x01(\t\x12\x18\n\x10toolchain_digest\x18\x05 \x01(\t\x12\x17\n\x0f\x65nv_seal_digest\x18\x06 \x01(\t\"e\n\x08Topology\x12\x13\n\x0b\x61\x63\x63\x65lerator\x18\x01 \x01(\t\x12\x11\n\tgpu_count\x18\x02 \x01(\x05\x12\x18\n\x10\x65xecution_groups\x18\x03 \x01(\x05\x12\x17\n\x0f\x64\x65vice_ordinals\x18\x04 \x03(\x05\"B\n\x11\x43omponentManifest\x12-\n\x05slots\x18\x01 \x03(\x0b\x32\x1e.cozy.scheduler.v1.SlotBinding\"\xd2\x01\n\x0bSlotBinding\x12\x0c\n\x04slot\x18\x01 \x01(\t\x12\x0b\n\x03ref\x18\x02 \x01(\t\x12\x17\n\x0fsnapshot_digest\x18\x03 \x01(\t\x12\x33\n\ncomponents\x18\x04 \x03(\x0b\x32\x1f.cozy.scheduler.v1.ComponentRef\x12\x11\n\tobjective\x18\x05 \x01(\t\x12\x11\n\tdistilled\x18\x06 \x01(\x08\x12\x18\n\x10\x64istilled_status\x18\x07 \x01(\t\x12\x1a\n\x12inference_defaults\x18\x08 \x01(\t\"G\n\x0c\x43omponentRef\x12\x11\n\tcomponent\x18\x01 \x01(\t\x12\x0b\n\x03ref\x18\x02 \x01(\t\x12\x17\n\x0fsnapshot_digest\x18\x03 \x01(\t\"p\n\x0e\x41\x64\x61pterBinding\x12\x0c\n\x04slot\x18\x01 \x01(\t\x12\x0b\n\x03ref\x18\x02 \x01(\t\x12\x17\n\x0fsnapshot_digest\x18\x03 \x01(\t\x12\x0e\n\x06weight\x18\x04 \x01(\x01\x12\x1a\n\x12inference_defaults\x18\x05 \x01(\t\"0\n\x0e\x43onfigSnapshot\x12\x0e\n\x06values\x18\x01 \x01(\x0c\x12\x0e\n\x06\x64igest\x18\x02 \x01(\t\"U\n\x12\x43ontractIdentities\x12\x1f\n\x17quality_contract_digest\x18\x01 \x01(\t\x12\x1e\n\x16runtime_formula_digest\x18\x02 \x01(\t\"D\n\x0e\x45xplorationRef\x12\x16\n\x0e\x65xploration_id\x18\x01 \x01(\t\x12\x1a\n\x12operator_intent_id\x18\x02 \x01(\t\"z\n\rInputAssetRef\x12\x10\n\x08\x61sset_id\x18\x01 \x01(\t\x12\x12\n\nsource_ref\x18\x02 \x01(\t\x12\x0e\n\x06\x64igest\x18\x03 \x01(\t\x12\x12\n\nsize_bytes\x18\x04 \x01(\x03\x12\x0c\n\x04kind\x18\x05 \x01(\t\x12\x11\n\tmime_type\x18\x06 \x01(\t\"\xa6\x02\n\rDeliveryGrant\x12\x18\n\x10\x63\x61pability_token\x18\x01 \x01(\t\x12\x17\n\x0f\x65xpires_unix_ms\x18\x02 \x01(\x03\x12\r\n\x05\x65poch\x18\x03 \x01(\x04\x12\x42\n\tsnapshots\x18\x04 \x03(\x0b\x32/.cozy.scheduler.v1.DeliveryGrant.SnapshotsEntry\x12\x37\n\x0cinput_assets\x18\x05 \x03(\x0b\x32!.cozy.scheduler.v1.PresignedAsset\x1aV\n\x0eSnapshotsEntry\x12\x0b\n\x03key\x18\x01 \x01(\t\x12\x33\n\x05value\x18\x02 \x01(\x0b\x32$.cozy.scheduler.v1.PresignedSnapshot:\x02\x38\x01\"D\n\x11PresignedSnapshot\x12/\n\x05\x66iles\x18\x01 \x03(\x0b\x32 .cozy.scheduler.v1.PresignedFile\"\x95\x01\n\rPresignedFile\x12\x0c\n\x04path\x18\x01 \x01(\t\x12\x12\n\nsize_bytes\x18\x02 \x01(\x03\x12\x0e\n\x06\x64igest\x18\x03 \x01(\t\x12\x0b\n\x03url\x18\x04 \x01(\t\x12\x18\n\x10\x63hunk_size_bytes\x18\x05 \x01(\x03\x12+\n\x06\x63hunks\x18\x06 \x03(\x0b\x32\x1b.cozy.scheduler.v1.ChunkRef\"/\n\x0ePresignedAsset\x12\x10\n\x08\x61sset_id\x18\x01 \x01(\t\x12\x0b\n\x03url\x18\x02 \x01(\t\"2\n\x0bJobAccepted\x12\x12\n\nrequest_id\x18\x01 \x01(\t\x12\x0f\n\x07\x61ttempt\x18\x02 \x01(\x03\"\x88\x02\n\tJobResult\x12\x12\n\nrequest_id\x18\x01 \x01(\t\x12\x0f\n\x07\x61ttempt\x18\x02 \x01(\x03\x12,\n\x06status\x18\x03 \x01(\x0e\x32\x1c.cozy.scheduler.v1.JobStatus\x12\x10\n\x06inline\x18\x04 \x01(\x0cH\x00\x12\x12\n\x08\x62lob_ref\x18\x05 \x01(\tH\x00\x12\x14\n\x0csafe_message\x18\x06 \x01(\t\x12.\n\x07metrics\x18\x07 \x01(\x0b\x32\x1d.cozy.scheduler.v1.JobMetrics\x12\x32\n\x0b\x61\x64justments\x18\x08 \x03(\x0b\x32\x1d.cozy.scheduler.v1.AdjustmentB\x08\n\x06output\"O\n\nAdjustment\x12\r\n\x05\x66ield\x18\x01 \x01(\t\x12\x11\n\trequested\x18\x02 \x01(\t\x12\x0f\n\x07\x61pplied\x18\x03 \x01(\t\x12\x0e\n\x06reason\x18\x04 \x01(\t\"\xce\x05\n\nJobMetrics\x12\x12\n\nruntime_ms\x18\x01 \x01(\x03\x12\x10\n\x08queue_ms\x18\x02 \x01(\x03\x12\x18\n\x10rss_at_end_bytes\x18\x03 \x01(\x03\x12\x17\n\x0fpeak_vram_bytes\x18\x04 \x01(\x03\x12\x1c\n\x14\x63oncurrency_at_start\x18\x05 \x01(\x05\x12\x1f\n\x17output_media_duration_s\x18\x06 \x01(\x01\x12\x14\n\x0cinput_tokens\x18\x07 \x01(\x03\x12\x1b\n\x13input_cached_tokens\x18\x08 \x01(\x03\x12\x15\n\routput_tokens\x18\t \x01(\x03\x12\x14\n\x0coutput_count\x18\n \x01(\x03\x12\x14\n\x0cslot_held_ms\x18\x0b \x01(\x03\x12\x18\n\x10\x66inalize_wall_ms\x18\x0c \x01(\x03\x12\x0c\n\x04lane\x18\r \x01(\t\x12\x46\n\rruntime_terms\x18\x0e \x03(\x0b\x32/.cozy.scheduler.v1.JobMetrics.RuntimeTermsEntry\x12<\n\x08stage_ms\x18\x0f \x03(\x0b\x32*.cozy.scheduler.v1.JobMetrics.StageMsEntry\x12\x14\n\x0cserving_mode\x18\x10 \x01(\t\x12\x17\n\x0fserved_cell_ref\x18\x11 \x01(\t\x12\x1d\n\x15served_eager_fallback\x18\x12 \x01(\x08\x12\x17\n\x0f\x66\x61llback_reason\x18\x13 \x01(\t\x12\n\n\x02sm\x18\x14 \x01(\t\x12\r\n\x05steps\x18\x15 \x01(\x03\x12\r\n\x05width\x18\x16 \x01(\x03\x12\x0e\n\x06height\x18\x17 \x01(\x03\x1a\x33\n\x11RuntimeTermsEntry\x12\x0b\n\x03key\x18\x01 \x01(\t\x12\r\n\x05value\x18\x02 \x01(\x01:\x02\x38\x01\x1a.\n\x0cStageMsEntry\x12\x0b\n\x03key\x18\x01 \x01(\t\x12\r\n\x05value\x18\x02 \x01(\x03:\x02\x38\x01\"c\n\x0bJobProgress\x12\x12\n\nrequest_id\x18\x01 \x01(\t\x12\x0f\n\x07\x61ttempt\x18\x02 \x01(\x03\x12\x0b\n\x03seq\x18\x03 \x01(\x03\x12\x0c\n\x04\x64\x61ta\x18\x04 \x01(\x0c\x12\x14\n\x0c\x63ontent_type\x18\x05 \x01(\t\"0\n\tCancelJob\x12\x12\n\nrequest_id\x18\x01 \x01(\t\x12\x0f\n\x07\x61ttempt\x18\x02 \x01(\x03\"\xa6\x01\n\x07ModelOp\x12*\n\x02op\x18\x01 \x01(\x0e\x32\x1e.cozy.scheduler.v1.ModelOpKind\x12\x0b\n\x03ref\x18\x02 \x01(\t\x12-\n\x08snapshot\x18\x03 \x01(\x0b\x32\x1b.cozy.scheduler.v1.Snapshot\x12\x14\n\x0coperation_id\x18\x04 \x01(\t\x12\x1d\n\x15target_incarnation_id\x18\x05 \x01(\t\"\x9e\x04\n\nModelEvent\x12\x0b\n\x03ref\x18\x01 \x01(\t\x12,\n\x05state\x18\x02 \x01(\x0e\x32\x1d.cozy.scheduler.v1.ModelState\x12\x12\n\nvram_bytes\x18\x03 \x01(\x03\x12\r\n\x05\x65rror\x18\x04 \x01(\t\x12\x12\n\nbytes_done\x18\x05 \x01(\x03\x12\x13\n\x0b\x62ytes_total\x18\x06 \x01(\x03\x12\x13\n\x0b\x64uration_ms\x18\x07 \x01(\x03\x12\x12\n\ncache_hits\x18\x08 \x01(\x03\x12\x14\n\x0c\x63\x61\x63he_misses\x18\t \x01(\x03\x12\x10\n\x08warmup_s\x18\n \x01(\x01\x12\x1f\n\x17host_ram_required_bytes\x18\x0b \x01(\x03\x12\'\n\x1fhost_ram_available_before_bytes\x18\x0c \x01(\x03\x12&\n\x1ehost_ram_available_after_bytes\x18\r \x01(\x03\x12\x1d\n\x15host_ram_evicted_refs\x18\x0e \x03(\t\x12$\n\x1chost_ram_capacity_generation\x18\x0f \x01(\x04\x12\x17\n\x0fsnapshot_digest\x18\x10 \x01(\t\x12\x1c\n\x14residency_generation\x18\x11 \x01(\x04\x12\x14\n\x0coperation_id\x18\x12 \x01(\t\x12\x1d\n\x15target_incarnation_id\x18\x13 \x01(\t\x12\x15\n\rnetwork_bytes\x18\x14 \x01(\x03\"\xf4\x02\n\x0e\x41\x63tivityUpdate\x12\x0c\n\x04kind\x18\x01 \x01(\t\x12\r\n\x05phase\x18\x02 \x01(\t\x12\x0c\n\x04step\x18\x03 \x01(\x03\x12\x13\n\x0btotal_steps\x18\x04 \x01(\x03\x12\x0b\n\x03seq\x18\x05 \x01(\x04\x12/\n\x05state\x18\x06 \x01(\x0e\x32 .cozy.scheduler.v1.ActivityState\x12\r\n\x05\x65rror\x18\x07 \x01(\t\x12\x0e\n\x06\x64\x65tail\x18\x08 \x01(\t\x12\x1a\n\x12updated_at_unix_ms\x18\t \x01(\x03\x12\x0f\n\x07\x63ounter\x18\n \x01(\t\x12\x14\n\x0c\x63ounter_unit\x18\x0b \x01(\t\x12\x14\n\x0c\x63ounter_done\x18\x0c \x01(\x01\x12\x15\n\rcounter_total\x18\r \x01(\x01\x12\x12\n\nrate_per_s\x18\x0e \x01(\x01\x12\x14\n\x0cself_stalled\x18\x0f \x01(\x08\x12\x16\n\x0estalled_for_ms\x18\x10 \x01(\x03\x12\x13\n\x0b\x64uration_ms\x18\x11 \x01(\x03\"\xe2\x02\n\tBootPhase\x12\x0f\n\x07\x62oot_id\x18\x01 \x01(\t\x12\x0f\n\x07ordinal\x18\x02 \x01(\x03\x12\x16\n\x0eparent_ordinal\x18\x03 \x01(\x03\x12\r\n\x05phase\x18\x04 \x01(\t\x12\x10\n\x08terminal\x18\x05 \x01(\x08\x12\x1a\n\x12started_at_unix_ms\x18\x06 \x01(\x03\x12\x13\n\x0b\x64uration_ms\x18\x07 \x01(\x03\x12\x19\n\x11process_uptime_ms\x18\x08 \x01(\x03\x12\r\n\x05\x62ytes\x18\t \x01(\x03\x12\x0e\n\x06source\x18\n \x01(\t\x12\x0b\n\x03ref\x18\x0b \x01(\t\x12\x15\n\rartifact_kind\x18\x0c \x01(\t\x12\x14\n\x0c\x61rtifact_key\x18\r \x01(\t\x12\x10\n\x08\x66unction\x18\x0e \x01(\t\x12\x0f\n\x07outcome\x18\x0f \x01(\t\x12\x0e\n\x06reason\x18\x10 \x01(\t\x12\x0e\n\x06\x64\x65tail\x18\x11 \x01(\t\x12\x12\n\ncumulative\x18\x12 \x01(\x08\"\xad\x01\n\rFnUnavailable\x12\x15\n\rfunction_name\x18\x01 \x01(\t\x12\x0e\n\x06reason\x18\x02 \x01(\t\x12\x0e\n\x06\x64\x65tail\x18\x03 \x01(\t\x12\x38\n\x04\x61xes\x18\x04 \x03(\x0b\x32*.cozy.scheduler.v1.FnUnavailable.AxesEntry\x1a+\n\tAxesEntry\x12\x0b\n\x03key\x18\x01 \x01(\t\x12\r\n\x05value\x18\x02 \x01(\t:\x02\x38\x01\"\x8d\x01\n\nFnDegraded\x12\x15\n\rfunction_name\x18\x01 \x01(\t\x12\x0e\n\x06wanted\x18\x02 \x01(\t\x12\x0b\n\x03ran\x18\x03 \x01(\t\x12\x0e\n\x06reason\x18\x04 \x01(\t\x12\x1e\n\x16\x65st_latency_multiplier\x18\x05 \x01(\x01\x12\x1b\n\x13recommended_vram_gb\x18\x06 \x01(\x01\"\x1c\n\x05\x44rain\x12\x13\n\x0b\x64\x65\x61\x64line_ms\x18\x01 \x01(\x03\"6\n\x0cTokenRefresh\x12\r\n\x05token\x18\x01 \x01(\t\x12\x17\n\x0f\x65xpires_at_unix\x18\x02 \x01(\x03*Q\n\x0fProtocolVersion\x12 \n\x1cPROTOCOL_VERSION_UNSPECIFIED\x10\x00\x12\x1c\n\x18PROTOCOL_VERSION_CURRENT\x10\x01*y\n\rResidencyTier\x12\x1e\n\x1aRESIDENCY_TIER_UNSPECIFIED\x10\x00\x12\x17\n\x13RESIDENCY_TIER_DISK\x10\x01\x12\x16\n\x12RESIDENCY_TIER_RAM\x10\x02\x12\x17\n\x13RESIDENCY_TIER_VRAM\x10\x03*\xf1\x01\n\x11\x44\x65siredIntentKind\x12#\n\x1f\x44\x45SIRED_INTENT_KIND_UNSPECIFIED\x10\x00\x12#\n\x1f\x44\x45SIRED_INTENT_KIND_MATERIALIZE\x10\x01\x12&\n\"DESIRED_INTENT_KIND_FUNCTION_READY\x10\x02\x12$\n DESIRED_INTENT_KIND_CONFIG_APPLY\x10\x03\x12%\n!DESIRED_INTENT_KIND_COMPILE_ADOPT\x10\x04\x12\x1d\n\x19\x44\x45SIRED_INTENT_KIND_DRAIN\x10\x05*\x95\x02\n\x12\x44\x65siredIntentCause\x12$\n DESIRED_INTENT_CAUSE_UNSPECIFIED\x10\x00\x12\"\n\x1e\x44\x45SIRED_INTENT_CAUSE_COLD_BOOT\x10\x01\x12 \n\x1c\x44\x45SIRED_INTENT_CAUSE_REQUEST\x10\x02\x12$\n DESIRED_INTENT_CAUSE_PREPOSITION\x10\x03\x12&\n\"DESIRED_INTENT_CAUSE_CONFIG_CHANGE\x10\x04\x12$\n DESIRED_INTENT_CAUSE_REPLACEMENT\x10\x05\x12\x1f\n\x1b\x44\x45SIRED_INTENT_CAUSE_RETIRE\x10\x06*|\n\x11GoalReceiptStatus\x12#\n\x1fGOAL_RECEIPT_STATUS_UNSPECIFIED\x10\x00\x12 \n\x1cGOAL_RECEIPT_STATUS_ACCEPTED\x10\x01\x12 \n\x1cGOAL_RECEIPT_STATUS_REJECTED\x10\x02*\xc9\x02\n\x15LifecycleIntentStatus\x12\'\n#LIFECYCLE_INTENT_STATUS_UNSPECIFIED\x10\x00\x12$\n LIFECYCLE_INTENT_STATUS_ACCEPTED\x10\x01\x12#\n\x1fLIFECYCLE_INTENT_STATUS_WAITING\x10\x02\x12#\n\x1fLIFECYCLE_INTENT_STATUS_RUNNING\x10\x03\x12%\n!LIFECYCLE_INTENT_STATUS_SUCCEEDED\x10\x04\x12\"\n\x1eLIFECYCLE_INTENT_STATUS_FAILED\x10\x05\x12$\n LIFECYCLE_INTENT_STATUS_CANCELED\x10\x06\x12&\n\"LIFECYCLE_INTENT_STATUS_SUPERSEDED\x10\x07*\xc9\x08\n\x14LifecycleIntentStage\x12&\n\"LIFECYCLE_INTENT_STAGE_UNSPECIFIED\x10\x00\x12%\n!LIFECYCLE_INTENT_STAGE_VALIDATING\x10\x01\x12+\n\'LIFECYCLE_INTENT_STAGE_WAIT_TENANT_IDLE\x10\x02\x12(\n$LIFECYCLE_INTENT_STAGE_WAIT_REF_LOCK\x10\x03\x12)\n%LIFECYCLE_INTENT_STAGE_WAIT_LOAD_LOCK\x10\x04\x12(\n$LIFECYCLE_INTENT_STAGE_WAIT_GPU_SLOT\x10\x05\x12(\n$LIFECYCLE_INTENT_STAGE_WAIT_SNAPSHOT\x10\x06\x12-\n)LIFECYCLE_INTENT_STAGE_WAIT_DISK_HEADROOM\x10\x07\x12(\n$LIFECYCLE_INTENT_STAGE_WAIT_HOST_RAM\x10\x08\x12-\n)LIFECYCLE_INTENT_STAGE_WAIT_NETWORK_RETRY\x10\t\x12+\n\'LIFECYCLE_INTENT_STAGE_WAIT_REPLACEMENT\x10\n\x12#\n\x1fLIFECYCLE_INTENT_STAGE_FETCHING\x10\x0b\x12$\n LIFECYCLE_INTENT_STAGE_VERIFYING\x10\x0c\x12\"\n\x1eLIFECYCLE_INTENT_STAGE_ON_DISK\x10\r\x12\'\n#LIFECYCLE_INTENT_STAGE_LOADING_HOST\x10\x0e\x12)\n%LIFECYCLE_INTENT_STAGE_LOADING_DEVICE\x10\x0f\x12\"\n\x1eLIFECYCLE_INTENT_STAGE_WARMING\x10\x10\x12$\n LIFECYCLE_INTENT_STAGE_COMPILING\x10\x11\x12#\n\x1fLIFECYCLE_INTENT_STAGE_ADOPTING\x10\x12\x12/\n+LIFECYCLE_INTENT_STAGE_CONFIG_MATERIALIZING\x10\x13\x12\x33\n/LIFECYCLE_INTENT_STAGE_CONFIG_BINDINGS_APPLYING\x10\x14\x12,\n(LIFECYCLE_INTENT_STAGE_CONFIG_BOOT_STALE\x10\x15\x12 \n\x1cLIFECYCLE_INTENT_STAGE_READY\x10\x16\x12#\n\x1fLIFECYCLE_INTENT_STAGE_DRAINING\x10\x17\x12%\n!LIFECYCLE_INTENT_STAGE_FINALIZING\x10\x18\x12#\n\x1fLIFECYCLE_INTENT_STAGE_FLUSHING\x10\x19*\xc0\x03\n\x13LifecycleWaitReason\x12%\n!LIFECYCLE_WAIT_REASON_UNSPECIFIED\x10\x00\x12%\n!LIFECYCLE_WAIT_REASON_TENANT_WORK\x10\x01\x12\"\n\x1eLIFECYCLE_WAIT_REASON_REF_LOCK\x10\x02\x12#\n\x1fLIFECYCLE_WAIT_REASON_LOAD_LOCK\x10\x03\x12\"\n\x1eLIFECYCLE_WAIT_REASON_GPU_SLOT\x10\x04\x12\"\n\x1eLIFECYCLE_WAIT_REASON_SNAPSHOT\x10\x05\x12\'\n#LIFECYCLE_WAIT_REASON_DISK_HEADROOM\x10\x06\x12\"\n\x1eLIFECYCLE_WAIT_REASON_HOST_RAM\x10\x07\x12\'\n#LIFECYCLE_WAIT_REASON_NETWORK_RETRY\x10\x08\x12%\n!LIFECYCLE_WAIT_REASON_REPLACEMENT\x10\t\x12-\n)LIFECYCLE_WAIT_REASON_SINGLE_FLIGHT_OWNER\x10\n*\xa6\x05\n\x12LifecycleErrorCode\x12$\n LIFECYCLE_ERROR_CODE_UNSPECIFIED\x10\x00\x12\x30\n,LIFECYCLE_ERROR_CODE_MISSING_MANDATORY_FIELD\x10\x01\x12\x30\n,LIFECYCLE_ERROR_CODE_WORKER_SESSION_MISMATCH\x10\x02\x12)\n%LIFECYCLE_ERROR_CODE_RELEASE_MISMATCH\x10\x03\x12/\n+LIFECYCLE_ERROR_CODE_COMMAND_SEQ_REGRESSION\x10\x04\x12-\n)LIFECYCLE_ERROR_CODE_COMMAND_SEQ_CONFLICT\x10\x05\x12*\n&LIFECYCLE_ERROR_CODE_CONFIG_REGRESSION\x10\x06\x12)\n%LIFECYCLE_ERROR_CODE_UNKNOWN_FUNCTION\x10\x07\x12\x32\n.LIFECYCLE_ERROR_CODE_SNAPSHOT_IDENTITY_MISSING\x10\x08\x12+\n\'LIFECYCLE_ERROR_CODE_UNSUPPORTED_INTENT\x10\t\x12\x32\n.LIFECYCLE_ERROR_CODE_UNKNOWN_MANDATORY_COMMAND\x10\n\x12\x31\n-LIFECYCLE_ERROR_CODE_PROTOCOL_UNREPORTED_WAIT\x10\x0b\x12\x35\n1LIFECYCLE_ERROR_CODE_CONFIG_SNAPSHOT_WRITE_FAILED\x10\x0c\x12%\n!LIFECYCLE_ERROR_CODE_DRAIN_FAILED\x10\r*\xe1\x01\n\x17\x46unctionCapabilityState\x12)\n%FUNCTION_CAPABILITY_STATE_UNSPECIFIED\x10\x00\x12#\n\x1f\x46UNCTION_CAPABILITY_STATE_READY\x10\x01\x12&\n\"FUNCTION_CAPABILITY_STATE_APPLYING\x10\x02\x12(\n$FUNCTION_CAPABILITY_STATE_BOOT_STALE\x10\x03\x12$\n FUNCTION_CAPABILITY_STATE_FAILED\x10\x04*\xdf\x01\n\x16\x43onfigApplicationState\x12(\n$CONFIG_APPLICATION_STATE_UNSPECIFIED\x10\x00\x12%\n!CONFIG_APPLICATION_STATE_APPLYING\x10\x01\x12&\n\"CONFIG_APPLICATION_STATE_CONVERGED\x10\x02\x12\'\n#CONFIG_APPLICATION_STATE_BOOT_STALE\x10\x03\x12#\n\x1f\x43ONFIG_APPLICATION_STATE_FAILED\x10\x04*\x9b\x02\n\x14\x44rainLifecycleStatus\x12&\n\"DRAIN_LIFECYCLE_STATUS_UNSPECIFIED\x10\x00\x12#\n\x1f\x44RAIN_LIFECYCLE_STATUS_ACCEPTED\x10\x01\x12#\n\x1f\x44RAIN_LIFECYCLE_STATUS_DRAINING\x10\x02\x12%\n!DRAIN_LIFECYCLE_STATUS_FINALIZING\x10\x03\x12#\n\x1f\x44RAIN_LIFECYCLE_STATUS_FLUSHING\x10\x04\x12\"\n\x1e\x44RAIN_LIFECYCLE_STATUS_DRAINED\x10\x05\x12!\n\x1d\x44RAIN_LIFECYCLE_STATUS_FAILED\x10\x06*v\n\x0bStorageTier\x12\x1c\n\x18STORAGE_TIER_UNSPECIFIED\x10\x00\x12\x1a\n\x16STORAGE_TIER_CONTAINER\x10\x01\x12\x17\n\x13STORAGE_TIER_VOLUME\x10\x02\x12\x14\n\x10STORAGE_TIER_NFS\x10\x03*\xd8\x01\n\x0bWorkerPhase\x12\x1c\n\x18WORKER_PHASE_UNSPECIFIED\x10\x00\x12\x18\n\x14WORKER_PHASE_BOOTING\x10\x01\x12#\n\x1fWORKER_PHASE_DOWNLOADING_MODELS\x10\x02\x12\"\n\x1eWORKER_PHASE_LOADING_PIPELINES\x10\x03\x12\x18\n\x14WORKER_PHASE_WARMING\x10\x04\x12\x16\n\x12WORKER_PHASE_READY\x10\x05\x12\x16\n\x12WORKER_PHASE_ERROR\x10\x06*V\n\nOutputMode\x12\x1b\n\x17OUTPUT_MODE_UNSPECIFIED\x10\x00\x12\x13\n\x0fOUTPUT_MODE_URL\x10\x01\x12\x16\n\x12OUTPUT_MODE_INLINE\x10\x02*\x82\x01\n\nWeightLane\x12\x1b\n\x17WEIGHT_LANE_UNSPECIFIED\x10\x00\x12\x14\n\x10WEIGHT_LANE_BF16\x10\x01\x12\x15\n\x11WEIGHT_LANE_W8A16\x10\x02\x12\x14\n\x10WEIGHT_LANE_W8A8\x10\x03\x12\x14\n\x10WEIGHT_LANE_W4A4\x10\x04*V\n\x08\x41rmShape\x12\x19\n\x15\x41RM_SHAPE_UNSPECIFIED\x10\x00\x12\x18\n\x14\x41RM_SHAPE_BRANCHLESS\x10\x01\x12\x15\n\x11\x41RM_SHAPE_ADAPTER\x10\x02*\x86\x01\n\rSteadyBackend\x12\x1e\n\x1aSTEADY_BACKEND_UNSPECIFIED\x10\x00\x12\x1b\n\x17STEADY_BACKEND_AOT_CELL\x10\x01\x12\x19\n\x15STEADY_BACKEND_DYNAMO\x10\x02\x12\x1d\n\x19STEADY_BACKEND_EAGER_ONLY\x10\x03*\x9b\x01\n\tJobStatus\x12\x1a\n\x16JOB_STATUS_UNSPECIFIED\x10\x00\x12\x11\n\rJOB_STATUS_OK\x10\x01\x12\x16\n\x12JOB_STATUS_INVALID\x10\x02\x12\x18\n\x14JOB_STATUS_RETRYABLE\x10\x03\x12\x14\n\x10JOB_STATUS_FATAL\x10\x04\x12\x17\n\x13JOB_STATUS_CANCELED\x10\x05*S\n\x0bModelOpKind\x12\x1d\n\x19MODEL_OP_KIND_UNSPECIFIED\x10\x00\x12%\n!MODEL_OP_KIND_ADOPT_COMPILE_CACHE\x10\x01*\x82\x02\n\nModelState\x12\x1b\n\x17MODEL_STATE_UNSPECIFIED\x10\x00\x12\x1b\n\x17MODEL_STATE_DOWNLOADING\x10\x01\x12\x17\n\x13MODEL_STATE_ON_DISK\x10\x02\x12\x16\n\x12MODEL_STATE_IN_RAM\x10\x03\x12\x17\n\x13MODEL_STATE_IN_VRAM\x10\x04\x12\x17\n\x13MODEL_STATE_EVICTED\x10\x05\x12\x16\n\x12MODEL_STATE_FAILED\x10\x06\x12\x17\n\x13MODEL_STATE_ADOPTED\x10\x07\x12&\n\"MODEL_STATE_HOST_CAPACITY_PROGRESS\x10\x08*\x84\x01\n\rActivityState\x12\x1e\n\x1a\x41\x43TIVITY_STATE_UNSPECIFIED\x10\x00\x12\x1a\n\x16\x41\x43TIVITY_STATE_RUNNING\x10\x01\x12\x1c\n\x18\x41\x43TIVITY_STATE_COMPLETED\x10\x02\x12\x19\n\x15\x41\x43TIVITY_STATE_FAILED\x10\x03\x32g\n\x0fWorkerScheduler\x12T\n\x07\x43onnect\x12 .cozy.scheduler.v1.WorkerMessage\x1a#.cozy.scheduler.v1.SchedulerMessage(\x01\x30\x01\x42YZWgithub.com/cozy-creator/tensorhub/internal/orchestrator/grpc/pb/schedulerv1;schedulerv1b\x06proto3') +DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n\x16worker_scheduler.proto\x12\x11\x63ozy.scheduler.v1\"\xf7\x05\n\rWorkerMessage\x12)\n\x05hello\x18\x01 \x01(\x0b\x32\x18.cozy.scheduler.v1.HelloH\x00\x12\x34\n\x0bstate_delta\x18\x02 \x01(\x0b\x32\x1d.cozy.scheduler.v1.StateDeltaH\x00\x12\x36\n\x0cjob_accepted\x18\x03 \x01(\x0b\x32\x1e.cozy.scheduler.v1.JobAcceptedH\x00\x12\x32\n\njob_result\x18\x04 \x01(\x0b\x32\x1c.cozy.scheduler.v1.JobResultH\x00\x12\x36\n\x0cjob_progress\x18\x05 \x01(\x0b\x32\x1e.cozy.scheduler.v1.JobProgressH\x00\x12\x34\n\x0bmodel_event\x18\x06 \x01(\x0b\x32\x1d.cozy.scheduler.v1.ModelEventH\x00\x12:\n\x0e\x66n_unavailable\x18\x07 \x01(\x0b\x32 .cozy.scheduler.v1.FnUnavailableH\x00\x12\x34\n\x0b\x66n_degraded\x18\x08 \x01(\x0b\x32\x1d.cozy.scheduler.v1.FnDegradedH\x00\x12<\n\x0f\x61\x63tivity_update\x18\t \x01(\x0b\x32!.cozy.scheduler.v1.ActivityUpdateH\x00\x12\x44\n\x13hardware_unsuitable\x18\n \x01(\x0b\x32%.cozy.scheduler.v1.HardwareUnsuitableH\x00\x12\x36\n\x0cgoal_receipt\x18\x0b \x01(\x0b\x32\x1e.cozy.scheduler.v1.GoalReceiptH\x00\x12\x42\n\x12lifecycle_snapshot\x18\x0c \x01(\x0b\x32$.cozy.scheduler.v1.LifecycleSnapshotH\x00\x12\x32\n\nboot_phase\x18\r \x01(\x0b\x32\x1c.cozy.scheduler.v1.BootPhaseH\x00\x42\x05\n\x03msg\"\xb2\x03\n\x10SchedulerMessage\x12\x30\n\thello_ack\x18\x01 \x01(\x0b\x32\x1b.cozy.scheduler.v1.HelloAckH\x00\x12,\n\x07run_job\x18\x02 \x01(\x0b\x32\x19.cozy.scheduler.v1.RunJobH\x00\x12\x32\n\ncancel_job\x18\x03 \x01(\x0b\x32\x1c.cozy.scheduler.v1.CancelJobH\x00\x12.\n\x08model_op\x18\x04 \x01(\x0b\x32\x1a.cozy.scheduler.v1.ModelOpH\x00\x12)\n\x05\x64rain\x18\x05 \x01(\x0b\x32\x18.cozy.scheduler.v1.DrainH\x00\x12\x38\n\rtoken_refresh\x18\x06 \x01(\x0b\x32\x1f.cozy.scheduler.v1.TokenRefreshH\x00\x12\x34\n\x0brun_attempt\x18\x07 \x01(\x0b\x32\x1d.cozy.scheduler.v1.RunAttemptH\x00\x12\x38\n\rserve_posture\x18\x08 \x01(\x0b\x32\x1f.cozy.scheduler.v1.ServePostureH\x00\x42\x05\n\x03msg\"\xb3\x03\n\x05Hello\x12<\n\x10protocol_version\x18\x01 \x01(\x0e\x32\".cozy.scheduler.v1.ProtocolVersion\x12\x11\n\tworker_id\x18\x02 \x01(\t\x12\x12\n\nrelease_id\x18\x03 \x01(\t\x12\x35\n\tresources\x18\x04 \x01(\x0b\x32\".cozy.scheduler.v1.WorkerResources\x12,\n\x05state\x18\x05 \x01(\x0b\x32\x1d.cozy.scheduler.v1.StateDelta\x12\x31\n\x06models\x18\x06 \x03(\x0b\x32!.cozy.scheduler.v1.ModelResidency\x12\x31\n\tin_flight\x18\x07 \x03(\x0b\x32\x1e.cozy.scheduler.v1.InFlightJob\x12\x1d\n\x15heartbeat_interval_ms\x18\x08 \x01(\x04\x12\x19\n\x11worker_session_id\x18\t \x01(\t\x12@\n\x12lifecycle_snapshot\x18\n \x01(\x0b\x32$.cozy.scheduler.v1.LifecycleSnapshot\"\xcb\x02\n\x0fWorkerResources\x12\x11\n\tgpu_count\x18\x01 \x01(\x05\x12\x18\n\x10vram_total_bytes\x18\x02 \x01(\x03\x12\x10\n\x08gpu_name\x18\x03 \x01(\t\x12\x0e\n\x06gpu_sm\x18\x04 \x01(\t\x12\x16\n\x0einstalled_libs\x18\x05 \x03(\t\x12\x14\n\x0cimage_digest\x18\x06 \x01(\t\x12\x12\n\ngit_commit\x18\x07 \x01(\t\x12\x13\n\x0binstance_id\x18\x08 \x01(\t\x12\x32\n\x0bhost_canary\x18\t \x01(\x0b\x32\x1d.cozy.scheduler.v1.HostCanary\x12\x15\n\rtorch_version\x18\n \x01(\t\x12\x1a\n\x12gen_worker_version\x18\x0b \x01(\t\x12\x13\n\x0bworker_mode\x18\x0c \x01(\t\x12\x16\n\x0e\x64river_version\x18\r \x01(\t\"\xa2\x02\n\x12HardwareUnsuitable\x12\x11\n\tworker_id\x18\x01 \x01(\t\x12\x12\n\nrelease_id\x18\x02 \x01(\t\x12\x14\n\x0creason_class\x18\x03 \x01(\t\x12\x0e\n\x06\x64\x65tail\x18\x04 \x01(\t\x12\x16\n\x0e\x64river_version\x18\x05 \x01(\t\x12\x10\n\x08gpu_name\x18\x06 \x01(\t\x12\x15\n\rtorch_version\x18\x07 \x01(\t\x12\x1a\n\x12torch_cuda_version\x18\x08 \x01(\t\x12\x1a\n\x12gen_worker_version\x18\t \x01(\t\x12\x14\n\x0cimage_digest\x18\n \x01(\t\x12\x13\n\x0binstance_id\x18\x0b \x01(\t\x12\x1b\n\x13reported_at_unix_ms\x18\x0c \x01(\x03\"\x88\x02\n\nHostCanary\x12\x13\n\x0bmemcpy_gbps\x18\x01 \x01(\x01\x12\x10\n\x08\x64\x32h_gbps\x18\x02 \x01(\x01\x12\x17\n\x0fpinned_alloc_ok\x18\x03 \x01(\x08\x12\x17\n\x0f\x63pu_single_mbps\x18\x04 \x01(\x01\x12\x16\n\x0e\x63pu_multi_mbps\x18\x05 \x01(\x01\x12\r\n\x05vcpus\x18\x06 \x01(\x05\x12\x14\n\x0cram_total_gb\x18\x07 \x01(\x01\x12\x13\n\x0b\x64uration_ms\x18\x08 \x01(\x03\x12\x14\n\x0cinterconnect\x18\t \x01(\t\x12\x11\n\tpeer_gbps\x18\n \x01(\x01\x12\x13\n\x0bpeer_access\x18\x0b \x01(\x08\x12\x11\n\ttopo_link\x18\x0c \x01(\t\"\x98\x01\n\x0eModelResidency\x12\x0b\n\x03ref\x18\x01 \x01(\t\x12.\n\x04tier\x18\x02 \x01(\x0e\x32 .cozy.scheduler.v1.ResidencyTier\x12\x12\n\nvram_bytes\x18\x03 \x01(\x03\x12\x17\n\x0fsnapshot_digest\x18\x04 \x01(\t\x12\x1c\n\x14residency_generation\x18\x05 \x01(\x04\"2\n\x0bInFlightJob\x12\x12\n\nrequest_id\x18\x01 \x01(\t\x12\x0f\n\x07\x61ttempt\x18\x02 \x01(\x03\"\x9f\x02\n\x08HelloAck\x12<\n\x10protocol_version\x18\x01 \x01(\x0e\x32\".cozy.scheduler.v1.ProtocolVersion\x12\x15\n\rfile_base_url\x18\x02 \x01(\t\x12\x37\n\x0bresolutions\x18\x03 \x03(\x0b\x32\".cozy.scheduler.v1.ModelResolution\x12>\n\x11\x64\x65sired_residency\x18\x04 \x01(\x0b\x32#.cozy.scheduler.v1.DesiredResidency\x12\x45\n\x15\x64\x65sired_state_command\x18\x05 \x01(\x0b\x32&.cozy.scheduler.v1.DesiredStateCommand\"\xaf\x02\n\x10\x44\x65siredResidency\x12\x12\n\ngeneration\x18\x01 \x01(\x04\x12\x11\n\tdisk_refs\x18\x02 \x03(\t\x12/\n\x03hot\x18\x03 \x03(\x0b\x32\".cozy.scheduler.v1.DesiredInstance\x12\x45\n\tsnapshots\x18\x04 \x03(\x0b\x32\x32.cozy.scheduler.v1.DesiredResidency.SnapshotsEntry\x12\x12\n\nrelease_id\x18\x05 \x01(\t\x12\x19\n\x11\x63onfig_generation\x18\x06 \x01(\x04\x1aM\n\x0eSnapshotsEntry\x12\x0b\n\x03key\x18\x01 \x01(\t\x12*\n\x05value\x18\x02 \x01(\x0b\x32\x1b.cozy.scheduler.v1.Snapshot:\x02\x38\x01\"\x99\x03\n\x13\x44\x65siredStateCommand\x12\x19\n\x11worker_session_id\x18\x01 \x01(\t\x12\x13\n\x0b\x63ommand_seq\x18\x02 \x01(\x04\x12\x0f\n\x07goal_id\x18\x03 \x01(\t\x12\x12\n\nrelease_id\x18\x04 \x01(\t\x12\x19\n\x11\x63onfig_generation\x18\x05 \x01(\x04\x12\x15\n\rconfig_digest\x18\x06 \x01(\x0c\x12\x19\n\x11issued_at_unix_ms\x18\x07 \x01(\x03\x12\x19\n\x11\x61\x63\x63\x65pt_by_unix_ms\x18\x08 \x01(\x03\x12\x1f\n\x17\x66irst_action_by_unix_ms\x18\t \x01(\x03\x12\x31\n\x07intents\x18\n \x03(\x0b\x32 .cozy.scheduler.v1.DesiredIntent\x12\x11\n\tmandatory\x18\x0b \x01(\x08\x12\x42\n\x16\x63hanged_config_classes\x18\x0c \x01(\x0b\x32\".cozy.scheduler.v1.ConfigClassMask\x12\x1a\n\x12parameter_snapshot\x18\r \x01(\x0c\"\x95\x03\n\rDesiredIntent\x12\x11\n\tintent_id\x18\x01 \x01(\t\x12\x32\n\x04kind\x18\x02 \x01(\x0e\x32$.cozy.scheduler.v1.DesiredIntentKind\x12\x34\n\x05\x63\x61use\x18\x03 \x01(\x0e\x32%.cozy.scheduler.v1.DesiredIntentCause\x12\x15\n\rfunction_name\x18\x04 \x01(\t\x12\x0b\n\x03ref\x18\x05 \x01(\t\x12\x17\n\x0fsnapshot_digest\x18\x06 \x01(\x0c\x12\x36\n\x0c\x64\x65sired_tier\x18\x07 \x01(\x0e\x32 .cozy.scheduler.v1.ResidencyTier\x12\x16\n\x0e\x62inding_digest\x18\x08 \x01(\x0c\x12\x18\n\x10parent_intent_id\x18\t \x01(\t\x12;\n\x10waiting_requests\x18\n \x03(\x0b\x32!.cozy.scheduler.v1.RequestAttempt\x12\x10\n\x08priority\x18\x0b \x01(\x05\x12\x11\n\tmandatory\x18\x0c \x01(\x08\"5\n\x0eRequestAttempt\x12\x12\n\nrequest_id\x18\x01 \x01(\t\x12\x0f\n\x07\x61ttempt\x18\x02 \x01(\x03\"\xd0\x02\n\x0bGoalReceipt\x12\x19\n\x11worker_session_id\x18\x01 \x01(\t\x12\x13\n\x0b\x63ommand_seq\x18\x02 \x01(\x04\x12\x0f\n\x07goal_id\x18\x03 \x01(\t\x12\x12\n\nrelease_id\x18\x04 \x01(\t\x12\x34\n\x06status\x18\x05 \x01(\x0e\x32$.cozy.scheduler.v1.GoalReceiptStatus\x12\x39\n\nerror_code\x18\x06 \x01(\x0e\x32%.cozy.scheduler.v1.LifecycleErrorCode\x12\x36\n\nrejections\x18\x07 \x03(\x0b\x32\".cozy.scheduler.v1.IntentRejection\x12\x0e\n\x06\x64\x65tail\x18\x08 \x01(\t\x12\x1b\n\x13received_at_unix_ms\x18\t \x01(\x03\x12\x16\n\x0e\x63ommand_digest\x18\n \x01(\x0c\"o\n\x0fIntentRejection\x12\x11\n\tintent_id\x18\x01 \x01(\t\x12\x39\n\nerror_code\x18\x02 \x01(\x0e\x32%.cozy.scheduler.v1.LifecycleErrorCode\x12\x0e\n\x06\x64\x65tail\x18\x03 \x01(\t\"R\n\x11LifecycleProgress\x12\x0c\n\x04\x64one\x18\x01 \x01(\x01\x12\r\n\x05total\x18\x02 \x01(\x01\x12\x0c\n\x04unit\x18\x03 \x01(\t\x12\x12\n\nrate_per_s\x18\x04 \x01(\x01\"\x98\x05\n\x0bIntentState\x12\x19\n\x11worker_session_id\x18\x01 \x01(\t\x12\x11\n\tstate_seq\x18\x02 \x01(\x04\x12\x0f\n\x07goal_id\x18\x03 \x01(\t\x12\x11\n\tintent_id\x18\x04 \x01(\t\x12\x12\n\nrelease_id\x18\x05 \x01(\t\x12\x19\n\x11\x63onfig_generation\x18\x06 \x01(\x04\x12\x38\n\x06status\x18\x07 \x01(\x0e\x32(.cozy.scheduler.v1.LifecycleIntentStatus\x12\x36\n\x05stage\x18\x08 \x01(\x0e\x32\'.cozy.scheduler.v1.LifecycleIntentStage\x12\x36\n\x06reason\x18\t \x01(\x0e\x32&.cozy.scheduler.v1.LifecycleWaitReason\x12\x15\n\rsince_unix_ms\x18\n \x01(\x03\x12\x1a\n\x12updated_at_unix_ms\x18\x0b \x01(\x03\x12\x1d\n\x15next_retry_at_unix_ms\x18\x0c \x01(\x03\x12\x1b\n\x13\x64\x65\x61\x64line_at_unix_ms\x18\r \x01(\x03\x12\x19\n\x11\x62locker_intent_id\x18\x0e \x01(\t\x12:\n\x0f\x62locker_request\x18\x0f \x01(\x0b\x32!.cozy.scheduler.v1.RequestAttempt\x12\x36\n\x08progress\x18\x10 \x01(\x0b\x32$.cozy.scheduler.v1.LifecycleProgress\x12\x39\n\nerror_code\x18\x11 \x01(\x0e\x32%.cozy.scheduler.v1.LifecycleErrorCode\x12\x0e\n\x06\x64\x65tail\x18\x12 \x01(\t\x12\x15\n\ractual_digest\x18\x13 \x01(\x0c\"\x83\x01\n\rModelIdentity\x12\x0b\n\x03ref\x18\x01 \x01(\t\x12\x17\n\x0fsnapshot_digest\x18\x02 \x01(\x0c\x12.\n\x04tier\x18\x03 \x01(\x0e\x32 .cozy.scheduler.v1.ResidencyTier\x12\x1c\n\x14residency_generation\x18\x04 \x01(\x04\"\xaa\x02\n\x12\x46unctionCapability\x12\x15\n\rfunction_name\x18\x01 \x01(\t\x12\x12\n\nrelease_id\x18\x02 \x01(\t\x12\x19\n\x11\x63onfig_generation\x18\x03 \x01(\x04\x12\x16\n\x0e\x62inding_digest\x18\x04 \x01(\x0c\x12\x0c\n\x04lane\x18\x05 \x01(\t\x12\x30\n\x06models\x18\x06 \x03(\x0b\x32 .cozy.scheduler.v1.ModelIdentity\x12%\n\x1d\x63ompile_target_incarnation_id\x18\x07 \x01(\t\x12\x39\n\x05state\x18\x08 \x01(\x0e\x32*.cozy.scheduler.v1.FunctionCapabilityState\x12\x14\n\x0cserving_tier\x18\t \x01(\t\"E\n\x0f\x43onfigClassMask\x12\x12\n\nparameters\x18\x01 \x01(\x08\x12\x10\n\x08\x62indings\x18\x02 \x01(\x08\x12\x0c\n\x04\x62oot\x18\x03 \x01(\x08\"\xf3\x02\n\x11\x43onfigApplication\x12\x12\n\nrelease_id\x18\x01 \x01(\t\x12\x19\n\x11target_generation\x18\x02 \x01(\x04\x12\x1b\n\x13received_generation\x18\x03 \x01(\x04\x12%\n\x1dparameter_snapshot_generation\x18\x04 \x01(\x04\x12 \n\x18\x62inding_ready_generation\x18\x05 \x01(\x04\x12\x17\n\x0f\x62oot_generation\x18\x06 \x01(\x04\x12\x38\n\x05state\x18\x07 \x01(\x0e\x32).cozy.scheduler.v1.ConfigApplicationState\x12;\n\x0fpending_classes\x18\x08 \x01(\x0b\x32\".cozy.scheduler.v1.ConfigClassMask\x12\x39\n\nerror_code\x18\t \x01(\x0e\x32%.cozy.scheduler.v1.LifecycleErrorCode\"\x89\x02\n\x0f\x44rainProjection\x12\x0f\n\x07goal_id\x18\x01 \x01(\t\x12\x11\n\tintent_id\x18\x02 \x01(\t\x12\x37\n\x06status\x18\x03 \x01(\x0e\x32\'.cozy.scheduler.v1.DrainLifecycleStatus\x12\x15\n\rsince_unix_ms\x18\x04 \x01(\x03\x12\x1a\n\x12updated_at_unix_ms\x18\x05 \x01(\x03\x12\x1b\n\x13\x64\x65\x61\x64line_at_unix_ms\x18\x06 \x01(\x03\x12\x39\n\nerror_code\x18\x07 \x01(\x0e\x32%.cozy.scheduler.v1.LifecycleErrorCode\x12\x0e\n\x06\x64\x65tail\x18\x08 \x01(\t\"\x8f\x03\n\x11LifecycleSnapshot\x12\x19\n\x11worker_session_id\x18\x01 \x01(\t\x12\x11\n\tstate_seq\x18\x02 \x01(\x04\x12/\n\x07intents\x18\x03 \x03(\x0b\x32\x1e.cozy.scheduler.v1.IntentState\x12;\n\x0c\x63\x61pabilities\x18\x04 \x03(\x0b\x32%.cozy.scheduler.v1.FunctionCapability\x12@\n\x12\x63onfig_application\x18\x05 \x01(\x0b\x32$.cozy.scheduler.v1.ConfigApplication\x12\x35\n\rgoal_receipts\x18\x06 \x03(\x0b\x32\x1e.cozy.scheduler.v1.GoalReceipt\x12\x31\n\x05\x64rain\x18\x07 \x01(\x0b\x32\".cozy.scheduler.v1.DrainProjection\x12\x14\n\x0c\x66ull_replace\x18\x08 \x01(\x08\x12\x1c\n\x14generated_at_unix_ms\x18\t \x01(\x03\"Y\n\x0f\x44\x65siredInstance\x12\x15\n\rfunction_name\x18\x01 \x01(\t\x12/\n\x06models\x18\x02 \x03(\x0b\x32\x1f.cozy.scheduler.v1.ModelBinding\"e\n\x0fModelResolution\x12\x0b\n\x03ref\x18\x01 \x01(\t\x12\x14\n\x0cresolved_ref\x18\x02 \x01(\t\x12\x0c\n\x04\x63\x61st\x18\x03 \x01(\t\x12\x0c\n\x04lane\x18\x04 \x01(\t\x12\x13\n\x0blane_pinned\x18\x05 \x01(\x08\"\x98\x03\n\nStateDelta\x12-\n\x05phase\x18\x01 \x01(\x0e\x32\x1e.cozy.scheduler.v1.WorkerPhase\x12\x1b\n\x13\x61vailable_functions\x18\x02 \x03(\t\x12\x19\n\x11loading_functions\x18\x03 \x03(\t\x12\x17\n\x0f\x66ree_vram_bytes\x18\x04 \x01(\x03\x12\x17\n\x0f\x66inalizing_jobs\x18\x05 \x01(\x05\x12%\n\x1dobserved_residency_generation\x18\x06 \x01(\x04\x12\x39\n\x0f\x63ompile_targets\x18\x07 \x03(\x0b\x32 .cozy.scheduler.v1.CompileTarget\x12\x33\n\x0c\x63\x65ll_lookups\x18\x08 \x03(\x0b\x32\x1d.cozy.scheduler.v1.CellLookup\x12\x36\n\ndisk_usage\x18\t \x01(\x0b\x32\".cozy.scheduler.v1.DiskUsageReport\x12\"\n\x1aobserved_config_generation\x18\n \x01(\x04\"\xac\x01\n\x10StorageTierUsage\x12,\n\x04tier\x18\x01 \x01(\x0e\x32\x1e.cozy.scheduler.v1.StorageTier\x12\x12\n\nmount_path\x18\x02 \x01(\t\x12\x13\n\x0btotal_bytes\x18\x03 \x01(\x03\x12\x12\n\nfree_bytes\x18\x04 \x01(\x03\x12\x12\n\nused_bytes\x18\x05 \x01(\x03\x12\x19\n\x11reclaimable_bytes\x18\x06 \x01(\x03\"b\n\x0f\x44iskUsageReport\x12\x32\n\x05tiers\x18\x01 \x03(\x0b\x32#.cozy.scheduler.v1.StorageTierUsage\x12\x1b\n\x13\x63\x61pacity_generation\x18\x02 \x01(\x04\".\n\nCellLookup\x12\x0e\n\x06\x66\x61mily\x18\x01 \x01(\t\x12\x10\n\x08\x63\x65ll_key\x18\x02 \x01(\t\"\xcc\x03\n\rCompileTarget\x12\x16\n\x0eincarnation_id\x18\x01 \x01(\t\x12\x0e\n\x06\x66\x61mily\x18\x02 \x01(\t\x12\x1c\n\x14pipeline_weight_lane\x18\x03 \x01(\t\x12\x13\n\x0blora_bucket\x18\x04 \x01(\x05\x12\x17\n\x0f\x63ontract_digest\x18\x05 \x01(\t\x12\x1a\n\x12\x61\x63tive_compile_ref\x18\x06 \x01(\t\x12&\n\x1e\x61\x63tive_compile_snapshot_digest\x18\x07 \x01(\t\x12\x16\n\x0e\x66unction_names\x18\x08 \x03(\t\x12?\n\x0emodel_bindings\x18\t \x03(\x0b\x32\'.cozy.scheduler.v1.CompileTargetBinding\x12\x1a\n\x12requested_cell_key\x18\n \x01(\t\x12T\n\x13requested_cell_axes\x18\x0b \x03(\x0b\x32\x37.cozy.scheduler.v1.CompileTarget.RequestedCellAxesEntry\x1a\x38\n\x16RequestedCellAxesEntry\x12\x0b\n\x03key\x18\x01 \x01(\t\x12\r\n\x05value\x18\x02 \x01(\t:\x02\x38\x01\"J\n\x14\x43ompileTargetBinding\x12\x0c\n\x04slot\x18\x01 \x01(\t\x12\x0b\n\x03ref\x18\x02 \x01(\t\x12\x17\n\x0fsnapshot_digest\x18\x03 \x01(\t\"\x8c\x05\n\x06RunJob\x12\x12\n\nrequest_id\x18\x01 \x01(\t\x12\x0f\n\x07\x61ttempt\x18\x02 \x01(\x03\x12\x15\n\rfunction_name\x18\x03 \x01(\t\x12\x15\n\rinput_payload\x18\x04 \x01(\x0c\x12\x12\n\ntimeout_ms\x18\x05 \x01(\x03\x12\x0b\n\x03org\x18\x06 \x01(\t\x12\x12\n\ninvoker_id\x18\x07 \x01(\t\x12\x18\n\x10\x63\x61pability_token\x18\x08 \x01(\t\x12\x32\n\x0boutput_mode\x18\t \x01(\x0e\x32\x1d.cozy.scheduler.v1.OutputMode\x12\x33\n\x07\x63ompute\x18\n \x01(\x0b\x32\".cozy.scheduler.v1.ResolvedCompute\x12/\n\x06models\x18\x0b \x03(\x0b\x32\x1f.cozy.scheduler.v1.ModelBinding\x12;\n\tsnapshots\x18\x0c \x03(\x0b\x32(.cozy.scheduler.v1.RunJob.SnapshotsEntry\x12\x45\n\x10required_compile\x18\r \x01(\x0b\x32+.cozy.scheduler.v1.RequiredCompileExecution\x12\x0c\n\x04lane\x18\x0e \x01(\t\x12\x33\n\x0cinput_assets\x18\x0f \x03(\x0b\x32\x1d.cozy.scheduler.v1.InputAsset\x12\x19\n\x11\x63onfig_generation\x18\x10 \x01(\x04\x12\x15\n\rconfig_params\x18\x11 \x01(\x0c\x1aM\n\x0eSnapshotsEntry\x12\x0b\n\x03key\x18\x01 \x01(\t\x12*\n\x05value\x18\x02 \x01(\x0b\x32\x1b.cozy.scheduler.v1.Snapshot:\x02\x38\x01\"w\n\nInputAsset\x12\x10\n\x08\x61sset_id\x18\x01 \x01(\t\x12\x12\n\nsource_ref\x18\x02 \x01(\t\x12\x0e\n\x06\x62lake3\x18\x03 \x01(\t\x12\x12\n\nsize_bytes\x18\x04 \x01(\x03\x12\x0c\n\x04kind\x18\x05 \x01(\t\x12\x11\n\tmime_type\x18\x06 \x01(\t\"\x82\x01\n\x18RequiredCompileExecution\x12\x1d\n\x15target_incarnation_id\x18\x01 \x01(\t\x12\x10\n\x08\x63\x65ll_ref\x18\x02 \x01(\t\x12\x1c\n\x14\x63\x65ll_snapshot_digest\x18\x03 \x01(\t\x12\x17\n\x0f\x63ontract_digest\x18\x04 \x01(\t\"9\n\x0fResolvedCompute\x12\x13\n\x0b\x61\x63\x63\x65lerator\x18\x01 \x01(\t\x12\x11\n\tgpu_index\x18\x02 \x01(\x05\"\xac\x02\n\x0cModelBinding\x12\x0c\n\x04slot\x18\x01 \x01(\t\x12\x0b\n\x03ref\x18\x02 \x01(\t\x12-\n\x05loras\x18\x03 \x03(\x0b\x32\x1e.cozy.scheduler.v1.LoraOverlay\x12\x1a\n\x12inference_defaults\x18\x04 \x01(\t\x12\x43\n\ncomponents\x18\x05 \x03(\x0b\x32/.cozy.scheduler.v1.ModelBinding.ComponentsEntry\x12\x11\n\tobjective\x18\x06 \x01(\t\x12\x11\n\tdistilled\x18\x07 \x01(\x08\x12\x18\n\x10\x64istilled_status\x18\x08 \x01(\t\x1a\x31\n\x0f\x43omponentsEntry\x12\x0b\n\x03key\x18\x01 \x01(\t\x12\r\n\x05value\x18\x02 \x01(\t:\x02\x38\x01\"F\n\x0bLoraOverlay\x12\x0b\n\x03ref\x18\x01 \x01(\t\x12\x0e\n\x06weight\x18\x02 \x01(\x01\x12\x1a\n\x12inference_defaults\x18\x03 \x01(\t\"J\n\x08Snapshot\x12\x0e\n\x06\x64igest\x18\x01 \x01(\t\x12.\n\x05\x66iles\x18\x02 \x03(\x0b\x32\x1f.cozy.scheduler.v1.SnapshotFile\"\xa4\x01\n\x0cSnapshotFile\x12\x0c\n\x04path\x18\x01 \x01(\t\x12\x12\n\nsize_bytes\x18\x02 \x01(\x03\x12\x0e\n\x06\x62lake3\x18\x03 \x01(\t\x12\x0b\n\x03url\x18\x04 \x01(\t\x12\x0e\n\x06\x64igest\x18\x05 \x01(\t\x12\x18\n\x10\x63hunk_size_bytes\x18\x06 \x01(\x03\x12+\n\x06\x63hunks\x18\x07 \x03(\x0b\x32\x1b.cozy.scheduler.v1.ChunkRef\"4\n\x08\x43hunkRef\x12\x0e\n\x06sha256\x18\x01 \x01(\t\x12\x0b\n\x03url\x18\x02 \x01(\t\x12\x0b\n\x03len\x18\x03 \x01(\x03\"\xb3\x01\n\nRunAttempt\x12-\n\x07\x61ttempt\x18\x01 \x01(\x0b\x32\x1c.cozy.scheduler.v1.AttemptId\x12.\n\x04spec\x18\x02 \x01(\x0b\x32 .cozy.scheduler.v1.ExecutionSpec\x12/\n\x05grant\x18\x03 \x01(\x0b\x32 .cozy.scheduler.v1.DeliveryGrant\x12\x15\n\rinput_payload\x18\x04 \x01(\x0c\"0\n\tAttemptId\x12\x12\n\nrequest_id\x18\x01 \x01(\t\x12\x0f\n\x07\x61ttempt\x18\x02 \x01(\x03\"\xc4\x05\n\rExecutionSpec\x12\x0e\n\x06\x64igest\x18\x01 \x01(\t\x12\x14\n\x0cspec_version\x18\x02 \x01(\r\x12\x33\n\x07release\x18\x03 \x01(\x0b\x32\".cozy.scheduler.v1.EndpointRelease\x12\x15\n\rfunction_name\x18\x04 \x01(\t\x12\x38\n\x0enumerical_lane\x18\x05 \x01(\x0b\x32 .cozy.scheduler.v1.ExecutionLane\x12#\n\x03\x61rm\x18\x06 \x01(\x0b\x32\x16.cozy.scheduler.v1.Arm\x12-\n\x08topology\x18\x07 \x01(\x0b\x32\x1b.cozy.scheduler.v1.Topology\x12\x38\n\ncomponents\x18\x08 \x01(\x0b\x32$.cozy.scheduler.v1.ComponentManifest\x12\x33\n\x08\x61\x64\x61pters\x18\t \x03(\x0b\x32!.cozy.scheduler.v1.AdapterBinding\x12\x31\n\x06\x63onfig\x18\n \x01(\x0b\x32!.cozy.scheduler.v1.ConfigSnapshot\x12\x38\n\tcontracts\x18\x0b \x01(\x0b\x32%.cozy.scheduler.v1.ContractIdentities\x12\x36\n\x0b\x65xploration\x18\x0c \x01(\x0b\x32!.cozy.scheduler.v1.ExplorationRef\x12\x33\n\x0b\x61ttribution\x18\r \x01(\x0b\x32\x1e.cozy.scheduler.v1.Attribution\x12\x32\n\x0boutput_mode\x18\x0e \x01(\x0e\x32\x1d.cozy.scheduler.v1.OutputMode\x12\x36\n\x0cinput_assets\x18\x0f \x03(\x0b\x32 .cozy.scheduler.v1.InputAssetRef\"s\n\x0f\x45ndpointRelease\x12\x0b\n\x03org\x18\x01 \x01(\t\x12\x10\n\x08\x65ndpoint\x18\x02 \x01(\t\x12\x12\n\nrelease_id\x18\x03 \x01(\t\x12\x14\n\x0cimage_digest\x18\x04 \x01(\t\x12\x17\n\x0f\x63ode_closure_id\x18\x05 \x01(\t\".\n\x0b\x41ttribution\x12\x0b\n\x03org\x18\x01 \x01(\t\x12\x12\n\ninvoker_id\x18\x02 \x01(\t\"R\n\rExecutionLane\x12.\n\x07weights\x18\x01 \x01(\x0e\x32\x1d.cozy.scheduler.v1.WeightLane\x12\x11\n\tmandatory\x18\x02 \x01(\x08\"\xd7\x01\n\x03\x41rm\x12\x1d\n\x15graph_contract_digest\x18\x01 \x01(\t\x12*\n\x05shape\x18\x02 \x01(\x0e\x32\x1b.cozy.scheduler.v1.ArmShape\x12\x1b\n\x13\x61\x64\x61pter_rank_bucket\x18\x03 \x01(\x05\x12\x31\n\x07\x62\x61\x63kend\x18\x04 \x01(\x0e\x32 .cozy.scheduler.v1.SteadyBackend\x12\x35\n\x08\x61rtifact\x18\x05 \x01(\x0b\x32#.cozy.scheduler.v1.ArtifactIdentity\"\x98\x01\n\x10\x41rtifactIdentity\x12\x10\n\x08\x63\x65ll_ref\x18\x01 \x01(\t\x12\x16\n\x0e\x63ontent_digest\x18\x02 \x01(\t\x12\x10\n\x08\x63\x65ll_key\x18\x03 \x01(\t\x12\x15\n\rpublisher_org\x18\x04 \x01(\t\x12\x18\n\x10toolchain_digest\x18\x05 \x01(\t\x12\x17\n\x0f\x65nv_seal_digest\x18\x06 \x01(\t\"e\n\x08Topology\x12\x13\n\x0b\x61\x63\x63\x65lerator\x18\x01 \x01(\t\x12\x11\n\tgpu_count\x18\x02 \x01(\x05\x12\x18\n\x10\x65xecution_groups\x18\x03 \x01(\x05\x12\x17\n\x0f\x64\x65vice_ordinals\x18\x04 \x03(\x05\"B\n\x11\x43omponentManifest\x12-\n\x05slots\x18\x01 \x03(\x0b\x32\x1e.cozy.scheduler.v1.SlotBinding\"\xd2\x01\n\x0bSlotBinding\x12\x0c\n\x04slot\x18\x01 \x01(\t\x12\x0b\n\x03ref\x18\x02 \x01(\t\x12\x17\n\x0fsnapshot_digest\x18\x03 \x01(\t\x12\x33\n\ncomponents\x18\x04 \x03(\x0b\x32\x1f.cozy.scheduler.v1.ComponentRef\x12\x11\n\tobjective\x18\x05 \x01(\t\x12\x11\n\tdistilled\x18\x06 \x01(\x08\x12\x18\n\x10\x64istilled_status\x18\x07 \x01(\t\x12\x1a\n\x12inference_defaults\x18\x08 \x01(\t\"G\n\x0c\x43omponentRef\x12\x11\n\tcomponent\x18\x01 \x01(\t\x12\x0b\n\x03ref\x18\x02 \x01(\t\x12\x17\n\x0fsnapshot_digest\x18\x03 \x01(\t\"p\n\x0e\x41\x64\x61pterBinding\x12\x0c\n\x04slot\x18\x01 \x01(\t\x12\x0b\n\x03ref\x18\x02 \x01(\t\x12\x17\n\x0fsnapshot_digest\x18\x03 \x01(\t\x12\x0e\n\x06weight\x18\x04 \x01(\x01\x12\x1a\n\x12inference_defaults\x18\x05 \x01(\t\"0\n\x0e\x43onfigSnapshot\x12\x0e\n\x06values\x18\x01 \x01(\x0c\x12\x0e\n\x06\x64igest\x18\x02 \x01(\t\"U\n\x12\x43ontractIdentities\x12\x1f\n\x17quality_contract_digest\x18\x01 \x01(\t\x12\x1e\n\x16runtime_formula_digest\x18\x02 \x01(\t\"D\n\x0e\x45xplorationRef\x12\x16\n\x0e\x65xploration_id\x18\x01 \x01(\t\x12\x1a\n\x12operator_intent_id\x18\x02 \x01(\t\"z\n\rInputAssetRef\x12\x10\n\x08\x61sset_id\x18\x01 \x01(\t\x12\x12\n\nsource_ref\x18\x02 \x01(\t\x12\x0e\n\x06\x64igest\x18\x03 \x01(\t\x12\x12\n\nsize_bytes\x18\x04 \x01(\x03\x12\x0c\n\x04kind\x18\x05 \x01(\t\x12\x11\n\tmime_type\x18\x06 \x01(\t\"\xa6\x02\n\rDeliveryGrant\x12\x18\n\x10\x63\x61pability_token\x18\x01 \x01(\t\x12\x17\n\x0f\x65xpires_unix_ms\x18\x02 \x01(\x03\x12\r\n\x05\x65poch\x18\x03 \x01(\x04\x12\x42\n\tsnapshots\x18\x04 \x03(\x0b\x32/.cozy.scheduler.v1.DeliveryGrant.SnapshotsEntry\x12\x37\n\x0cinput_assets\x18\x05 \x03(\x0b\x32!.cozy.scheduler.v1.PresignedAsset\x1aV\n\x0eSnapshotsEntry\x12\x0b\n\x03key\x18\x01 \x01(\t\x12\x33\n\x05value\x18\x02 \x01(\x0b\x32$.cozy.scheduler.v1.PresignedSnapshot:\x02\x38\x01\"D\n\x11PresignedSnapshot\x12/\n\x05\x66iles\x18\x01 \x03(\x0b\x32 .cozy.scheduler.v1.PresignedFile\"\x95\x01\n\rPresignedFile\x12\x0c\n\x04path\x18\x01 \x01(\t\x12\x12\n\nsize_bytes\x18\x02 \x01(\x03\x12\x0e\n\x06\x64igest\x18\x03 \x01(\t\x12\x0b\n\x03url\x18\x04 \x01(\t\x12\x18\n\x10\x63hunk_size_bytes\x18\x05 \x01(\x03\x12+\n\x06\x63hunks\x18\x06 \x03(\x0b\x32\x1b.cozy.scheduler.v1.ChunkRef\"/\n\x0ePresignedAsset\x12\x10\n\x08\x61sset_id\x18\x01 \x01(\t\x12\x0b\n\x03url\x18\x02 \x01(\t\"2\n\x0bJobAccepted\x12\x12\n\nrequest_id\x18\x01 \x01(\t\x12\x0f\n\x07\x61ttempt\x18\x02 \x01(\x03\"\x88\x02\n\tJobResult\x12\x12\n\nrequest_id\x18\x01 \x01(\t\x12\x0f\n\x07\x61ttempt\x18\x02 \x01(\x03\x12,\n\x06status\x18\x03 \x01(\x0e\x32\x1c.cozy.scheduler.v1.JobStatus\x12\x10\n\x06inline\x18\x04 \x01(\x0cH\x00\x12\x12\n\x08\x62lob_ref\x18\x05 \x01(\tH\x00\x12\x14\n\x0csafe_message\x18\x06 \x01(\t\x12.\n\x07metrics\x18\x07 \x01(\x0b\x32\x1d.cozy.scheduler.v1.JobMetrics\x12\x32\n\x0b\x61\x64justments\x18\x08 \x03(\x0b\x32\x1d.cozy.scheduler.v1.AdjustmentB\x08\n\x06output\"O\n\nAdjustment\x12\r\n\x05\x66ield\x18\x01 \x01(\t\x12\x11\n\trequested\x18\x02 \x01(\t\x12\x0f\n\x07\x61pplied\x18\x03 \x01(\t\x12\x0e\n\x06reason\x18\x04 \x01(\t\"\xce\x05\n\nJobMetrics\x12\x12\n\nruntime_ms\x18\x01 \x01(\x03\x12\x10\n\x08queue_ms\x18\x02 \x01(\x03\x12\x18\n\x10rss_at_end_bytes\x18\x03 \x01(\x03\x12\x17\n\x0fpeak_vram_bytes\x18\x04 \x01(\x03\x12\x1c\n\x14\x63oncurrency_at_start\x18\x05 \x01(\x05\x12\x1f\n\x17output_media_duration_s\x18\x06 \x01(\x01\x12\x14\n\x0cinput_tokens\x18\x07 \x01(\x03\x12\x1b\n\x13input_cached_tokens\x18\x08 \x01(\x03\x12\x15\n\routput_tokens\x18\t \x01(\x03\x12\x14\n\x0coutput_count\x18\n \x01(\x03\x12\x14\n\x0cslot_held_ms\x18\x0b \x01(\x03\x12\x18\n\x10\x66inalize_wall_ms\x18\x0c \x01(\x03\x12\x0c\n\x04lane\x18\r \x01(\t\x12\x46\n\rruntime_terms\x18\x0e \x03(\x0b\x32/.cozy.scheduler.v1.JobMetrics.RuntimeTermsEntry\x12<\n\x08stage_ms\x18\x0f \x03(\x0b\x32*.cozy.scheduler.v1.JobMetrics.StageMsEntry\x12\x14\n\x0cserving_mode\x18\x10 \x01(\t\x12\x17\n\x0fserved_cell_ref\x18\x11 \x01(\t\x12\x1d\n\x15served_eager_fallback\x18\x12 \x01(\x08\x12\x17\n\x0f\x66\x61llback_reason\x18\x13 \x01(\t\x12\n\n\x02sm\x18\x14 \x01(\t\x12\r\n\x05steps\x18\x15 \x01(\x03\x12\r\n\x05width\x18\x16 \x01(\x03\x12\x0e\n\x06height\x18\x17 \x01(\x03\x1a\x33\n\x11RuntimeTermsEntry\x12\x0b\n\x03key\x18\x01 \x01(\t\x12\r\n\x05value\x18\x02 \x01(\x01:\x02\x38\x01\x1a.\n\x0cStageMsEntry\x12\x0b\n\x03key\x18\x01 \x01(\t\x12\r\n\x05value\x18\x02 \x01(\x03:\x02\x38\x01\"c\n\x0bJobProgress\x12\x12\n\nrequest_id\x18\x01 \x01(\t\x12\x0f\n\x07\x61ttempt\x18\x02 \x01(\x03\x12\x0b\n\x03seq\x18\x03 \x01(\x03\x12\x0c\n\x04\x64\x61ta\x18\x04 \x01(\x0c\x12\x14\n\x0c\x63ontent_type\x18\x05 \x01(\t\"0\n\tCancelJob\x12\x12\n\nrequest_id\x18\x01 \x01(\t\x12\x0f\n\x07\x61ttempt\x18\x02 \x01(\x03\"\xa6\x01\n\x07ModelOp\x12*\n\x02op\x18\x01 \x01(\x0e\x32\x1e.cozy.scheduler.v1.ModelOpKind\x12\x0b\n\x03ref\x18\x02 \x01(\t\x12-\n\x08snapshot\x18\x03 \x01(\x0b\x32\x1b.cozy.scheduler.v1.Snapshot\x12\x14\n\x0coperation_id\x18\x04 \x01(\t\x12\x1d\n\x15target_incarnation_id\x18\x05 \x01(\t\"\x9e\x04\n\nModelEvent\x12\x0b\n\x03ref\x18\x01 \x01(\t\x12,\n\x05state\x18\x02 \x01(\x0e\x32\x1d.cozy.scheduler.v1.ModelState\x12\x12\n\nvram_bytes\x18\x03 \x01(\x03\x12\r\n\x05\x65rror\x18\x04 \x01(\t\x12\x12\n\nbytes_done\x18\x05 \x01(\x03\x12\x13\n\x0b\x62ytes_total\x18\x06 \x01(\x03\x12\x13\n\x0b\x64uration_ms\x18\x07 \x01(\x03\x12\x12\n\ncache_hits\x18\x08 \x01(\x03\x12\x14\n\x0c\x63\x61\x63he_misses\x18\t \x01(\x03\x12\x10\n\x08warmup_s\x18\n \x01(\x01\x12\x1f\n\x17host_ram_required_bytes\x18\x0b \x01(\x03\x12\'\n\x1fhost_ram_available_before_bytes\x18\x0c \x01(\x03\x12&\n\x1ehost_ram_available_after_bytes\x18\r \x01(\x03\x12\x1d\n\x15host_ram_evicted_refs\x18\x0e \x03(\t\x12$\n\x1chost_ram_capacity_generation\x18\x0f \x01(\x04\x12\x17\n\x0fsnapshot_digest\x18\x10 \x01(\t\x12\x1c\n\x14residency_generation\x18\x11 \x01(\x04\x12\x14\n\x0coperation_id\x18\x12 \x01(\t\x12\x1d\n\x15target_incarnation_id\x18\x13 \x01(\t\x12\x15\n\rnetwork_bytes\x18\x14 \x01(\x03\"\xf4\x02\n\x0e\x41\x63tivityUpdate\x12\x0c\n\x04kind\x18\x01 \x01(\t\x12\r\n\x05phase\x18\x02 \x01(\t\x12\x0c\n\x04step\x18\x03 \x01(\x03\x12\x13\n\x0btotal_steps\x18\x04 \x01(\x03\x12\x0b\n\x03seq\x18\x05 \x01(\x04\x12/\n\x05state\x18\x06 \x01(\x0e\x32 .cozy.scheduler.v1.ActivityState\x12\r\n\x05\x65rror\x18\x07 \x01(\t\x12\x0e\n\x06\x64\x65tail\x18\x08 \x01(\t\x12\x1a\n\x12updated_at_unix_ms\x18\t \x01(\x03\x12\x0f\n\x07\x63ounter\x18\n \x01(\t\x12\x14\n\x0c\x63ounter_unit\x18\x0b \x01(\t\x12\x14\n\x0c\x63ounter_done\x18\x0c \x01(\x01\x12\x15\n\rcounter_total\x18\r \x01(\x01\x12\x12\n\nrate_per_s\x18\x0e \x01(\x01\x12\x14\n\x0cself_stalled\x18\x0f \x01(\x08\x12\x16\n\x0estalled_for_ms\x18\x10 \x01(\x03\x12\x13\n\x0b\x64uration_ms\x18\x11 \x01(\x03\"\xe2\x02\n\tBootPhase\x12\x0f\n\x07\x62oot_id\x18\x01 \x01(\t\x12\x0f\n\x07ordinal\x18\x02 \x01(\x03\x12\x16\n\x0eparent_ordinal\x18\x03 \x01(\x03\x12\r\n\x05phase\x18\x04 \x01(\t\x12\x10\n\x08terminal\x18\x05 \x01(\x08\x12\x1a\n\x12started_at_unix_ms\x18\x06 \x01(\x03\x12\x13\n\x0b\x64uration_ms\x18\x07 \x01(\x03\x12\x19\n\x11process_uptime_ms\x18\x08 \x01(\x03\x12\r\n\x05\x62ytes\x18\t \x01(\x03\x12\x0e\n\x06source\x18\n \x01(\t\x12\x0b\n\x03ref\x18\x0b \x01(\t\x12\x15\n\rartifact_kind\x18\x0c \x01(\t\x12\x14\n\x0c\x61rtifact_key\x18\r \x01(\t\x12\x10\n\x08\x66unction\x18\x0e \x01(\t\x12\x0f\n\x07outcome\x18\x0f \x01(\t\x12\x0e\n\x06reason\x18\x10 \x01(\t\x12\x0e\n\x06\x64\x65tail\x18\x11 \x01(\t\x12\x12\n\ncumulative\x18\x12 \x01(\x08\"\xad\x01\n\rFnUnavailable\x12\x15\n\rfunction_name\x18\x01 \x01(\t\x12\x0e\n\x06reason\x18\x02 \x01(\t\x12\x0e\n\x06\x64\x65tail\x18\x03 \x01(\t\x12\x38\n\x04\x61xes\x18\x04 \x03(\x0b\x32*.cozy.scheduler.v1.FnUnavailable.AxesEntry\x1a+\n\tAxesEntry\x12\x0b\n\x03key\x18\x01 \x01(\t\x12\r\n\x05value\x18\x02 \x01(\t:\x02\x38\x01\"\x8d\x01\n\nFnDegraded\x12\x15\n\rfunction_name\x18\x01 \x01(\t\x12\x0e\n\x06wanted\x18\x02 \x01(\t\x12\x0b\n\x03ran\x18\x03 \x01(\t\x12\x0e\n\x06reason\x18\x04 \x01(\t\x12\x1e\n\x16\x65st_latency_multiplier\x18\x05 \x01(\x01\x12\x1b\n\x13recommended_vram_gb\x18\x06 \x01(\x01\"\x1c\n\x05\x44rain\x12\x13\n\x0b\x64\x65\x61\x64line_ms\x18\x01 \x01(\x03\"6\n\x0cTokenRefresh\x12\r\n\x05token\x18\x01 \x01(\t\x12\x17\n\x0f\x65xpires_at_unix\x18\x02 \x01(\x03\"A\n\x0cServePosture\x12\x12\n\neager_only\x18\x01 \x01(\x08\x12\x0e\n\x06reason\x18\x02 \x01(\t\x12\r\n\x05\x61\x63tor\x18\x03 \x01(\t*Q\n\x0fProtocolVersion\x12 \n\x1cPROTOCOL_VERSION_UNSPECIFIED\x10\x00\x12\x1c\n\x18PROTOCOL_VERSION_CURRENT\x10\x01*y\n\rResidencyTier\x12\x1e\n\x1aRESIDENCY_TIER_UNSPECIFIED\x10\x00\x12\x17\n\x13RESIDENCY_TIER_DISK\x10\x01\x12\x16\n\x12RESIDENCY_TIER_RAM\x10\x02\x12\x17\n\x13RESIDENCY_TIER_VRAM\x10\x03*\xf1\x01\n\x11\x44\x65siredIntentKind\x12#\n\x1f\x44\x45SIRED_INTENT_KIND_UNSPECIFIED\x10\x00\x12#\n\x1f\x44\x45SIRED_INTENT_KIND_MATERIALIZE\x10\x01\x12&\n\"DESIRED_INTENT_KIND_FUNCTION_READY\x10\x02\x12$\n DESIRED_INTENT_KIND_CONFIG_APPLY\x10\x03\x12%\n!DESIRED_INTENT_KIND_COMPILE_ADOPT\x10\x04\x12\x1d\n\x19\x44\x45SIRED_INTENT_KIND_DRAIN\x10\x05*\x95\x02\n\x12\x44\x65siredIntentCause\x12$\n DESIRED_INTENT_CAUSE_UNSPECIFIED\x10\x00\x12\"\n\x1e\x44\x45SIRED_INTENT_CAUSE_COLD_BOOT\x10\x01\x12 \n\x1c\x44\x45SIRED_INTENT_CAUSE_REQUEST\x10\x02\x12$\n DESIRED_INTENT_CAUSE_PREPOSITION\x10\x03\x12&\n\"DESIRED_INTENT_CAUSE_CONFIG_CHANGE\x10\x04\x12$\n DESIRED_INTENT_CAUSE_REPLACEMENT\x10\x05\x12\x1f\n\x1b\x44\x45SIRED_INTENT_CAUSE_RETIRE\x10\x06*|\n\x11GoalReceiptStatus\x12#\n\x1fGOAL_RECEIPT_STATUS_UNSPECIFIED\x10\x00\x12 \n\x1cGOAL_RECEIPT_STATUS_ACCEPTED\x10\x01\x12 \n\x1cGOAL_RECEIPT_STATUS_REJECTED\x10\x02*\xc9\x02\n\x15LifecycleIntentStatus\x12\'\n#LIFECYCLE_INTENT_STATUS_UNSPECIFIED\x10\x00\x12$\n LIFECYCLE_INTENT_STATUS_ACCEPTED\x10\x01\x12#\n\x1fLIFECYCLE_INTENT_STATUS_WAITING\x10\x02\x12#\n\x1fLIFECYCLE_INTENT_STATUS_RUNNING\x10\x03\x12%\n!LIFECYCLE_INTENT_STATUS_SUCCEEDED\x10\x04\x12\"\n\x1eLIFECYCLE_INTENT_STATUS_FAILED\x10\x05\x12$\n LIFECYCLE_INTENT_STATUS_CANCELED\x10\x06\x12&\n\"LIFECYCLE_INTENT_STATUS_SUPERSEDED\x10\x07*\xc9\x08\n\x14LifecycleIntentStage\x12&\n\"LIFECYCLE_INTENT_STAGE_UNSPECIFIED\x10\x00\x12%\n!LIFECYCLE_INTENT_STAGE_VALIDATING\x10\x01\x12+\n\'LIFECYCLE_INTENT_STAGE_WAIT_TENANT_IDLE\x10\x02\x12(\n$LIFECYCLE_INTENT_STAGE_WAIT_REF_LOCK\x10\x03\x12)\n%LIFECYCLE_INTENT_STAGE_WAIT_LOAD_LOCK\x10\x04\x12(\n$LIFECYCLE_INTENT_STAGE_WAIT_GPU_SLOT\x10\x05\x12(\n$LIFECYCLE_INTENT_STAGE_WAIT_SNAPSHOT\x10\x06\x12-\n)LIFECYCLE_INTENT_STAGE_WAIT_DISK_HEADROOM\x10\x07\x12(\n$LIFECYCLE_INTENT_STAGE_WAIT_HOST_RAM\x10\x08\x12-\n)LIFECYCLE_INTENT_STAGE_WAIT_NETWORK_RETRY\x10\t\x12+\n\'LIFECYCLE_INTENT_STAGE_WAIT_REPLACEMENT\x10\n\x12#\n\x1fLIFECYCLE_INTENT_STAGE_FETCHING\x10\x0b\x12$\n LIFECYCLE_INTENT_STAGE_VERIFYING\x10\x0c\x12\"\n\x1eLIFECYCLE_INTENT_STAGE_ON_DISK\x10\r\x12\'\n#LIFECYCLE_INTENT_STAGE_LOADING_HOST\x10\x0e\x12)\n%LIFECYCLE_INTENT_STAGE_LOADING_DEVICE\x10\x0f\x12\"\n\x1eLIFECYCLE_INTENT_STAGE_WARMING\x10\x10\x12$\n LIFECYCLE_INTENT_STAGE_COMPILING\x10\x11\x12#\n\x1fLIFECYCLE_INTENT_STAGE_ADOPTING\x10\x12\x12/\n+LIFECYCLE_INTENT_STAGE_CONFIG_MATERIALIZING\x10\x13\x12\x33\n/LIFECYCLE_INTENT_STAGE_CONFIG_BINDINGS_APPLYING\x10\x14\x12,\n(LIFECYCLE_INTENT_STAGE_CONFIG_BOOT_STALE\x10\x15\x12 \n\x1cLIFECYCLE_INTENT_STAGE_READY\x10\x16\x12#\n\x1fLIFECYCLE_INTENT_STAGE_DRAINING\x10\x17\x12%\n!LIFECYCLE_INTENT_STAGE_FINALIZING\x10\x18\x12#\n\x1fLIFECYCLE_INTENT_STAGE_FLUSHING\x10\x19*\xc0\x03\n\x13LifecycleWaitReason\x12%\n!LIFECYCLE_WAIT_REASON_UNSPECIFIED\x10\x00\x12%\n!LIFECYCLE_WAIT_REASON_TENANT_WORK\x10\x01\x12\"\n\x1eLIFECYCLE_WAIT_REASON_REF_LOCK\x10\x02\x12#\n\x1fLIFECYCLE_WAIT_REASON_LOAD_LOCK\x10\x03\x12\"\n\x1eLIFECYCLE_WAIT_REASON_GPU_SLOT\x10\x04\x12\"\n\x1eLIFECYCLE_WAIT_REASON_SNAPSHOT\x10\x05\x12\'\n#LIFECYCLE_WAIT_REASON_DISK_HEADROOM\x10\x06\x12\"\n\x1eLIFECYCLE_WAIT_REASON_HOST_RAM\x10\x07\x12\'\n#LIFECYCLE_WAIT_REASON_NETWORK_RETRY\x10\x08\x12%\n!LIFECYCLE_WAIT_REASON_REPLACEMENT\x10\t\x12-\n)LIFECYCLE_WAIT_REASON_SINGLE_FLIGHT_OWNER\x10\n*\xa6\x05\n\x12LifecycleErrorCode\x12$\n LIFECYCLE_ERROR_CODE_UNSPECIFIED\x10\x00\x12\x30\n,LIFECYCLE_ERROR_CODE_MISSING_MANDATORY_FIELD\x10\x01\x12\x30\n,LIFECYCLE_ERROR_CODE_WORKER_SESSION_MISMATCH\x10\x02\x12)\n%LIFECYCLE_ERROR_CODE_RELEASE_MISMATCH\x10\x03\x12/\n+LIFECYCLE_ERROR_CODE_COMMAND_SEQ_REGRESSION\x10\x04\x12-\n)LIFECYCLE_ERROR_CODE_COMMAND_SEQ_CONFLICT\x10\x05\x12*\n&LIFECYCLE_ERROR_CODE_CONFIG_REGRESSION\x10\x06\x12)\n%LIFECYCLE_ERROR_CODE_UNKNOWN_FUNCTION\x10\x07\x12\x32\n.LIFECYCLE_ERROR_CODE_SNAPSHOT_IDENTITY_MISSING\x10\x08\x12+\n\'LIFECYCLE_ERROR_CODE_UNSUPPORTED_INTENT\x10\t\x12\x32\n.LIFECYCLE_ERROR_CODE_UNKNOWN_MANDATORY_COMMAND\x10\n\x12\x31\n-LIFECYCLE_ERROR_CODE_PROTOCOL_UNREPORTED_WAIT\x10\x0b\x12\x35\n1LIFECYCLE_ERROR_CODE_CONFIG_SNAPSHOT_WRITE_FAILED\x10\x0c\x12%\n!LIFECYCLE_ERROR_CODE_DRAIN_FAILED\x10\r*\xe1\x01\n\x17\x46unctionCapabilityState\x12)\n%FUNCTION_CAPABILITY_STATE_UNSPECIFIED\x10\x00\x12#\n\x1f\x46UNCTION_CAPABILITY_STATE_READY\x10\x01\x12&\n\"FUNCTION_CAPABILITY_STATE_APPLYING\x10\x02\x12(\n$FUNCTION_CAPABILITY_STATE_BOOT_STALE\x10\x03\x12$\n FUNCTION_CAPABILITY_STATE_FAILED\x10\x04*\xdf\x01\n\x16\x43onfigApplicationState\x12(\n$CONFIG_APPLICATION_STATE_UNSPECIFIED\x10\x00\x12%\n!CONFIG_APPLICATION_STATE_APPLYING\x10\x01\x12&\n\"CONFIG_APPLICATION_STATE_CONVERGED\x10\x02\x12\'\n#CONFIG_APPLICATION_STATE_BOOT_STALE\x10\x03\x12#\n\x1f\x43ONFIG_APPLICATION_STATE_FAILED\x10\x04*\x9b\x02\n\x14\x44rainLifecycleStatus\x12&\n\"DRAIN_LIFECYCLE_STATUS_UNSPECIFIED\x10\x00\x12#\n\x1f\x44RAIN_LIFECYCLE_STATUS_ACCEPTED\x10\x01\x12#\n\x1f\x44RAIN_LIFECYCLE_STATUS_DRAINING\x10\x02\x12%\n!DRAIN_LIFECYCLE_STATUS_FINALIZING\x10\x03\x12#\n\x1f\x44RAIN_LIFECYCLE_STATUS_FLUSHING\x10\x04\x12\"\n\x1e\x44RAIN_LIFECYCLE_STATUS_DRAINED\x10\x05\x12!\n\x1d\x44RAIN_LIFECYCLE_STATUS_FAILED\x10\x06*v\n\x0bStorageTier\x12\x1c\n\x18STORAGE_TIER_UNSPECIFIED\x10\x00\x12\x1a\n\x16STORAGE_TIER_CONTAINER\x10\x01\x12\x17\n\x13STORAGE_TIER_VOLUME\x10\x02\x12\x14\n\x10STORAGE_TIER_NFS\x10\x03*\xd8\x01\n\x0bWorkerPhase\x12\x1c\n\x18WORKER_PHASE_UNSPECIFIED\x10\x00\x12\x18\n\x14WORKER_PHASE_BOOTING\x10\x01\x12#\n\x1fWORKER_PHASE_DOWNLOADING_MODELS\x10\x02\x12\"\n\x1eWORKER_PHASE_LOADING_PIPELINES\x10\x03\x12\x18\n\x14WORKER_PHASE_WARMING\x10\x04\x12\x16\n\x12WORKER_PHASE_READY\x10\x05\x12\x16\n\x12WORKER_PHASE_ERROR\x10\x06*V\n\nOutputMode\x12\x1b\n\x17OUTPUT_MODE_UNSPECIFIED\x10\x00\x12\x13\n\x0fOUTPUT_MODE_URL\x10\x01\x12\x16\n\x12OUTPUT_MODE_INLINE\x10\x02*\x82\x01\n\nWeightLane\x12\x1b\n\x17WEIGHT_LANE_UNSPECIFIED\x10\x00\x12\x14\n\x10WEIGHT_LANE_BF16\x10\x01\x12\x15\n\x11WEIGHT_LANE_W8A16\x10\x02\x12\x14\n\x10WEIGHT_LANE_W8A8\x10\x03\x12\x14\n\x10WEIGHT_LANE_W4A4\x10\x04*V\n\x08\x41rmShape\x12\x19\n\x15\x41RM_SHAPE_UNSPECIFIED\x10\x00\x12\x18\n\x14\x41RM_SHAPE_BRANCHLESS\x10\x01\x12\x15\n\x11\x41RM_SHAPE_ADAPTER\x10\x02*\x86\x01\n\rSteadyBackend\x12\x1e\n\x1aSTEADY_BACKEND_UNSPECIFIED\x10\x00\x12\x1b\n\x17STEADY_BACKEND_AOT_CELL\x10\x01\x12\x19\n\x15STEADY_BACKEND_DYNAMO\x10\x02\x12\x1d\n\x19STEADY_BACKEND_EAGER_ONLY\x10\x03*\x9b\x01\n\tJobStatus\x12\x1a\n\x16JOB_STATUS_UNSPECIFIED\x10\x00\x12\x11\n\rJOB_STATUS_OK\x10\x01\x12\x16\n\x12JOB_STATUS_INVALID\x10\x02\x12\x18\n\x14JOB_STATUS_RETRYABLE\x10\x03\x12\x14\n\x10JOB_STATUS_FATAL\x10\x04\x12\x17\n\x13JOB_STATUS_CANCELED\x10\x05*S\n\x0bModelOpKind\x12\x1d\n\x19MODEL_OP_KIND_UNSPECIFIED\x10\x00\x12%\n!MODEL_OP_KIND_ADOPT_COMPILE_CACHE\x10\x01*\x82\x02\n\nModelState\x12\x1b\n\x17MODEL_STATE_UNSPECIFIED\x10\x00\x12\x1b\n\x17MODEL_STATE_DOWNLOADING\x10\x01\x12\x17\n\x13MODEL_STATE_ON_DISK\x10\x02\x12\x16\n\x12MODEL_STATE_IN_RAM\x10\x03\x12\x17\n\x13MODEL_STATE_IN_VRAM\x10\x04\x12\x17\n\x13MODEL_STATE_EVICTED\x10\x05\x12\x16\n\x12MODEL_STATE_FAILED\x10\x06\x12\x17\n\x13MODEL_STATE_ADOPTED\x10\x07\x12&\n\"MODEL_STATE_HOST_CAPACITY_PROGRESS\x10\x08*\x84\x01\n\rActivityState\x12\x1e\n\x1a\x41\x43TIVITY_STATE_UNSPECIFIED\x10\x00\x12\x1a\n\x16\x41\x43TIVITY_STATE_RUNNING\x10\x01\x12\x1c\n\x18\x41\x43TIVITY_STATE_COMPLETED\x10\x02\x12\x19\n\x15\x41\x43TIVITY_STATE_FAILED\x10\x03\x32g\n\x0fWorkerScheduler\x12T\n\x07\x43onnect\x12 .cozy.scheduler.v1.WorkerMessage\x1a#.cozy.scheduler.v1.SchedulerMessage(\x01\x30\x01\x42YZWgithub.com/cozy-creator/tensorhub/internal/orchestrator/grpc/pb/schedulerv1;schedulerv1b\x06proto3') _globals = globals() _builder.BuildMessageAndEnumDescriptors(DESCRIPTOR, _globals) @@ -48,216 +48,218 @@ _globals['_JOBMETRICS_STAGEMSENTRY']._serialized_options = b'8\001' _globals['_FNUNAVAILABLE_AXESENTRY']._loaded_options = None _globals['_FNUNAVAILABLE_AXESENTRY']._serialized_options = b'8\001' - _globals['_PROTOCOLVERSION']._serialized_start=16215 - _globals['_PROTOCOLVERSION']._serialized_end=16296 - _globals['_RESIDENCYTIER']._serialized_start=16298 - _globals['_RESIDENCYTIER']._serialized_end=16419 - _globals['_DESIREDINTENTKIND']._serialized_start=16422 - _globals['_DESIREDINTENTKIND']._serialized_end=16663 - _globals['_DESIREDINTENTCAUSE']._serialized_start=16666 - _globals['_DESIREDINTENTCAUSE']._serialized_end=16943 - _globals['_GOALRECEIPTSTATUS']._serialized_start=16945 - _globals['_GOALRECEIPTSTATUS']._serialized_end=17069 - _globals['_LIFECYCLEINTENTSTATUS']._serialized_start=17072 - _globals['_LIFECYCLEINTENTSTATUS']._serialized_end=17401 - _globals['_LIFECYCLEINTENTSTAGE']._serialized_start=17404 - _globals['_LIFECYCLEINTENTSTAGE']._serialized_end=18501 - _globals['_LIFECYCLEWAITREASON']._serialized_start=18504 - _globals['_LIFECYCLEWAITREASON']._serialized_end=18952 - _globals['_LIFECYCLEERRORCODE']._serialized_start=18955 - _globals['_LIFECYCLEERRORCODE']._serialized_end=19633 - _globals['_FUNCTIONCAPABILITYSTATE']._serialized_start=19636 - _globals['_FUNCTIONCAPABILITYSTATE']._serialized_end=19861 - _globals['_CONFIGAPPLICATIONSTATE']._serialized_start=19864 - _globals['_CONFIGAPPLICATIONSTATE']._serialized_end=20087 - _globals['_DRAINLIFECYCLESTATUS']._serialized_start=20090 - _globals['_DRAINLIFECYCLESTATUS']._serialized_end=20373 - _globals['_STORAGETIER']._serialized_start=20375 - _globals['_STORAGETIER']._serialized_end=20493 - _globals['_WORKERPHASE']._serialized_start=20496 - _globals['_WORKERPHASE']._serialized_end=20712 - _globals['_OUTPUTMODE']._serialized_start=20714 - _globals['_OUTPUTMODE']._serialized_end=20800 - _globals['_WEIGHTLANE']._serialized_start=20803 - _globals['_WEIGHTLANE']._serialized_end=20933 - _globals['_ARMSHAPE']._serialized_start=20935 - _globals['_ARMSHAPE']._serialized_end=21021 - _globals['_STEADYBACKEND']._serialized_start=21024 - _globals['_STEADYBACKEND']._serialized_end=21158 - _globals['_JOBSTATUS']._serialized_start=21161 - _globals['_JOBSTATUS']._serialized_end=21316 - _globals['_MODELOPKIND']._serialized_start=21318 - _globals['_MODELOPKIND']._serialized_end=21401 - _globals['_MODELSTATE']._serialized_start=21404 - _globals['_MODELSTATE']._serialized_end=21662 - _globals['_ACTIVITYSTATE']._serialized_start=21665 - _globals['_ACTIVITYSTATE']._serialized_end=21797 + _globals['_PROTOCOLVERSION']._serialized_start=16340 + _globals['_PROTOCOLVERSION']._serialized_end=16421 + _globals['_RESIDENCYTIER']._serialized_start=16423 + _globals['_RESIDENCYTIER']._serialized_end=16544 + _globals['_DESIREDINTENTKIND']._serialized_start=16547 + _globals['_DESIREDINTENTKIND']._serialized_end=16788 + _globals['_DESIREDINTENTCAUSE']._serialized_start=16791 + _globals['_DESIREDINTENTCAUSE']._serialized_end=17068 + _globals['_GOALRECEIPTSTATUS']._serialized_start=17070 + _globals['_GOALRECEIPTSTATUS']._serialized_end=17194 + _globals['_LIFECYCLEINTENTSTATUS']._serialized_start=17197 + _globals['_LIFECYCLEINTENTSTATUS']._serialized_end=17526 + _globals['_LIFECYCLEINTENTSTAGE']._serialized_start=17529 + _globals['_LIFECYCLEINTENTSTAGE']._serialized_end=18626 + _globals['_LIFECYCLEWAITREASON']._serialized_start=18629 + _globals['_LIFECYCLEWAITREASON']._serialized_end=19077 + _globals['_LIFECYCLEERRORCODE']._serialized_start=19080 + _globals['_LIFECYCLEERRORCODE']._serialized_end=19758 + _globals['_FUNCTIONCAPABILITYSTATE']._serialized_start=19761 + _globals['_FUNCTIONCAPABILITYSTATE']._serialized_end=19986 + _globals['_CONFIGAPPLICATIONSTATE']._serialized_start=19989 + _globals['_CONFIGAPPLICATIONSTATE']._serialized_end=20212 + _globals['_DRAINLIFECYCLESTATUS']._serialized_start=20215 + _globals['_DRAINLIFECYCLESTATUS']._serialized_end=20498 + _globals['_STORAGETIER']._serialized_start=20500 + _globals['_STORAGETIER']._serialized_end=20618 + _globals['_WORKERPHASE']._serialized_start=20621 + _globals['_WORKERPHASE']._serialized_end=20837 + _globals['_OUTPUTMODE']._serialized_start=20839 + _globals['_OUTPUTMODE']._serialized_end=20925 + _globals['_WEIGHTLANE']._serialized_start=20928 + _globals['_WEIGHTLANE']._serialized_end=21058 + _globals['_ARMSHAPE']._serialized_start=21060 + _globals['_ARMSHAPE']._serialized_end=21146 + _globals['_STEADYBACKEND']._serialized_start=21149 + _globals['_STEADYBACKEND']._serialized_end=21283 + _globals['_JOBSTATUS']._serialized_start=21286 + _globals['_JOBSTATUS']._serialized_end=21441 + _globals['_MODELOPKIND']._serialized_start=21443 + _globals['_MODELOPKIND']._serialized_end=21526 + _globals['_MODELSTATE']._serialized_start=21529 + _globals['_MODELSTATE']._serialized_end=21787 + _globals['_ACTIVITYSTATE']._serialized_start=21790 + _globals['_ACTIVITYSTATE']._serialized_end=21922 _globals['_WORKERMESSAGE']._serialized_start=46 _globals['_WORKERMESSAGE']._serialized_end=805 _globals['_SCHEDULERMESSAGE']._serialized_start=808 - _globals['_SCHEDULERMESSAGE']._serialized_end=1184 - _globals['_HELLO']._serialized_start=1187 - _globals['_HELLO']._serialized_end=1622 - _globals['_WORKERRESOURCES']._serialized_start=1625 - _globals['_WORKERRESOURCES']._serialized_end=1956 - _globals['_HARDWAREUNSUITABLE']._serialized_start=1959 - _globals['_HARDWAREUNSUITABLE']._serialized_end=2249 - _globals['_HOSTCANARY']._serialized_start=2252 - _globals['_HOSTCANARY']._serialized_end=2516 - _globals['_MODELRESIDENCY']._serialized_start=2519 - _globals['_MODELRESIDENCY']._serialized_end=2671 - _globals['_INFLIGHTJOB']._serialized_start=2673 - _globals['_INFLIGHTJOB']._serialized_end=2723 - _globals['_HELLOACK']._serialized_start=2726 - _globals['_HELLOACK']._serialized_end=3013 - _globals['_DESIREDRESIDENCY']._serialized_start=3016 - _globals['_DESIREDRESIDENCY']._serialized_end=3319 - _globals['_DESIREDRESIDENCY_SNAPSHOTSENTRY']._serialized_start=3242 - _globals['_DESIREDRESIDENCY_SNAPSHOTSENTRY']._serialized_end=3319 - _globals['_DESIREDSTATECOMMAND']._serialized_start=3322 - _globals['_DESIREDSTATECOMMAND']._serialized_end=3731 - _globals['_DESIREDINTENT']._serialized_start=3734 - _globals['_DESIREDINTENT']._serialized_end=4139 - _globals['_REQUESTATTEMPT']._serialized_start=4141 - _globals['_REQUESTATTEMPT']._serialized_end=4194 - _globals['_GOALRECEIPT']._serialized_start=4197 - _globals['_GOALRECEIPT']._serialized_end=4533 - _globals['_INTENTREJECTION']._serialized_start=4535 - _globals['_INTENTREJECTION']._serialized_end=4646 - _globals['_LIFECYCLEPROGRESS']._serialized_start=4648 - _globals['_LIFECYCLEPROGRESS']._serialized_end=4730 - _globals['_INTENTSTATE']._serialized_start=4733 - _globals['_INTENTSTATE']._serialized_end=5397 - _globals['_MODELIDENTITY']._serialized_start=5400 - _globals['_MODELIDENTITY']._serialized_end=5531 - _globals['_FUNCTIONCAPABILITY']._serialized_start=5534 - _globals['_FUNCTIONCAPABILITY']._serialized_end=5832 - _globals['_CONFIGCLASSMASK']._serialized_start=5834 - _globals['_CONFIGCLASSMASK']._serialized_end=5903 - _globals['_CONFIGAPPLICATION']._serialized_start=5906 - _globals['_CONFIGAPPLICATION']._serialized_end=6277 - _globals['_DRAINPROJECTION']._serialized_start=6280 - _globals['_DRAINPROJECTION']._serialized_end=6545 - _globals['_LIFECYCLESNAPSHOT']._serialized_start=6548 - _globals['_LIFECYCLESNAPSHOT']._serialized_end=6947 - _globals['_DESIREDINSTANCE']._serialized_start=6949 - _globals['_DESIREDINSTANCE']._serialized_end=7038 - _globals['_MODELRESOLUTION']._serialized_start=7040 - _globals['_MODELRESOLUTION']._serialized_end=7141 - _globals['_STATEDELTA']._serialized_start=7144 - _globals['_STATEDELTA']._serialized_end=7552 - _globals['_STORAGETIERUSAGE']._serialized_start=7555 - _globals['_STORAGETIERUSAGE']._serialized_end=7727 - _globals['_DISKUSAGEREPORT']._serialized_start=7729 - _globals['_DISKUSAGEREPORT']._serialized_end=7827 - _globals['_CELLLOOKUP']._serialized_start=7829 - _globals['_CELLLOOKUP']._serialized_end=7875 - _globals['_COMPILETARGET']._serialized_start=7878 - _globals['_COMPILETARGET']._serialized_end=8338 - _globals['_COMPILETARGET_REQUESTEDCELLAXESENTRY']._serialized_start=8282 - _globals['_COMPILETARGET_REQUESTEDCELLAXESENTRY']._serialized_end=8338 - _globals['_COMPILETARGETBINDING']._serialized_start=8340 - _globals['_COMPILETARGETBINDING']._serialized_end=8414 - _globals['_RUNJOB']._serialized_start=8417 - _globals['_RUNJOB']._serialized_end=9069 - _globals['_RUNJOB_SNAPSHOTSENTRY']._serialized_start=3242 - _globals['_RUNJOB_SNAPSHOTSENTRY']._serialized_end=3319 - _globals['_INPUTASSET']._serialized_start=9071 - _globals['_INPUTASSET']._serialized_end=9190 - _globals['_REQUIREDCOMPILEEXECUTION']._serialized_start=9193 - _globals['_REQUIREDCOMPILEEXECUTION']._serialized_end=9323 - _globals['_RESOLVEDCOMPUTE']._serialized_start=9325 - _globals['_RESOLVEDCOMPUTE']._serialized_end=9382 - _globals['_MODELBINDING']._serialized_start=9385 - _globals['_MODELBINDING']._serialized_end=9685 - _globals['_MODELBINDING_COMPONENTSENTRY']._serialized_start=9636 - _globals['_MODELBINDING_COMPONENTSENTRY']._serialized_end=9685 - _globals['_LORAOVERLAY']._serialized_start=9687 - _globals['_LORAOVERLAY']._serialized_end=9757 - _globals['_SNAPSHOT']._serialized_start=9759 - _globals['_SNAPSHOT']._serialized_end=9833 - _globals['_SNAPSHOTFILE']._serialized_start=9836 - _globals['_SNAPSHOTFILE']._serialized_end=10000 - _globals['_CHUNKREF']._serialized_start=10002 - _globals['_CHUNKREF']._serialized_end=10054 - _globals['_RUNATTEMPT']._serialized_start=10057 - _globals['_RUNATTEMPT']._serialized_end=10236 - _globals['_ATTEMPTID']._serialized_start=10238 - _globals['_ATTEMPTID']._serialized_end=10286 - _globals['_EXECUTIONSPEC']._serialized_start=10289 - _globals['_EXECUTIONSPEC']._serialized_end=10997 - _globals['_ENDPOINTRELEASE']._serialized_start=10999 - _globals['_ENDPOINTRELEASE']._serialized_end=11114 - _globals['_ATTRIBUTION']._serialized_start=11116 - _globals['_ATTRIBUTION']._serialized_end=11162 - _globals['_EXECUTIONLANE']._serialized_start=11164 - _globals['_EXECUTIONLANE']._serialized_end=11246 - _globals['_ARM']._serialized_start=11249 - _globals['_ARM']._serialized_end=11464 - _globals['_ARTIFACTIDENTITY']._serialized_start=11467 - _globals['_ARTIFACTIDENTITY']._serialized_end=11619 - _globals['_TOPOLOGY']._serialized_start=11621 - _globals['_TOPOLOGY']._serialized_end=11722 - _globals['_COMPONENTMANIFEST']._serialized_start=11724 - _globals['_COMPONENTMANIFEST']._serialized_end=11790 - _globals['_SLOTBINDING']._serialized_start=11793 - _globals['_SLOTBINDING']._serialized_end=12003 - _globals['_COMPONENTREF']._serialized_start=12005 - _globals['_COMPONENTREF']._serialized_end=12076 - _globals['_ADAPTERBINDING']._serialized_start=12078 - _globals['_ADAPTERBINDING']._serialized_end=12190 - _globals['_CONFIGSNAPSHOT']._serialized_start=12192 - _globals['_CONFIGSNAPSHOT']._serialized_end=12240 - _globals['_CONTRACTIDENTITIES']._serialized_start=12242 - _globals['_CONTRACTIDENTITIES']._serialized_end=12327 - _globals['_EXPLORATIONREF']._serialized_start=12329 - _globals['_EXPLORATIONREF']._serialized_end=12397 - _globals['_INPUTASSETREF']._serialized_start=12399 - _globals['_INPUTASSETREF']._serialized_end=12521 - _globals['_DELIVERYGRANT']._serialized_start=12524 - _globals['_DELIVERYGRANT']._serialized_end=12818 - _globals['_DELIVERYGRANT_SNAPSHOTSENTRY']._serialized_start=12732 - _globals['_DELIVERYGRANT_SNAPSHOTSENTRY']._serialized_end=12818 - _globals['_PRESIGNEDSNAPSHOT']._serialized_start=12820 - _globals['_PRESIGNEDSNAPSHOT']._serialized_end=12888 - _globals['_PRESIGNEDFILE']._serialized_start=12891 - _globals['_PRESIGNEDFILE']._serialized_end=13040 - _globals['_PRESIGNEDASSET']._serialized_start=13042 - _globals['_PRESIGNEDASSET']._serialized_end=13089 - _globals['_JOBACCEPTED']._serialized_start=13091 - _globals['_JOBACCEPTED']._serialized_end=13141 - _globals['_JOBRESULT']._serialized_start=13144 - _globals['_JOBRESULT']._serialized_end=13408 - _globals['_ADJUSTMENT']._serialized_start=13410 - _globals['_ADJUSTMENT']._serialized_end=13489 - _globals['_JOBMETRICS']._serialized_start=13492 - _globals['_JOBMETRICS']._serialized_end=14210 - _globals['_JOBMETRICS_RUNTIMETERMSENTRY']._serialized_start=14111 - _globals['_JOBMETRICS_RUNTIMETERMSENTRY']._serialized_end=14162 - _globals['_JOBMETRICS_STAGEMSENTRY']._serialized_start=14164 - _globals['_JOBMETRICS_STAGEMSENTRY']._serialized_end=14210 - _globals['_JOBPROGRESS']._serialized_start=14212 - _globals['_JOBPROGRESS']._serialized_end=14311 - _globals['_CANCELJOB']._serialized_start=14313 - _globals['_CANCELJOB']._serialized_end=14361 - _globals['_MODELOP']._serialized_start=14364 - _globals['_MODELOP']._serialized_end=14530 - _globals['_MODELEVENT']._serialized_start=14533 - _globals['_MODELEVENT']._serialized_end=15075 - _globals['_ACTIVITYUPDATE']._serialized_start=15078 - _globals['_ACTIVITYUPDATE']._serialized_end=15450 - _globals['_BOOTPHASE']._serialized_start=15453 - _globals['_BOOTPHASE']._serialized_end=15807 - _globals['_FNUNAVAILABLE']._serialized_start=15810 - _globals['_FNUNAVAILABLE']._serialized_end=15983 - _globals['_FNUNAVAILABLE_AXESENTRY']._serialized_start=15940 - _globals['_FNUNAVAILABLE_AXESENTRY']._serialized_end=15983 - _globals['_FNDEGRADED']._serialized_start=15986 - _globals['_FNDEGRADED']._serialized_end=16127 - _globals['_DRAIN']._serialized_start=16129 - _globals['_DRAIN']._serialized_end=16157 - _globals['_TOKENREFRESH']._serialized_start=16159 - _globals['_TOKENREFRESH']._serialized_end=16213 - _globals['_WORKERSCHEDULER']._serialized_start=21799 - _globals['_WORKERSCHEDULER']._serialized_end=21902 + _globals['_SCHEDULERMESSAGE']._serialized_end=1242 + _globals['_HELLO']._serialized_start=1245 + _globals['_HELLO']._serialized_end=1680 + _globals['_WORKERRESOURCES']._serialized_start=1683 + _globals['_WORKERRESOURCES']._serialized_end=2014 + _globals['_HARDWAREUNSUITABLE']._serialized_start=2017 + _globals['_HARDWAREUNSUITABLE']._serialized_end=2307 + _globals['_HOSTCANARY']._serialized_start=2310 + _globals['_HOSTCANARY']._serialized_end=2574 + _globals['_MODELRESIDENCY']._serialized_start=2577 + _globals['_MODELRESIDENCY']._serialized_end=2729 + _globals['_INFLIGHTJOB']._serialized_start=2731 + _globals['_INFLIGHTJOB']._serialized_end=2781 + _globals['_HELLOACK']._serialized_start=2784 + _globals['_HELLOACK']._serialized_end=3071 + _globals['_DESIREDRESIDENCY']._serialized_start=3074 + _globals['_DESIREDRESIDENCY']._serialized_end=3377 + _globals['_DESIREDRESIDENCY_SNAPSHOTSENTRY']._serialized_start=3300 + _globals['_DESIREDRESIDENCY_SNAPSHOTSENTRY']._serialized_end=3377 + _globals['_DESIREDSTATECOMMAND']._serialized_start=3380 + _globals['_DESIREDSTATECOMMAND']._serialized_end=3789 + _globals['_DESIREDINTENT']._serialized_start=3792 + _globals['_DESIREDINTENT']._serialized_end=4197 + _globals['_REQUESTATTEMPT']._serialized_start=4199 + _globals['_REQUESTATTEMPT']._serialized_end=4252 + _globals['_GOALRECEIPT']._serialized_start=4255 + _globals['_GOALRECEIPT']._serialized_end=4591 + _globals['_INTENTREJECTION']._serialized_start=4593 + _globals['_INTENTREJECTION']._serialized_end=4704 + _globals['_LIFECYCLEPROGRESS']._serialized_start=4706 + _globals['_LIFECYCLEPROGRESS']._serialized_end=4788 + _globals['_INTENTSTATE']._serialized_start=4791 + _globals['_INTENTSTATE']._serialized_end=5455 + _globals['_MODELIDENTITY']._serialized_start=5458 + _globals['_MODELIDENTITY']._serialized_end=5589 + _globals['_FUNCTIONCAPABILITY']._serialized_start=5592 + _globals['_FUNCTIONCAPABILITY']._serialized_end=5890 + _globals['_CONFIGCLASSMASK']._serialized_start=5892 + _globals['_CONFIGCLASSMASK']._serialized_end=5961 + _globals['_CONFIGAPPLICATION']._serialized_start=5964 + _globals['_CONFIGAPPLICATION']._serialized_end=6335 + _globals['_DRAINPROJECTION']._serialized_start=6338 + _globals['_DRAINPROJECTION']._serialized_end=6603 + _globals['_LIFECYCLESNAPSHOT']._serialized_start=6606 + _globals['_LIFECYCLESNAPSHOT']._serialized_end=7005 + _globals['_DESIREDINSTANCE']._serialized_start=7007 + _globals['_DESIREDINSTANCE']._serialized_end=7096 + _globals['_MODELRESOLUTION']._serialized_start=7098 + _globals['_MODELRESOLUTION']._serialized_end=7199 + _globals['_STATEDELTA']._serialized_start=7202 + _globals['_STATEDELTA']._serialized_end=7610 + _globals['_STORAGETIERUSAGE']._serialized_start=7613 + _globals['_STORAGETIERUSAGE']._serialized_end=7785 + _globals['_DISKUSAGEREPORT']._serialized_start=7787 + _globals['_DISKUSAGEREPORT']._serialized_end=7885 + _globals['_CELLLOOKUP']._serialized_start=7887 + _globals['_CELLLOOKUP']._serialized_end=7933 + _globals['_COMPILETARGET']._serialized_start=7936 + _globals['_COMPILETARGET']._serialized_end=8396 + _globals['_COMPILETARGET_REQUESTEDCELLAXESENTRY']._serialized_start=8340 + _globals['_COMPILETARGET_REQUESTEDCELLAXESENTRY']._serialized_end=8396 + _globals['_COMPILETARGETBINDING']._serialized_start=8398 + _globals['_COMPILETARGETBINDING']._serialized_end=8472 + _globals['_RUNJOB']._serialized_start=8475 + _globals['_RUNJOB']._serialized_end=9127 + _globals['_RUNJOB_SNAPSHOTSENTRY']._serialized_start=3300 + _globals['_RUNJOB_SNAPSHOTSENTRY']._serialized_end=3377 + _globals['_INPUTASSET']._serialized_start=9129 + _globals['_INPUTASSET']._serialized_end=9248 + _globals['_REQUIREDCOMPILEEXECUTION']._serialized_start=9251 + _globals['_REQUIREDCOMPILEEXECUTION']._serialized_end=9381 + _globals['_RESOLVEDCOMPUTE']._serialized_start=9383 + _globals['_RESOLVEDCOMPUTE']._serialized_end=9440 + _globals['_MODELBINDING']._serialized_start=9443 + _globals['_MODELBINDING']._serialized_end=9743 + _globals['_MODELBINDING_COMPONENTSENTRY']._serialized_start=9694 + _globals['_MODELBINDING_COMPONENTSENTRY']._serialized_end=9743 + _globals['_LORAOVERLAY']._serialized_start=9745 + _globals['_LORAOVERLAY']._serialized_end=9815 + _globals['_SNAPSHOT']._serialized_start=9817 + _globals['_SNAPSHOT']._serialized_end=9891 + _globals['_SNAPSHOTFILE']._serialized_start=9894 + _globals['_SNAPSHOTFILE']._serialized_end=10058 + _globals['_CHUNKREF']._serialized_start=10060 + _globals['_CHUNKREF']._serialized_end=10112 + _globals['_RUNATTEMPT']._serialized_start=10115 + _globals['_RUNATTEMPT']._serialized_end=10294 + _globals['_ATTEMPTID']._serialized_start=10296 + _globals['_ATTEMPTID']._serialized_end=10344 + _globals['_EXECUTIONSPEC']._serialized_start=10347 + _globals['_EXECUTIONSPEC']._serialized_end=11055 + _globals['_ENDPOINTRELEASE']._serialized_start=11057 + _globals['_ENDPOINTRELEASE']._serialized_end=11172 + _globals['_ATTRIBUTION']._serialized_start=11174 + _globals['_ATTRIBUTION']._serialized_end=11220 + _globals['_EXECUTIONLANE']._serialized_start=11222 + _globals['_EXECUTIONLANE']._serialized_end=11304 + _globals['_ARM']._serialized_start=11307 + _globals['_ARM']._serialized_end=11522 + _globals['_ARTIFACTIDENTITY']._serialized_start=11525 + _globals['_ARTIFACTIDENTITY']._serialized_end=11677 + _globals['_TOPOLOGY']._serialized_start=11679 + _globals['_TOPOLOGY']._serialized_end=11780 + _globals['_COMPONENTMANIFEST']._serialized_start=11782 + _globals['_COMPONENTMANIFEST']._serialized_end=11848 + _globals['_SLOTBINDING']._serialized_start=11851 + _globals['_SLOTBINDING']._serialized_end=12061 + _globals['_COMPONENTREF']._serialized_start=12063 + _globals['_COMPONENTREF']._serialized_end=12134 + _globals['_ADAPTERBINDING']._serialized_start=12136 + _globals['_ADAPTERBINDING']._serialized_end=12248 + _globals['_CONFIGSNAPSHOT']._serialized_start=12250 + _globals['_CONFIGSNAPSHOT']._serialized_end=12298 + _globals['_CONTRACTIDENTITIES']._serialized_start=12300 + _globals['_CONTRACTIDENTITIES']._serialized_end=12385 + _globals['_EXPLORATIONREF']._serialized_start=12387 + _globals['_EXPLORATIONREF']._serialized_end=12455 + _globals['_INPUTASSETREF']._serialized_start=12457 + _globals['_INPUTASSETREF']._serialized_end=12579 + _globals['_DELIVERYGRANT']._serialized_start=12582 + _globals['_DELIVERYGRANT']._serialized_end=12876 + _globals['_DELIVERYGRANT_SNAPSHOTSENTRY']._serialized_start=12790 + _globals['_DELIVERYGRANT_SNAPSHOTSENTRY']._serialized_end=12876 + _globals['_PRESIGNEDSNAPSHOT']._serialized_start=12878 + _globals['_PRESIGNEDSNAPSHOT']._serialized_end=12946 + _globals['_PRESIGNEDFILE']._serialized_start=12949 + _globals['_PRESIGNEDFILE']._serialized_end=13098 + _globals['_PRESIGNEDASSET']._serialized_start=13100 + _globals['_PRESIGNEDASSET']._serialized_end=13147 + _globals['_JOBACCEPTED']._serialized_start=13149 + _globals['_JOBACCEPTED']._serialized_end=13199 + _globals['_JOBRESULT']._serialized_start=13202 + _globals['_JOBRESULT']._serialized_end=13466 + _globals['_ADJUSTMENT']._serialized_start=13468 + _globals['_ADJUSTMENT']._serialized_end=13547 + _globals['_JOBMETRICS']._serialized_start=13550 + _globals['_JOBMETRICS']._serialized_end=14268 + _globals['_JOBMETRICS_RUNTIMETERMSENTRY']._serialized_start=14169 + _globals['_JOBMETRICS_RUNTIMETERMSENTRY']._serialized_end=14220 + _globals['_JOBMETRICS_STAGEMSENTRY']._serialized_start=14222 + _globals['_JOBMETRICS_STAGEMSENTRY']._serialized_end=14268 + _globals['_JOBPROGRESS']._serialized_start=14270 + _globals['_JOBPROGRESS']._serialized_end=14369 + _globals['_CANCELJOB']._serialized_start=14371 + _globals['_CANCELJOB']._serialized_end=14419 + _globals['_MODELOP']._serialized_start=14422 + _globals['_MODELOP']._serialized_end=14588 + _globals['_MODELEVENT']._serialized_start=14591 + _globals['_MODELEVENT']._serialized_end=15133 + _globals['_ACTIVITYUPDATE']._serialized_start=15136 + _globals['_ACTIVITYUPDATE']._serialized_end=15508 + _globals['_BOOTPHASE']._serialized_start=15511 + _globals['_BOOTPHASE']._serialized_end=15865 + _globals['_FNUNAVAILABLE']._serialized_start=15868 + _globals['_FNUNAVAILABLE']._serialized_end=16041 + _globals['_FNUNAVAILABLE_AXESENTRY']._serialized_start=15998 + _globals['_FNUNAVAILABLE_AXESENTRY']._serialized_end=16041 + _globals['_FNDEGRADED']._serialized_start=16044 + _globals['_FNDEGRADED']._serialized_end=16185 + _globals['_DRAIN']._serialized_start=16187 + _globals['_DRAIN']._serialized_end=16215 + _globals['_TOKENREFRESH']._serialized_start=16217 + _globals['_TOKENREFRESH']._serialized_end=16271 + _globals['_SERVEPOSTURE']._serialized_start=16273 + _globals['_SERVEPOSTURE']._serialized_end=16338 + _globals['_WORKERSCHEDULER']._serialized_start=21924 + _globals['_WORKERSCHEDULER']._serialized_end=22027 # @@protoc_insertion_point(module_scope) diff --git a/src/gen_worker/pb/worker_scheduler_pb2.pyi b/src/gen_worker/pb/worker_scheduler_pb2.pyi index fe4af328..e5b64fb4 100644 --- a/src/gen_worker/pb/worker_scheduler_pb2.pyi +++ b/src/gen_worker/pb/worker_scheduler_pb2.pyi @@ -394,7 +394,7 @@ class WorkerMessage(_message.Message): def __init__(self, hello: _Optional[_Union[Hello, _Mapping]] = ..., state_delta: _Optional[_Union[StateDelta, _Mapping]] = ..., job_accepted: _Optional[_Union[JobAccepted, _Mapping]] = ..., job_result: _Optional[_Union[JobResult, _Mapping]] = ..., job_progress: _Optional[_Union[JobProgress, _Mapping]] = ..., model_event: _Optional[_Union[ModelEvent, _Mapping]] = ..., fn_unavailable: _Optional[_Union[FnUnavailable, _Mapping]] = ..., fn_degraded: _Optional[_Union[FnDegraded, _Mapping]] = ..., activity_update: _Optional[_Union[ActivityUpdate, _Mapping]] = ..., hardware_unsuitable: _Optional[_Union[HardwareUnsuitable, _Mapping]] = ..., goal_receipt: _Optional[_Union[GoalReceipt, _Mapping]] = ..., lifecycle_snapshot: _Optional[_Union[LifecycleSnapshot, _Mapping]] = ..., boot_phase: _Optional[_Union[BootPhase, _Mapping]] = ...) -> None: ... class SchedulerMessage(_message.Message): - __slots__ = ("hello_ack", "run_job", "cancel_job", "model_op", "drain", "token_refresh", "run_attempt") + __slots__ = ("hello_ack", "run_job", "cancel_job", "model_op", "drain", "token_refresh", "run_attempt", "serve_posture") HELLO_ACK_FIELD_NUMBER: _ClassVar[int] RUN_JOB_FIELD_NUMBER: _ClassVar[int] CANCEL_JOB_FIELD_NUMBER: _ClassVar[int] @@ -402,6 +402,7 @@ class SchedulerMessage(_message.Message): DRAIN_FIELD_NUMBER: _ClassVar[int] TOKEN_REFRESH_FIELD_NUMBER: _ClassVar[int] RUN_ATTEMPT_FIELD_NUMBER: _ClassVar[int] + SERVE_POSTURE_FIELD_NUMBER: _ClassVar[int] hello_ack: HelloAck run_job: RunJob cancel_job: CancelJob @@ -409,7 +410,8 @@ class SchedulerMessage(_message.Message): drain: Drain token_refresh: TokenRefresh run_attempt: RunAttempt - def __init__(self, hello_ack: _Optional[_Union[HelloAck, _Mapping]] = ..., run_job: _Optional[_Union[RunJob, _Mapping]] = ..., cancel_job: _Optional[_Union[CancelJob, _Mapping]] = ..., model_op: _Optional[_Union[ModelOp, _Mapping]] = ..., drain: _Optional[_Union[Drain, _Mapping]] = ..., token_refresh: _Optional[_Union[TokenRefresh, _Mapping]] = ..., run_attempt: _Optional[_Union[RunAttempt, _Mapping]] = ...) -> None: ... + serve_posture: ServePosture + def __init__(self, hello_ack: _Optional[_Union[HelloAck, _Mapping]] = ..., run_job: _Optional[_Union[RunJob, _Mapping]] = ..., cancel_job: _Optional[_Union[CancelJob, _Mapping]] = ..., model_op: _Optional[_Union[ModelOp, _Mapping]] = ..., drain: _Optional[_Union[Drain, _Mapping]] = ..., token_refresh: _Optional[_Union[TokenRefresh, _Mapping]] = ..., run_attempt: _Optional[_Union[RunAttempt, _Mapping]] = ..., serve_posture: _Optional[_Union[ServePosture, _Mapping]] = ...) -> None: ... class Hello(_message.Message): __slots__ = ("protocol_version", "worker_id", "release_id", "resources", "state", "models", "in_flight", "heartbeat_interval_ms", "worker_session_id", "lifecycle_snapshot") @@ -1694,3 +1696,13 @@ class TokenRefresh(_message.Message): token: str expires_at_unix: int def __init__(self, token: _Optional[str] = ..., expires_at_unix: _Optional[int] = ...) -> None: ... + +class ServePosture(_message.Message): + __slots__ = ("eager_only", "reason", "actor") + EAGER_ONLY_FIELD_NUMBER: _ClassVar[int] + REASON_FIELD_NUMBER: _ClassVar[int] + ACTOR_FIELD_NUMBER: _ClassVar[int] + eager_only: bool + reason: str + actor: str + def __init__(self, eager_only: _Optional[bool] = ..., reason: _Optional[str] = ..., actor: _Optional[str] = ...) -> None: ... diff --git a/src/gen_worker/procsplit/parent.py b/src/gen_worker/procsplit/parent.py index 45aece4f..0c0a1ea3 100644 --- a/src/gen_worker/procsplit/parent.py +++ b/src/gen_worker/procsplit/parent.py @@ -1686,8 +1686,11 @@ async def on_message(self, msg: pb.SchedulerMessage) -> None: if not any_link: asyncio.create_task(self._drain_without_child(), name="drain-no-child") return - # hello_ack handled above; everything else (model_op, token_refresh, …) - # is worker-wide desired state: broadcast to every group. + # hello_ack handled above; everything else (model_op, token_refresh, + # serve_posture, …) is worker-wide desired state: broadcast to every + # group. pgw#1142's eager-only order is worker-wide by definition — + # the compiled serving it suppresses lives in the CHILDREN, and every + # one of them has to hear it, so it takes this path unchanged. payload = msg.SerializeToString() delivered = False for slot in self._slots: diff --git a/src/gen_worker/serve_posture.py b/src/gen_worker/serve_posture.py new file mode 100644 index 00000000..ba94a17b --- /dev/null +++ b/src/gen_worker/serve_posture.py @@ -0,0 +1,228 @@ +"""The operator's EAGER-ONLY command (DESIGN-RULINGS §4.32 item 4). + +Paul, verbatim: *"The consumer should have the ability to turn off compile +entirely, and serve-eager only, if the compile is broken or they just don't +care. I.e., they can send some sort of command to the worker to serve eager +rather than serve compiled."* + +This module holds the order and nothing else: the enforcement lives at the +three places that already decide compiled-vs-eager, so there is one posture +with one token rather than a fourth mechanism. + +WHY IT IS A COMMAND +------------------- +Not an env var (§1.17 / Paul's standing rule: an env may carry a VALUE, never a +DECISION), and not a second config surface. Two eager postures already exist +and neither can answer the operator's question: + +* ``STEADY_BACKEND_EAGER_ONLY`` on the Plan's ``ExecutionSpec`` is chosen by + the hub at DISPATCH — it cannot be flipped on a pod that is already serving, + and there is nothing to un-flip. It reports ``hub_ordered_eager``. +* pgw#714's ``operator_eager_pin`` is the hub-resolved lane, pinned in config. + Same shape: durable policy, decided before the pod serves. + +The missing capability — the one Paul named — is turning compiled serving off +on a LIVE worker, and back on. It arrives as ``ServePosture`` on the existing +scheduler stream (or, on cozy-local, from the CLI), never as an environment. + +THE THREE SCOPE DECISIONS, and why +---------------------------------- +**Whole-worker, not per-function.** The operator's question is "is compile +broken on this pod". A per-function switch needs a second addressing vocabulary +(which function? on which execution group? under which release?) for a case +nobody has had; it stays future scope, and the ONE global order is what every +enforcement point reads, so adding a scope later narrows this rather than +forking it. + +**Runtime, not boot-time.** "The compile is broken" is discovered while +serving. A boot-only switch would mean recycling the pod to use it, which is +the cost the switch exists to avoid. + +**Reversible — and that decides the enforcement point.** ``suppress`` must not +unwrap an armed cell: unwrapping is destructive (``aot_serve.unwrap`` restores +the captured forward and the arm is gone for the boot), so a worker that +un-suppressed would serve eager anyway until something re-armed it. Instead the +suppression is read AT THE CALL — ``aot_serve``'s wrapper answers from the +eager forward while the artifact stays armed — so releasing it resumes compiled +serving on the very next request, with no re-arm, no re-materialize and no +re-mint. + +COMPOSITION WITH §4.31's STICKY DE-ARM +-------------------------------------- +Same posture, two triggers, and they must never be read as each other: + +=================== ========================== ========================== + §4.31 sticky de-arm §4.32 eager-only command +=================== ========================== ========================== +trigger a cell-attributable serve an operator, explicitly + failure (guard refusal, + artifact crash) +scope THAT cell the whole worker +lifetime sticky for the boot until released +reversible no — it is evidence yes — it is policy +token ``compiled_degraded`` &c. ``operator_eager_only`` +=================== ========================== ========================== + +Releasing the command deliberately does NOT resurrect a de-armed cell. A +de-arm recorded that this artifact failed on this machine; policy has no +standing to overrule evidence. +""" + +from __future__ import annotations + +import logging +import threading +import time +from dataclasses import dataclass +from typing import Optional + +from .cell_adopt import EagerPhase + +logger = logging.getLogger(__name__) + +#: The wire token for "an operator ordered this worker eager". Shares the +#: `EagerPhase` vocabulary — the hub groups `worker_activity_events` and joins +#: request rows on these values — and is DISTINCT from every automatic eager +#: reason so a suppressed worker can never be read as a broken adopt path. +REASON: str = EagerPhase.OPERATOR_EAGER_ONLY.value + +#: Activity `phase` tokens for the two transitions. Countable hub-side. +PHASE_SUPPRESSED = "eager_only_engaged" +PHASE_RELEASED = "eager_only_released" + + +@dataclass(frozen=True) +class EagerOnlyOrder: + """The standing order. ``active`` False is the default posture.""" + + active: bool = False + actor: str = "" + reason: str = "" + at: float = 0.0 + + def describe(self) -> str: + who = self.actor or "an operator" + why = f": {self.reason}" if self.reason else "" + if not self.active: + return f"compiled serving permitted (last change by {who}{why})" + return f"compiled serving suppressed by {who}{why}" + + +_LOCK = threading.Lock() +_ORDER = EagerOnlyOrder() + + +def order() -> EagerOnlyOrder: + """The current standing order — a snapshot, safe to read from any thread.""" + with _LOCK: + return _ORDER + + +def eager_only() -> bool: + """True when an operator has ordered this worker to serve eager only.""" + return _ORDER.active + + +def block() -> str: + """Why arming is refused right now, or ``""``. + + Shaped for :func:`compile_cache.arming_block`, which is the ONE + precondition authority: routing through it means the command suppresses + adoption, JIT intake, cold compile and every self-mint with one check + rather than with a check per call site. + """ + current = order() + if not current.active: + return "" + who = current.actor or "an operator" + why = f" ({current.reason})" if current.reason else "" + return (f"{who} ordered this worker to serve EAGER ONLY{why} " + f"(§4.32 item 4); release the order to arm again") + + +def apply_command( + eager_only_flag: bool, *, actor: str = "", reason: str = "", +) -> bool: + """Apply one ``ServePosture`` command. Returns True when the posture MOVED. + + Idempotent by design: the hub replays the order to a reconnecting worker + and cozy-local may say ``on`` twice, and neither should produce a second + transition event. A repeat with a NEW actor/reason still refreshes the + record (who last touched it is the operator-visible fact) without claiming + a transition. + """ + global _ORDER + now = time.time() + with _LOCK: + previous = _ORDER + _ORDER = EagerOnlyOrder( + active=bool(eager_only_flag), + actor=str(actor or "").strip(), + reason=str(reason or "").strip(), + at=now, + ) + moved = previous.active != _ORDER.active + current = _ORDER + if not moved: + logger.info("serve-posture: unchanged — %s", current.describe()) + return False + if current.active: + logger.warning( + "serve-posture: EAGER ONLY by order — %s. Nothing will arm or " + "mint, and armed cells stay armed but are not called; releasing " + "the order resumes compiled serving without a re-arm.", + current.describe()) + else: + logger.warning( + "serve-posture: eager-only order RELEASED — %s. Armed cells serve " + "compiled again from the next request; cells de-armed for cause " + "(§4.31) stay de-armed.", current.describe()) + _emit_transition(current) + return True + + +def _emit_transition(current: EagerOnlyOrder) -> None: + """Confess the transition on the wire. + + Imported here rather than at module import: ``compile_cache`` reads this + module from its precondition authority, and ``activity`` pulls the whole + protobuf/progress stack behind it. + """ + from . import activity as activity_mod + + phase = PHASE_SUPPRESSED if current.active else PHASE_RELEASED + detail = ( + f"actor={current.actor or '(unnamed)'} " + f"reason={current.reason or '(none given)'} — " + + ("compiled serving is suppressed on this worker by operator order " + "(§4.32 item 4); every request serves eager and the posture token " + f"is {REASON}" + if current.active else + "the operator's eager-only order is released; armed cells serve " + "compiled again and arming decisions run normally") + ) + try: + activity_mod.emit_event( + activity_mod.KIND_SERVE_POSTURE, detail, phase=phase) + except Exception: # noqa: BLE001 — a confession must never break a command + logger.debug("serve-posture event emission failed", exc_info=True) + + +def reset(order_: Optional[EagerOnlyOrder] = None) -> None: + """Restore the default posture (tests, and the CLI's own teardown).""" + global _ORDER + with _LOCK: + _ORDER = order_ or EagerOnlyOrder() + + +__all__ = [ + "EagerOnlyOrder", + "PHASE_RELEASED", + "PHASE_SUPPRESSED", + "REASON", + "apply_command", + "block", + "eager_only", + "order", + "reset", +] diff --git a/src/gen_worker/serving_mode.py b/src/gen_worker/serving_mode.py index af56d128..16728da7 100644 --- a/src/gen_worker/serving_mode.py +++ b/src/gen_worker/serving_mode.py @@ -31,6 +31,7 @@ from . import aot_serve from . import compile_cache +from . import serve_posture from .cell_adopt import EagerPhase logger = logging.getLogger(__name__) @@ -83,6 +84,11 @@ POSTURE_GRAPH_BREAK = EagerPhase.GRAPH_BREAK.value #: pgw#1082: the declaration named a dynamic range its own inputs leave. POSTURE_DECLARED_RANGE_EXCEEDED = EagerPhase.DECLARED_RANGE_EXCEEDED.value +#: pgw#1142 / §4.32 item 4: an operator ordered this worker eager-only. The one +#: posture in this list that is a DECISION rather than a condition, and the one +#: that can be taken back — a request row carrying it says the platform was +#: asked for eager, not that anything failed to arm. +POSTURE_OPERATOR_EAGER_ONLY = EagerPhase.OPERATOR_EAGER_ONLY.value #: Step-count field names, in precedence order. Matches warmup._STEP_FIELDS so #: the boot warmup and the served request agree on what "steps" means. @@ -236,6 +242,22 @@ def resolve( fell_back = bool(reason) if not reason and mode == MODE_EAGER: reason = str(eager_posture or "").strip() + if serve_posture.eager_only(): + # pgw#1142 / §4.32 item 4: an operator ordered eager. The cell may + # still be ARMED — deliberately, so the order can be taken back + # without a re-arm — and `classify_mode` reads exactly that armed + # artifact, so without this the request would report `aot_cell` for a + # forward the artifact never ran. That is the wire lie pgw#1082/#1093 + # spent two pods closing, arriving from the opposite direction. + # + # It outranks the per-request fallback classes because it PRECEDES + # them: nothing was dispatched to a compiled callable, so no guard + # could miss and no ingress could refuse. And it is not a fallback — + # nothing fell back, there was nothing to fall back FROM — so the + # compiled-vs-eager comparison keeps its meaning. + mode = MODE_EAGER + reason = POSTURE_OPERATOR_EAGER_ONLY + fell_back = False return ServedIdentity( serving_mode=mode, served_cell_ref=str(active_compile_ref or "").strip(), diff --git a/tests/test_eager_only_command_pgw1142.py b/tests/test_eager_only_command_pgw1142.py new file mode 100644 index 00000000..be747d06 --- /dev/null +++ b/tests/test_eager_only_command_pgw1142.py @@ -0,0 +1,432 @@ +"""pgw#1142 / DESIGN-RULINGS §4.32 item 4 — the operator's eager-only command. + +Paul, verbatim: *"The consumer should have the ability to turn off compile +entirely, and serve-eager only, if the compile is broken or they just don't +care. I.e., they can send some sort of command to the worker to serve eager +rather than serve compiled."* + +Four properties, and every one of them is a separate way the feature can be +built wrong: + +1. **It suppresses.** Nothing arms, nothing mints, and an already-armed cell is + not called. +2. **It is REVERSIBLE.** Releasing it resumes compiled serving on the next + request with no re-arm — which is only true if the suppression is read at + the CALL rather than by unwrapping the artifact. +3. **It is DISTINCT from §4.31's sticky de-arm.** Same eager posture, two + triggers: releasing an operator order must not resurrect a cell that + de-armed itself for cause, and their tokens must never collide. +4. **It is legible.** A suppressed worker's request rows say + ``operator_eager_only`` — not ``aot_cell`` (the cell is still armed, and + naive classification reads exactly that), and not any of the tokens that + mean something failed. +""" + +from __future__ import annotations + +import argparse +import asyncio +import types +from pathlib import Path +from typing import Any, Dict, List + +import pytest + +from gen_worker import activity +from gen_worker import aot_serve +from gen_worker import boot_adopt +from gen_worker import compile_cache as cc +from gen_worker import fleet_cells +from gen_worker import lifecycle as lifecycle_mod +from gen_worker import serve_posture +from gen_worker import serving_mode +from gen_worker.cell_adopt import EagerPhase +from gen_worker.cli import serve as serve_cli +from gen_worker.pb import worker_scheduler_pb2 as pb + + +# --------------------------------------------------------------------------- +# fixtures +# --------------------------------------------------------------------------- + + +@pytest.fixture(autouse=True) +def default_posture() -> Any: + """The order is process-global; no test may leak one into the next.""" + serve_posture.reset() + yield + serve_posture.reset() + + +@pytest.fixture() +def events(monkeypatch: pytest.MonkeyPatch) -> List[Any]: + captured: List[Any] = [] + monkeypatch.setattr(activity, "_emit", captured.append) + return captured + + +class FakeTensor: + def __init__(self, shape: Any, dtype: str = "torch.bfloat16") -> None: + self.shape = tuple(shape) + self.dtype = dtype + + +class FakePackage: + """Stands in for ``AOTICompiledModel``; records every invocation.""" + + def __init__(self) -> None: + self.invocations: List[Any] = [] + self.loaded: Dict[str, Any] | None = None + self.raises = False + + def get_constant_fqns(self) -> List[str]: + return ["conv_in.weight"] + + def load_constants(self, values: Any, check_full_update: bool = False, + user_managed: bool = False) -> None: + self.loaded = dict(values) + + def __call__(self, *args: Any, **kwargs: Any) -> Any: + self.invocations.append((args, kwargs)) + if self.raises: + raise RuntimeError("cuda kernel launch failed") + return "ARTIFACT_OUTPUT" + + +class FakeModule: + def __init__(self) -> None: + self.device = "cpu" + self.eager_calls = 0 + + def state_dict(self) -> Dict[str, Any]: + return {"conv_in.weight": FakeTensor([1])} + + def forward(self, *args: Any, **kwargs: Any) -> Any: + self.eager_calls += 1 + return "EAGER_OUTPUT" + + +class FakePipeline: + def __init__(self, module: FakeModule) -> None: + self.unet = module + + +class Cfg: + family = "sdxl-base" + lora_bucket = 0 + targets = ("unet",) + regional = False + + +META: Dict[str, Any] = { + "format": aot_serve.ARTIFACT_FORMAT, "kind": aot_serve.ARTIFACT_KIND, + "family": "sdxl-base", "precision": "w8a8", "sku": "l4", "sm": "sm_89", + "torch": "2.13.0+cu130", "cuda": "13.0", "cell_key": "deadbeef", + "lora_bucket": 0, +} + +ENTRY = { + "target": "unet", "fork": [], "class_dims": [], + "inputs": [{"name": "sample", "position": 0, "dtype": "bfloat16", + "shape": [2, 4, "h", "w"]}], + "symbols": {"h": [64, 160], "w": [64, 160]}, + "constants": [{"fqn": "conv_in.weight", "source": aot_serve.SOURCE_STATE_DICT, + "dtype": "bfloat16", "shape": [320, 4, 3, 3]}], + "graph": {}, +} + + +def _armed_module() -> tuple[FakeModule, FakePackage, FakePipeline]: + """A module wrapped by the real ``aot_serve`` swap, artifact armed.""" + module, package = FakeModule(), FakePackage() + pipeline = FakePipeline(module) + runner = aot_serve.ArtifactRunner( + package=package, + contract=aot_serve.contract_from_meta(ENTRY), + constants=aot_serve.constants_from_meta(ENTRY), + module_name="unet", entry="unet/g") + runner.bind(module.state_dict(), {}) + aot_serve.wrap_module(module, runner, META) + setattr(pipeline, "_cozy_aot", { + "meta": META, "module": module, + "state": getattr(module, "_cozy_aot")["state"]}) + return module, package, pipeline + + +def _call() -> tuple[Any, ...]: + return (FakeTensor([2, 4, 128, 128]),) + + +# --------------------------------------------------------------------------- +# 1. the order itself +# --------------------------------------------------------------------------- + + +def test_default_posture_permits_compiled_serving() -> None: + assert serve_posture.eager_only() is False + assert serve_posture.block() == "" + + +def test_the_order_engages_and_is_released() -> None: + assert serve_posture.apply_command( + True, actor="paul", reason="the compile is broken") is True + assert serve_posture.eager_only() is True + block = serve_posture.block() + assert "paul" in block and "the compile is broken" in block + + assert serve_posture.apply_command(False, actor="paul") is True + assert serve_posture.eager_only() is False + assert serve_posture.block() == "" + + +def test_repeating_the_order_is_not_a_transition() -> None: + """The hub replays it to a reconnecting worker; that is not an event.""" + assert serve_posture.apply_command(True, actor="hub") is True + assert serve_posture.apply_command(True, actor="hub") is False + assert serve_posture.order().active is True + + +def test_each_transition_confesses_on_the_wire(events: List[Any]) -> None: + serve_posture.apply_command(True, actor="paul", reason="broken") + serve_posture.apply_command(True, actor="paul", reason="broken") # no-op + serve_posture.apply_command(False, actor="paul") + + kinds = [(e.kind, e.phase) for e in events] + assert kinds == [ + (activity.KIND_SERVE_POSTURE, serve_posture.PHASE_SUPPRESSED), + (activity.KIND_SERVE_POSTURE, serve_posture.PHASE_RELEASED), + ] + # The operator's words reach the hub — an order nobody can explain is an + # unexplained fleet-wide eager posture six months later. + assert "broken" in events[0].detail + assert serve_posture.REASON in events[0].detail + + +def test_the_token_is_its_own(events: List[Any]) -> None: + """It must never be countable with a failure class or with a plan order.""" + assert serve_posture.REASON == EagerPhase.OPERATOR_EAGER_ONLY.value + assert serve_posture.REASON == serving_mode.POSTURE_OPERATOR_EAGER_ONLY + assert serve_posture.REASON == boot_adopt.OPERATOR_EAGER_ONLY + assert boot_adopt.OPERATOR_EAGER_ONLY in boot_adopt.GATE_REASONS + for other in (EagerPhase.COMPILED_DEGRADED, EagerPhase.HUB_ORDERED_EAGER, + EagerPhase.BOOT_ENDED_UNCOMPILED, EagerPhase.UNCOMPILED): + assert serve_posture.REASON != other.value + # ...and it is not the plan's `eager_only` backend token either: one is a + # standing order about this POD, the other is one dispatch's backend. + assert serve_posture.REASON != "eager_only" + + +# --------------------------------------------------------------------------- +# 2. it suppresses — arming, minting, and the ordered arm +# --------------------------------------------------------------------------- + + +def test_arming_block_names_the_order() -> None: + """The ONE precondition authority answers it, so every arming route does.""" + pipeline = FakePipeline(FakeModule()) + # Whatever else this host blocks on (no CUDA in CI), it is not the order. + assert "EAGER ONLY" not in cc.arming_block( + pipeline, Cfg(), cache_ready=True, allow_cold=True) + serve_posture.apply_command(True, actor="paul", reason="don't care") + # And the order is named FIRST: it is the reason an operator asked for, + # so it must not be masked by an environment fact behind it. + assert cc.arming_block( + pipeline, Cfg(), cache_ready=True, allow_cold=True + ).startswith("paul ordered this worker to serve EAGER ONLY") + + +def test_the_fleet_policy_is_not_even_entered( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """Not merely "does not arm": it must not resolve, download or mint. + + `arming_block` would refuse the arm at the far end anyway. This asserts the + cost is not paid — a hub round trip and a materialize, and on a miss a + self-mint child that compiles for minutes before rediscovering the refusal. + """ + def _explode(*_a: Any, **_k: Any) -> Any: + raise AssertionError("the arming policy ran under an eager-only order") + + monkeypatch.setattr(fleet_cells, "_arming_policy", _explode) + serve_posture.apply_command(True, actor="paul") + outcome = fleet_cells.enable_compiled(FakePipeline(FakeModule()), Cfg()) + assert outcome.armed is False + assert outcome.eager_reason == EagerPhase.OPERATOR_EAGER_ONLY + + +def test_an_ordered_aot_arm_is_obeyed_as_eager_not_refused() -> None: + """The hub named an exact cell; the operator said eager. + + The operator wins, and the shape of winning matters: `OrderedArmError` + would fail the attempt typed and take the function down, which is the + opposite of "serve eager instead". + """ + serve_posture.apply_command(True, actor="paul", reason="broken") + outcome = fleet_cells.arm_ordered( + FakePipeline(FakeModule()), Cfg(), None, + backend="aot_cell", artifact=Path("/nonexistent/cell.pt2"), + delivered_ref="root/family-sdxl-base#ck1-abc", + delivered_digest="sha256:abc", expected=None, publisher_org="root") + assert outcome.armed is False + assert outcome.eager_reason == EagerPhase.OPERATOR_EAGER_ONLY + + +# --------------------------------------------------------------------------- +# 3. serve-time suppression, and reversibility +# --------------------------------------------------------------------------- + + +def test_an_armed_cell_serves_eager_under_the_order_and_stays_armed() -> None: + module, package, pipeline = _armed_module() + + assert module.forward(*_call()) == "ARTIFACT_OUTPUT" + assert len(package.invocations) == 1 + + serve_posture.apply_command(True, actor="paul", reason="broken") + assert module.forward(*_call()) == "EAGER_OUTPUT" + assert module.eager_calls == 1 + # The artifact was not reached, and it was not thrown away. + assert len(package.invocations) == 1 + assert aot_serve.is_armed(pipeline) is True + + +def test_releasing_the_order_resumes_compiled_serving_with_no_rearm() -> None: + """The property that decides the enforcement point. + + Unwrapping the artifact would have produced the same first half and a + permanently eager worker afterwards. + """ + module, package, pipeline = _armed_module() + serve_posture.apply_command(True, actor="paul", reason="broken") + assert module.forward(*_call()) == "EAGER_OUTPUT" + + serve_posture.apply_command(False, actor="paul", reason="it was the driver") + assert module.forward(*_call()) == "ARTIFACT_OUTPUT" + assert len(package.invocations) == 1 + assert module.eager_calls == 1 + + +# --------------------------------------------------------------------------- +# 4. distinct from §4.31's sticky de-arm +# --------------------------------------------------------------------------- + + +def test_releasing_the_order_does_not_resurrect_a_de_armed_cell() -> None: + """De-arm is EVIDENCE; the order is POLICY. Policy does not overrule it.""" + module, package, pipeline = _armed_module() + package.raises = True + + # §4.31: a cell-attributable failure serves this request eager and de-arms + # the cell, sticky for the boot. + assert module.forward(*_call()) == "EAGER_OUTPUT" + assert getattr(module, "_cozy_aot")["state"]["failed"] is True + + serve_posture.apply_command(True, actor="paul") + serve_posture.apply_command(False, actor="paul") + package.raises = False + assert module.forward(*_call()) == "EAGER_OUTPUT" + # Two forwards, two eager answers: the failing one (which the tenant still + # got a correct result from) and the one after the order was released. + assert module.eager_calls == 2 + + +def test_the_two_triggers_report_different_reasons() -> None: + de_armed = serving_mode.resolve( + active_compile_ref="", eager_posture=EagerPhase.COMPILED_DEGRADED.value) + assert de_armed.fallback_reason == EagerPhase.COMPILED_DEGRADED.value + + serve_posture.apply_command(True, actor="paul") + ordered = serving_mode.resolve( + active_compile_ref="", eager_posture=EagerPhase.COMPILED_DEGRADED.value) + assert ordered.fallback_reason == serve_posture.REASON + + +# --------------------------------------------------------------------------- +# 5. telemetry: a suppressed worker is not a broken adopt path +# --------------------------------------------------------------------------- + + +def test_a_suppressed_request_is_not_reported_as_compiled() -> None: + ref = "root/family-sdxl-base#ck1-abc" + aot_serve.note_aot_key("ck1-abc") + armed = serving_mode.resolve(active_compile_ref=ref, sm="89") + assert armed.serving_mode == serving_mode.MODE_AOT_CELL + + serve_posture.apply_command(True, actor="paul", reason="broken") + suppressed = serving_mode.resolve(active_compile_ref=ref, sm="89") + assert suppressed.serving_mode == serving_mode.MODE_EAGER + assert suppressed.fallback_reason == serve_posture.REASON + # Not a FALLBACK: nothing fell back, so the compiled-vs-eager comparison + # keeps its meaning. + assert suppressed.served_eager_fallback is False + # The cell is still named — an operator has to be able to see WHICH cell + # is standing by. + assert suppressed.served_cell_ref == ref + + +def test_a_real_guard_miss_still_reports_itself() -> None: + """The order must not swallow the per-request fallback vocabulary.""" + missed = serving_mode.resolve( + active_compile_ref="root/family-sdxl-base#ck1-abc", guard_missed=True) + assert missed.fallback_reason == serving_mode.FALLBACK_GUARD_MISS + assert missed.served_eager_fallback is True + + +# --------------------------------------------------------------------------- +# 6. the wire: the command arrives on the existing control channel +# --------------------------------------------------------------------------- + + +def test_the_scheduler_command_applies_in_both_directions() -> None: + """The real `Lifecycle.on_message` branch, over a real protobuf. + + Bound to a bare namespace on purpose: the branch must not depend on worker + state, because a worker holding a broken cell is exactly the worker whose + state may be unhealthy. + """ + handler = lifecycle_mod.Lifecycle.on_message.__get__( + types.SimpleNamespace()) + + asyncio.run(handler(pb.SchedulerMessage( + serve_posture=pb.ServePosture( + eager_only=True, actor="admin:paul", reason="ie#657 cells crash")))) + assert serve_posture.eager_only() is True + assert serve_posture.order().actor == "admin:paul" + + asyncio.run(handler(pb.SchedulerMessage( + serve_posture=pb.ServePosture(eager_only=False, actor="admin:paul")))) + assert serve_posture.eager_only() is False + + +def test_the_command_rides_the_existing_oneof() -> None: + msg = pb.SchedulerMessage(serve_posture=pb.ServePosture(eager_only=True)) + assert msg.WhichOneof("msg") == "serve_posture" + # Round-trips: the hub and the worker share one generated contract. + assert pb.SchedulerMessage.FromString( + msg.SerializeToString()).serve_posture.eager_only is True + + +# --------------------------------------------------------------------------- +# 7. the cozy-local CLI half +# --------------------------------------------------------------------------- + + +def test_the_serve_socket_carries_a_posture_control_frame() -> None: + frame = serve_cli._parse_frame( + b'{"posture":{"eager_only":true,"reason":"broken"}}') + assert frame == {"kind": "posture", "eager_only": True, "reason": "broken"} + + off = serve_cli._parse_frame(b'{"posture":{"eager_only":false}}') + assert off == {"kind": "posture", "eager_only": False, "reason": ""} + + bad = serve_cli._parse_frame(b'{"posture":{"eager_only":"yes"}}') + assert bad["kind"] == "error" + + +def test_eager_only_is_a_distinct_flag_from_eager() -> None: + """`--eager` is about when setup() runs. Overloading it would be a trap.""" + parser = argparse.ArgumentParser() + serve_cli.add_subparser(parser.add_subparsers(dest="command")) + args = parser.parse_args(["serve", "--eager-only"]) + assert args.eager_only is True + assert args.eager is False diff --git a/tests/test_silent_failure_audit_pgw824.py b/tests/test_silent_failure_audit_pgw824.py index cc88a4dc..acfa6d3d 100644 --- a/tests/test_silent_failure_audit_pgw824.py +++ b/tests/test_silent_failure_audit_pgw824.py @@ -234,6 +234,13 @@ def test_the_eager_phase_values_are_a_wire_contract() -> None: # as `worker_function_unavailable reason=compile_cell_failed` and the # pod was reaped and replaced (three pods, 2026-08-11). "ADOPTED_CELL_REFUSED": "adopted_cell_refused", + # pgw#1142 / §4.32 item 4: an OPERATOR ordered this worker eager-only. + # The only member of this vocabulary that is a reversible DECISION + # rather than a condition, and the reason it needs its own value: + # counting it with `hub_ordered_eager` would merge a standing order + # about a pod with one dispatch's backend, and counting it with any + # token above would put a supported control into a defect population. + "OPERATOR_EAGER_ONLY": "operator_eager_only", } # The join the ArmOutcome docstring claims: a delegated mint's decline and # the serving posture that describes it are ONE token, not two that happen @@ -245,6 +252,8 @@ def test_the_eager_phase_values_are_a_wire_contract() -> None: assert EagerPhase.GRAPH_BREAK == serving_mode.POSTURE_GRAPH_BREAK assert (EagerPhase.DECLARED_RANGE_EXCEEDED == serving_mode.POSTURE_DECLARED_RANGE_EXCEEDED) + assert (EagerPhase.OPERATOR_EAGER_ONLY + == serving_mode.POSTURE_OPERATOR_EAGER_ONLY) def test_the_posture_tokens_are_the_enum_and_not_a_second_spelling() -> None: