Repository navigation
S25 final integration: sampler islands on spot, synced trainers to samplers, round cut, stage-1 rehearsal, agentic RL, eval suite - #279
Draft
michaellchung wants to merge 96 commits into
Draft
michaellchung wants to merge 96 commits into
michaellchung wants to merge 96 commits into
Conversation
verl's TQ agent loop stores AgentLoopOutput.as_dict(), so provenance sits inside each row's extra_fields, not under its own name; the per-source metrics of the TQ path were all 'unknown'. Also resolve the train handles of a bare TrainingWorker (worker.engine). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… add the T-G3 GPU harness convert_to_hf emits SGLang weight-update names (model.layers.*), while the GLM-5.3 checkpoint and the quantization layout keep the text model under model.language_model.; hf_slices now renames with checkpoint_name (same rule as Miles Glm5NextBridge._nest_language_model). tools/probes/g3_miles: Modal H100!:2 harness for N7.5 (TE vs local forward/backward), N8.2 (in-trainer sharded requantization vs single process) and N7.1d (distributed optimizer + CPU offload cut round trip). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex 0.145.0 only speaks the Responses API (wire_api = "chat" is refused), so the records proxy is its only path to a chat-template engine. A local run of the real Codex binary against the proxy found three gaps: - Codex sends its sub-agent tools as a namespace and a hosted web_search tool; the proxy and gateway now leave those out of the prompt and still refuse other non-function tool types. - Harbor's codex agent asks for reasoning summaries by default; the yeto trial builders now pass reasoning_summary=none and web_search=disabled. - Codex sends its instructions and a developer message as two system messages; the Qwen3.8 template accepts one, so the new qwen38 template merges leading system messages. A template refusal is now a TemplateError (HTTP 400) instead of an unhandled error (HTTP 500 that Codex retries). Register the qwen38 template (Qwen3.8 XML tool calls, thinking kept in every turn) and add scripts/modal_records_endpoint.py, which serves the SGLang adapter plus the records proxy on one pinned Modal GPU behind a secret path prefix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…uant fixes - glm5_next recipe reads n_routed_experts and first_k_dense_replace from the checkpoint instead of hard-coding GLM5.3-Flash (288 experts, 3 dense layers). - CheckpointLayout.load falls back to the index and shard headers when the template directory is a reference checkpoint without layout.json; stage 1 passes the round-0 MXFP4 export as --template, which failed before. - Re-quantizer --frozen-base/--frozen-pattern: untrained tensors the trainer does not export (e.g. the vision tower) come from the bf16 source; any other missing tensor still fails. - sglang_load_report: a hook for SGLang's GLM5.3 model file that renames routed-expert weight_packed tensors and reports, per rank, the expert matrices that reached a weight loader; the sglang-server adapter reads it for post-load check 1 (it reported no count before, so the check failed). - PostLoadChecks.require_fixed_answers (default True) for a cut-down model that cannot answer; the other fixed-prompt problems still fail. - Island flag --miles-commit (full SHA) for an image that carries another Miles commit than the pin; stage 1 passes the plan's commit when full. - Stage 1: store_backend modal-volume (store hook env on every node), digest-pinned images, tp<N> topologies, sampler python/extra_env/checks, model num_experts/first_k_dense_replace. - examples/stage1/glm5next-mini-modal.yaml + modal-mini-inventory.yaml and the Modal driver tools/probes/s25_g1 (prep, smoke, launch). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pert fusion under MXFP4 SGLang fuses the BF16 shared expert into the routed FusedMoE as one more expert; with the MXFP4 MoE method that slot is packed U8 and the BF16 shared weights do not fit (shape error at load, Modal smoke-2). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ame passed to the container Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…reverse Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TokenTurn carries the engine's finish reason (the records recorder fills it)
and HarborAgentLoop reports length_capped_turns in extra_fields. A Codex
trajectory whose reply was cut at max_tokens otherwise looked like a normal
episode ("truncated" only covers the trainer's response width). Record the
first model-driven Codex runs in the Harbor README.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The container imports yeto from the mounted workdir, which is not on sys.path before _image_paths() runs. The check action loads the template from the models Volume and imports the engine and proxy modules on a CPU container, so a broken image or path fails before a GPU starts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…argv with parse_args The ports placement check refuses --rollout-num-gpus (0 included) without --rl-placement fixed-partition, and fixed-partition needs >= 1 rollout GPU, so the trainer-only island argv the plan generated never started (Modal run-1). The plan's argv check now runs the island's parse_args with its combination rules, not only the bare parser. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… Modal runner interpreter With HF_HUB_OFFLINE=1 the island cannot load a Hub dataset id (Modal run-2); a plan may name the dataset staged on the node instead (no revision flag). The Modal Volume store commits and reloads through the Modal runner's interpreter (YETO_MODAL_PYTHON/YETO_MODAL_SYSPATH, as yeto.modal_runner sets); without them every reload failed and was only logged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ore the in-place CE, TE-to-TE name map, host-copy pin report Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ads one round behind - AutoBridge cannot read glm5_next (transformers 5.12.1 has no such model type; Modal run-3). Miles trains it through its plugin spec, so the island takes a read-only provider view of config.json (models/glm5_next), as it does for Flash-Next. - A trainer that reads the sample buffer needs --rl-max-policy-age >= 1 and an algorithm spec that tolerates it; stage 1 passes age 1 and trainer.miles.algorithm_spec (default decoupled-ppo-c2: GRPO + token TIS). - CPU test of the island path from the plan argv to the Miles argv. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An open file on a Modal Volume blocks every reload in that container (run-3: the re-quantizer could not reload while its own log was open there). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The trainer-only island (partitioned-serial) refuses to start without --rl-expected-algorithm-sha256 (Modal run-5). The plan computes it from the island argv with the launcher's resolve_ports_algorithm. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… actors connect_island_ray forwards the sample-buffer and store variables to the Ray actors only when RAY_ADDRESS is set; without it the buffer rollout function in the RolloutExecutor found no YETO_SAMPLE_BUFFER_MAX_POLICY_AGE (Modal run-6). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e in-trainer N8.2 result On 2xH100 (T-G3) a GPU expert slice with W4A8 AWQ failed (the AWQ factors live on the host), while the host-copy path was byte-equal to the single-process output for W4A8 (R2 AWQ) and MXFP4. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Miles loads an HF checkpoint only through Megatron-Bridge, which has no glm5_next bridge (Modal run-7). The recipe now uses --megatron-to-hf-mode raw and requires --megatron-ref-load; stage 1 takes model.bf16_source.torch_dist, adds a convert step (Miles tools/convert_hf_to_torch_dist.py on trainer node 0) and reports a gap when the path is unset. The Modal driver converts once into the models Volume. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Miles' convert_hf_to_torch_dist.py rejects the run flag --qkv-format (Modal run-8). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…YTHONPATH The editable megatron_core install does not expose megatron.post_training, which Megatron's get_model imports in the converter (Modal run-9). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t on PYTHONPATH Megatron's get_model imports megatron.post_training in the trainer actors too, which the ports image's editable megatron_core does not expose (Modal run-10). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Under --debug-train-only there is no weight updater and no rollout engine: align_weight_version is a no-op and MilesPublisher skips the engine update and version stamp; the ReleasingPublisher's release is the publication (Modal run-11 stopped at both). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…trainer A pure sampler island on spot capacity can be reclaimed and started again in a new container (Modal reschedules the input; sky relaunches the cluster). The new incarnation now continues where the buffer says the old one stopped: - every index entry carries a sampler_round record (sampler id, incarnation, rollout id, data cursor after the round, last group of the round); - yeto.rl.engine.sampler_resume reads the resume point: the next rollout id after the newest round with an entry, and that round's data cursor; a round already in the buffer is never written again (RoundGuard), and no prompt is drawn twice (the Miles data source is put at the cursor); - on the reclaim marker (the Modal runner writes it for sampler islands) the island starts no new round, writes no more groups, writes rl_sampler_leave with reason PREEMPTED and a per-incarnation member record in the store, then waits for the cloud to stop the container; a new incarnation writes rl_sampler_join and loads the newest policy release before its first round. --rl-island-sample-role now takes one trainer and one or more samplers. With several samplers on a shared directory (a Modal Volume, where flock does not hold across containers) each sampler writes its own shard file and the trainer merges the shards into index.jsonl (one store commit per refresh); the data ends once every named shard has ended. The trainer scheduler is refused with several samplers (the head does not read the shards). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A verl trainer island now trains from groups a separate sampler island wrote to the durable sample buffer, and releases each policy to the store for the samplers. Only the verl adapter layer changes; verl is not patched. - buffer_feed: the fully_async rollout port's queue calls served from the buffer (versions mapped through the island's VersionMap, so the RoundCollector keeps its max_policy_age judgement); kept groups become padded verl rows; every pulled buffer position is logged once. - buffer_ports: BufferReleasePublisher merges the LoRA into the base and releases full weights (the engine sampler loads full checkpoints); the base policy v0 (B all zero) is not released. - merged_release: W + B@A in float32, stored in the base dtype, same file layout. - buffer_trainer_runner / verl_main.main_buffer_trainer: trainer-only set-up (no rollouter, no MessageQueue) with the fully_async trainer actor plus yeto_add_group / yeto_param_version. - island_entry: --rl-sample-role trainer plans the buffer trainer; other roles are refused (the sampler is the engine sampler island). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tepped state (S25 N7.6b, N7.6c) N7.6b: the round-cut controller (RoundCutCheckpoint, CutSource) moves from the Miles adapter to yeto/rl/engine/round_cut.py, so both islands run one controller; the Miles modules re-export it and keep their in-flight export check. The data cursor of a cut now comes from the cut's own cursor_fields (verl: sample_offset). adapters/verl/round_cuts.py wires it into the verl sync island the way the Miles island does: --rl-resume-store / --rl-resume-state-dir / --rl-cut-every / --rl-cut-keep / --rl-stop-after-rounds / --rl-resume-allow-config-change, a ResumeController that restores the store before the driver is built (the driver's ledger is the restored one), a preemption marker handler, and the trainer_remote commanded mode whose stop cuts with the SCHEDULED trigger. Synced islands and the fully_async path are refused. N7.6c: the verl rank digest is taken over optimizer.param_groups and optimizer.state (the tensors the optimizer steps), not the module state dict. A regression test injects an optimizer that steps a build-time snapshot after the load and requires the restore to be refused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mports no CUDA code The worker_process_setup_hook runs before Ray sets an actor's CUDA_VISIBLE_DEVICES; importing verl there initialised CUDA early and put both FSDP ranks on GPU 0 (NCCL Duplicate GPU detected, T-N109 run a). The hook now installs an import hook that loads the estimator right after verl's core_algos executes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ripts check first); count each over-age drop once Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…obal_steps) verl's V1 trainer reads min_global_steps / max_global_steps from every sample's extra fields (T-N109 run b failed on None). The records proxy exposes the trajectory being served in a context variable; VerlRolloutEngine keeps the global_steps of each reply per trajectory and HarborAgentLoop reports their range. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two launches from one checkout (a trainer and a sampler launch on one store, S25 T-EVAL E4) unpack the same digest at the same time; the later rename hit ENOTEMPTY and the trainer launch exited 1. Keep the existing tree when it verifies, drop the staging copy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ups from consumed minus trained; over-age check reads rl_group_policies per round Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…trainer A pure sampler island on spot capacity can be reclaimed and started again in a new container (Modal reschedules the input; sky relaunches the cluster). The new incarnation now continues where the buffer says the old one stopped: - every index entry carries a sampler_round record (sampler id, incarnation, rollout id, data cursor after the round, last group of the round); - yeto.rl.engine.sampler_resume reads the resume point: the next rollout id after the newest round with an entry, and that round's data cursor; a round already in the buffer is never written again (RoundGuard), and no prompt is drawn twice (the Miles data source is put at the cursor); - on the reclaim marker (the Modal runner writes it for sampler islands) the island starts no new round, writes no more groups, writes rl_sampler_leave with reason PREEMPTED and a per-incarnation member record in the store, then waits for the cloud to stop the container; a new incarnation writes rl_sampler_join and loads the newest policy release before its first round. --rl-island-sample-role now takes one trainer and one or more samplers. With several samplers on a shared directory (a Modal Volume, where flock does not hold across containers) each sampler writes its own shard file and the trainer merges the shards into index.jsonl (one store commit per refresh); the data ends once every named shard has ended. The trainer scheduler is refused with several samplers (the head does not read the shards). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The cloud stops the container seconds after its signal, while the sampler loop may still be inside a round or its throttle sleep. A daemon watcher now writes rl_sampler_leave (reason PREEMPTED, rounds done, the rollout id in flight) and the member record as soon as the marker appears; the exit path leaves only when the watcher has not. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Modal signals the whole container: in T-C1 run a the sampler process got SIGINT next to the runner, raised KeyboardInterrupt inside its round throttle and left as INFRA_FAILED. A pair sampler island (the launcher sets the marker path) now handles SIGINT/SIGTERM on its main thread by writing the reclaim marker, so the watcher leaves as PREEMPTED and the island waits for the container to stop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A pair's sampler islands can now use spot capacity while the trainer stays on demand. The sampler island copy gets spot billing without the legacy --spot volume (its durable state is the sample buffer); the flag is refused outside --rl-island-sample-role and together with --spot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In T-C1 run b the Modal runner and the sampler's handler wrote the marker through the same temp name at the same moment; the rename failed and the handler raised inside the island's main thread. The temp name is now per process and a failed write is printed, not raised. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d 0; shared records env names The records backend marks the last sampled turn when the next prompt no longer fits max_model_len (context_full); with the agent's non-zero exit the outer layer re-classifies the trajectory as POLICY_LIMIT, reward 0, as for a malformed tool call (main agent decision). The records-proxy env names move to gateway.records_env so the Miles adapter no longer imports the verl one (import boundary test). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cords by object A spot sampler island on Nebius gets the yeto-reclaim-notice host unit: its notice file is the sampler's reclaim marker, and the sampler writes the ack file once its LEAVE is stored, so the VM stops only after the LEAVE. Member records go through the store's object API when it has one (the staged s3/OSS mirror does not carry sample-buffer/, so the record stayed on the island disk). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
T-G1B g1b-1 lost the engine sampler's container; Modal ran it again and the new container started at round 0, wrote payloads under the same names (r0-row0, ...) and overwrote the first container's buffer payloads and tape. The verl buffer path's engine sampler island has the same gap. - The engine sampler island reuses the neutral sampler_resume layer (C1): each process is one incarnation, reads the resume point from the buffer, starts after the newest written round, refuses written rounds (RoundGuard), writes rl_sampler_join / rl_sampler_leave and a member record per incarnation, and stops writing on the reclaim marker. - Sample ids are <sampler>-<incarnation>-r<round>-<prompt row>, and the buffer refuses a payload name that is already stored. - Each incarnation writes its own tape (events.<incarnation>.jsonl). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The word-diversity rehearsal reward was gamed in T-G1B g1b-1: one 512-token "word" without spaces scored 1.0 for every sample, so round 2 had no gradient. yeto.rl.token_diversity_reward scores distinct token ids over response tokens. The engine sampler's Miles reward sample now carries tokens and response_length as a Miles Sample does. The G1C plan uses the new reward with lr 1e-4. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
g1b-1 held a sampler H100 about 30 min after both trainers had finished. orchestrate now cancels every call past a whole-run limit, and the sampler 5 min after both trainers returned 0. A sampler container Modal runs again writes its own log directory. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
T-G1B g1b-1 lost the engine sampler's container; Modal ran it again and the new container started at round 0, wrote payloads under the same names (r0-row0, ...) and overwrote the first container's buffer payloads and tape. The verl buffer path's engine sampler island has the same gap. - The engine sampler island reuses the neutral sampler_resume layer (C1): each process is one incarnation, reads the resume point from the buffer, starts after the newest written round, refuses written rounds (RoundGuard), writes rl_sampler_join / rl_sampler_leave and a member record per incarnation, and stops writing on the reclaim marker. - Sample ids are <sampler>-<incarnation>-r<round>-<prompt row>, and the buffer refuses a payload name that is already stored. - Each incarnation writes its own tape (events.<incarnation>.jsonl). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit a4b9962)
…kipped); eval E4 reads it and cross-checks consumed minus trained Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… reclaim, hold until the container stops; E2 checks continuous trainer versions - engine_sampler_entry.run: optional caller stop (leaves FINISHED) and on_event hook; build_engine_sampler separates the loop stop from the round reclaim. - engine_sampler_entry.hold_after_reclaim: a reclaimed process waits for the cloud to stop the container instead of exiting cleanly (main() calls it). - tools/eval: E2-verl is supported; the sampler function sets the reclaim marker and calls entry.run; judge_e2 adds the versions check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ed container may die before the function ends) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pack's entries carry mtime 0. With a remote sky API server, sky zips the workdir it uploads and ZIP refuses timestamps before 1980, so every launch with a code pack failed before provisioning. The digest covers names and bytes only; a tree unpacked earlier is stamped when it is reused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…reclaim signals every process, and the island must finish its round and leave PREEMPTED Found by eval-e2-verl-e2v-a: the reclaimed SGLang server died under round 2 and the reclaimed incarnation ended with an engine error instead of a clean leave. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The cluster yaml lives on the API server, so reading the SSH credentials from
it failed ('Cluster yaml ... not found') and the reclaim unit was never
installed. Without a local yaml, ask the server for the cluster's credentials
(sky status with credentials, as the sky CLI does) and write the key locally.
Checked against a live Nebius cluster.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…able bits sky uploads the workdir to a remote API server as a zip that the server unpacks without file modes, so every executable file reached the island as 0644 and the island refused the tree (code pack digest mismatch). The pack info now lists the executable files; verify_tree sets their bit again before it rebuilds the digest. Packs without the list are left alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… a spec Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The spot sampler commits and a4b9962 are patch-identical to commits already on this branch (C1, bd58b18); their conflicts keep this side, which also carries the later C1b/E2V changes. The SGLang topology check keeps the general tp<N>/dp<N> rule, which covers the rehearsal's tp1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Body:
Summary
Integrates the S25 work after #278 (s25-n71d) into one branch:
s25-e2v (integ4: G2/G3/G4, C1-C4, C3V, T-EVAL, E2V) + s25-c1b + s25-g1c + s25-n109.
89 non-merge commits. Some C1 commits appear twice (once from s25-c1, once
cherry-picked into s25-g1c); the patches are identical and the merge keeps one tree state.
Sampler islands and spot capacity (C1, C1b, E2V, G1C)
--rl-sampler-spot: spot sampler islands with an on-demand trainer; several samplers per trainer; the trainer's rounds split evenly over the samplers.rl_buffer_stale_skipped).Trainer islands (C2, C3, C3V, C4, N7.6b/c)
Stage 1 (GLM5.3-Next) rehearsal fixes (G1, G1C, G2-G4)
Agentic RL (N10.9)
Evaluation suite (T-EVAL)
tools/eval(list/run/judge/report): capabilities E1-E5 x Miles/verl on Modal A10G, Qwen3-0.6B.Merge notes
Completionusage checked against the G1Cprompt_token_idsfield.Tests
The first run had 1 failure in test_delta_protocol (unchanged file, timing); 5 single re-runs passed, the second full run passed.
yeto-stage1 plan examples/stage1/glm53-flash-ppu.yaml: rc=0; 2 known GAPs (PPU trainer image N0.2, docker device args N0.2b).python -m tools.eval list: lists 10 cells.Not verified
nebius compute instance stop).🤖 Generated with Claude Code