Repository navigation
S26 E4/E5: phase timing, analysis tools, release writer Volume optimization - #289
Draft
michaellchung wants to merge 64 commits into
Draft
michaellchung wants to merge 64 commits into
michaellchung wants to merge 64 commits into
Conversation
Eight arms per model preset ({bare,yeto} x {verl,miles} x {sync,async})
generated from one shared spec with a control fingerprint. Bare arms run
the frameworks' own entry points (verl main_ppo / fully_async_main,
Miles train.py / train_async.py) in the same images yeto uses; yeto arms
run verl island_entry and the yeto launcher. Parsers normalize verl
FileLogger, Miles perf lines and yeto tapes into per-round records
(missing fields are marked, never estimated); the report gives
yeto-vs-bare deltas with min-max range verdicts. Presets: Qwen3-0.6B
(E0.1) and Qwen3-4B-Instruct-2507 (E0.2). CLI: eff-plan, eff-run,
eff-parse, eff-report.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Allow --pipeline 1..P with --policy-sweep-fragments so fragment k+1 uploads while fragment k merges, checkpoints and broadcasts; commits stay in strict step order. Add --async-checkpoint (policy sweep only): the committed state is snapshotted and written on a blocking task, and the round is broadcast only after the checkpoint is durable. Checkpoint bytes, versions and the semantic profile hash are unchanged; the tape gains checkpoint_ms. Launcher flags --rl-sync-fragment-pipeline and --rl-sync-async-checkpoint (default off). Adds bit-equality, failure and real-syncer tests and a CPU micro-benchmark (tools/bench_sync_boundary.py). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ric recipe; TE flash fails on sm86) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add session dtype 4 (f32 session, bf16 PUSH deltas): INIT/BCAST/FINAL, masters and momentum stay f32; the client rounds f32 delta bytes to bf16 (round-to-nearest-even, streamed across part boundaries). The dtype is bound into the role-stream session contract. Launcher --rl-sync-push-dtype, island --sweep-push-dtype (default f32). Allow --checkpoint-every N (1 or a multiple of the fragment count) for policy sweeps: only checkpointed steps are durable before their broadcast, on both the synchronous and async paths; the interval is part of the semantic profile hash. Launcher --rl-sync-checkpoint-every (default 1). E3 launcher options are refused outside the full-parameter preset. Adds bf16 error-bound, interval/resume and real-syncer tests, and a third micro-benchmark profile. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…spec Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add session dtype 5 (f32 session, block-scaled int8 PUSH deltas): u32 block size, then per block an f32 absmax/127 scale and one i8 per value; error is at most absmax/254 per element. Launcher --rl-sync-push-int8-block (default 128) is bound into the role-stream contract. Add session flag 0x80 (bf16 downlink deltas) for f32-bulk sessions. Syncer and learners hold a bit-identical f32 anchor per fragment; BCAST is FULL f32 or bf16(theta - anchor), with a FULL refresh every N versions (--bcast-delta-full-every), FULL re-sends on HELLO/reconnect, and a SHA-256 check of the anchor after every update. A mismatch drops the connection and the reconnect restores the anchor. SyncerClient decodes before adapters see the payload: the Miles streaming sink is wrapped, the strict bridge (verl and Miles LoRA) gets f32 bytes. Masters, checkpoints and FINAL stay exact. Add --rl-sync-overlap for the full-parameter sweep: an island pushes round k, trains round k+1 at once, and at the next boundary rebases in place, theta <- G_k + (local - base), with G_k as the anchor so no delta is counted twice. The launcher raises --rl-max-policy-age to at least 1 before its checks; the island refuses overlap below that age. Adds Rust and Python codec/anchor tests, real-syncer int8/downlink/resync tests, a strict-bridge downlink test, and coordinator overlap tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…LoRA and /dev/shm NCCL host buffers The verl image applies yeto's patch_verl, whose sender hook imports yeto, so the bare arms now build the image without it. The fork's disaggregated checkpoint engine drops the PEFT config of unmerged LoRA params; the bare async arm sends merged weights instead. On Modal the trainer allocated NCCL host buffers with cuMem while the receiver fell back to /dev/shm and failed to attach (NCCL_DEBUG=INFO run); NCCL_CUMEM_HOST_ENABLE=0 keeps both on /dev/shm. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sweep Pre-GPU check of --rl-sync-overlap on the Miles role stream found three problems: - The launcher raised --rl-max-policy-age to 1, but the dense sweep island refuses any age other than 0, so an overlap run could never start; and an age >= 1 also turns on Miles' fully_async rollout (partial rollout when colocated). Overlap is not rollout staleness: the driver publishes the boundary's cut before the next generation, so rollouts stay on-policy and only the local weights lag the global policy by one outer step. Keep the age at 0 (refuse others at launch) and bind the lag into the session contract only. - After a rebase the applied cut has local step generation 0, and Miles pushes only cuts with progress, so the second sweep's push failed. The submission is now the rebased cut relabeled with the rounds trained since the round base (same values, hashes and refs). - The boundary submitted once; a PULL that arrived later waited a whole local round. The overlap boundary now submits the whole sweep before training on, and the launcher requires --rl-sync-fragment-pipeline equal to --fragments so the syncer sends all PULLs at once. Adds Miles role-stream tests (real adapter and role stream, CPU tensors, fake trainer/Ray/syncer): exact-once rounds through rebase_fragment, and syncer restarts while the overlapped round trains (before commit, after commit with a lost broadcast, partly committed sweep). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…over rounds that report the value The fork logs a step's training metrics one parameter version late and never logs the last step's, so rows are keyed by training/global_step and the step without metrics keeps its trained batch from the config. Throughput rates use the steady rounds that report the value; the report lists partial counts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… does not) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y Ray worker On Modal one weight-sync process could allocate NCCL host buffers with cuMem while its peer fell back to /dev/shm and then failed to attach (eff01-yeto-verl-async-e; NCCL_DEBUG=INFO in eff01-bare-verl-async-d). NCCL_CUMEM_HOST_ENABLE=0 unless the caller sets it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The overlap rebase uses the bf16-tracked anchor and pushes theta - anchor against that same anchor, so the syncer's f32 masters still receive every round's progress exactly once and no DownlinkResync happens. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ains without dropout --rl-gradient-accumulation-fusion lets a run keep Megatron's fused wgrad (the Miles recipes' default); off keeps the historical argv. The generic recipe passed no dropout, so Megatron's 0.1 applied during RL training while the Miles recipes pass 0.0; it now passes 0.0 as the Qwen3.5 recipe does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No deterministic inference, constant LR, full recompute, dynamic batching, gradient-accumulation fusion, and an algorithm spec with the recipe's clip range (0.2 / 0.28); S26 GOAL §6 makes the bare framework's settings the reference. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… store) miles_local.sh pulls a copy of the repo whose test fixtures are tapes of other runs; their rl_local_round rows replaced the run's rewards (all four yeto+Miles runs showed the same fixture curve with reward 0 in round 0). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… missing The trainer's timing_s/gen and yeto's driver generate phase only pull finished samples from the rollouter queue. Bare verl logs the rollouter's active time per parameter version; yeto logs none per round, so its value stays missing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…from the rollouter E0.1 found yeto+Miles 77 % (sync) and 252 % (async) slower than bare Miles. The tape shows the gap is the per-round round cut: the arm sets a checkpoint store, so a cut runs after every publish (5.7-12.9 s: trainer wake, export, sleep, Modal volume commit), and in fully_async mode the cut also stops Miles' background worker, so generation never runs ahead of training. The bare arms keep no checkpoint; cut only at the end (--rl-cut-every rounds). yeto+verl async generation time was missing: the tape's rl_publication_check.rollouter_timing carries the rollouter's active time per policy version, the same metric the bare arm reports. Read it (version v closes round v - 1). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… path VerlPolicyState.export_digest returns a trainer-resident handle: the workers validate the LoRA like canonical_state, compute the policy digests and the bf16 send checksum, and only those cross Ray. Both verl publishers take the send checksum from the handle; other states are hashed on the driver as before. materialize falls back to the full export; YETO_RL_PUBLISH_FASTPATH=0 keeps the old path. Tests pin hash and checksum equality with the full export. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… background E0.1 measured a periodic round cut at 5.7-12.9 s per round on the critical path of a colocated Miles island: wake the offloaded trainer (0.7 s), export (1.8 s), sleep it again (1.0 s), then a synchronous store commit (1.5-8 s). - prepare(): when a cut is due at the next safe point, the driver writes the trainer part right before it offloads a colocated trainer for the publication (phase "checkpoint"), for the state it is about to publish. The safe point keeps it only if its own context names the same cut (id, progress incl. policy hash, data cursor, ledger, outer token, in-flight); otherwise the unpointed files are dropped and the cut is taken as before. - Background commit: a periodic cut writes the snapshot and LATEST, then a thread commits and prunes (ResumeController.commit_checkpoint_store). The next cut joins it first and reports rl_cut_committed. Pruning only follows a successful commit, so a failed or lost commit leaves the previous cut as the committed one. The final cut, and a run with a sample buffer that commits the same store, stay inline. YETO_RL_CUT_BACKGROUND_COMMIT=0 turns it off. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A round cut on a Miles fully_async island stopped the background worker and requeued every untrained prompt into the live data source (P0-S decision 3). With a cut every round the worker restarted each round, so generation never ran ahead of training (E0.1: 8 worker starts in 8 rounds, policy age 0, 2.15 s of serial generation per round). snapshot_untrained() reads the data source buffer, every drawn untrained group (reset copies) and the data cursor in one step on the worker's event loop, where the worker draws; the cut carries them and the cursor read with them, and the live run is untouched. A resume still regenerates exactly the drawn, untrained prompts (decision 3 unchanged); the resume manifest's data watermark is that cursor. requeue_untrained() stays for its other callers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
e0f833d made the generic Miles recipe pass --attention-dropout 0.0 and --hidden-dropout 0.0 (the bare Miles recipe's values); the seven decoupling golden samples were not regenerated and failed on miles_argv. The only difference is those four argv items (and the argv hash); no algorithm or contract hash changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
E0.1: the yeto+Miles sync generate span was 0.46 s longer than Miles' perf/rollout_time. 0.44 s of it is --offload-rollout (release_memory_occupation), which Miles times outside rollout_time but MilesRolloutPool.generate ran inside the driver's generate phase. The driver now asks the pool to defer it and runs it as phase "rollout_offload"; callers that use the pool directly keep the old in-generate offload. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
final (default) cuts once at the end like the bare arms keep no checkpoint; every-round is the product default. Kept out of the control fingerprint so the bare runs stay comparable. The yeto+verl arm has no checkpoint store in either mode. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With bf16 downlink deltas, the last BCAST of a fragment can carry the same
version as its FINAL. Islands store one copy per (fragment, version) and
refuse a second copy with other bytes, so the delta-tracked BCAST and the
exact FINAL collided ('authoritative fragment replay changed bytes'), the
islands redialed in a loop and finalization timed out (s26-e3-p06-D8).
Broadcasts of a fragment's last version now always go out FULL.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GPU canary e1a-yeto-miles-async-1 (every-round cuts, after 74016eb and 3b00fe5): overlap is back (one fully_async worker start), but each cut still held the round for 2.1 s writing and hashing the 283 MB shard, so a round took 5.3 s against 4.15 s bare (78 %). MilesTrainerGroup.start_cut captures every rank's state at the step boundary (CPU copies) and the ranks write, fsync and hash their shards in a thread (cut_plugin.save_cut_shard background=True); finish_cut collects the summaries and commits the manifest. A periodic cut now only captures at its safe point; the shard write, manifest, pointer, store snapshot and commit complete before the next cut (join_commit, rl_cut_committed). Until then the previous cut stays the pointed and committed one; a failed completion drops the unpointed files. The store's first cut, the final cut, runs with a sample buffer or a critic, and YETO_RL_CUT_BACKGROUND_COMMIT=0 stay inline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he sample filter hook GPU e1a2-yeto-miles-async-1: with the shards written in the background a cut held the safe point 0.2 s, but joining the previous Modal volume commit (1.8-9.9 s) at the next cut still held the round. A periodic cut is now skipped (rl_cut_skipped) while that commit runs; the final cut waits for it. The same run spent 0.9 s per round between Miles' 'Finish rollout' and the train data (0.04 s in E0.1): the yeto sample filter hook now prints its per-step timing to find it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
state_digest copied every tensor into a bytes object before hashing, holding the GIL for the copy. Hashing the buffer gives the same digest (checked for float32, bfloat16, empty tensors and the numpy RNG arrays) and lets hashlib release the GIL, so a background cut writer does not stall the trainer's own thread (E1 GPU: a colocated offload took 4.0 s instead of 1.3 s while the shard was written). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
e1a3-yeto-miles-async-1 still held each round 5-6 s: join_commit completed the pending cut, started its store commit and then joined that same commit. A periodic cut now completes the pending cut and starts its commit without waiting (the commit still running at the next safe point skips that cut); only the final cut waits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fully_async round metadata took 1.1 s per round on GPU (0.09 s in round 0). Print the seconds of round id, group records, carry-over fields, submitted groups and the sink put with the hook timing line. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When the prepare of a periodic cut met a running commit, and the commit was done by the safe point, the cut was taken there by waking the offloaded colocated trainer (~2 s, Modal A10G sync run e1a7). Such a cut is now skipped (rl_cut_skipped) when a committed cut exists; the next safe point cuts again. Cuts whose prepare was not declined, the first cut and the final cut are taken as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The yeto+Miles arm ran in one 8-CPU / 64 GiB container whatever its card count; the bare arms (modal_eff) get 8 CPU and 48 GiB per card, i.e. 16 CPU for the two-card fully_async arm. The yeto fully_async trainer took 6-11 % more time per token than the bare one on the Modal A10G E1 runs while the one-card sync arms matched, with and without periodic cuts. The arm now asks for 8 CPU and 48 GiB per card. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A periodic cut whose store sync is deferred copied the state dir into the store
only at the next safe point, after one more round had changed it: the snapshot
held the ledger of round N+1 with the pointer of the cut at round N, and a
resume from it was refused ('the restored ledger's consumed groups (64, ...)
differ from the pointer's (56, ...)', Modal A10G resume run e1c, cut written by
e1a7). The deferred cut now stages a copy of the state dir at the cut (a few KB:
ledger, journal) and the background sync snapshots that copy, with the same
pointer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…stics E3 showed sweeps waiting on the slowest island's upload and one C32 run where a stream stopped moving with every socket still open. Island-side SyncerClient changes (all default off): - --wan-stall-timeout S: a watchdog fails the connection group when an outbound stream holds bytes, or an inbound chunked message stops growing, for S seconds; sockets also get TCP_USER_TIMEOUT=S. Recovery uses the existing redial path: the syncer replays its PULLs and the island pushes the fragment again at the same (global_step, round_attempt). - --wan-stream-schedule adaptive: each 4 MiB chunk goes to the data stream with the earliest estimated finish (backlog / EWMA rate) instead of round robin, so slow or stalled streams carry less. No wire or syncer change. - rl_full_parameter_round now records per-stream upload rates, the wait after this island's upload drained, and stall counts; scalars go to W&B on rl/step. A real-syncer test stalls one stream mid-sweep (rr and adaptive) and checks the redial keeps versions and finals bit for bit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A background cut took its adapter and optimizer copies to the host at the step boundary (0.12-0.14 s per round on A10, one wait per tensor). It now clones them device-to-device there and the writer thread moves them to the host, then computes the N7.6 resume state from the lrs and scheduler progress read at the boundary. The shard is byte-identical to the inline host path. The device path needs CUDA parameters and free memory for twice the copies plus 1 GiB; otherwise, or on a device OOM, the capture takes the host path. Each rank reports its mode and reason; rl_round_cut carries them as cut_capture. YETO_RL_CUT_DEVICE_COPY=0 turns the device path off. The Miles fork takes export_named_optimizer_state(device=None) on branch yeto/s26-e1-device-export (466b89b92, not pushed); for the pinned image the same entries are built from the fork's optimizer_slots. Tests compare both with the fork's host export. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The trainer-side policy digest copied each LoRA tensor to the host on its own (about 400 waits for Qwen3-0.6B). stage_f32_on_host converts and checks the tensors on the GPU and copies them into one reusable pinned buffer with one synchronization. The values, and so both digests, are bit for bit those of the per-tensor copies; a non-finite tensor is named as before. verl: the worker digest stages the same way, validates its own copies without a second clone, and computes the bf16 send checksum in a thread beside the two digest streams. Only the digest path stages; full exports keep their own tensors. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Under max_policy_age > 0 the hook read the policy-age limit from the sink actor once per group and twice per trajectory record, each time with a ray.get_actor lookup: about 70 actor calls a round, 0.03 s of the 0.045 s hook on A10. one_sink_view memoizes sink reads and the actor handle within one run of record_trained_groups, the all-samples hook and the buffer filter. A write from the same thread drops the value from the view; other writers are seen from the next run on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Modal A10G containers report NVIDIA A10 or, sometimes, NVIDIA A10G; the A10G trained about 12 % slower (E1 run e1a9). The bare and yeto+verl containers now refuse an A10G when the spec asks for the A10 the other runs had, parsed runs record the GPU names from NVML, and the report refuses to compare runs on different GPU models. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hash_seconds and export_seconds are wall times and may differ between two calls on the same tensors (seen on yeto-cpu-test). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With --offload-train the colocated trainer sleeps right after the cut is prepared, and the writer thread then failed to read the device copies (CUDA error: invalid argument; 5 of 6 cuts refused in GPU run e1g-yeto-miles-sync-1, the run itself went on). Such a trainer now takes the host path, reported as reason offload_train. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The background cut writer ran torch.save and four sha256 passes in a thread of the trainer process while the next step trained; each trained token was about 8 % slower in those rounds (GPU runs e1g/e1h). The trainer thread now copies the device snapshot into shared host memory and a per-rank spawn writer process (nice 10, one intra-op thread) serializes, hashes and summarizes it. A writer-process failure removes its partial file and the thread writes the same shard; the summary's writer field and rl_cut_committed record the mode and the fallback. YETO_RL_CUT_WRITER_PROCESS=0 keeps the thread. Strings are interned before torch.save so the shard bytes depend on values only: a shard written in the writer process is byte-identical to the inline one (tests compare file bytes, digests and the N7.6 state). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Modal A10G containers are an NVIDIA A10 or an A10G, and the A10G trains
about 12 % slower (E1 run e1a9). --modal-gpu-name MODEL fails an island
at container start unless every GPU nvidia-smi reports is that model
("NVIDIA A10" refuses "NVIDIA A10G"). The efficiency eval passes it
for the yeto+Miles arm with the model the bare arms require, so every
arm runs on the same card.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In sync colocated mode the trainer hashed the staged policy (two sha256 streams, about 0.1 s) before it could offload. export_digest now takes deferred=True: the trainer returns once the tensors are staged in its pinned buffer and hashes them in a thread; the state's DeferredDigest collects the result (collect_digest plugin) on first read, at the publication. The values are those of the eager digest; a new export waits for an uncollected hash before it reuses the buffer. The cut context reads the policy hash last. The driver defers only for a colocated trainer that offloads to publish (YETO_RL_DEFERRED_DIGEST=0 turns it off). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…elaunch s26-e3b-p06-C32a: learner 0's connection group died before the syncer applied its INIT, the client never sent INIT again, and no island got the initial cut. s26-e3b runs also showed that a slow island is slow on every stream (container level), which no stream scheduling fixes. - SyncerClient.send_initial_fragments/init_replay_due/ mark_initial_cut_received: learner 0 replays INIT on every new connection group until the initial cut arrives (the syncer ignores INIT it already applied). TCP_USER_TIMEOUT is set only after the initial cut, so the multi-GB INIT and first broadcast are never reset by the kernel. - MSG_BW_PROBE (26): an HMAC-sealed probe connection to the legacy syncer uploads N bytes, gets an ack, downloads M bytes; never part of a session, at most 64 concurrent, 1 GiB per direction, 300 s. - --wan-probe-min-mbps / --wan-probe-min-down-mbps (default off): Miles islands probe the actor syncer before loading the model, record rl_wan_probe, and exit 75 below the minimum. The fleet controller requests a new container up to --wan-probe-relaunches times (default 2), also under --no-island-relaunch, since the island never joined the session. Real-syncer test: INIT interrupted between fragments is replayed and the finals match an undisturbed run bit for bit; the probe works on the same port and a wrong key is refused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…uster arms) 0 keeps the E0 shape and its control fingerprint; async splits the total evenly between rollout and trainer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t mid-INIT Only the legacy syncer serves MSG_BW_PROBE. With --rl-elastic every probe would fail, every island would be judged slow and relaunched until its budget ran out, so the launch now refuses the probe flags there. Tests: - real syncer: one data socket of learner 0's group is shut down between the INIT of fragment 0 and fragment 1; the client replaces the group, replays INIT, and the final versions and bytes equal an undisturbed run. - slow-container decision: threshold equality accepts, upload just below rejects, slow download rejects, a zero download minimum is not checked, and a stream error rejects a fast link. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… for the yeto arms E11 trains on DAPO-Math-17k (gsm8k saturates Qwen3-4B-Instruct-2507). eff_prep reads a JSONL data_file and keeps the first train_rows rows; the yeto+Miles arms read the same rows as a local task file (EXTRA_FILES in miles_local.sh). The 2-island arm splits every round's prompts between the islands. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t memory fraction The bare Miles loader puts model_args_utils on the path (qwen3-4B-Instruct-2507.py imports it: e11-bare-miles-async-1). The yeto+verl arm passes the spec's mem_fraction as YETO_VERL_GPU_MEM; at the 0.5 default vLLM cannot fit the 4B model's 262144-token context (e02-yeto-verl-sync-1). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Legacy scheduling with LoRA Miles islands hung in the first policy apply (e11-yeto-2i-1); elastic is the multi-island LoRA path with real-card runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…E11 arms Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…preset A single --rl-single-island-no-sync island with both roles may now train every parameter (--tuning full); before, only sampler/trainer islands could. The efficiency spec gains tuning and tensor_parallel fields and the e11-qwen3-0.6b-full preset (trainer TP 2, so DP stays 1 as Miles full-parameter islands with engines require). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…land arm eff-run kills a run when no training step line arrives for 15 min (30 min before the first step), in the container for the bare and yeto+verl arms and in miles_local.sh for the yeto+Miles arm. yeto_miles_full_islands_launcher_args builds the 2-island full-parameter arm with the E3 C sync configuration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Trainer side: PolicyReleaseStore.release records the seconds of each phase (read LATEST, export, digest, weight commit, LATEST commit, prune); ReleasingPublisher adds the publish to its own engines, released_at and the release size to rl_policy_released. Sampler side: PolicyFollower records its phases and when it first read each release (seen_at); EngineReleaseLoader records stage, lock, engine load, stamp, unlock, warm-up, checksum and probe seconds; rl_sampler_policy_loaded carries them with released_at. rl_sampler_round carries generate_s. --rl-sampler-fused-stage (default off, LoRA sampler only): the staging copy of the adapter computes the release digest, so a release is read from the store once instead of twice. The LoRA copy now happens before the update lock in both modes; engine calls and their order are unchanged. tools/eval/sampler_islands.py: E4 scale report (rounds, rates, preemption recovery, speedup vs a baseline run) and E5 latency report (per release and sampler island) from island tapes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eport Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… not a round Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PolicyReleaseStore(cached_latest=True, the default): the writer of a release directory remembers the release it wrote last instead of reloading the store (a Modal Volume reload took 2-27 s per release in E4/E5) to read LATEST. It reloads for its first release in a process (a resumed trainer continues the seq), after a failed release, and when the LATEST it sees without a reload is not the one it wrote (another writer). Weights, record and LATEST go out in one commit; readers already load only complete releases (check_complete), so an early-visible LATEST only delays the load. --rl-release-reload-latest (trainer or convertible node) restores the earlier behavior. rl_policy_released carries latest_source. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Defaults
Timing is additive. Fused LoRA stage is off by default. The release-writer change (single commit, remembered LATEST) is on the normal path.
Branch dependencies
Stacked on s26-e11-cluster (adds 4 commits: 189df10, f183d75, 72adc7c, dd88b26). Merge that PR first.
Tests
GPU evidence
Not verified
🤖 Generated with Claude Code