Repository navigation

S26 E0.2/E11: cluster comparison tools and full-parameter no-sync single island - #288

Merged

michaellchung merged 60 commits into

mainfrom

s26-e11-cluster

Oct 26, 2026

Merged

S26 E0.2/E11: cluster comparison tools and full-parameter no-sync single island#288
michaellchung merged 60 commits into
mainfrom
s26-e11-cluster

Commits on Oct 20, 2026
- tools/eval: add efficiency evaluation (bare vs yeto, verl and Miles)
  
  chowhsu
  and
  claude
  committed
  5a01732
- syncer: pipeline policy-sweep fragments and async per-round checkpoints
  
  chowhsu
  and
  claude
  committed
  8358c1b
- tools/eval: bare Miles trains with unfused attention (match yeto generic recipe; TE flash fails on sm86)
  
  chowhsu
  and
  claude
  committed
  261ba2e
- syncer: bf16 sweep pushes and sweep-aligned checkpoint intervals
  
  chowhsu
  and
  claude
  committed
  eb9e229
- tools/eval: yeto+verl async arm passes the staleness-2 TIS algorithm spec
  
  chowhsu
  and
  claude
  committed
  c3c9d09
- syncer: int8 sweep pushes, bf16 downlink deltas, sync/train overlap
  
  chowhsu
  and
  claude
  committed
  1a783d0
- tools/eval: bare verl arms run the unpatched fork; async uses merged LoRA and /dev/shm NCCL host buffers
  
  chowhsu
  and
  claude
  committed
  63dc064
- overlap: keep rollout policy age 0, relabel rebased cuts, push whole sweep
  
  chowhsu
  and
  claude
  committed
  8af78fa
- tools/eval: parse bare verl fully_async rows by training step; rates over rounds that report the value
  
  chowhsu
  and
  claude
  committed
  052c31e
- tools/eval: bare Miles arms do not shuffle prompts (yeto's Miles argv does not)
  
  chowhsu
  and
  claude
  committed
  c9245e5
- verl adapter: fully_async keeps NCCL host buffers on /dev/shm in every Ray worker
  
  chowhsu
  and
  claude
  committed
  73442ad
- tools/eval: read the yeto+Miles launcher exit code into the parsed run
  
  chowhsu
  and
  claude
  committed
  0122f4e
- test: overlap island with bf16 downlink deltas against the real syncer
  
  chowhsu
  and
  claude
  committed
  152374b
- miles adapter: opt-in gradient-accumulation fusion; generic recipe trains without dropout
  
  chowhsu
  and
  claude
  committed
  e0f833d
- tools/eval: yeto+Miles arm takes the bare Miles recipe settings
  
  chowhsu
  and
  claude
  committed
  6a4263e
- tools/eval: read only the run's own tapes (skip the repo copy and the store)
  
  chowhsu
  and
  claude
  committed
  ff893bc
- tools/eval: verl fully_async generation time from the rollouter, else missing
  
  chowhsu
  and
  claude
  committed
  ba17953
Commits on Oct 21, 2026
- eff eval: final round cut only for yeto+Miles; verl async generation from the rollouter
  
  chowhsu
  and
  claude
  committed
  a4b1104
- verl: hash the LoRA on the trainer for the single-island publish fast path
  
  chowhsu
  and
  claude
  committed
  77ce9cc
- round cut: write before the trainer offloads; commit the store in the background
  
  chowhsu
  and
  claude
  committed
  74016eb
- fully_async cut: record untrained prompts without stopping the worker
  
  chowhsu
  and
  claude
  committed
  3b00fe5
- golden: regenerate decoupling samples for the explicit Miles dropout 0.0
  
  chowhsu
  and
  claude
  committed
  84d43ce
- driver: time the rollout offload after generation as its own phase
  
  chowhsu
  and
  claude
  committed
  36eb805
- eff eval: --checkpoint final|every-round for the yeto+Miles arm
  
  chowhsu
  and
  claude
  committed
  dfe8b91
- syncer: send each fragment's FINAL version as an exact FULL broadcast
  
  chowhsu
  and
  claude
  committed
  bc8f53b
- round cut: write the trainer shards in the background for periodic cuts
  
  chowhsu
  and
  claude
  committed
  8ef1018
- round cut: skip a periodic cut while the previous commit runs; time the sample filter hook
  
  chowhsu
  and
  claude
  committed
  de275ef
- cut digests: hash tensor buffers without a bytes copy
  
  chowhsu
  and
  claude
  committed
  77b460c
- round cut: a periodic cut does not wait for the store commit it starts
  
  chowhsu
  and
  claude
  committed
  86d8872
- rollout metadata hook: time its steps (E1 diagnosis)
  
  chowhsu
  and
  claude
  committed
  0f615ab
- fully_async: estimate the cross-version IS truncation off the critical path
  
  chowhsu
  and
  claude
  committed
  427d142
- test: the fully_async round report fake takes build_metadata's new keyword
  
  chowhsu
  and
  claude
  committed
  4015cc5
- round cuts: commit the deferred cut fully off the critical path
  
  chowhsu
  and
  claude
  committed
  1502e7d
- driver: take the terminal load probe off the round's critical path
  
  chowhsu
  and
  claude
  committed
  ae82993
- round cuts: collect a skipped prepared cut's writer at the next cut
  
  chowhsu
  and
  claude
  committed
  31a951a
- round cuts: do not wake an offloaded trainer for a declined periodic cut
  
  chowhsu
  and
  claude
  committed
  0a11aa1
- eval: give the yeto+Miles arm the bare arms' container resources
  
  chowhsu
  and
  claude
  committed
  f19a055
- round cuts: sync a deferred cut's state dir as it was at the cut
  
  chowhsu
  and
  claude
  committed
  19e595a
- wan: per-stream stall timeout, rate-aware chunk striping, sync diagnostics
  
  chowhsu
  and
  claude
  committed
  3dcbec8
- round cuts: copy a background cut's state on the device first
  
  chowhsu
  and
  claude
  committed
  8154509
- publish fast path: stage the digest's tensors with one GPU wait
  
  chowhsu
  and
  claude
  committed
  13d53ab
- rollout metadata hook: read each sink value once per hook run
  
  chowhsu
  and
  claude
  committed
  19f2648
- eval: tell an NVIDIA A10 from an A10G
  
  chowhsu
  and
  claude
  committed
  4a1c245
- test: compare staged digests without their timing fields
  
  chowhsu
  and
  claude
  committed
  1fde82f
- round cuts: keep a sleeping trainer's cut on the host path
  
  chowhsu
  and
  claude
  committed
  d583c59
- round cuts: serialize and hash background shards in a writer process
  
  chowhsu
  and
  claude
  committed
  2f3d79c
- launcher: pin a Modal island to one GPU model with --modal-gpu-name
  
  chowhsu
  and
  claude
  committed
  27f66dd
- publish fastpath: hash the policy while a colocated trainer offloads
  
  chowhsu
  and
  claude
  committed
  d8d159c
- wan: replay INIT after a mid-init redial, link probe with container relaunch
  
  chowhsu
  and
  claude
  committed
  2509fde
- Merge s26-e3-sync into s26-e11-cluster
  
  chowhsu
  and
  claude
  committed
  329472f
- eval efficiency: spec.cards sets the per-run card total (E11 whole-cluster arms)
  
  chowhsu
  and
  claude
  committed
  87c298a
- wan: refuse the link probe with the elastic syncer; test a lost socket mid-INIT
  
  chowhsu
  and
  claude
  committed
  823b2e8
- eval efficiency: E11 preset and the 2-island yeto+Miles launcher args
  
  chowhsu
  and
  claude
  committed
  140aefa
- eval efficiency: JSONL datasets, a train-row cap and local task files for the yeto arms
  
  chowhsu
  and
  claude
  committed
  b0dfb1c
- eval efficiency: derived Miles model scripts and the yeto+verl rollout memory fraction
  
  chowhsu
  and
  claude
  committed
  392368b
- eval efficiency: the E11 2-island arm uses elastic island scheduling
  
  chowhsu
  and
  claude
  committed
  5593787
- eval efficiency: cluster.py, reward against samples and time for the E11 arms
  
  chowhsu
  and
  claude
  committed
  ae1a26e
- Merge commit '823b2e82' into s26-e11-cluster
  chowhsu
  committed
  f1b26ff
- Full-parameter no-sync single island and the E11 0.6B full-parameter preset
  
  chowhsu
  and
  claude
  committed
  fb520a6
- eval efficiency: no-progress watchdog and the E11 full-parameter 2-island arm
  
  chowhsu
  and
  claude
  committed
  8783b25

Commits

Commits on Oct 20, 2026

Commits on Oct 21, 2026