Repository navigation

S26 E1: single-island overhead (background round cuts, publish fast path, profilers) - #287

Closed

michaellchung wants to merge 55 commits into

s25-finalfrom

s26-e1-overhead

Closed

S26 E1: single-island overhead (background round cuts, publish fast path, profilers)#287
michaellchung wants to merge 55 commits into
s25-finalfrom
s26-e1-overhead

Commits on Oct 20, 2026
- tools/eval: add efficiency evaluation (bare vs yeto, verl and Miles)
  
  chowhsu
  and
  claude
  committed
  5a01732
- tools/eval: bare Miles trains with unfused attention (match yeto generic recipe; TE flash fails on sm86)
  
  chowhsu
  and
  claude
  committed
  261ba2e
- tools/eval: yeto+verl async arm passes the staleness-2 TIS algorithm spec
  
  chowhsu
  and
  claude
  committed
  c3c9d09
- tools/eval: bare verl arms run the unpatched fork; async uses merged LoRA and /dev/shm NCCL host buffers
  
  chowhsu
  and
  claude
  committed
  63dc064
- tools/eval: parse bare verl fully_async rows by training step; rates over rounds that report the value
  
  chowhsu
  and
  claude
  committed
  052c31e
- tools/eval: bare Miles arms do not shuffle prompts (yeto's Miles argv does not)
  
  chowhsu
  and
  claude
  committed
  c9245e5
- verl adapter: fully_async keeps NCCL host buffers on /dev/shm in every Ray worker
  
  chowhsu
  and
  claude
  committed
  73442ad
- tools/eval: read the yeto+Miles launcher exit code into the parsed run
  
  chowhsu
  and
  claude
  committed
  0122f4e
- miles adapter: opt-in gradient-accumulation fusion; generic recipe trains without dropout
  
  chowhsu
  and
  claude
  committed
  e0f833d
- tools/eval: yeto+Miles arm takes the bare Miles recipe settings
  
  chowhsu
  and
  claude
  committed
  6a4263e
- tools/eval: read only the run's own tapes (skip the repo copy and the store)
  
  chowhsu
  and
  claude
  committed
  ff893bc
- tools/eval: verl fully_async generation time from the rollouter, else missing
  
  chowhsu
  and
  claude
  committed
  ba17953
Commits on Oct 21, 2026
- eff eval: final round cut only for yeto+Miles; verl async generation from the rollouter
  
  chowhsu
  and
  claude
  committed
  a4b1104
- verl: hash the LoRA on the trainer for the single-island publish fast path
  
  chowhsu
  and
  claude
  committed
  77ce9cc
- round cut: write before the trainer offloads; commit the store in the background
  
  chowhsu
  and
  claude
  committed
  74016eb
- fully_async cut: record untrained prompts without stopping the worker
  
  chowhsu
  and
  claude
  committed
  3b00fe5
- golden: regenerate decoupling samples for the explicit Miles dropout 0.0
  
  chowhsu
  and
  claude
  committed
  84d43ce
- driver: time the rollout offload after generation as its own phase
  
  chowhsu
  and
  claude
  committed
  36eb805
- eff eval: --checkpoint final|every-round for the yeto+Miles arm
  
  chowhsu
  and
  claude
  committed
  dfe8b91
- round cut: write the trainer shards in the background for periodic cuts
  
  chowhsu
  and
  claude
  committed
  8ef1018
- round cut: skip a periodic cut while the previous commit runs; time the sample filter hook
  
  chowhsu
  and
  claude
  committed
  de275ef
- cut digests: hash tensor buffers without a bytes copy
  
  chowhsu
  and
  claude
  committed
  77b460c
- round cut: a periodic cut does not wait for the store commit it starts
  
  chowhsu
  and
  claude
  committed
  86d8872
- rollout metadata hook: time its steps (E1 diagnosis)
  
  chowhsu
  and
  claude
  committed
  0f615ab
- fully_async: estimate the cross-version IS truncation off the critical path
  
  chowhsu
  and
  claude
  committed
  427d142
- test: the fully_async round report fake takes build_metadata's new keyword
  
  chowhsu
  and
  claude
  committed
  4015cc5
- round cuts: commit the deferred cut fully off the critical path
  
  chowhsu
  and
  claude
  committed
  1502e7d
- driver: take the terminal load probe off the round's critical path
  
  chowhsu
  and
  claude
  committed
  ae82993
- round cuts: collect a skipped prepared cut's writer at the next cut
  
  chowhsu
  and
  claude
  committed
  31a951a
- round cuts: do not wake an offloaded trainer for a declined periodic cut
  
  chowhsu
  and
  claude
  committed
  0a11aa1
- eval: give the yeto+Miles arm the bare arms' container resources
  
  chowhsu
  and
  claude
  committed
  f19a055
- round cuts: sync a deferred cut's state dir as it was at the cut
  
  chowhsu
  and
  claude
  committed
  19e595a
- round cuts: copy a background cut's state on the device first
  
  chowhsu
  and
  claude
  committed
  8154509
- publish fast path: stage the digest's tensors with one GPU wait
  
  chowhsu
  and
  claude
  committed
  13d53ab
- rollout metadata hook: read each sink value once per hook run
  
  chowhsu
  and
  claude
  committed
  19f2648
- eval: tell an NVIDIA A10 from an A10G
  
  chowhsu
  and
  claude
  committed
  4a1c245
- test: compare staged digests without their timing fields
  
  chowhsu
  and
  claude
  committed
  1fde82f
- round cuts: keep a sleeping trainer's cut on the host path
  
  chowhsu
  and
  claude
  committed
  d583c59
- round cuts: serialize and hash background shards in a writer process
  
  chowhsu
  and
  claude
  committed
  2f3d79c
- launcher: pin a Modal island to one GPU model with --modal-gpu-name
  
  chowhsu
  and
  claude
  committed
  27f66dd
- publish fastpath: hash the policy while a colocated trainer offloads
  
  chowhsu
  and
  claude
  committed
  d8d159c
- eval: opt-in trainer step profile (--rl-profile-trainer)
  
  chowhsu
  and
  claude
  committed
  1c0df83
- round cuts: stage a background cut's device copies on the host at the step
  
  chowhsu
  and
  claude
  committed
  fe5e612
- Revert "round cuts: stage a background cut's device copies on the host at the step"
  
  chowhsu
  and
  claude
  committed
  40cf73a
- round cuts: hand a background cut to its writer in one shared buffer
  
  chowhsu
  and
  claude
  committed
  74c2f6f
- eval: sample the island driver per phase under --rl-profile-trainer
  
  chowhsu
  and
  claude
  committed
  b77491c
- eval: sample the thread that drives the phases, not the enabling one
  
  chowhsu
  and
  claude
  committed
  92f33bc
- eval: phase profile labels carry the caller chain
  
  chowhsu
  and
  claude
  committed
  0dec43d
- WIP: verify publications only on cut, first and every-K rounds
  
  chowhsu
  and
  claude
  committed
  2ec20f7
- round cuts: device-copy capture for a trainer that sleeps after the cut
  
  chowhsu
  and
  claude
  committed
  b664b9d
- round cuts: default interval 4; verify publications at that interval
  
  chowhsu
  and
  claude
  committed
  e79520d
- driver: mark only unverified publications in the publication event
  
  chowhsu
  and
  claude
  committed
  096bdf6
- test: keep the phase profiler off after the profile tests
  
  chowhsu
  and
  claude
  committed
  becb3a3
- round cuts: commit the store's first cut in the background too
  
  chowhsu
  and
  claude
  committed
  2f7e288
- test: complete deferred first cuts in the recovery abstraction tests
  
  chowhsu
  and
  claude
  committed
  233f139

Commits

Commits on Oct 20, 2026

Commits on Oct 21, 2026