Repository navigation

S25 final integration: sampler islands on spot, synced trainers to samplers, round cut, stage-1 rehearsal, agentic RL, eval suite - #279

Merged

michaellchung merged 96 commits into

mainfrom

s25-final

Oct 26, 2026

Merged

S25 final integration: sampler islands on spot, synced trainers to samplers, round cut, stage-1 rehearsal, agentic RL, eval suite#279
michaellchung merged 96 commits into
mainfrom
s25-final

Commits on Oct 20, 2026
- Read verl TransferQueue provenance from the extra_fields record
  
  chowhsu
  and
  claude
  committed
  e81cf39
- Map Miles convert_to_hf names to checkpoint names in quant hf_slices; add the T-G3 GPU harness
  
  chowhsu
  and
  claude
  committed
  480f96f
- Serve Harbor's codex agent through the records proxy with Qwen3.8
  
  chowhsu
  and
  claude
  committed
  d62e554
- Stage-1 mini rehearsal on Modal: plan, SGLang expert load report, requant fixes
  
  chowhsu
  and
  claude
  committed
  3e6604a
- s25_g1 driver: print and keep the prep summary
  
  chowhsu
  and
  claude
  committed
  5df375d
- Stage-1 plan: sampler.extra_args; mini plan disables SGLang shared-expert fusion under MXFP4
  
  chowhsu
  and
  claude
  committed
  f0bbfd9
- T-G3 harness: Megatron-LM on PYTHONPATH, model_args_utils path, GPU name passed to the container
  
  chowhsu
  and
  claude
  committed
  a875e23
- s25_g1 driver: a failed sampler cancels the running trainer, and the reverse
  
  chowhsu
  and
  claude
  committed
  e80d9fe
- Report model calls that hit their token cap on the verl Harbor path
  
  chowhsu
  and
  claude
  committed
  e5e8919
- Add a CPU check to the Modal records endpoint and fix its import path
  
  chowhsu
  and
  claude
  committed
  4c05531
- Stage 1: no --rollout-num-gpus on the trainer-only island; check the argv with parse_args
  
  chowhsu
  and
  claude
  committed
  0ff31dc
- Stage 1: task.data_local for offline nodes; s25_g1 driver exports the Modal runner interpreter
  
  chowhsu
  and
  claude
  committed
  aca8dab
- T-G3 harness: symmetric failure handling in N8.2, logits captured before the in-place CE, TE-to-TE name map, host-copy pin report
  
  chowhsu
  and
  claude
  committed
  7314e2b
- Miles island: glm5_next geometry from config.json; stage 1 trainer reads one round behind
  
  chowhsu
  and
  claude
  committed
  6f41656
- s25_g1 driver: logs on container disk, mirrored to the Volume
  
  chowhsu
  and
  claude
  committed
  80da1a1
- Stage 1: pin the island's AlgorithmSpec hash like the launcher
  
  chowhsu
  and
  claude
  committed
  f483de5
- Stage 1: export RAY_ADDRESS for the island so its env reaches the Ray actors
  
  chowhsu
  and
  claude
  committed
  ab3e172
- T-G3 harness: extra continuous and resume samples for the N7.1d spread
  
  chowhsu
  and
  claude
  committed
  e875afd
- Hand host copies to the requantizer from the Miles adapter; record the in-trainer N8.2 result
  
  chowhsu
  and
  claude
  committed
  55cd48d
- glm5_next trains from a Megatron torch_dist checkpoint in raw mode
  
  chowhsu
  and
  claude
  committed
  d44a44b
- Stage 1: the torch_dist converter takes the model arguments only
  
  chowhsu
  and
  claude
  committed
  d1fc221
- Stage 1: run the torch_dist converter with the Megatron checkout on PYTHONPATH
  
  chowhsu
  and
  claude
  committed
  db07dc1
- s25_g1 driver: convert-only entrypoint (2 cards, no sampler)
  
  chowhsu
  and
  claude
  committed
  bd81274
- Merge S25 Modal GPU test fixes (G2, G3, G4)
  
  chowhsu
  and
  claude
  committed
  40b3536
- Stage 1: trainer.miles.extra_env; mini plan puts the Megatron checkout on PYTHONPATH
  
  chowhsu
  and
  claude
  committed
  648f47c
- Trainer-only Miles island publishes without engines
  
  chowhsu
  and
  claude
  committed
  abf0de2
- Resume a replaced spot sampler island and allow several samplers per trainer
  
  chowhsu
  and
  claude
  committed
  0be12c3
- verl: trainer island over the sample buffer (S25 C2)
  
  chowhsu
  and
  claude
  committed
  79130ce
- Wire the yeto round-cut controller into the verl island; digest the stepped state (S25 N7.6b, N7.6c)
  
  chowhsu
  and
  claude
  committed
  39aa81f
- Split the trainer's rounds evenly over several sampler islands
  
  chowhsu
  and
  claude
  committed
  a9fad17
- Let synced trainer islands feed sampler islands through merged releases
  
  chowhsu
  and
  claude
  committed
  b651fe0
- s25_g1 driver: the re-quantizer runs in its own CPU container
  
  chowhsu
  and
  claude
  committed
  af2303b
- Leave at once on the reclaim marker from a watcher thread
  
  chowhsu
  and
  claude
  committed
  bf94fa7
- T-C4 GPU probes: verl SCHEDULED cut + resume chain, Miles stale-master injection phase; record the round-cut wiring
  
  chowhsu
  and
  claude
  committed
  7c21f7f
- Driver accepts a publication without members on an engineless pool
  
  chowhsu
  and
  claude
  committed
  07ec4ca
- Make the in-flight export check a CutSource class attribute; fix the A10G name assert
  
  chowhsu
  and
  claude
  committed
  9e0ec82
- Re-quantizer accepts the text-model prefix of Miles' raw glm5_next export
  
  chowhsu
  and
  claude
  committed
  6361010
- verl cut: load the trainable state non-strictly (frozen LoRA base weights are not in the cut)
  
  chowhsu
  and
  claude
  committed
  58c2e32
- Round cut walks a ChainedOptimizer through its members
  
  chowhsu
  and
  claude
  committed
  615ebfc
- Mini plan: capture SGLang CUDA graphs up to batch 16 only
  
  chowhsu
  and
  claude
  committed
  6f2a056
- Turn the reclaim signal into the marker inside the sampler island
  
  chowhsu
  and
  claude
  committed
  7a25a04
- T-C4 verl probe: fresh container per phase and a Volume reload before it
  
  chowhsu
  and
  claude
  committed
  8e8253e
- Add --rl-sampler-spot: spot sampler islands, on-demand trainer
  
  chowhsu
  and
  claude
  committed
  94b0a60
- Mini plan: the SGLang flag is --cuda-graph-max-bs-decode
  
  chowhsu
  and
  claude
  committed
  8105fc4
- verl island: order-independent prompt digest per round; record the first-step resume check
  
  chowhsu
  and
  claude
  committed
  45006c7
- Cut coverage: dense tensors are covered over the EP ranks of a (tp, pp)
  
  chowhsu
  and
  claude
  committed
  51a6ebf
- s25_g1 driver: every launch starts from an empty run store
  
  chowhsu
  and
  claude
  committed
  cfc2fb4
- Never raise from the sampler's reclaim signal handler
  
  chowhsu
  and
  claude
  committed
  2c482b6
- Merge verl sampler island (C2) into synced-trainers branch (C3)
  
  chowhsu
  and
  claude
  committed
  eb99fc1
- Synced verl trainer islands feed sampler islands; merged releases over object stores
  
  chowhsu
  and
  claude
  committed
  3eb1780
- Register the qwen3_2507 chat template (Qwen3-4B-Instruct-2507) with token alignment tests
  
  chowhsu
  and
  claude
  committed
  e449334
- verl image: add yeto's object-store client (boto3) for s3/oss stores
  
  chowhsu
  and
  claude
  committed
  4de5c78
- Records endpoint: model presets (Qwen3-4B-Instruct-2507 for N10.9)
  
  chowhsu
  and
  claude
  committed
  e4a7740
- Island keeps the default verify_miles_revision call without --miles-commit; allowlist the requant frozen flags
  
  chowhsu
  and
  claude
  committed
  ef6400c
- Merge s25-c1-spot-sampler into s25-integ4
  
  chowhsu
  and
  claude
  committed
  ba86cf0
- Merge s25-c4-cut-ctl into s25-integ4
  
  chowhsu
  and
  claude
  committed
  e6a232e
- verl agentic path: records proxy over verl's rollout client, path-secret recorder, agentic entry
  
  chowhsu
  and
  claude
  committed
  ac4f852
- Agentic outcome: a malformed model reply that ends the agent is a trained 0, not ABORTED
  
  chowhsu
  and
  claude
  committed
  5f7d529
- G1B: dense word-diversity reward plan and per-release tensor digests
  
  chowhsu
  and
  claude
  committed
  e44e4ed
- Miles Harbor path with the records proxy as recorder (N10.9)
  
  chowhsu
  and
  claude
  committed
  12b7cd7
- Add the RL capability evaluation suite (tools/eval)
  
  chowhsu
  and
  claude
  committed
  f158400
- tools/eval: price A10G containers with their CPU and memory requests
  
  chowhsu
  and
  claude
  committed
  c05b6ca
- tools/eval: one detached app per verl run; Modal 1.6 call timeouts
  
  chowhsu
  and
  claude
  committed
  8a9a659
- tools/eval: no image-level PYTHONPATH on the verl image; stop the run's app by id
  
  chowhsu
  and
  claude
  committed
  78d6327
- tools/eval: check the preflight before claiming the run dir
  
  chowhsu
  and
  claude
  committed
  934e505
- tools/eval: ship the verl harness functions by value (tools is not in the images)
  
  chowhsu
  and
  claude
  committed
  8dd269f
- verl agentic entry: register yeto_grpo lazily so the Ray setup hook imports no CUDA code
  
  chowhsu
  and
  claude
  committed
  2d01736
- tools/eval: Miles launches skip the launcher thread preflight (the scripts check first); count each over-age drop once
  
  chowhsu
  and
  claude
  committed
  4895f1a
- verl agentic path: report each sample's weight step range (min/max_global_steps)
  
  chowhsu
  and
  claude
  committed
  25b418a
- Miles records generate: take the router address from Miles' arguments
  
  chowhsu
  and
  claude
  committed
  c8efa15
- codepack: reuse a tree another launch unpacked first instead of failing
  
  chowhsu
  and
  claude
  committed
  8731a0f
- tools/eval: E4 counts the Miles buffer reader's passed-over stale groups from consumed minus trained; over-age check reads rl_group_policies per round
  
  chowhsu
  and
  claude
  committed
  7e031aa
- Resume a replaced spot sampler island and allow several samplers per trainer
  
  chowhsu
  and
  claude
  committed
  8b2a031
- Split the trainer's rounds evenly over several sampler islands
  
  chowhsu
  and
  claude
  committed
  c7812a8
- Leave at once on the reclaim marker from a watcher thread
  
  chowhsu
  and
  claude
  committed
  8ffc381
- Turn the reclaim signal into the marker inside the sampler island
  
  chowhsu
  and
  claude
  committed
  e47429a
- Add --rl-sampler-spot: spot sampler islands, on-demand trainer
  
  chowhsu
  and
  claude
  committed
  944c6ee
- Never raise from the sampler's reclaim signal handler
  
  chowhsu
  and
  claude
  committed
  31af37f
- Agentic outcome: a conversation that outgrows the context is a trained 0; shared records env names
  
  chowhsu
  and
  claude
  committed
  00064e4
- Nebius spot sampler island: host unit forwards the VM stop, member records by object
  
  chowhsu
  and
  claude
  committed
  d9330ea
- Resume a replaced engine sampler island; never overwrite a payload
  
  chowhsu
  and
  claude
  committed
  a4b9962
- Token-diversity test reward; engine sampler rewards see token ids
  
  chowhsu
  and
  claude
  committed
  7346fc2
- G1 driver: wall-clock limits and one log directory per sampler container
  
  chowhsu
  and
  claude
  committed
  e7125c8
- Resume a replaced engine sampler island; never overwrite a payload
  
  chowhsu
  and
  claude
  committed
  bd58b18
- Miles buffer reader tapes passed-over stale groups (rl_buffer_stale_skipped); eval E4 reads it and cross-checks consumed minus trained
  
  chowhsu
  and
  claude
  committed
  0e29001
- Eval E2-verl runs the production engine sampler entry: resume after a reclaim, hold until the container stops; E2 checks continuous trainer versions
  
  chowhsu
  and
  claude
  committed
  58329f9
- tests: the Miles reader stub reports its stale groups
  
  chowhsu
  and
  claude
  committed
  c34612b
- eval E2-verl: save the sampler tape when the island leaves (a reclaimed container may die before the function ends)
  
  chowhsu
  and
  claude
  committed
  434435b
- codepack: unpacked trees get a 2000-01-01 mtime so sky can zip them
  
  chowhsu
  and
  claude
  committed
  3e1c833
- Engine server adapters start the server with SIGINT ignored: a cloud reclaim signals every process, and the island must finish its round and leave PREEMPTED
  
  chowhsu
  and
  claude
  committed
  3bf6524
- nebius_reclaim: host unit install works with a remote sky API server
  
  chowhsu
  and
  claude
  committed
  37c00f6
- codepack: the pack lists its executables; verify restores lost executable bits
  
  chowhsu
  and
  claude
  committed
  89985b8
- launcher: the spot sampler env tolerates an island task built without a spec
  
  chowhsu
  and
  claude
  committed
  58e10d4
- Merge s25-c1b: real spot preemption and remote sky API fixes
  
  chowhsu
  and
  claude
  committed
  d7958c0
- Merge s25-g1c: stage-1 mini rehearsal fixes and G1C
  
  chowhsu
  and
  claude
  committed
  ae34153
- Merge s25-n109: agentic RL chain (Harbor TB2.1 on verl and Miles)
  
  chowhsu
  and
  claude
  committed
  5937baa

Commits

Commits on Oct 20, 2026