Skip to content

rl/syncer: one syncer with async multi-island sync for LoRA and full-parameter RL #292

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
michaellchung merged 263 commits into from
Oct 26, 2026
Merged

rl/syncer: one syncer with async multi-island sync for LoRA and full-parameter RL #292

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
263 commits
Select commit Hold shift + click to select a range
cc8eba8
Relay a rank>0 Modal reclaim to the rank-0 driver; one launch for syn…
chowhsu Oct 20, 2026
3469d0c
Reclaim marker keeps its single-node format; only a relayed marker na…
chowhsu Oct 20, 2026
6b74178
E6: engine sampler tier settings, dense expert count, CPU proofs
chowhsu Oct 20, 2026
a215af7
E6: load Miles process_utils from its file in the env test
chowhsu Oct 20, 2026
d1c91c1
A merging trainer creates the sample-buffer index (verl pair with sev…
chowhsu Oct 20, 2026
18f2c94
verl buffer trainer releases the per-shard read positions
chowhsu Oct 20, 2026
2e125ed
Test: pass the shard trainer buffer by identity (an empty buffer is f…
chowhsu Oct 20, 2026
e3e94c1
verl buffer trainer: the buffer's end ends the run FINISHED
chowhsu Oct 20, 2026
621cac6
Engine sampler island: a finished run ends its tape with rl_learner_f…
chowhsu Oct 20, 2026
1c23a60
Engine sampler tape records carry the launcher island id
chowhsu Oct 20, 2026
7ee7aab
fully_async round cuts: the rollout pool exports no in-flight work
chowhsu Oct 20, 2026
af35378
verl sync island trains on every card of its container; multi-card Mo…
chowhsu Oct 21, 2026
f6bdc9f
Miles sampler: forward the data-slot env to the Ray rollout workers; …
chowhsu Oct 21, 2026
0b64895
verl vLLM readback: one tmp file per replica process
chowhsu Oct 21, 2026
0752eb4
Head syncer waits only for a synced group's trainers, not its sampler…
chowhsu Oct 21, 2026
4166006
verl: refuse expert parallelism and non-FSDP2 strategy with the E7 re…
chowhsu Oct 21, 2026
c711302
Syncer quorum counts a synced group's trainers only
chowhsu Oct 21, 2026
b1739c1
Dry-run plan sizes the role syncers like the real launch (synced group)
chowhsu Oct 21, 2026
782ea0a
Speed up boat sandbox start-up: VM reuse pool, setup retries, phase t…
chowhsu Oct 21, 2026
ab2c39f
Cache the pinned Codex build on boat VMs so Harbor skips the Codex in…
chowhsu Oct 21, 2026
92ebb28
Agentic Harbor path: suspend between rounds, TB2 lists, curriculum, C…
chowhsu Oct 21, 2026
4ce7af1
Agentic Harbor path: relay port ranges, metrics without a verdict, TB…
chowhsu Oct 21, 2026
df855e9
Keep curl and ripgrep in the container when the Codex cache skips Har…
chowhsu Oct 21, 2026
743ff69
Provide node and npm with the boat Codex cache, as Harbor's install d…
chowhsu Oct 21, 2026
4854dda
E6: fixes from the H100 gate runs (fused ignore aliases, token scorin…
chowhsu Oct 21, 2026
ffe86d0
E6: refuse tier 1 on verl, set verl max_model_len, native-FP8 gate pr…
chowhsu Oct 21, 2026
d9a2703
E6: pin the FP8-fixed ports image; tier 1 trainer FP8 source (Miles n…
chowhsu Oct 21, 2026
7f100f4
E6: Miles RL drivers for the tier groups (Modal H100!)
chowhsu Oct 21, 2026
8d1df81
E6: full-parameter pair needs a fixed-partition sampler island
chowhsu Oct 21, 2026
e3da116
E6: Dockerfile and tests follow the new image pins; neutral flag help
chowhsu Oct 21, 2026
2011906
E6: pair islands are 2xH100 each (fixed partition applies to every is…
chowhsu Oct 21, 2026
aa3b97c
E6: tier 2 also runs as one launch per role (--rl-sample-role); pair …
chowhsu Oct 21, 2026
d7c79be
E6: per-round parser of the Miles RL runs
chowhsu Oct 21, 2026
be553cf
Modal store: no reload while a writer holds uncommitted files
chowhsu Oct 21, 2026
beaf4e0
E6: pair trainer keeps the Adam state on the host (OOM on one H100)
chowhsu Oct 21, 2026
1fe2c76
Modal store: the write guard is a container-wide flock
chowhsu Oct 21, 2026
f4cd14e
E6: tier 1 on Miles defaults to the trainer FP8 source
chowhsu Oct 21, 2026
bce94af
RL: full parameters on one no-sync island with dedicated engine cards
chowhsu Oct 21, 2026
8fe48c9
E6: pin the ports image with the SGLang compressed-tensors FP8 reload
chowhsu Oct 21, 2026
f74ac99
E6: neutral fp8-source help; full-parameter dedicated-engine group is…
chowhsu Oct 21, 2026
dfa4a58
E6: full-parameter dedicated-engine group runs without round cuts
chowhsu Oct 21, 2026
5171966
Merge branch 's26-e04-imgassert' into s26-agentic-merge
chowhsu Oct 21, 2026
52a19bc
Merge branch 's26-e8-agentic-eff' into s26-agentic-merge
chowhsu Oct 21, 2026
156b5ec
Merge s26-e01-effeval into E0.3 ablation branch
chowhsu Oct 21, 2026
ca78ca0
Merge s26-e3-sync into E0.3 ablation branch
chowhsu Oct 21, 2026
755375e
Merge s26-e6-quant into E0.3 ablation branch
chowhsu Oct 21, 2026
2e1d8d3
E0.3: bias-stacking ablation drivers (Qwen3-0.6B, Miles, layers L0-L3)
chowhsu Oct 21, 2026
a78e73a
E0.3: head driver plan line without STEPS
chowhsu Oct 21, 2026
30b4748
Merge s26-e95-agentic: turn suspend/resume, gate relay, TB2.1 lists
chowhsu Oct 21, 2026
da087f4
Match Harbor's tool layout on a boat Codex cache hit
chowhsu Oct 21, 2026
a5abd8a
verl agentic entry: --fully-async selects verl's fully_async_main
chowhsu Oct 21, 2026
1c0df83
eval: opt-in trainer step profile (--rl-profile-trainer)
chowhsu Oct 21, 2026
6d3d3f7
Harbor layer start lock: exact local builds, duplicate dist-info
chowhsu Oct 21, 2026
ab2e507
Harbor layer lock: record a twice-installed package without a version
chowhsu Oct 21, 2026
cd25a91
verl agentic entry: run fully_async_main as __main__
chowhsu Oct 21, 2026
84011ee
E6: hold the store writer guard across a round cut; cuts with host Adam
chowhsu Oct 21, 2026
91adbef
E0.3: per-round parser; multi-island layers replace slow containers w…
chowhsu Oct 21, 2026
fe5e612
round cuts: stage a background cut's device copies on the host at the…
chowhsu Oct 21, 2026
3c9ee84
E0.3 v2: 256 samples per round, no checkpoint store so fully_async ke…
chowhsu Oct 21, 2026
3f169ce
verl agentic: log aborted-call resends and one line per trajectory
chowhsu Oct 21, 2026
40cf73a
Revert "round cuts: stage a background cut's device copies on the hos…
chowhsu Oct 21, 2026
92cb83c
E0.3: parser groups v2 run ids by layer
chowhsu Oct 21, 2026
74c2f6f
round cuts: hand a background cut to its writer in one shared buffer
chowhsu Oct 21, 2026
9079c7e
Merge s26-e1-overhead into E0.3 ablation branch
chowhsu Oct 21, 2026
9774103
verl agentic workers: print yeto log records to stderr
chowhsu Oct 21, 2026
b77491c
eval: sample the island driver per phase under --rl-profile-trainer
chowhsu Oct 21, 2026
6961531
verl fully-async: turn multi_turn on for agent-loop runs
chowhsu Oct 21, 2026
6824d4c
boat pool: short TTL for idle VMs, delete pool VMs at process exit
chowhsu Oct 21, 2026
8cd4805
Miles Harbor agent: test-only extra hold before the gate reopens
chowhsu Oct 21, 2026
fedd74b
records endpoint: SGLang topology tp1 matches TP 1
chowhsu Oct 21, 2026
92f33bc
eval: sample the thread that drives the phases, not the enabling one
chowhsu Oct 21, 2026
0dec43d
eval: phase profile labels carry the caller chain
chowhsu Oct 21, 2026
2ec20f7
WIP: verify publications only on cut, first and every-K rounds
chowhsu Oct 21, 2026
b664b9d
round cuts: device-copy capture for a trainer that sleeps after the cut
chowhsu Oct 21, 2026
e79520d
round cuts: default interval 4; verify publications at that interval
chowhsu Oct 21, 2026
1882b42
R3: run Codex through a yeto model provider with a long SSE idle timeout
chowhsu Oct 21, 2026
b0335e4
Miles Harbor agent: per-trajectory outcome and suspension log (YETO_A…
chowhsu Oct 21, 2026
71b2fd6
Miles Harbor path: delete boat pool VMs at RolloutExecutor.dispose()
chowhsu Oct 21, 2026
096bdf6
driver: mark only unverified publications in the publication event
chowhsu Oct 21, 2026
74e6904
Merge branch 's26-e1-overhead' into s26-e2-async
chowhsu Oct 21, 2026
8778950
WIP: S26 E2 sweep driver, --rl-async-buffer-rounds, adaptive TIS
chowhsu Oct 21, 2026
becb3a3
test: keep the phase profiler off after the profile tests
chowhsu Oct 21, 2026
fd6ff26
RL: --rl-async-buffer-rounds and opt-in adaptive TIS (AIS) correction
chowhsu Oct 21, 2026
189df10
E5: time the policy release phases; optional fused LoRA stage
chowhsu Oct 21, 2026
09582a7
probes: S26 E2 driver buffer override and staleness parsing
chowhsu Oct 21, 2026
2ae24bf
Test the agentic stage-3 argv of the Miles island entry
chowhsu Oct 21, 2026
f183d75
E4: trainer driver phase seconds and buffer wait share in the scale r…
chowhsu Oct 21, 2026
e22ed34
probes: S26 E2 reuses the E0.3 task file
chowhsu Oct 21, 2026
117bd99
probes: keep the S26 GSM8K task files under data/ (inputs, not run logs)
chowhsu Oct 21, 2026
0f7f7a8
tests: do not leak the phase sampler from the --rl-profile-trainer sw…
chowhsu Oct 21, 2026
2f7e288
round cuts: commit the store's first cut in the background too
chowhsu Oct 21, 2026
ad67598
Revert "tests: do not leak the phase sampler from the --rl-profile-tr…
chowhsu Oct 21, 2026
e5b5ef0
test: keep the phase profiler off after the profile tests
chowhsu Oct 21, 2026
233f139
test: complete deferred first cuts in the recovery abstraction tests
chowhsu Oct 21, 2026
29b80e9
RL: fully_async buffer depth defaults to the policy-age limit
chowhsu Oct 21, 2026
b799606
E0.3: keep the GSM8K task inputs under data/ (inputs, not run logs)
chowhsu Oct 21, 2026
1a3014f
E0.3: pass the task file as an explicit local path
chowhsu Oct 21, 2026
c38355c
E0.3: stop a single-island run after 15 minutes without a new round
chowhsu Oct 22, 2026
2ab01f8
E6: 3c rerun cuts every 5 rounds; stop both apps after 15 min without…
chowhsu Oct 22, 2026
5d2f139
E0.3: compare two runs on the prompts both trained in the same rounds
chowhsu Oct 22, 2026
ae435f2
E0.3: parser groups v2m run ids by layer
chowhsu Oct 22, 2026
72adc7c
E4 tool: merge launch-log events; a preempted round with no groups is…
chowhsu Oct 22, 2026
dd88b26
E5: a release writer keeps the LATEST it wrote and commits once
chowhsu Oct 22, 2026
dc7f596
probes: S26 E2 driver passes the buffer flag only when a group names one
chowhsu Oct 22, 2026
d4f9590
Move fork and image references from michaellchung to agentenv
chowhsu Oct 22, 2026
90616cb
Keep the vendored MIS file byte-identical; retarget the legacy image …
chowhsu Oct 22, 2026
d5612df
rl: expose the outer-merge settings as RL options and test them on th…
chowhsu Oct 22, 2026
9ddd4f6
syncer: dynamic island membership in the fragment-round server
chowhsu Oct 22, 2026
8a4102d
syncer: stream full-parameter head memory with push folding and COW c…
chowhsu Oct 22, 2026
a69a397
syncer: optional delayed Nesterov outer optimizer
chowhsu Oct 22, 2026
5fe0c82
rl: round cuts for synced islands, timed cut interval, background cuts
chowhsu Oct 22, 2026
a45f08e
rl eval: offline evaluation of round cuts and cut-to-HF conversion
chowhsu Oct 22, 2026
3845696
rl: count --rl-sync-max-base-lag in rounds
chowhsu Oct 22, 2026
4181104
rl: full-parameter sampler engine partition and ordered sampler weigh…
chowhsu Oct 22, 2026
3a209ea
syncer: describe the current wiring of the whole-vector service loop
chowhsu Oct 22, 2026
0093fb2
rl: asynchronous fragment rounds for the full-parameter preset
chowhsu Oct 22, 2026
8e8c47d
syncer: describe the fragment pipeline, grace window and merge modes …
chowhsu Oct 22, 2026
d018330
rl: full-parameter sync layout for MoE islands with expert parallelism
chowhsu Oct 22, 2026
1a5e1f7
syncer, rl: peer recovery, syncer-epoch fence and island re-JOIN on l…
chowhsu Oct 22, 2026
1bc64bf
syncer: streaming weighted fold and a fragment-sharded head
chowhsu Oct 22, 2026
79c4f14
rl: int4 pushes for full-parameter sync with optional island error fe…
chowhsu Oct 22, 2026
bafe4e8
rl: round cuts and test-only round delay for full-parameter islands
chowhsu Oct 22, 2026
a3cc01f
launch: syncer VM type and disk options; RL weighted fold and async c…
chowhsu Oct 22, 2026
a1f0b33
rl, syncer: policy versions, catch-up and quorum start for full-param…
chowhsu Oct 22, 2026
df5a961
rl: sharded syncer head for full-parameter asynchronous rounds
chowhsu Oct 22, 2026
b5e1a35
rl, launch: peer recovery for full-parameter Miles islands
chowhsu Oct 22, 2026
e660bf0
rl, syncer: restart push quantization blocks at tensor boundaries by …
chowhsu Oct 22, 2026
09ff929
test: make two port and timing races deterministic
chowhsu Oct 22, 2026
abfb488
rl: optional per-expert routed-token weights for MoE full-parameter m…
chowhsu Oct 22, 2026
f6757e8
rl: SGLang sampler weight updates during generation
chowhsu Oct 22, 2026
cb1cfde
rl: ports image on Miles ed5f64fd7 with host-Adam round cuts
chowhsu Oct 22, 2026
7744d52
rl: hold a store writer guard across a round cut; Volume reloads wait…
chowhsu Oct 22, 2026
590e09b
rl, syncer: island pools on the fragment-round syncer; membership opt…
chowhsu Oct 22, 2026
a758946
test: fake role streams promise in-order broadcasts like the Miles st…
chowhsu Oct 22, 2026
feb0628
rl: async sync defaults for multi-island RL, strict control mode
chowhsu Oct 22, 2026
a08f178
rl: a restarted full-parameter island joins a running async pool
chowhsu Oct 22, 2026
75eac8c
syncer: finalize as soon as the last round commits
chowhsu Oct 22, 2026
057a44a
modal: accept a sharded head's comma-separated syncer address list in…
chowhsu Oct 22, 2026
2ab9b6f
miles config: full-parameter sync accepts optimizer CPU offload with …
chowhsu Oct 22, 2026
8e0fc69
syncer: carry late pushes into the next round in async mode; pace gra…
chowhsu Oct 22, 2026
fce646b
syncer, rl: heartbeats keep an island's lease from HELLO through startup
chowhsu Oct 22, 2026
5634a33
test: real syncer control for tensor-aligned push blocks
chowhsu Oct 22, 2026
c6d68c1
Ports image: Miles cc693c693 (full-parameter sync with distributed op…
chowhsu Oct 22, 2026
6d57467
Full-parameter layout: split a parameter above the fragment bound int…
chowhsu Oct 22, 2026
391769c
Miles plan: pass flat_ranges only for split fragments; refuse a split…
chowhsu Oct 22, 2026
70c3dcd
Miles cut restore: rebase full-parameter scheduler anchor after the load
chowhsu Oct 22, 2026
fb76c72
Ports image: Miles 0b64b40c1 (flat-range fragment split, scheduler an…
chowhsu Oct 22, 2026
b571033
Full-parameter local init: digest-checked version 0 without the initi…
chowhsu Oct 22, 2026
0c2fc16
verl: readback undoes the served model's vLLM prefix renaming (Qwen3.…
chowhsu Oct 22, 2026
492a563
rl: stop local training only when every fragment's last round is ready
chowhsu Oct 23, 2026
d6dd6bf
rl: a retried round attempt supersedes the older PULL permit
chowhsu Oct 23, 2026
60239ff
Island pool with peer recovery: one option for a replacement island's…
chowhsu Oct 23, 2026
7e3e8d1
syncer: send rejoin catch-ups one fragment per scheduler pass
chowhsu Oct 23, 2026
968aaa3
wan_streams: start an idle stream's stall clock at its enqueue
chowhsu Oct 23, 2026
6157577
rl islands: size the Modal disk and stop duplicating local-init anchors
chowhsu Oct 23, 2026
40c3ee4
Merge s27-pool-peer: island pool with peer recovery
chowhsu Oct 23, 2026
0b61700
Merge s27-island-disk: catch-up pacing, stream stall timer, island di…
chowhsu Oct 23, 2026
ff3044e
Miles cut restore: digest the param_layout like the save does
chowhsu Oct 23, 2026
92dcdb1
rl: commit write-only island cuts inline when the trainer reads the s…
chowhsu Oct 23, 2026
1ae040f
rl: resume a relaunched synced trainer from its buffer lane and the s…
chowhsu Oct 23, 2026
3eb7a96
Nemotron-H parameter rules and the verl Megatron GRPO plan
chowhsu Oct 23, 2026
b1dfaa4
Nemotron-H model FLOPs for MFU and a Modal verl GRPO smoke tool
chowhsu Oct 23, 2026
76c1818
verl Nemotron image: build mamba-ssm CUDA, drop flash_attn.cute, no r…
chowhsu Oct 23, 2026
ac2a7dc
Miles recipe for the Nemotron-H family (hybrid Mamba-2 + MoE)
chowhsu Oct 23, 2026
391f7ed
Accept ungated (squared ReLU) MoE experts in the expert partition check
chowhsu Oct 23, 2026
7f50753
rl: per-fragment full-parameter apply and async pipelines up to 8
chowhsu Oct 23, 2026
abcd7b7
Ports image: Miles b13c170ce (per-fragment full-parameter apply)
chowhsu Oct 23, 2026
8848adb
rl: detect the resident-fragment apply through Miles' forwarding grou…
chowhsu Oct 23, 2026
c4186d5
rl/miles: allow-listed trainer-speed flags for Nemotron-H MFU runs
chowhsu Oct 23, 2026
388bfe2
rl/miles: train extra argv also splits on commas (one launcher word)
chowhsu Oct 23, 2026
681b414
rl: island wait timeouts, async pipelines up to 60, terminal retire
chowhsu Oct 23, 2026
b8f41f6
rl ports: --context-parallel for the Miles trainer (S27 G-AC long seq…
chowhsu Oct 23, 2026
fdd468a
tests: fixed-roster quorum timeout keeps pushes and attempt on the re…
chowhsu Oct 23, 2026
aa04c6a
multinode: the expert group shares TP x CP x DP ranks (CP-aware train…
chowhsu Oct 23, 2026
a62580e
miles config: map parallel.context_parallel (S27 G-AC)
chowhsu Oct 23, 2026
3c1ad34
rl: deferred capture, batched partial-quorum applies, faster pushes
chowhsu Oct 23, 2026
b252153
run_flags: keep the new flag help backend-neutral
chowhsu Oct 23, 2026
2b25c30
Ports image: Miles aa6b887d8 (deferred full-parameter export)
chowhsu Oct 23, 2026
46fde5e
rl: numeric final-answer reward for one-number labels (DAPO-Math rows)
chowhsu Oct 23, 2026
dfdc9b3
Merge s27-reward-fix (46fde5ea) into s27-health
chowhsu Oct 23, 2026
7648c55
Merge s27-gy-mfu (388bfe25) into s27-health
chowhsu Oct 23, 2026
2ccae80
Merge s27-gac-long (a62580ed) into s27-health
chowhsu Oct 23, 2026
ca8a80b
rl run config: word the CP divisibility check without a backend name
chowhsu Oct 23, 2026
73874f5
rl: island lead cap, sync diagnostics and pushes computed ahead
chowhsu Oct 23, 2026
116a8c5
Merge s27-sync-lead (73874f5e) into s27-health
chowhsu Oct 23, 2026
abac778
Count Modal containers per island slot in the container-set guard
chowhsu Oct 23, 2026
face74c
rl: look up the Nemotron-H recipe through the backend registry
chowhsu Oct 23, 2026
869078a
rl: pin Miles d1616ba12 (per-sample seq_idx for THD packs) and image …
chowhsu Oct 23, 2026
bd82486
Merge s26-e04-imgassert: Harbor layer version lock and start check
chowhsu Oct 24, 2026
18e17a0
Merge s26-e7-verl-megatron: verl refuses EP and non-FSDP2 strategies
chowhsu Oct 24, 2026
f4235da
Merge s26-e45: release phase timings, one-commit release writer, scal…
chowhsu Oct 24, 2026
cead61d
Merge s26-e1-overhead: already in S27 (5fe0c82b); keep cut interval d…
chowhsu Oct 24, 2026
f782572
Merge s26-e6-quant: rollout quantization tiers on the S27 pins
chowhsu Oct 24, 2026
aa1be9a
Merge s26-e03-ablation: bias-stacking ablation drivers and task inputs
chowhsu Oct 24, 2026
2c17cb1
Merge s26-e2-async: --rl-async-buffer-rounds, adaptive TIS option, E2…
chowhsu Oct 24, 2026
8166500
Merge s26-e9-robust: reclaim relay, data slots, buffer-end FINISHED, …
chowhsu Oct 24, 2026
3db7cae
Merge s26-agentic-merge: Codex harness removal, Harbor agentic path, …
chowhsu Oct 24, 2026
467d1d9
Merge s26-agentenv-migrate: keep the S27 fork and image references fo…
chowhsu Oct 24, 2026
4c702a5
rl: several sampler islands draw disjoint prompt blocks
chowhsu Oct 24, 2026
e90be7b
E5: time the policy release phases; optional fused LoRA stage
chowhsu Oct 21, 2026
da6e900
E4: trainer driver phase seconds and buffer wait share in the scale r…
chowhsu Oct 21, 2026
2c94f60
E4 tool: merge launch-log events; a preempted round with no groups is…
chowhsu Oct 22, 2026
2754919
E5: a release writer keeps the LATEST it wrote and commits once
chowhsu Oct 22, 2026
21a48e3
push codec: cache-blocked int4 encoder and carry-only part regrouping
chowhsu Oct 24, 2026
c5e6746
full-parameter sync: push in the background while the island trains
chowhsu Oct 24, 2026
922f388
rl: pin Miles 8edea35a7 (zero-copy full-parameter chunks) and image 8…
chowhsu Oct 24, 2026
cbc2a6e
rl: a sampler round reaches the store with one commit
chowhsu Oct 24, 2026
475cd61
rl: one owner-affine fit for islands and planning; accept latent MoE …
chowhsu Oct 24, 2026
12d474a
rl: read Nemotron-H VL configs as their language model
chowhsu Oct 24, 2026
9071639
rl: offline full-parameter layout of a Miles Nemotron-H island
chowhsu Oct 24, 2026
ba720ee
rl: build the 120B Nemotron-H providers with Miles' bridges
chowhsu Oct 24, 2026
87ddb26
rl: allow multi-node full-parameter islands with dedicated engines
chowhsu Oct 24, 2026
3b64f08
Merge s27-sync-overlap (background push, zero-copy chunks, int4 codec…
chowhsu Oct 24, 2026
6e571b3
rl: read the Nemotron-H checkpoint config once when registering bridges
chowhsu Oct 24, 2026
67f0907
tests: the 120B island argv carries the per-layer MoE mask
chowhsu Oct 24, 2026
d7cfbbc
rl: pin Miles b7868074c (Nemotron-H VL language-only bridge) and imag…
chowhsu Oct 24, 2026
ec94ddf
rl: buffer payloads keep integer arrays narrow; the trainer reads pay…
chowhsu Oct 24, 2026
c37fb1c
rl: a trainer island stages full-weight releases and uploads them in …
chowhsu Oct 24, 2026
b4b3e2c
launch: a relaunch into a stopped Modal app abandons the island at once
chowhsu Oct 24, 2026
73893e9
rl: the default round-cut interval is timed, with cuts capped at 5% o…
chowhsu Oct 24, 2026
e000f7a
rl: a staged release records the trainer's buffer position after its …
chowhsu Oct 24, 2026
a276ed7
rl: YETO_RL_RELEASE_EXPORT=miles stages releases with Miles' own export
chowhsu Oct 24, 2026
4dc9aca
modal: an island container moves its ephemeral ports off Miles' port …
chowhsu Oct 24, 2026
b95e1f9
rl: write Megatron's per-layer --moe-layer-freq mask without spaces
chowhsu Oct 24, 2026
2fb0194
rl: split --data rows across training islands; relaunched full-parame…
chowhsu Oct 24, 2026
29cfbef
rl: trajectory reward records carry the sample's data row
chowhsu Oct 24, 2026
adead9b
tools: check a run's data split from its island tapes
chowhsu Oct 24, 2026
b26643d
rl: data positions are one file per record; a refused write never sto…
chowhsu Oct 24, 2026
7c3dc3f
rl: the external-learner data split warning names the remedy that app…
chowhsu Oct 24, 2026
f9318d1
tests: compare the strict-mode plans with the data split off
chowhsu Oct 24, 2026
26a4454
rl: full-parameter sync at DP>1 (data-parallel replicas)
chowhsu Oct 24, 2026
e62497d
rl: Nemotron-H training defaults from the S27 MFU measurements
chowhsu Oct 24, 2026
d93c97f
rl: a full-parameter sampler island skips the DP>1 sync check
chowhsu Oct 24, 2026
0c96eb4
Merge s27-task-assign: automatic per-island row split and data positions
chowhsu Oct 24, 2026
c35c926
Merge s27-dp-sync: full-parameter sync with DP > 1 and Nemotron-H tra…
chowhsu Oct 24, 2026
8874370
Merge s27-120b-prep and s27-120b-gaw: Nemotron-H 120B Super VL island…
chowhsu Oct 24, 2026
76f067d
rl: pin Miles ebc6519af (S27 closing merge) and image ebc6519-2fa8801
chowhsu Oct 24, 2026
b3d1746
Revert "rl: several sampler islands draw disjoint prompt blocks"
chowhsu Oct 24, 2026
4b55bab
Merge s27-disagg-fast (with 4c702a55 reverted): faster trainer + samp…
chowhsu Oct 24, 2026
b0ed66c
tests: DP > 1 refusals pin a pre-replica Miles
chowhsu Oct 24, 2026
3b0bafc
rl: offline Nemotron-H layout with data-parallel replica owners
chowhsu Oct 24, 2026
edabf4f
syncer, launcher: manifest-only final cut for full-parameter asynchro…
chowhsu Oct 24, 2026
8cab8eb
full-parameter sync: per-fragment boundary, staggered first pushes, A…
chowhsu Oct 24, 2026
0c4750a
launcher: ACK-only final cut by default only in --rl-sync-mode async
chowhsu Oct 24, 2026
6504d53
Merge s27-sync-boundary: per-fragment boundary, staggered first pushe…
chowhsu Oct 24, 2026
63afafa
rl: tape every trained data row of a round (rl_round_data_rows)
chowhsu Oct 24, 2026
72e9eef
rl: pin Miles 59aea07f9 (per-fragment round boundary) and image 59aea…
chowhsu Oct 24, 2026
fadbffc
rl: --rl-export-hf-rounds writes the island's model as HF directories
chowhsu Oct 24, 2026
33cce36
launcher: print the --rl-export-hf-rounds warning to stderr
chowhsu Oct 24, 2026
718b2f1
hf_export: first export to a Modal Volume no longer fails; keep the m…
chowhsu Oct 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 1 addition & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,7 @@ Tested — a completed Yeto fine-tuning run on real hardware:
| `nemotron3-nano` | `nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16` | 61 |
| `nemotron3-super` | `nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16` | 240 |
| `nemotron3-ultra` | `nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16` | 1100 |
| `nemotron35-super-vl` | `nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16` | 249 |
| `glm45-air` | `zai-org/GLM-4.5-Air` | 212 |
| `glm46` | `zai-org/GLM-4.6` | 714 |
| `llama4-scout` | `meta-llama/Llama-4-Scout-17B-16E-Instruct` | 218 |
Expand Down
Loading
Loading