Repository navigation
S26 E2: async buffer depth defaults to the policy-age limit; opt-in adaptive TIS - #285
Closed
michaellchung wants to merge 100 commits into
Closed
michaellchung wants to merge 100 commits into
michaellchung wants to merge 100 commits into
Conversation
Eight arms per model preset ({bare,yeto} x {verl,miles} x {sync,async})
generated from one shared spec with a control fingerprint. Bare arms run
the frameworks' own entry points (verl main_ppo / fully_async_main,
Miles train.py / train_async.py) in the same images yeto uses; yeto arms
run verl island_entry and the yeto launcher. Parsers normalize verl
FileLogger, Miles perf lines and yeto tapes into per-round records
(missing fields are marked, never estimated); the report gives
yeto-vs-bare deltas with min-max range verdicts. Presets: Qwen3-0.6B
(E0.1) and Qwen3-4B-Instruct-2507 (E0.2). CLI: eff-plan, eff-run,
eff-parse, eff-report.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Allow --pipeline 1..P with --policy-sweep-fragments so fragment k+1 uploads while fragment k merges, checkpoints and broadcasts; commits stay in strict step order. Add --async-checkpoint (policy sweep only): the committed state is snapshotted and written on a blocking task, and the round is broadcast only after the checkpoint is durable. Checkpoint bytes, versions and the semantic profile hash are unchanged; the tape gains checkpoint_ms. Launcher flags --rl-sync-fragment-pipeline and --rl-sync-async-checkpoint (default off). Adds bit-equality, failure and real-syncer tests and a CPU micro-benchmark (tools/bench_sync_boundary.py). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ric recipe; TE flash fails on sm86) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add session dtype 4 (f32 session, bf16 PUSH deltas): INIT/BCAST/FINAL, masters and momentum stay f32; the client rounds f32 delta bytes to bf16 (round-to-nearest-even, streamed across part boundaries). The dtype is bound into the role-stream session contract. Launcher --rl-sync-push-dtype, island --sweep-push-dtype (default f32). Allow --checkpoint-every N (1 or a multiple of the fragment count) for policy sweeps: only checkpointed steps are durable before their broadcast, on both the synchronous and async paths; the interval is part of the semantic profile hash. Launcher --rl-sync-checkpoint-every (default 1). E3 launcher options are refused outside the full-parameter preset. Adds bf16 error-bound, interval/resume and real-syncer tests, and a third micro-benchmark profile. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…spec Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add session dtype 5 (f32 session, block-scaled int8 PUSH deltas): u32 block size, then per block an f32 absmax/127 scale and one i8 per value; error is at most absmax/254 per element. Launcher --rl-sync-push-int8-block (default 128) is bound into the role-stream contract. Add session flag 0x80 (bf16 downlink deltas) for f32-bulk sessions. Syncer and learners hold a bit-identical f32 anchor per fragment; BCAST is FULL f32 or bf16(theta - anchor), with a FULL refresh every N versions (--bcast-delta-full-every), FULL re-sends on HELLO/reconnect, and a SHA-256 check of the anchor after every update. A mismatch drops the connection and the reconnect restores the anchor. SyncerClient decodes before adapters see the payload: the Miles streaming sink is wrapped, the strict bridge (verl and Miles LoRA) gets f32 bytes. Masters, checkpoints and FINAL stay exact. Add --rl-sync-overlap for the full-parameter sweep: an island pushes round k, trains round k+1 at once, and at the next boundary rebases in place, theta <- G_k + (local - base), with G_k as the anchor so no delta is counted twice. The launcher raises --rl-max-policy-age to at least 1 before its checks; the island refuses overlap below that age. Adds Rust and Python codec/anchor tests, real-syncer int8/downlink/resync tests, a strict-bridge downlink test, and coordinator overlap tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add a backend-neutral rollout quantization setting (--rl-rollout-quant-tier 0..3, one flag definition for launcher and island): - tier 0: BF16 (default, no change); - tier 1: the sampler engine quantizes in-scope linears to FP8 itself and keeps an FP8 KV cache. Miles/SGLang: --sglang-quantization fp8 and --sglang-kv-cache-dtype fp8_e4m3, with the ignore list passed through SGLANG_FP8_IGNORED_LAYERS in the Ray job runtime_env. verl/vLLM: verl's own FP8 rollout config with our explicit ignored_layers through engine_kwargs.vllm.hf_overrides; - tier 2: the trainer side converts each BF16 release to FP8 W8A8 per channel (compressed-tensors float-quantized, the FP8_DYNAMIC preset) and sampler islands load it. Reuses the N8 requant pipeline with a new scheme whose target layout is derived from the BF16 base and the scope rule; - tier 3: 4-bit, interface only (refused at launch). Scope rule (yeto/rl/quant/scope.py): a dense model quantizes only its FFN linears, a MoE model only its routed experts; router, shared experts, lm_head, embeddings, norms and attention stay BF16. The launcher refuses tier 3, cards without FP8 and tier 2 without sampler islands or with LoRA; island entries refuse a tier 1 plan that would quantize an out-of-scope module; sampler loaders refuse an FP8 release outside the scope before touching the engine. Add yeto.rl.quant.gate: logprob difference (mean, p50/p90/p99), TIS truncation share and tokens per second of a quantized engine against BF16 on one prompt set, with configurable criteria and the report written to disk. CPU only. FP8 values equal compressed-tensors 0.18 bit for bit; the ignore lists checked against SGLang 2fa88018's own skip functions. Not run on a GPU. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Read the cards of --gpu with yeto.gpu_spec and accept tiers 1 and 2 only on cards with FP8 tensor cores (L4, L40S, RTX-6000-Ada, RTX-PRO-6000, H100, H200, B200); every other card is refused before launch. - Add the yeto_rollout_quant_tier runtime attribute only for a quantized tier, so tier-0 Miles launches keep their golden argv and runtime attributes. - Test the Miles translation and the island-task forwarding of the tier. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…LoRA and /dev/shm NCCL host buffers The verl image applies yeto's patch_verl, whose sender hook imports yeto, so the bare arms now build the image without it. The fork's disaggregated checkpoint engine drops the PEFT config of unmerged LoRA params; the bare async arm sends merged weights instead. On Modal the trainer allocated NCCL host buffers with cuMem while the receiver fell back to /dev/shm and failed to attach (NCCL_DEBUG=INFO run); NCCL_CUMEM_HOST_ENABLE=0 keeps both on /dev/shm. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sweep Pre-GPU check of --rl-sync-overlap on the Miles role stream found three problems: - The launcher raised --rl-max-policy-age to 1, but the dense sweep island refuses any age other than 0, so an overlap run could never start; and an age >= 1 also turns on Miles' fully_async rollout (partial rollout when colocated). Overlap is not rollout staleness: the driver publishes the boundary's cut before the next generation, so rollouts stay on-policy and only the local weights lag the global policy by one outer step. Keep the age at 0 (refuse others at launch) and bind the lag into the session contract only. - After a rebase the applied cut has local step generation 0, and Miles pushes only cuts with progress, so the second sweep's push failed. The submission is now the rebased cut relabeled with the rounds trained since the round base (same values, hashes and refs). - The boundary submitted once; a PULL that arrived later waited a whole local round. The overlap boundary now submits the whole sweep before training on, and the launcher requires --rl-sync-fragment-pipeline equal to --fragments so the syncer sends all PULLs at once. Adds Miles role-stream tests (real adapter and role stream, CPU tensors, fake trainer/Ray/syncer): exact-once rounds through rebase_fragment, and syncer restarts while the overlapped round trains (before commit, after commit with a lost broadcast, partly committed sweep). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Engine sampler islands: the launch layer writes rollout_quant {tier, apply}
into the sampler spec (stage1.sampler_spec); the entry applies it on the
node (sampler_engines.quant_tier). Tier 1 on sglang-server fills
--quantization fp8, --kv-cache-dtype fp8_e4m3 and SGLANG_FP8_IGNORED_LAYERS
from the checked online plan; vllm-server is refused (stand-alone vLLM online
FP8 has no ignore list). Tier 2 converts the base to the FP8 scheme and sets
quant_scheme, expected_format, release_dir and the routed-expert count. A spec
that already quantizes is refused. Stage 1 refuses a tier on top of its own
quant section.
- Post-load expert_count: an expected count of 0 (dense model) passes without
an engine report; the release index count is checked before the load.
- requant: drop a tied lm_head.weight that an exporter writes although the
base ties it to the embeddings (tie_word_embeddings).
- Tests: the ignore list reaches the process started by Miles'
launch_bound_subprocess; Miles' convert_qwen2_to_hf names equal the
Qwen3-4B-Instruct-2507 index names (fixture from the real index).
- Probe (Modal CPU, verl image): Hydra overrides -> RolloutConfig -> verl's
CLI builder -> vLLM 0.29 argument parser -> Fp8Config. The received
ignored_layers equal the sent list; qkv_proj and o_proj skipped,
gate_up_proj and down_proj quantized; kv_cache_dtype fp8.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Importing it as a package put the real miles package into sys.modules and broke a later test that fakes miles (test_rl_reward_success). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…over rounds that report the value The fork logs a step's training metrics one parameter version late and never logs the last step's, so rows are keyed by training/global_step and the step without metrics keeps its trained batch from the config. Throughput rates use the steady rounds that report the value; the report lists partial counts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… does not) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y Ray worker On Modal one weight-sync process could allocate NCCL host buffers with cuMem while its peer fell back to /dev/shm and then failed to attach (eff01-yeto-verl-async-e; NCCL_DEBUG=INFO in eff01-bare-verl-async-d). NCCL_CUMEM_HOST_ENABLE=0 unless the caller sets it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The overlap rebase uses the bf16-tracked anchor and pushes theta - anchor against that same anchor, so the syncer's f32 masters still receive every round's progress exactly once and no DownlinkResync happens. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ains without dropout --rl-gradient-accumulation-fusion lets a run keep Megatron's fused wgrad (the Miles recipes' default); off keeps the historical argv. The generic recipe passed no dropout, so Megatron's 0.1 applied during RL training while the Miles recipes pass 0.0; it now passes 0.0 as the Qwen3.5 recipe does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No deterministic inference, constant LR, full recompute, dynamic batching, gradient-accumulation fusion, and an algorithm spec with the recipe's clip range (0.2 / 0.28); S26 GOAL §6 makes the bare framework's settings the reference. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… store) miles_local.sh pulls a copy of the repo whose test fixtures are tapes of other runs; their rl_local_round rows replaced the run's rewards (all four yeto+Miles runs showed the same fixture curve with reward 0 in round 0). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… missing The trainer's timing_s/gen and yeto's driver generate phase only pull finished samples from the rollouter queue. Bare verl logs the rollouter's active time per parameter version; yeto logs none per round, so its value stays missing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…from the rollouter E0.1 found yeto+Miles 77 % (sync) and 252 % (async) slower than bare Miles. The tape shows the gap is the per-round round cut: the arm sets a checkpoint store, so a cut runs after every publish (5.7-12.9 s: trainer wake, export, sleep, Modal volume commit), and in fully_async mode the cut also stops Miles' background worker, so generation never runs ahead of training. The bare arms keep no checkpoint; cut only at the end (--rl-cut-every rounds). yeto+verl async generation time was missing: the tape's rl_publication_check.rollouter_timing carries the rollouter's active time per policy version, the same metric the bare arm reports. Read it (version v closes round v - 1). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… path VerlPolicyState.export_digest returns a trainer-resident handle: the workers validate the LoRA like canonical_state, compute the policy digests and the bf16 send checksum, and only those cross Ray. Both verl publishers take the send checksum from the handle; other states are hashed on the driver as before. materialize falls back to the full export; YETO_RL_PUBLISH_FASTPATH=0 keeps the old path. Tests pin hash and checksum equality with the full export. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… background E0.1 measured a periodic round cut at 5.7-12.9 s per round on the critical path of a colocated Miles island: wake the offloaded trainer (0.7 s), export (1.8 s), sleep it again (1.0 s), then a synchronous store commit (1.5-8 s). - prepare(): when a cut is due at the next safe point, the driver writes the trainer part right before it offloads a colocated trainer for the publication (phase "checkpoint"), for the state it is about to publish. The safe point keeps it only if its own context names the same cut (id, progress incl. policy hash, data cursor, ledger, outer token, in-flight); otherwise the unpointed files are dropped and the cut is taken as before. - Background commit: a periodic cut writes the snapshot and LATEST, then a thread commits and prunes (ResumeController.commit_checkpoint_store). The next cut joins it first and reports rl_cut_committed. Pruning only follows a successful commit, so a failed or lost commit leaves the previous cut as the committed one. The final cut, and a run with a sample buffer that commits the same store, stay inline. YETO_RL_CUT_BACKGROUND_COMMIT=0 turns it off. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A round cut on a Miles fully_async island stopped the background worker and requeued every untrained prompt into the live data source (P0-S decision 3). With a cut every round the worker restarted each round, so generation never ran ahead of training (E0.1: 8 worker starts in 8 rounds, policy age 0, 2.15 s of serial generation per round). snapshot_untrained() reads the data source buffer, every drawn untrained group (reset copies) and the data cursor in one step on the worker's event loop, where the worker draws; the cut carries them and the cursor read with them, and the live run is untouched. A resume still regenerates exactly the drawn, untrained prompts (decision 3 unchanged); the resume manifest's data watermark is that cursor. requeue_untrained() stays for its other callers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
e0f833d made the generic Miles recipe pass --attention-dropout 0.0 and --hidden-dropout 0.0 (the bare Miles recipe's values); the seven decoupling golden samples were not regenerated and failed on miles_argv. The only difference is those four argv items (and the argv hash); no algorithm or contract hash changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
E0.1: the yeto+Miles sync generate span was 0.46 s longer than Miles' perf/rollout_time. 0.44 s of it is --offload-rollout (release_memory_occupation), which Miles times outside rollout_time but MilesRolloutPool.generate ran inside the driver's generate phase. The driver now asks the pool to defer it and runs it as phase "rollout_offload"; callers that use the pool directly keep the old in-generate offload. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
final (default) cuts once at the end like the bare arms keep no checkpoint; every-round is the product default. Kept out of the control fingerprint so the bare runs stay comparable. The yeto+verl arm has no checkpoint store in either mode. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With bf16 downlink deltas, the last BCAST of a fragment can carry the same
version as its FINAL. Islands store one copy per (fragment, version) and
refuse a second copy with other bytes, so the delta-tracked BCAST and the
exact FINAL collided ('authoritative fragment replay changed bytes'), the
islands redialed in a loop and finalization timed out (s26-e3-p06-D8).
Broadcasts of a fragment's last version now always go out FULL.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GPU canary e1a-yeto-miles-async-1 (every-round cuts, after 74016eb and 3b00fe5): overlap is back (one fully_async worker start), but each cut still held the round for 2.1 s writing and hashing the 283 MB shard, so a round took 5.3 s against 4.15 s bare (78 %). MilesTrainerGroup.start_cut captures every rank's state at the step boundary (CPU copies) and the ranks write, fsync and hash their shards in a thread (cut_plugin.save_cut_shard background=True); finish_cut collects the summaries and commits the manifest. A periodic cut now only captures at its safe point; the shard write, manifest, pointer, store snapshot and commit complete before the next cut (join_commit, rl_cut_committed). Until then the previous cut stays the pointed and committed one; a failed completion drops the unpointed files. The store's first cut, the final cut, runs with a sample buffer or a critic, and YETO_RL_CUT_BACKGROUND_COMMIT=0 stay inline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Observation only, for E1: around each train_one_step the trainer process prints the step's wall time, main-thread and process CPU time, garbage collector pauses, CUDA caching-allocator counters and stack samples of every Python thread. The launcher forwards the flag to the island, the island sets it on the Miles args, and eff-run --profile-trainer passes it for the yeto+Miles arms (not in the control fingerprint). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ith the link probe Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… step The writer thread copied the device snapshot to the host tensor by tensor while the next round trained: about 400 synchronous device-to-host copies interleaved with the training kernels, the suspected cause of the slower training (profile run e1j-yeto-miles-async-a: the step grew while GC, allocator and main-thread CPU stayed as in the final-only run). The capture now copies the device snapshot into one reused pinned host buffer with a single synchronization, drops the device copies, and the writer thread copies the staged views into shared memory (the buffer is reused only after that). Since nothing reads device memory after the capture returns, a trainer that sleeps right after the cut (--offload-train) takes the device path too. Tests compare the shard bytes and summaries of staged, unstaged and inline cuts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eps its background groups Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t at the step" This reverts commit fe5e612. GPU runs e1k-yeto-miles-async-2 and e1k-yeto-miles-sync-1: the trained tokens were not faster (0.553 ms per token, as before) and the step-boundary capture got slower (async 0.033 -> 0.08 s, sync host path 0.16-0.28 -> 0.30 s). The interleaved device-to-host copies were not the cause of the slower training. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Moving each tensor to shared memory passed one file descriptor per tensor (about 400 per cut) through the trainer process's resource sharer, and its queue feeder and server threads ran while the next round trained (profile run e1j-yeto-miles-async-a: about 15 % of the step's stack samples; none in the final-only run). The writer thread now copies every contiguous tensor that owns its whole storage into one shared uint8 buffer and sends offsets; the writer process rebuilds each tensor as its own copy. Other tensors are shared one by one as before. The shard bytes and summaries are unchanged (tests compare them with the inline cut). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Observation only, for E1: a daemon thread samples the driver thread's stack every 5 ms and, at each driver phase change, prints the finished phase's wall time and its most frequent yeto/miles/ray frames. Without the flag the phase hook does nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GPU run e1n-yeto-miles-async-p: the island enables the sampler from the thread that starts asyncio, but the driver runs its phases on another thread, so every sample showed the event loop. The sampler now binds to the thread of the first phase report. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A sample is labelled by its innermost four yeto/miles/ray frames, so a wait inside Ray shows which driver call waits (GPU run e1o-yeto-miles-sync-p only showed the Ray frame). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Untested work in progress (E1 phase 8): verify_publication, the unverified identity of a skipped round, --rl-verify-every-round. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Under --offload-train the cut capture took host copies tensor by tensor (0.16-0.28 s at the step boundary, GPU runs e1i/e1l sync). It now copies the state device-to-device and stages the copies into one reused pinned host buffer with a single synchronization before the capture returns, so nothing reads device memory after the trainer sleeps. The writer thread copies the staged views into the shared hand-over buffer and frees the staging buffer. Shard bytes and summaries equal the inline cut's (tests). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The product default round-cut interval is now 4 (--rl-cut-every 1 cuts every round). A publication hashes the policy and reads the engine weight checksums back only on the first publication, on every round that is a multiple of the cut interval (every round that keeps a cut) and, with --rl-verify-every-round, on every round. Other publications skip both but still check, by weights version, that the trainer holds the policy; their identity is a 64-hex value derived from the trainer weights version (never a content hash), and the publication event says sync/publication_verified false. A cut refuses an unverified publication, so resume always sees a content hash. Help texts and the rl-resume-from-checkpoint spec and design say so; the efficiency eval gains --checkpoint default (no --rl-cut-every). Tests: which rounds verify, the switch, the skipped read-back, the weights-version check on an unverified round, and the cut refusal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Verified publications keep the event as it was, so recorded golden tapes stay byte-identical (test_decoupling_golden, test_rl_driver_profiles). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
--rl-async-buffer-rounds K sizes the Miles fully_async finished-group buffer (K rounds of groups) so generation can run up to K rounds ahead; refused without overlap. yeto.rl.algos.adaptive_tis implements AIS (arXiv:2605.13907) as a custom TIS function under the named mechanism adaptive_tis; it is not declared on any engine until GPU evidence exists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…itch test The switch test started the process-wide sampler and left it running, so a later CLI test read 'yeto phase profile' lines in its JSON output. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ainer switch test" This reverts commit 0f7f7a8.
A test enabled the process-wide driver phase sampler and left it on; it then printed on every later driver phase and broke the JSON output that test_rl_reconfig_e1::test_command_inbox_and_cli parses (failed in the full suite only). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Miles' own default buffer (2 rounds of groups) at --rl-max-policy-age 1 dropped about half of the generated groups as too old (S26 E2 sweep). Without --rl-async-buffer-rounds the overlapped Miles island now sets the buffer to A_hard rounds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Closing: all commits of |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
S26 E2: async defaults and tooling.
--rl-async-buffer-rounds K(maps to Miles--async-data-buffer-capacity-factor; rejected without overlap or K<1).yeto/rl/algos/adaptive_tis.py); not declared by any engine.tools/probes/s26_e2/; GSM8K task files underdata/gsm8k/(inputs, not run logs).Defaults
Branch dependencies
This is a stacked branch. s25-final..HEAD contains the E0.3 ablation, E0.1 efficiency eval, E1 overhead, E6 quantized-rollout work and E3 sync work up to 2509fde. It depends on the s26-e03-ablation PR and on s26-e1-overhead and s26-e6-quant, which are not pushed separately. Merge or review those first; the E2-only commits are fd6ff26, 8778950, 09582a7, e22ed34, 117bd99, e5b5ef0, 29b80e9 (plus test-leak fixes).
Test evidence
Remote full suite (yeto-cpu-test): 6896 passed, 0 failed at 29b80e9. 7 new unit tests for adaptive TIS; tests for the buffer flag in tests/test_rl_policy_versions.py.
Note: the 5 GPU runs ran before the final 0-failure run; later fixes touched only tests and data file paths.
GPU evidence
Qwen3-0.6B, full-parameter, Miles, 1 train + 1 engine H100, GSM8K, 256 samples per round, 10 rounds, token-level TIS, seeds 17/29.
Not verified
🤖 Generated with Claude Code