Skip to content

docs(golden-al): document SGLang and TRT-LLM synthetic acceptance knobs#2330

Merged
functionstackx merged 1 commit into
mainfrom
docs/golden-al-sglang-trtllm-env-vars
Jul 25, 2026
Merged

docs(golden-al): document SGLang and TRT-LLM synthetic acceptance knobs#2330
functionstackx merged 1 commit into
mainfrom
docs/golden-al-sglang-trtllm-env-vars

Conversation

@functionstackx

Copy link
Copy Markdown
Collaborator

Summary

The golden AL fairness guidelines only showed the vLLM way of injecting the committed golden acceptance length (synthetic_acceptance_length). This PR documents the equivalents for the other two engines, as requested in #2309 (review):

  • golden_al_distribution/README.md (+ README_zh.md kept in parity):
    • SGLang: SGLANG_SIMULATE_ACC_LEN: '<AL>' with SGLANG_SIMULATE_ACC_METHOD: match-expected and SGLANG_SIMULATE_ACC_TOKEN_MODE: real-draft-token, set in the server environment (aggregated_environment / decode_environment in srt-slurm YAMLs, or exported in benchmark scripts) — matching existing DSv4 agentic recipes.
    • TensorRT-LLM: TLLM_SPEC_DECODE_FORCE_NUM_ACCEPTED_TOKENS, with the off-by-one called out: it counts accepted draft tokens only (excludes the bonus/verification token), so it must be set to golden AL minus 1 — e.g. AL 3.5 → 2.5. Fractional semantics (integer part always accepted, fractional part = probability of one extra draft token) verified against the linked TRT-LLM source.
  • .github/codeowner-signoff-verify-prompt.md (Check 10):
    • (a) adds a TRT-LLM bullet with the AL−1 rule alongside the existing SGLang/vLLM checks;
    • (b) instructs the verifier to compare the pinned TRT-LLM value plus 1 against the golden curve, so correct configs don't false-fail.

Test plan

  • Docs-only change — rendered markdown checked locally
  • Env var names/values cross-checked against existing agentic recipes (benchmarks/multi_node/srt-slurm-recipes/sglang/deepseek-v4/agentic/*.yaml) and TRT-LLM source

🤖 Generated with Claude Code

The golden AL README only showed vLLM's synthetic_acceptance_length.
Add the SGLang env vars (SGLANG_SIMULATE_ACC_LEN / _METHOD / _TOKEN_MODE)
and TRT-LLM's TLLM_SPEC_DECODE_FORCE_NUM_ACCEPTED_TOKENS, including the
off-by-one: the TRT-LLM value counts accepted draft tokens only and
excludes the bonus token, so it must be set to golden AL minus 1
(fractional allowed, e.g. AL 3.5 -> 2.5).

Also teach the codeowner sign-off verify prompt (Check 10) to accept
TRT-LLM configs and to compare the pinned TRT-LLM value plus 1 against
the golden curve. Keep README_zh.md in parity.

Requested in
#2309 (review)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@functionstackx
functionstackx requested a review from a team July 25, 2026 04:05
@functionstackx
functionstackx merged commit ec7168f into main Jul 25, 2026
4 checks passed
@functionstackx
functionstackx deleted the docs/golden-al-sglang-trtllm-env-vars branch July 25, 2026 04:07

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — docs-only change, no code paths affected.

Reviewed the new SGLang and TRT-LLM synthetic-acceptance documentation in golden_al_distribution/README.md/README_zh.md and the corresponding Check 10 updates in the codeowner-signoff verify prompt. The TRT-LLM AL-1 rule is applied consistently in both places (doc example and the verifier's compare-plus-1 instruction), and the Chinese translation stays in parity with the English source.

Extended reasoning...

Overview

Docs-only PR adding SGLang and TensorRT-LLM equivalents to the vLLM synthetic-acceptance-length documentation in golden_al_distribution/README.md (+ Chinese parity file), plus a matching update to Check 10 of .github/codeowner-signoff-verify-prompt.md (an LLM-prompt template consumed by a verification workflow, not executable code).

Security risks

None. No code, config, CI logic, or permissions are touched — only markdown prose and an LLM prompt template used for downstream sign-off audits.

Level of scrutiny

Low. This is exactly the kind of small, self-contained, mechanical documentation change that does not need human review: it adds a bullet describing an env var and its off-by-one semantics, cross-checked against the linked TRT-LLM source, and keeps the two README files in parity. The bug-hunting system found no issues, and the one candidate (TRT-LLM AL-1 rule disabling forcing at golden AL=1.0) was already investigated and refuted.

Other factors

The doc claim (AL minus 1) and the verify-prompt claim (pinned value plus 1 should equal golden AL) are algebraically consistent with each other, and the fractional-acceptance semantics described match the linked TRT-LLM source referenced in the PR description. No outstanding review comments in the timeline to address.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

1 participant