fix(verl): classify trajectory prefix mismatch diagnostics

[Daniel Peng (original4422)]

Summary

Related to #645. Classify the existing, capped mismatch diagnostics without changing token-prefix merge decisions:

token-prefix failure → still split the training row
  decoded prefix preserved → retoken_mismatch
  known delimiters/whitespace restore prefix → template_mismatch
  otherwise → others_mismatch

Template normalization recognizes only literal <think>, </think>, <tool_call> and </tool_call> delimiters plus whitespace. It retains their content and unknown tags, rejects an empty normalized context, and falls back to other without a tokenizer or on decoding failure. These flags are heuristics, not proof of the underlying cause. The table schema, 100-row cap and training metrics are unchanged.

Evidence

• Before: new public-adapter regressions produced 4 failures and 6 passing negative controls; every mismatch was labeled other.
  After: all 10 cases pass, including content changes, unknown delimiters, empty normalized context, decoding failure and absent-tokenizer controls.
• uv run --locked --no-sync pytest -v --durations=20 tests: 159 passed, 3 dependency warnings (child-only inherited proxy variables removed for localhost tests).
• A separate original-package probe used genuine Tokenizers/Transformers, CPU Torch/VERL and real offline W&B tables. All tensors and metrics in its five cases match the pre-change baseline, including strict-prefix merging, masks, rewards, log probabilities and routed experts; input rollouts remain unchanged.
• Full configured pre-commit run --all-files --show-diff-on-failure, ruff check ., ruff format --check ., python scripts/check_headers.py, and pyright: passed.
• uv build --no-sources and uv run --locked --no-sync mkdocs build --strict: passed.

This bounded change does not cover arbitrary tokenizer special tokens, semantically equivalent tool-call JSON, Qwen corpus evaluation, optional diagnostic counters or merge-precondition documentation from the broader proposal. It keeps unrecognized cases as other.

Developed and validated with OpenAI Codex assistance. No GPU training, model inference or corpus-level classification/performance evaluation was run.

Merge Danger

Door: two-way

Blast Radius: diagnostics

This only changes flags on already-recorded mismatch rows. Strict token-prefix splitting and training batches are preserved; unfamiliar template changes conservatively remain other.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Daniel Peng (original4422)]

@microsoft-github-policy-service agree