Skip to content

Add native Qwen3 classifier export and vLLM serving - #8

Merged
akanyaani merged 2 commits into
FrontiersMindAI:mainfrom
OthmanAdi:codex/native-classifier-export
Oct 27, 2026
Merged

akanyaani merged 2 commits into
FrontiersMindAI:mainfrom
OthmanAdi:codex/native-classifier-export

Conversation

@OthmanAdi

Copy link
Copy Markdown
Contributor

A trained Qwen3 code-readout checkpoint currently has to use DecisionFormer's Transformers server because the existing vLLM adapter accepts the bare Qwen3.5/3.8 decision layout. This adds an explicit export to Qwen3ForSequenceClassification and a matching vLLM pooling path.

decisionformer export MODEL OUTPUT --format sequence-classification copies the merged backbone and independently trained answer matrix into standard model.* and score.weight tensors. It preserves the tokenizer, prompt, code order, context limit and calibration metadata. The adapter requests raw classifier logits, slices to the active options, then applies the saved question-type temperature and existing response formatting.

The first conversion scope is merged, unquantized, causal Qwen3 with one code readout and last-token pooling. Exports use float32 weights, which approximately doubles backbone storage relative to 16-bit checkpoints. Existing canonical checkpoints and the Qwen3.5 serving path are retained. Incomplete weight loads are refused, and failed exports are staged and cleaned before publication without overwriting an existing destination.

Validation:

  • Full Linux CPU suite: 309 passed, 13 skipped. Ruff and git diff --check pass.
  • Actual vLLM CPU 0.32.0, PyTorch 2.13.0 and Transformers 5.20.2: a locally trained tiny Qwen3 checkpoint completed 16 decisions across four requests, including choice, yes/no, score, mixed lengths, JSON and empty states. Maximum active-logit difference was 6.71e-8, probability difference 2.15e-8, with no prompt or choice mismatches against the FP32 reference.
  • The separate opt-in actual-engine test also passed. Offline tests cover independent HF loading, learned-head preservation, padding, active-option normalization, per-type temperatures, single/sharded readout files, missing weights and failed writes.
  • Four Windows distributed-launcher tests fail before model loading because that PyTorch build lacks libuv support. The same launcher failure reproduces on unchanged main. The full suite above ran in Linux.

The guide includes reproduction commands and the pinned CPU container setup. examples/compare_classifier.py prints reference/exported answers and exits nonzero on drift. DECISIONFORMER_RUN_VLLM=1 enables the actual-engine test.

These checks establish the tested CPU interoperability path. GPU execution, model accuracy and throughput improvements were not measured, and no acceleration claim is made.

Preserve the trained code readout, prompt and calibration in a standard HF classifier layout. Add native vLLM serving, strict checkpoint validation, staged export publication, offline parity checks and an opt-in actual-engine test with reproducibility documentation.
@Rohit7sali

Rohit7sali commented Oct 26, 2026 •

Copy link
Copy Markdown

We are running some tests to check the PR, we will update soon.

Resolves conflicts with FrontiersMindAI#9 in the vLLM backend description and the
--backend help: Qwen3.5 only for code readouts, which need a vLLM
nightly; native Qwen3 classifier exports use vLLM's pooling runner.
@akanyaani
akanyaani merged commit 63e4c03 into FrontiersMindAI:main Oct 27, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants