Propose opt-in Qwen4-Exp ngram/main split contract

[Copilot]

Defines a proposed, opt-in contract for independently loadable Qwen4-Exp ngram and main components, while preserving unsplit defaults. The proposal makes the seam, state ownership, component selection, weight routing, and persistence behavior explicit for review.

• API and ABI: Specify text/VLM component keys, signatures, symbolic shapes, dtypes, PLE-layer ordering, and the proposed [L,B,S,D] embedding layout.
• Ownership and state: Keep PLE projections, gates, and convolution in main; define EOS/padding/history behavior and complete-state snapshot/restore/replay for S1 review.
• Weights and packaging: Describe selected build/load dependencies, save-time filtering, token-table/head identity constraints, and nested MTP persistence.
• Qualification: Propose tiny CPU targets for parity, mapping rollback, independent loading, package roundtrip, and later reduced generation; retain QSA tie and BF16 vision limitations.

package = mobius.build(
    model_id,
    execution_provider="onnx-standard",
    task=Qwen4ExpCausalLMTask(
        split_ngram=True,
        requested_components={"ngram", "main"},
    ),
)

All new interface choices remain proposals pending explicit maintainer approval.

• Fixes Design the opt-in Qwen4-Exp ngram/main API, state ABI, and weight contract #768

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ titaiwangms
❌ Copilot
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[github-actions]

Performance Comparison

Comparing b1e9c8e9 → db043755

Model	Metric	Baseline	Current	Delta
bert (feature-extraction)	model_size_bytes	359 KB	359 KB	+0.0%	⚪
bert (feature-extraction)	num_nodes	68	68	+0.0%	⚪
falcon	model_size_bytes	364 KB	364 KB	+0.0%	⚪
falcon	num_nodes	66	66	+0.0%	⚪
gemma2	model_size_bytes	428 KB	428 KB	+0.0%	⚪
gemma2	num_nodes	105	105	+0.0%	⚪
gpt2	model_size_bytes	324 KB	324 KB	+0.0%	⚪
gpt2	num_nodes	54	54	+0.0%	⚪
llama	model_size_bytes	425 KB	425 KB	+0.0%	⚪
llama	num_nodes	60	60	+0.0%	⚪
llama (static-cache)	model_size_bytes	425 KB	425 KB	+0.0%	⚪
llama (static-cache)	num_nodes	56	56	+0.0%	⚪
mamba (ssm-text-generation)	model_size_bytes	296 KB	296 KB	+0.0%	⚪
mamba (ssm-text-generation)	num_nodes	94	94	+0.0%	⚪
phi3	model_size_bytes	421 KB	421 KB	+0.0%	⚪
phi3	num_nodes	58	58	+0.0%	⚪
phi3 (static-cache)	model_size_bytes	421 KB	421 KB	+0.0%	⚪
phi3 (static-cache)	num_nodes	54	54	+0.0%	⚪
qwen2	model_size_bytes	425 KB	425 KB	+0.0%	⚪
qwen2	num_nodes	60	60	+0.0%	⚪
qwen2 (static-cache)	model_size_bytes	425 KB	425 KB	+0.0%	⚪
qwen2 (static-cache)	num_nodes	56	56	+0.0%	⚪
qwen3_5_moe (hybrid-text-generation)	model_size_bytes	506 KB	506 KB	+0.0%	⚪
qwen3_5_moe (hybrid-text-generation)	num_nodes	269	269	+0.0%	⚪
qwen3_5_text (hybrid-text-generation)	model_size_bytes	458 KB	458 KB	+0.0%	⚪
qwen3_5_text (hybrid-text-generation)	num_nodes	131	131	+0.0%	⚪
qwen3_5_vl (hybrid-qwen-vl)	model_size_bytes	977 KB	977 KB	+0.0%	⚪
qwen3_5_vl (hybrid-qwen-vl)	num_nodes	461	461	+0.0%	⚪
t5 (seq2seq)	model_size_bytes	836 KB	836 KB	+0.0%	⚪
t5 (seq2seq)	num_nodes	176	176	+0.0%	⚪
whisper (speech-to-text)	model_size_bytes	1008 KB	1008 KB	+0.0%	⚪
whisper (speech-to-text)	num_nodes	128	128	+0.0%	⚪

No performance regressions.

[Copilot]

🟡 Changes recommended

The proposed seam and persistence behavior conflict with parent requirements and current package key handling.

2 open findings

• Undefined target EP behavior for split mode · New
• Full attention mask violates minimal ngram seam contract · New

What changed in this PR

Documents a proposed opt-in Qwen4-Exp ngram/main split while preserving the unsplit default.

Changes:

• Defines component APIs, tensor/state ABI, and ownership.
• Specifies weight routing, persistence, and MTP interaction.
• Proposes qualification and roundtrip tests.

File	Description
docs/​design/​qwen4-exp-text-core.md	Adds the split-component contract proposal.

🧠 Review effort: Balanced

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[titaiwangms]

Static design review: changes requested

Reviewed head: 0e8b4d1bcdb7269b2697e8ce44a81a8279dfdd10.
Source baseline: Mobius 954e8f1053d64d377d8893e5c700d1b8380d584a.

This is the synthesis of a Copilot-assisted, five-role static review of this PR and companion #777. It assesses the proposed contract against the pinned source and #767-#775, not an implemented split. No tests, builds, model execution, or source changes were performed. The recommendation below concerns the document; neither merging it nor this review constitutes semantic approval to start N1.

1. Major: singleton save/load does not yet preserve the proposed component identity

Location: docs/design/qwen4-exp-text-core.md:449-455,473-480,520-521
Confidence: High

The proposal permits independently building or saving only main or ngram, selecting loads by those canonical keys, and preserving split keys through a filtered roundtrip. However, the existing persistence mechanism does not recover those keys for a singleton:

• _ComponentSaveLayout.uses_subdirectories uses component subdirectories only when more than one component is selected.
• The save loop writes a singleton to the root model.onnx.
• Flat-directory loading derives the key from the filename, yielding model, not main or ngram.

Thus, preserving the existing save behavior is insufficient to establish the proposed canonical-key roundtrip. This is a source-proven design gap, not a claim that an unimplemented split was executed and failed.

Minimal revision: Specify the narrow identity/layout extension: for example, split-only named directories, canonical-key recovery metadata, or explicitly different flat-load semantics. Preserve legacy defaults and distinguish singleton builds, a full package filtered to one component, and direct loading of a component subdirectory.

Other findings: Minor

Location in this document	Finding	Minimal revision
299,352-359	Adding attention_mask[B,P+S] moves masked-ID-to-EOS substitution into the ngram graph, whereas #767 specifies a mask-free lookup graph and host substitution. The alternative is not inherently invalid, but the parent/child boundary differs.	Either retain host preprocessing or explicitly identify this as a proposed contract change requiring approval and synchronization with #767.
255-256	The target EP is not defined, so the supported scope and behavior of split_ngram=True on other profiles are unspecified.	Name the supported export profile(s) and the explicit rejection or pending-decision policy for other profiles; do not imply an unapproved fallback.
514-525	The proposed test locations do not match #770/#771's mandatory entry points. The mapping plan also does not explicitly require value-based binding and proof that unrelated payloads are not accessed.	Align with src/mobius/models/qwen4_exp_split_test.py, the relevant src/mobius/integrations/transformers/_qwen4_exp_weights_test.py cases, and src/mobius/models/qwen4_exp_split_generation_test.py. Include same-shape/different-value sentinels and unrelated-source access assertions. Proposed tests need not already exist in this docs-only PR.
366-368	Saying the mask suppresses the PLE contribution can imply that the entire PLE output is zero at a masked position. In the pinned implementation, the mask zeros the current gated value and convolution input before the stateful convolution; historical convolution state can still contribute.	Explicitly preserve that ordering and avoid implying an additional post-convolution mask. See qwen4_exp.py:635-641.
538-540	Some pinned source ranges do not point to the claimed behavior. Main mask/EOS processing is at 1337-1359, VLM source ownership at 1930-1939, and the PLE citation should include the mask/convolution logic through 641.	Correct the source ranges. This is a traceability correction, not a finding that the underlying broad claims are false.

Findings ledger and minority positions

These are synthesis decisions, not unanimous reviewer agreement:

Issue	Raised by	Disposition
Singleton component identity	Code and Critical	Confirmed Major; directly checked against the pinned save/load branches.
Parent versus proposed mask boundary	Readability: Major; Deep: open choice; Integration: permissible refinement	Confirmed Minor, downgraded by the lead. The unacknowledged boundary change is real, but does not prove an invalid graph.
Cross-root/nested-MTP sharing mechanism	Readability: Major; Code and Integration: open question	Rejected as an established merge blocker, retained below as an implementation-readiness question. Ordinary text/VLM owners are listed, and companion #777 explicitly gates sharing integration.
Instance-level manifest/source-name/selected-buffer extensions	Integration: Major	Rejected as an established merge blocker, retained below. This proposal already allows narrow ownership and selected-binding extensions; missing implementation details alone do not invalidate a proposed API.
Verification target locations	Integration: Major; Code: Minor	Confirmed Minor, downgraded by the lead. Required entry points need alignment; proposed files not already existing is not itself a defect.

Open decisions before implementation

• Readability / Code / Integration: Define how shared token-table/head identities span the root manifest and nested mtp_head, including selected binding. Current same-manifest endpoint constraints do not by themselves establish cross-package coordination.
• Integration / Code: Specify instance-level manifest/source renames and selection-aware deterministic-buffer validation, including supported checkpoint formats and existing FP8 limitations.
• Readability: Settle cold-start, zero-length chunk, EOS-list, and non-binary-mask behavior. Do not collapse per-layer history without proof and approval.
• Critical: Decide whether component compatibility includes checkpoint/hash-config semantic identity, beyond matching ABI, layer order, dimensions, and dtype.
• Integration: State whether the opt-in is intentionally Python-only, and how saved/reloaded split packages remain recognizable to metadata/runtime consumers when config is not restored.
• Readability: Clarify selector-key/module-attribute/role mapping, non-None selection in unsplit mode, retained ple_input_ids bookkeeping, and the future main-tap request contract coordinated with #777.

Exclusions: No runtime numerical parity, fault-injection/rollback behavior, memory/performance measurements, official-weight qualification, quantization execution, native GenAI execution, or Olive validation. Existing QSA cutoff-tie limitations and the BF16 vision restriction remain unchanged. These documentation changes introduce no executable trust boundary; no new vulnerability claim is made.

[titaiwangms]

Addressed the documentation review in 5674408.

• Specify split-only metadata recovery for canonical singleton keys, preserving the existing flat layout and legacy loading. Include malformed/conflicting metadata, filtered saves, and direct component-directory cases.
• Restore the parent's mask-free ngram seam and host EOS preparation; clarify that PLE masking does not erase historical convolution contributions.
• Make the standard-only EP gate, explicit example arguments, Python-only scope, key/module/role mapping, and manifest/alias requirements unambiguous.
• Align the future Implement independently loadable standard-ONNX Qwen4-Exp ngram/main packages #770/Qualify reduced Qwen4-Exp split generation with real token feedback #771 test entry points, value-based mapping and unrelated-payload access assertions, offline fixtures, and pinned source ranges.

The second full static review identified the missing EP arguments in the revised examples; those were corrected and directly checked without starting a third review round. The PR-description example is aligned too.

Validation: git diff --check passed for both document changes, along with 40 targeted static assertions. No model tests, builds, downloads, or numerical qualification ran.

These are proposal/documentation fixes, not implemented loader or split behavior. Cross-package MTP sharing, semantic checkpoint/hash pairing, the main-tap selector, transaction semantics, predecessor gates, and explicit maintainer approvals remain open where previously required.

[Copilot]

🔵 Needs a closer look

The proposed public API, state ABI, and persistence contract require explicit maintainer acceptance.

0 open findings

2 resolved since last review

• Full attention mask violates minimal ngram seam contract
• Undefined target EP behavior for split mode

🧠 Review effort: Balanced