task: 细化 vLLM Ascend 知识蒸馏与知识库整理 #350

Activity

1. jiangkuaixue123

   added

   kind:taskBounded delivery task with a delegated sliceBounded delivery task with a delegated slice

   area:knowledgeKnowledge line: packs, skill library, knowledge admissionKnowledge line: packs, skill library, knowledge admission

   status:proposedWaiting for the decision named in the status blockWaiting for the decision named in the status block

   on Nov 4, 2026
2. amy-why-3459 commented on Nov 4, 2026

   

   amy-why-3459

   on Nov 4, 2026 – with Cursor

   Collaborator

   More actions

   感谢整理，方向很清楚。补充几点建议，方便确定首批范围：

   1. 先基于现有条目做盘点。 仓库里已经有不少 vLLM Ascend 相关内容，例如 hardware/ascend/ascend-aclgraph-capture-replay、ascend-model-adaptation、ascend-optimization-casebook、torchnpu-device-memory-lifecycle / torchnpu-pinned-host-memory-lifecycle，以及 hardware/host/host-memory-management/references/vllm-ascend-native-offload*，还有 docs/reviews/issue91-vllm-ascend-host-offload.md。建议覆盖矩阵直接以这些条目和 docs/reference-matrix.md、docs/skill-coverage.md 为起点，标出待细化、重复和缺口的部分，避免另起一套。
   2. 首批主题建议收窄。 现有内容在图模式（aclgraph）、模型适配和 host offload 上已有基础，可以优先在这几块做细粒度拆分；量化、分布式通信这类变化快、版本差异大的主题，建议等 readset 冻结后再单独开批。
   3. 版本锚点。 vLLM Ascend 和 vLLM 主线、CANN、torch_npu 版本强耦合，建议每个知识单元都写明这几项的版本组合，引用用 commit SHA 或 tag，不用分支名。
   4. 归属边界。 第 5 条完成标准里 vLLM 通用机制、插件特性、底层约束三层的划分很关键，建议在 PR 里给一个小表，说明每个新条目归哪个 canonical owner、交叉引用了哪些已有条目。
   5. 检索案例。 建议和 [Tracking] Skill Indexer: retrieval efficiency and accuracy #255 的检索评测输入格式对齐，这样代表性案例可以直接进回归，而不是只放在文档里。

   另外，issue 里提到需要维护者确认负责人，这一点还需要维护者来定。
3. Rohank143 commented on Nov 4, 2026

   

   Rohank143

   on Nov 4, 2026

   Last edited by Rohank143

   More actions

   📋 First-Batch Knowledge Inventory & Topic Coverage Proposal (Refs #350)

   Clarification Update (2026-11-04): See clarification comment below for precision on ascend-operator-tuning existing candidate status, decoupled historical pins, and exact Batch 1 requested boundaries.

   In response to the task proposal and the guidance provided by @amy-why-3459, here is the structured First-Batch Knowledge Inventory & Coverage Matrix addressing Completion Criteria 1–5:

   ---

   1. Inventory of Existing vLLM Ascend Knowledge Assets in Repository

   Starting from docs/reference-matrix.md, docs/skill-coverage.md, and canonical skill directories:

   1. hardware/ascend/ascend-aclgraph-capture-replay:
      • Source Pin: 8a8e1e05ecb64a30804c99c8f362909f9a0cb5bf (compilation/acl_graph.py, test_acl_graph.py).
      • Coverage: ACLGraph wrapper dispatch, input immutability, replay/update ordering, capture failure triage.
      • Status: To-Refine — Needs fine-grained rules on dynamic shape bucketing and multi-stream sync failure bounds. Kept distinct from ascend-graph-mode.
   2. hardware/ascend/ascend-model-adaptation:
      • Source Pin: Historical bc8115ed7fdc7e500320b5de70742ef049b4c1a2 (.agents/skills/vllm-ascend-model-adapter/).
      • Coverage: Checkpoint onboarding, FP8 paired-tensor loading, KV-head replication, multimodal validation order.
      • Status: To-Refine — Separate model-specific quirks from vLLM-Ascend general plugin features; update version anchors.
   3. hardware/host/host-memory-management (references/vllm-ascend-native-offload*):
      • Source Pin: 8a8e1e05ecb64a30804c99c8f362909f9a0cb5bf (7-file ledger, Apache-2.0 license).
      • Coverage: Event-query-gated CPU descriptor-pool reuse, Transfer ownership, asynchronous polling, RAM retention bounds.
      • Status: Covered (Reused by cross-reference) — Solid baseline established in [Tracking] SearchSpace distillation coverage and knowledge consolidation #91; reference doc vllm-ascend-native-offload.md serves as the authoritative pattern without rewriting.
   4. hardware/ascend/torchnpu-device-memory-lifecycle & torchnpu-pinned-host-memory-lifecycle:
      • Source Pin: 67511c53cd4c88014f6123de7097677e60e85faa (TorchNPU).
      • Coverage: Lower-level device/host memory allocation, stream-use lifetimes, HBM fragmentation.
      • Status: Covered (Bottom-layer constraint) — Serves as Tier 3 authority; cross-referenced rather than duplicated.
   5. hardware/ascend/ascend-operator-tuning:
      • Status: Existing Candidate (Source/readset review needed) — Already indexed in canonical skill-index.json (status: candidate, empirical_validation: not_run, provenance_readsets: []). Outside Batch 1 requested slice.
   6. hardware/ascend/ascend-optimization-casebook & ascend-graph-mode:
      • Coverage: Torchair graph paths and CANN profiling layers.
      • Status: Covered (Adjacent) — Provides profiling context.

   ---

   2. Topic Coverage & Status Matrix (Completion Criterion 1)

   Topic	Scope / Key Focus	Status	Target Canonical Owner	Source Baseline
   Graph Mode & ACLGraph	Warmup capture, buffer immutability, dynamic shape bucketing, recapture failure bounds	To-Refine (Batch 1)	hardware/ascend/ascend-aclgraph-capture-replay	vllm-ascend:compilation/acl_graph.py
   Model & Checkpoint Adaptation	FP8 paired weights loading, GQA KV-head replication, multimodal graph isolation	To-Refine (Batch 1)	hardware/ascend/ascend-model-adaptation	vllm-ascend:models/
   Host KV Offload & Descriptor Pooling	Event-query reclamation, Transfer lifetime vs physical RAM release	Covered (Reused by ref)	hardware/host/host-memory-management	references/vllm-ascend-native-offload.md
   Operator & Custom Kernels	Ascend custom op dispatch, flash attention / fused RMSNorm invocation	Existing candidate (readset needed)	hardware/ascend/ascend-operator-tuning	Outside Batch 1 requested slice
   Distributed / HCCL Communication	TP/PP collective mesh setup, HCOMM communicators	Deferred (Batch 2)	hardware/ascend/ascend-distributed-parallel	Awaiting CANN 8.0 frozen readset
   Quantization (W8A8 / SmoothQuant)	Quantization config mapping, calibration scale binding	Deferred (Batch 2)	hardware/ascend/ascend-optimization-casebook	Defer until readset freeze

   ---

   3. Three-Tier Responsibility & Attribution Boundaries (Completion Criterion 5)

   To prevent duplication and lexical conflicts:

   1. Tier 1 — vLLM Generic Mechanism (repositories/vllm/): PagedAttention block abstraction, scheduler preemption, worker RPC protocol. (Does not mention NPU or CANN specifics).
   2. Tier 2 — vLLM Ascend Plugin Features (hardware/ascend/ & hardware/host/): ACLGraph model runner, Ascend custom op dispatch, native host KV offload descriptor pool, paired weight converter.
   3. Tier 3 — Ascend / CANN / TorchNPU Bottom Constraints (hardware/ascend/): aclmdlExecute, geExecutor, HCCL mesh scratch, TorchNPU caching allocator, H2D streams. (Does not assume vLLM context).

   ---

   4. Historical Source Anchors (Independent, Not a Tested Compatibility Set, Criterion 4)

   • vLLM Ascend Baseline: vllm-project/vllm-ascend @ 8a8e1e05ecb64a30804c99c8f362909f9a0cb5bf (with historical documentation pin bc8115ed7fdc7e500320b5de70742ef049b4c1a2).
   • vLLM Upstream Pin: vllm-project/vllm @ tag v0.6.3 / v0.6.4.
   • CANN Toolkit: CANN 8.0.RC2.
   • Torch-NPU: torch_npu @ v2.4.0 (67511c53cd4c88014f6123de7097677e60e85faa).
     (These are separate historical anchors and are not claimed as a single verified compatible stack until verified against an exact matching readset).

   ---

   5. Proposed Retrieval Test Cases (Not Yet Executed Against #255 Evaluation Protocol)

   • Proposed Case 1 (ACLGraph): "ACLGraph capture fails during multimodal prefill with variable image resolution" $\rightarrow$ Expected Hit: hardware/ascend/ascend-aclgraph-capture-replay (Negative: ascend-model-adaptation).
   • Proposed Case 2 (Host Offload): "Host memory descriptor leak after vLLM Ascend CPU KV offload wait" $\rightarrow$ Expected Hit: hardware/host/host-memory-management (Negative: torchnpu-device-memory-lifecycle).
   • Proposed Case 3 (TorchNPU Allocator): "TorchNPU device memory fragmentation under paged attention block allocation" $\rightarrow$ Expected Hit: hardware/ascend/torchnpu-device-memory-lifecycle (Negative: ascend-aclgraph-capture-replay).

   ---

   Proposed Next Step & Precise Batch 1 Scope

   To keep the delegation request exact:

   1. ACLGraph capture/replay refinement under ascend-aclgraph-capture-replay (distinct from ascend-graph-mode).
   2. Model and checkpoint adaptation refinement under ascend-model-adaptation.
   3. Corresponding owner-local metadata and index updates, following existing schemas.

   Host KV offload remains an existing cross-reference, and Operator tuning remains outside this slice unless maintainers request otherwise.

   Could repository maintainers / @Dong1017 please confirm the exact Batch 1 boundary and authorize implementation delegation for this slice?
4. Rohank143 commented on Nov 4, 2026

   

   Rohank143

   on Nov 4, 2026

   More actions

   Thanks for the guidance. While checking the current canonical skill library, I found that ascend-operator-tuning already exists and is indexed. I'll correct its coverage status to distinguish an existing candidate from a missing knowledge area.

   To keep the proposed delegation precise, my requested Batch 1 implementation scope is:

   1. ACLGraph capture/replay refinement under the existing ascend-aclgraph-capture-replay owner (kept distinct from the separate ascend-graph-mode owner).
   2. Model and checkpoint adaptation refinement under the existing ascend-model-adaptation owner.
   3. Corresponding owner-local metadata and index updates, following the existing schema.

   Host KV offload will remain an existing cross-reference (hardware/host/host-memory-management/references/vllm-ascend-native-offload.md) unless maintainers request additional refinement. Operator tuning will remain outside this requested implementation slice.

   I will also keep the historical source pins separate (e.g. ACLGraph @ 8a8e1e05, Model Adaptation @ bc8115ed) and avoid claiming a verified compatible software stack until the exact readset and version combination are confirmed. The retrieval examples are proposed cases and have not yet been executed against the #255 evaluation protocol.

   Could maintainers please confirm the exact Batch 1 boundary, canonical source readset, and implementation and acceptance owners before I start implementation?
5. Dong1017 commented on Nov 4, 2026

   

   Dong1017

   on Nov 4, 2026

   Collaborator

   More actions

   Could maintainers please confirm the exact Batch 1 boundary, canonical source readset, and implementation and acceptance owners before I start implementation?

   Hi @jiangkuaixue123, can you share some insights when you have a chance? Do you agree to assign parts of work to @Rohank143?

   😄React with 😄1Reacted by Rohan Kakade
6. Rohank143 commented on Nov 4, 2026

   

   Rohank143

   on Nov 4, 2026

   More actions

   I'm interested in taking responsibility for the proposed Batch 1 scope (ACLGraph refinement, Model Adaptation refinement, and metadata updates).

   Before starting implementation, could you please confirm whether this is a paid assignment?

   If yes, please clarify the approved budget or expected reward, payment terms, and acceptance milestones.

   I'm happy to proceed once the scope, ownership, and compensation are mutually agreed upon.

   Thank you.
7. jiangkuaixue123 commented on Nov 4, 2026

   

   jiangkuaixue123

   on Nov 4, 2026 – with ChatGPT Codex Connector

   CollaboratorAuthor

   More actions

   @Dong1017 Yes, I agree to assign the proposed Batch 1 work to @Rohank143: ACLGraph capture/replay refinement, model and checkpoint adaptation refinement, and the corresponding owner-local metadata and index updates, as scoped in the clarification above.

   @Rohank143 Thanks for taking this on. The source readset and acceptance owner still need to be confirmed. Your question about compensation also remains to be resolved separately; this scope approval does not establish payment terms.
8. Dong1017 commented on Nov 4, 2026

   

   Dong1017

   on Nov 4, 2026

   Collaborator

   More actions

   Answering the compensation point @jiangkuaixue123 left open: this is an unpaid open-source project. InferRSI does not pay for contributions. There is no budget, no reward and no bounty program behind any Issue here, including this one. The scope approval above assigns work; it does not create payment terms, and no payment terms are coming.

   Thank you for asking before starting rather than after. That is the right order, and it means you can decide with the answer in hand. Taking Batch 1 is entirely optional — declining it now costs you nothing and counts against you in no way.

   If you do continue, the usual open-source terms apply: your commits keep your authorship and the Issue record links your work. I have given the same answer on #328 and #349, where the question about where a bounty was advertised is still open.
9. Rohank143 commented on Nov 5, 2026

   

   Rohank143

   on Nov 5, 2026

   Last edited by Rohank143

   More actions

   Thank you @Dong1017 @jiangkuaixue123 for clarifying the compensation terms.

   I understand and accept that this is an unpaid open-source contribution. I would still like to proceed with the approved Batch 1 scope.

   I confirm my willingness to work on:

   1. ACLGraph capture/replay refinement.
   2. Model and checkpoint adaptation refinement.
   3. Corresponding owner-local metadata and index updates.

   Before implementation, could you please confirm:

   • The canonical source readset and exact commit references for both areas.
   • The designated acceptance owner/reviewer and completion criteria.

   Once these are confirmed, I'll proceed with the implementation and submit a PR linked to Issue #350.

   Thank you for the opportunity.
10. Dong1017 commented on Nov 5, 2026

    

    Dong1017

    on Nov 5, 2026 – with ChatGPT Codex Connector

    Collaborator

    More actions

    @Rohank143 Thanks for confirming. I checked the existing owners at main 9dded7ad. There is a concrete difference in the retained source records:

    • ACLGraph capture/replay: the source ledger already lists three files at upstream 8a8e1e05ecb64a30804c99c8f362909f9a0cb5bf: vllm_ascend/compilation/acl_graph.py, docs/source/user_guide/feature_guide/graph_mode.md, and tests/ut/compilation/test_acl_graph.py, with blob hashes and the inspected scope.
    • Model/checkpoint adaptation: the routing record resolves the historical pin to bc8115ed7fdc7e500320b5de70742ef049b4c1a2, but explicitly records no bundled owner readset. The existing entry cites fp8-on-npu-lessons.md and multimodal-ep-aclgraph-lessons.md under .agents/skills/vllm-ascend-model-adapter/references/. Resolving that commit does not establish that the proposed implementation files have been inspected.

    These are starting references, not a newly selected Batch 1 readset or a verified compatible stack. The independent RC-history work in #354 does not automatically replace either owner's historical source boundary.

    @jiangkuaixue123, could you confirm whether Batch 1 should refine these retained pins or explicitly re-pin each owner, the complete model-adaptation readset, and the acceptance reviewer against this Issue's existing completion criteria? That would answer the remaining pre-implementation questions without expanding the agreed slice.

    Read-only source-record check; no upstream semantic re-audit, tests or device runs in this reply.