Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/reference-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,7 @@ not fabricated implementation SHAs or a version-matched runtime stack.
|---|---|---|---|
| Vidur — [2405.05465v2](https://arxiv.org/abs/2405.05465v2) | [A readset](../src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/references/papers-20261015/READSET.md); §§3–7, A.1–A.2 | mined — operator/queue model calibration and joint-SLO shortlist → [calibrate-serving-simulation](../src/inferrsi/data/skill_library/methods/calibrate-serving-simulation/SKILL.md), ceiling analysis | Empirical prediction, not physical bound; near-capacity CPU overhead and 12.65% maximum error in the specified appendix case; no current runtime calibration |
| Orca — OSDI 2022 proceedings | [Exact readset](../src/inferrsi/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/orca-readset.md); §§3–6, Algorithm 1 | mined — iteration eligibility, selective batching, FCFS/reservation → [assess-serving-preemption](../src/inferrsi/data/skill_library/methods/assess-serving-preemption/SKILL.md) | Historical engine/harness and normalized-median denominators; not present vLLM behavior or fairness proof |
| Sarathi-Serve — [OSDI 2024 proceedings, pp. 117–134](https://www.usenix.org/conference/osdi24/presentation/agrawal) | Fresh official-edition grounding, 2026-10-30; full text and artifact appendix; §§4.1–4.3, §5 setup, Figures 9/14 and Table 4 checked for the [case record](../src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.json) | mined — decode/partial-prefill/new-request token accounting and chunk-cost tradeoffs → [one optimization-search-policy case](../src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.md) | Source-derived candidate; empirical not_run. Combined design has worse P50 TTFT than hybrid-only in both Table 4 traces; no current-engine, universal-budget or fairness claim. Raw source reuse rights UNKNOWN; no archive recovery or novelty over unavailable supplier synthesis claimed. |
| FastServe — [2305.05920v3](https://arxiv.org/abs/2305.05920v3) | [Exact readset](../src/inferrsi/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/fastserve-readset.md) | mined — service priority, promotion and suspended-state host residency → serving-preemption candidate | Aggregate tail improvement is not tenant fairness; same-engine contrast distinct from historical-vLLM comparison |
| Llumnix — [2406.03243v1](https://arxiv.org/abs/2406.03243v1) | [Exact readset](../src/inferrsi/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/llumnix-readset.md) | mined — replica migration, virtual headroom, pre-copy/downtime and host staging → [evaluate-elastic-serving-placement](../src/inferrsi/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md) | Virtual headroom is not physical capacity; normal-class cost and evaluated-artifact fault-tolerance limit retained |
| LoongServe — [2404.09526v2](https://arxiv.org/abs/2404.09526v2) | [Exact readset](../src/inferrsi/data/skill_library/repositories/vllm/diagnose-vllm-admission-pressure/references/papers-20261015/references/loongserve-readset.md) | mined — elastic sequence groups, phase-dependent state placement and ongoing attention communication → elastic-placement candidate | Distributed device KV is not host RAM; group transitions have different migration costs; no current engine/NPU claim |
Expand All @@ -176,6 +177,9 @@ the supplied local directory. Its six-paper extraction and standalone candidates
are not imported or counted here. Existing DeepSpeed configuration evidence and
Mooncake transport discovery are not substitutes for those paper readsets.

The separate Sarathi-Serve case above uses a freshly pinned official edition;
it does not recover or import that archive or its supplier-authored synthesis.

### Negative findings (scoped to inspected paths and revisions)

- **Disk-level storage I/O mechanics** (mmap/pinned staging buffers,
Expand Down
2 changes: 1 addition & 1 deletion docs/searchspace-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ The primary compute and HBM resource. Most current work lives here.
| **Diagnose / Configure / Verify** | AKG static CUDA Affine loop tails: effective target/option gates, full point-loop bounds and protected input/output accesses | [kernel-writing-guide](../src/inferrsi/data/skill_library/methods/kernel-writing-guide/SKILL.md) | [Tail-guard reference](../src/inferrsi/data/skill_library/methods/kernel-writing-guide/references/akg-affine-tail-guards.md): separation precedence, non-divisible fixture and diagnostic boundary | AKG @ 5aa15f3fe4d856eb60cd3b27447e04f5f1fb4dff; source + FileCheck expectations only | Candidate; compiler/IR command, CUDA runtime and agent efficacy not run; no new measured coverage |
| **Optimize** | Operator-level: staged accuracy-gated tuning campaign (code-level → scheduling → instruction-level), metric→action routing; AscendC tiling design per operator class (reduction/broadcast/elewise/matmul/conversion) | `ascend-operator-tuning`, `ascendc-tiling-design` | PyPTO frontend/swimlane/incore playbooks; fullload-vs-split, four-element tiling template, chip-generation (DAV_2201/3510) routing | awesome-ascend-skills pypto-op-perf-tune + ascendc-tiling-design @ 75ec89c | 🟢 |
| **Optimize** | Casebook of past NPU optimization decisions: AI-CPU fallback, memory-bound, comm-exposed, graph-compilation, scheduling idle gaps | `ascend-optimization-casebook`, `inference-optimization-casebook` | Five bottleneck classes with symptom→diagnosis→action→evidence schema | MindIE-SD profiling-analyze + dit-perf-opt + perf-gate + dit-parallel-opt @ 8481d39b | 🟢 |
| **Optimize** | Bounded experiment selection, distinct mechanism hypotheses, incumbent preservation and final-artifact integrity | `optimization-search-policy` | Methodology provenance stays in Reference Matrix §2; no hardware coverage from agent-paper scores | vllm-evolve @ 43cc82c; scoped A/F paper amendments | 🟢 (procedure only) |
| **Optimize** | Bounded experiment selection, distinct mechanism hypotheses, incumbent preservation and final-artifact integrity | `optimization-search-policy` | [Sarathi chunk-budget case](../src/inferrsi/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.md): decode-first accounting, prefill overhead and TTFT/TBT tradeoffs; no universal budget. Methodology provenance stays in Reference Matrix §2; no hardware coverage from agent-paper scores | vllm-evolve @ 43cc82c; scoped A/F paper amendments; separate official OSDI 2024 Sarathi case/readset in Reference Matrix | 🟢 (source-derived candidate procedure; empirical not_run) |
| **Optimize** | Bounded multistream overlap: dependency-DAG constrained (data/state/event/collective_order edges; shared input ≠ dependency), minimal-first, execution-mode sync contract (eager streams vs TorchAir) | `cann-attribution-overlap` | Metric separation (wall / union-of-busy / summed durations / summed duration+wait; repeated-instance product = estimate); device-memory domain qualification; formula conflicts carried unresolved | cann/oam-tools @ 8440935f (pack-mediated, wave 6) | 🟢 |
| **Verify** | Evidence taxonomy (L0 correctness → L5 production), claim-evidence matching | `dense-performance-evidence`, `rsi-evidence`, `claim-contract` | InferRSI PASS/FAIL/INCONCLUSIVE protocol | InferRSI @ 4c8e1285 | 🟢 |
| **Diagnose / Verify** | Fixed-stage PyTorch/Ascend model report: bind current stage and actually loaded candidate to quality-validator output and comparable baseline | [ascend-performance-analysis](../src/inferrsi/data/skill_library/hardware/ascend/ascend-performance-analysis/SKILL.md) | [Stage-evidence rule](../src/inferrsi/data/skill_library/hardware/ascend/ascend-performance-analysis/references/cannbot-stage-evidence.md): workflow instructions versus registered marker/keyword checks; completion is not numerical validation | canonical cann/cannbot @ c0e2ad23ee1a73e3c0f84c8df1cc459e2aae3110; selected CANN-2.0 scope | Candidate procedure only; host/model/backend not_run, agent efficacy not evaluated; no new measured coverage |
Expand Down
4 changes: 2 additions & 2 deletions docs/skill-coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Generated from the registry, the skill sidecars and the two maintained
matrices. **Do not edit by hand**; regenerate after any registry, skill,
matrix or frozen-plan change.

- Input digest: `sha256:b6f5cec72e5e52fcc4113bf50c68612417688b40c54cc79cbd24a611db574f3c`
- Input digest: `sha256:bf6ac4fe724e1f91f7373ed7352852919749aa9cb1e53e8d27e68e3e1df4b8ff`
- Regenerate: `python tools/build_skill_coverage.py`
- Verify without writing: `python tools/build_skill_coverage.py --check`

Expand Down Expand Up @@ -129,7 +129,7 @@ optimization is fully covered or that any procedure has measured benefit.
| [inference-benchmark-design](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/SKILL.md) | common | Select task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents ... | **unknown** | — | — | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>Task-specific benchmark selection (issue #60) ([Issue #60 report](https://github.com/InferMatrixLab/InferRSI/issues/6 ...) | candidate · not_run · 2 readset(s) | no-SearchSpace-layer, no-applicability |
| [inference-optimization-casebook](../src/inferrsi/data/skill_library/methods/inference-optimization-casebook/SKILL.md) | common | How to structure and use a casebook of past optimization decisions as a decision-support tool. Extracts the casebook ... | **unknown** | — | 1 Accelerator · Optimize | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd) | candidate · not_run · 0 readset(s) | no-applicability, no-eval |
| [kernel-writing-guide](../src/inferrsi/data/skill_library/methods/kernel-writing-guide/SKILL.md) | common | Decision guide for writing custom inference kernels: when to write one, which framework to choose (Triton, CuTe DSL ... | **unknown** | — | 1 Accelerator · Optimize / Diagnose / Configure / Verify | 1 · Systems sources (Dao-AILab/causal-conv1d @ `cd81f0413cad2fc1e6f17e785ac39f59aae690cd` ...)<br>1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)<br>1 · Systems sources (huggingface/transformers @ `0dad7b822255a0ae261ec45ae937371e859ffd1a` ...)<br>1 · Systems sources (mindspore-ai/akg @ `5aa15f3fe4d856eb60cd3b27447e04f5f1fb4dff` ...)<br>1 · Systems sources (sgl-project/sglang @ `16176278d0180f1a1a578e53e6eef432577876c2` ...)<br>1 · Systems sources (vllm-project/vllm @ `537a79f0fc5ba564d2bef43d54915b2f80f9f77d` (root ...) | candidate · not_run · 7 readset(s) | no-applicability, no-eval |
| [optimization-search-policy](../src/inferrsi/data/skill_library/methods/optimization-search-policy/SKILL.md) | common | Policy for how an optimization agent should select, order, and stop optimization experiments. Covers candidate ... | **unknown** | — | 1 Accelerator · Optimize | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references (RRSI — arXiv 2609.24972v3)<br>2 · Methodology and corpus references (arXiv 2608.01867 (CRISP))<br>2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 6 readset(s) | no-applicability, no-eval |
| [optimization-search-policy](../src/inferrsi/data/skill_library/methods/optimization-search-policy/SKILL.md) | common | Policy for how an optimization agent should select, order, and stop optimization experiments. Covers candidate ... | **unknown** | — | 1 Accelerator · Optimize | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references (RRSI — arXiv 2609.24972v3)<br>2 · Methodology and corpus references (arXiv 2608.01867 (CRISP))<br>2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 7 readset(s) | no-applicability, no-eval |
| [performance-ceiling-analysis](../src/inferrsi/data/skill_library/methods/performance-ceiling-analysis/SKILL.md) | common | Estimate optimization headroom before modifying an inference system. Produces a three-level ceiling model (analytical ... | inference serving systems (LLM, diffusion, audio) | — | 1 Accelerator · Diagnose / Verify<br>5 OS · Configure | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)<br>1 · Systems sources (ebugger/Empirical-Roofline-Toolkit @ 0771fa9)<br>1 · Systems sources (hahnyuan/LLM-Viewer @ 1893e4b) | candidate · not_run · 16 readset(s) | no-eval |
| [plan-activation-residency](../src/inferrsi/data/skill_library/methods/plan-activation-residency/SKILL.md) | common | Propose training activation and gradient memory reductions by distinguishing lifetime reordering, address packing and ... | **unknown** | — | 1 Accelerator · Diagnose / Optimize / Verify | 1 · Systems sources (Checkmate — [1910.02653v3](https://arxiv.org/abs/1910.02653v3)) | candidate · not_run · 1 readset(s) | no-applicability, no-eval |
| [repository-evolution-analysis](../src/inferrsi/data/skill_library/methods/repository-evolution-analysis/SKILL.md) | common | Analyze a repository's commit history to build a capability-evolution model: what the system can do, how it got there ... | any software repository with a recoverable git history | — | — | 2 · Methodology and corpus references (LearningToCommit/LearningToCommit @ 7e5a1d7)<br>2 · Methodology and corpus references (yetone/magpie @ 77800032) | candidate · not_run · 0 readset(s) | no-SearchSpace-layer, no-eval |
Expand Down
4 changes: 2 additions & 2 deletions src/inferrsi/data/repo_packs/inferrsi/knowledge/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ These are knowledge abstracts (`--kind knowledge`), with namespaced package
IDs distinct from explicitly selected `checkout_ref` skills. Ascend/CUDA
procedures use their corresponding backend; shared methods use `common`.

Source snapshot: [bde70d82bb65cc2e5dcd0f9d33cda5d38091120b](https://github.com/InferMatrixLab/InferRSI/tree/bde70d82bb65cc2e5dcd0f9d33cda5d38091120b/src/inferrsi/data/skill_library).
Source snapshot: [c5e13944c06a8a03ab7015a8933981a983e42186](https://github.com/InferMatrixLab/InferRSI/tree/c5e13944c06a8a03ab7015a8933981a983e42186/src/inferrsi/data/skill_library).

```bash
inferrsi knowledge search 'profiling' --repo-type inferrsi --limit 50
Expand All @@ -29,7 +29,7 @@ contain the current input paths and bytes (Git checkout filters are respected).

```bash
python tools/build_skill_index.py --source-revision <committed-source-revision>
python tools/build_skill_index.py --source-revision bde70d82bb65cc2e5dcd0f9d33cda5d38091120b --check
python tools/build_skill_index.py --source-revision c5e13944c06a8a03ab7015a8933981a983e42186 --check
```

`--check` detects missing, stale and unexpected generated records without writing.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/bde70d82bb65cc2e5dcd0f9d33cda5d38091120b/src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md",
"commit": "bde70d82bb65cc2e5dcd0f9d33cda5d38091120b",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/c5e13944c06a8a03ab7015a8933981a983e42186/src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md",
"commit": "c5e13944c06a8a03ab7015a8933981a983e42186",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/metadata.json",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/bde70d82bb65cc2e5dcd0f9d33cda5d38091120b/src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md",
"commit": "bde70d82bb65cc2e5dcd0f9d33cda5d38091120b",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/c5e13944c06a8a03ab7015a8933981a983e42186/src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md",
"commit": "c5e13944c06a8a03ab7015a8933981a983e42186",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/metadata.json",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/bde70d82bb65cc2e5dcd0f9d33cda5d38091120b/src/inferrsi/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md",
"commit": "bde70d82bb65cc2e5dcd0f9d33cda5d38091120b",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/c5e13944c06a8a03ab7015a8933981a983e42186/src/inferrsi/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md",
"commit": "c5e13944c06a8a03ab7015a8933981a983e42186",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-distributed-parallel/metadata.json",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/bde70d82bb65cc2e5dcd0f9d33cda5d38091120b/src/inferrsi/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md",
"commit": "bde70d82bb65cc2e5dcd0f9d33cda5d38091120b",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/c5e13944c06a8a03ab7015a8933981a983e42186/src/inferrsi/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md",
"commit": "c5e13944c06a8a03ab7015a8933981a983e42186",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-graph-mode/metadata.json",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/bde70d82bb65cc2e5dcd0f9d33cda5d38091120b/src/inferrsi/data/skill_library/hardware/ascend/ascend-model-adaptation/SKILL.md",
"commit": "bde70d82bb65cc2e5dcd0f9d33cda5d38091120b",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/c5e13944c06a8a03ab7015a8933981a983e42186/src/inferrsi/data/skill_library/hardware/ascend/ascend-model-adaptation/SKILL.md",
"commit": "c5e13944c06a8a03ab7015a8933981a983e42186",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-model-adaptation/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-model-adaptation/metadata.json",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/bde70d82bb65cc2e5dcd0f9d33cda5d38091120b/src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md",
"commit": "bde70d82bb65cc2e5dcd0f9d33cda5d38091120b",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/c5e13944c06a8a03ab7015a8933981a983e42186/src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md",
"commit": "c5e13944c06a8a03ab7015a8933981a983e42186",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-profiling/metadata.json",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/bde70d82bb65cc2e5dcd0f9d33cda5d38091120b/src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-tuning/SKILL.md",
"commit": "bde70d82bb65cc2e5dcd0f9d33cda5d38091120b",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/c5e13944c06a8a03ab7015a8933981a983e42186/src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-tuning/SKILL.md",
"commit": "c5e13944c06a8a03ab7015a8933981a983e42186",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-tuning/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-tuning/metadata.json",
Expand Down
Loading
Loading