Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/reference-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -248,6 +248,7 @@ SSD/OS systems columns.
| JiusiServe/vllm-evolve @ 43cc82c | search | scheduling-algorithm evolution, search policy | `optimization-search-policy`, `dense-performance-evidence`, `inference-benchmark-design` |
| [OpenBMB/JustGoal repo + paired paper](https://github.com/OpenBMB/JustGoal/tree/f052fc0b28fc7f10b2ef1ae7aa4682aabbbe868d) @ `f052fc0b28fc7f10b2ef1ae7aa4682aabbbe868d` (repository Apache-2.0 default; NOTICE BSD exceptions; paper license UNKNOWN) | evaluation | Bounded direct inspection, 2026-11-02: complete 21-page paper/appendices and selected DSA wrapper/CUDA workspace/stream contracts; [exact readset and exclusions](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/justgoal-source.json). Source-only methodology; author-reported comparisons, released failures versus repaired timing and official/stricter oracle differences retained | [inference-benchmark-design reference](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/justgoal-evaluation-contracts.md); candidate/not_run, efficacy not_evaluated; no new implementation/backend coverage or full campaign runner |
| [AutoTuneBench repo + paper v1](https://github.com/li-ch/autotunebench/tree/64cc8d7ca5d0ee40c4e2189b2c2c8dfb1a2a7c2f) @ `64cc8d7ca5d0ee40c4e2189b2c2c8dfb1a2a7c2f`; [arXiv:2609.18123v1](https://arxiv.org/abs/2609.18123v1) (selected repository code/data MIT; paper CC BY 4.0) | evaluation | Bounded source inspection, 2026-11-02: treatment delivery, arm-validity precedence, exported protocol/check/record gaps, weak-denominator and anomaly counterexamples; complete 15-page paper, 6 figures, 7 tables and [exact selected readset/rights](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/autotunebench-source.json) | [inference-benchmark-design reference](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/autotunebench-trial-validity.md); source-derived candidate/not_run, efficacy not_evaluated; invalid-arm/source-median conflicts retained, no new controller or measured implementation coverage |
| [Xe-Forge repo + paper v1](https://github.com/IntelLabs/Xe-Forge/tree/6cd3ee98dfe32746b345a6abe521fa4cb9bdc187) @ `6cd3ee98dfe32746b345a6abe521fa4cb9bdc187`; [arXiv:2605.26118v1](https://arxiv.org/html/2605.26118v1) (repository Apache-2.0; paper CC BY 4.0) | evaluation | Bounded source inspection, 2026-11-02: 18-file trial identity/origin, external outcome, selection/finalization/reporting and baseline-oracle readset; complete paper HTML/appendix/tables/algorithm/listing/captions, unused graphic values unverified; [exact hashes, rights and exclusions](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/xeforge-source.json) | [`inference-benchmark-design` reference](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/xeforge-trial-evidence.md); candidate/not_run, efficacy not_evaluated; two measured meanings, named-verdict update limits and manual versus measured-only defaults retained; no runtime integration or whole-model/backend promotion |
| Dong1017/OmniRSI @ `codex/skill-eval-bootstrap` (82218b4) | evaluation | skill_distill / skill_eval harness, no_skill/human/distilled contract, Tier A pilot | `npu-skill-factory` (evaluation contract) |
| InferMatrixLab/InferRSI PR #9 @ 4ad551a9 | reference | `skill_library/` layout + knowledge records | `npu-skill-factory` (bundle layout) |
| RRSI — arXiv 2609.24972v3 | search; mined via reviewed Pack F | bounded harness proposals; calibrated noise/cost/novelty gates; grouped-ablation limits, replacing the earlier generic constraint summary | `optimization-search-policy`; [exact ledger](../src/inferrsi/data/skill_library/methods/optimization-search-policy/references/papers-20261015/references/source-notes.md#f1--rrsi) |
Expand Down
6 changes: 6 additions & 0 deletions docs/searchspace-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -600,6 +600,12 @@ NOASSERTION and InferRSI's repository license.
| --- | --- | --- | --- |
| Accelerator (methodology only) | Verify: trace treatment to the eligible consumer, apply frozen arm validity before interpreting a null, and separate declared protocol, reached checks and persisted gate evidence | [inference-benchmark-design reference](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/autotunebench-trial-validity.md) and [source record](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/autotunebench-source.json); AutoTuneBench repo + paired paper | Source-derived candidate / empirical not_run; paper measurements remain author-reported, efficacy not_evaluated. Contextual source bars are not InferRSI defaults. No new implementation, measured hardware/resource coverage, backend compatibility, or RAM/Storage/Network/OS coverage. |

## Xe-Forge trial-evidence guidance (source-derived)

| Resource | Stage and mechanism | Source and owner | Evidence boundary |
| --- | --- | --- | --- |
| Accelerator (methodology only) | Verify: bind saved candidate and comparator identities, distinguish tool origin from resolved performance, inspect host gate/ratio persistence, and follow manual versus measured-only delivery and baseline-oracle boundaries | [inference-benchmark-design reference](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/xeforge-trial-evidence.md) and [source record](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/references/xeforge-source.json); Xe-Forge repo + paired paper | Source-derived candidate / empirical not_run; paper/README results remain author-reported, efficacy not_evaluated. No new implementation, measured hardware/resource coverage, backend compatibility, whole-model/service promotion or RAM/Storage/Network/OS coverage; draft extensions remain proposals. |

## TorchNPU framework source procedures

| Resource | Stage / mechanism | Canonical owner | Evidence boundary |
Expand Down
4 changes: 2 additions & 2 deletions docs/skill-coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Generated from the registry, the skill sidecars and the two maintained
matrices. **Do not edit by hand**; regenerate after any registry, skill,
matrix or frozen-plan change.

- Input digest: `sha256:7640b171256091efd1514a33d5e434eef3feaed238fc92fe3eeba02fee94e9ff`
- Input digest: `sha256:0dcdead4b319f996fcf0f1a3125de5f1ae4717ba6c8aa34cb3f1fa6dbf5494f3`
- Regenerate: `python tools/build_skill_coverage.py`
- Verify without writing: `python tools/build_skill_coverage.py --check`

Expand Down Expand Up @@ -131,7 +131,7 @@ optimization is fully covered or that any procedure has measured benefit.
| [choose-parallel-training-plan](../src/inferrsi/data/skill_library/methods/choose-parallel-training-plan/SKILL.md) | common | Compare stage, device-mesh and operator-sharding candidates for static distributed training while accounting for ... | Comparing stage, device-mesh and operator-sharding candidates for static distributed ... | failure_modes, input_contract, produced_artifact, refusal_point<br>3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify<br>4 Network · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Alpa — [2201.12023v3](https://arxiv.org/abs/2201.12023v3)) | candidate · not_run · 1 readset(s) | no-eval |
| [dense-performance-evidence](../src/inferrsi/data/skill_library/methods/dense-performance-evidence/SKILL.md) | common | Taxonomy of performance evidence an optimization agent needs, ordered by collection cost and diagnostic power. Maps ... | Deciding which performance evidence to collect next and which claim each evidence level ... | failure_modes, input_contract, produced_artifact, refusal_point<br>4 advice | 1 Accelerator · Observe / Profile / Verify<br>2 RAM/Host · Verify<br>4 Network · Verify | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)<br>2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 0 readset(s) | — |
| [evaluate-elastic-serving-placement](../src/inferrsi/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md) | common | Compare migrating an LLM request between model replicas with changing its sequence-parallel execution group, including ... | Comparing migration of a running LLM request between compatible model replicas with ... | failure_modes, input_contract, produced_artifact, refusal_point<br>3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify<br>2 RAM/Host · Diagnose / Configure / Optimize / Verify<br>4 Network · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Llumnix — [2406.03243v1](https://arxiv.org/abs/2406.03243v1)) | candidate · not_run · 1 readset(s) | no-eval |
| [inference-benchmark-design](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/SKILL.md) | common | Select task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents ... | Selecting task quality gates, reference parity checks and performance workloads for text ... | failure_modes, input_contract, produced_artifact, refusal_point<br>5 advice | N/A: SKILL.md §§ Select the task and its evaluation contract, Statistical requirements, Shared-machine measurement discipline, and Search integrity and final-state verification design comparable evaluation; hardware identity and allocation isolation are comparison controls, not a hardware-resource mechanism. | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>Task-specific benchmark selection (issue #60) ([Issue #60 report](https://github.com/InferMatrixLab/InferRSI/issues/6 ...)<br>Task-specific benchmark selection (issue #60) ([NVIDIA/NeMo-Speech.cpp @ 20999cc9](https://github.com/NVIDIA/NeMo-Spe ...) | candidate · not_run · 7 readset(s) | — |
| [inference-benchmark-design](../src/inferrsi/data/skill_library/methods/inference-benchmark-design/SKILL.md) | common | Select task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents ... | Selecting task quality gates, reference parity checks and performance workloads for text ... | failure_modes, input_contract, produced_artifact, refusal_point<br>6 advice | N/A: SKILL.md §§ Select the task and its evaluation contract, Statistical requirements, Shared-machine measurement discipline, and Search integrity and final-state verification design comparable evaluation; hardware identity and allocation isolation are comparison controls, not a hardware-resource mechanism. | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references ([Xe-Forge repo + paper v1](https://github.com/IntelLabs/Xe-Forge/tree/ ...)<br>Task-specific benchmark selection (issue #60) ([Issue #60 report](https://github.com/InferMatrixLab/InferRSI/issues/6 ...)<br>Task-specific benchmark selection (issue #60) ([NVIDIA/NeMo-Speech.cpp @ 20999cc9](https://github.com/NVIDIA/NeMo-Spe ...) | candidate · not_run · 9 readset(s) | — |
| [inference-optimization-casebook](../src/inferrsi/data/skill_library/methods/inference-optimization-casebook/SKILL.md) | common | How to structure and use a casebook of past optimization decisions as a decision-support tool. Extracts the casebook ... | Structuring and querying a casebook of past optimization decisions as decision support ... | failure_modes, input_contract, produced_artifact, refusal_point<br>4 advice | 1 Accelerator · Optimize | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd) | candidate · not_run · 0 readset(s) | no-eval |
| [kernel-writing-guide](../src/inferrsi/data/skill_library/methods/kernel-writing-guide/SKILL.md) | common | Decision guide for writing custom inference kernels: when to write one, which framework to choose (Triton, CuTe DSL ... | Deciding whether to write a custom inference kernel, routing a kernel pattern to Triton ... | failure_modes, input_contract, produced_artifact, refusal_point<br>6 advice | 1 Accelerator · Optimize / Diagnose / Configure / Verify | 1 · Systems sources (Dao-AILab/causal-conv1d @ `cd81f0413cad2fc1e6f17e785ac39f59aae690cd` ...)<br>1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)<br>1 · Systems sources (huggingface/transformers @ `0dad7b822255a0ae261ec45ae937371e859ffd1a` ...)<br>1 · Systems sources (mindspore-ai/akg @ `5aa15f3fe4d856eb60cd3b27447e04f5f1fb4dff` ...)<br>1 · Systems sources (sgl-project/sglang @ `16176278d0180f1a1a578e53e6eef432577876c2` ...)<br>1 · Systems sources (vllm-project/vllm @ `537a79f0fc5ba564d2bef43d54915b2f80f9f77d` (root ...) | candidate · not_run · 7 readset(s) | no-eval |
| [optimization-search-policy](../src/inferrsi/data/skill_library/methods/optimization-search-policy/SKILL.md) | common | Policy for how an optimization agent should select, order, and stop optimization experiments. Covers candidate ... | Selecting, ordering and stopping optimization experiments: candidate generation ... | failure_modes, input_contract, produced_artifact, refusal_point<br>4 advice | 1 Accelerator · Optimize | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references (RRSI — arXiv 2609.24972v3)<br>2 · Methodology and corpus references (arXiv 2608.01867 (CRISP))<br>2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 9 readset(s) | no-eval |
Expand Down
4 changes: 2 additions & 2 deletions src/inferrsi/data/repo_packs/inferrsi/knowledge/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ These are knowledge abstracts (`--kind knowledge`), with namespaced package
IDs distinct from explicitly selected `checkout_ref` skills. Ascend/CUDA
procedures use their corresponding backend; shared methods use `common`.

Source snapshot: [7aa3344bf84c42467f43b3a927e1c7d13642bd55](https://github.com/InferMatrixLab/InferRSI/tree/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library).
Source snapshot: [bb4720992ac2a515adc09b645e1e3cfda47e9d69](https://github.com/InferMatrixLab/InferRSI/tree/bb4720992ac2a515adc09b645e1e3cfda47e9d69/src/inferrsi/data/skill_library).

```bash
inferrsi knowledge search 'profiling' --repo-type inferrsi --limit 50
Expand All @@ -29,7 +29,7 @@ contain the current input paths and bytes (Git checkout filters are respected).

```bash
python tools/build_skill_index.py --source-revision <committed-source-revision>
python tools/build_skill_index.py --source-revision 7aa3344bf84c42467f43b3a927e1c7d13642bd55 --check
python tools/build_skill_index.py --source-revision bb4720992ac2a515adc09b645e1e3cfda47e9d69 --check
```

`--check` detects missing, stale and unexpected generated records without writing.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md",
"commit": "7aa3344bf84c42467f43b3a927e1c7d13642bd55",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/bb4720992ac2a515adc09b645e1e3cfda47e9d69/src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md",
"commit": "bb4720992ac2a515adc09b645e1e3cfda47e9d69",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-aclgraph-capture-replay/metadata.json",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md",
"commit": "7aa3344bf84c42467f43b3a927e1c7d13642bd55",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/bb4720992ac2a515adc09b645e1e3cfda47e9d69/src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md",
"commit": "bb4720992ac2a515adc09b645e1e3cfda47e9d69",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-device-health/metadata.json",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md",
"commit": "7aa3344bf84c42467f43b3a927e1c7d13642bd55",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/bb4720992ac2a515adc09b645e1e3cfda47e9d69/src/inferrsi/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md",
"commit": "bb4720992ac2a515adc09b645e1e3cfda47e9d69",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-distributed-parallel/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-distributed-parallel/metadata.json",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md",
"commit": "7aa3344bf84c42467f43b3a927e1c7d13642bd55",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/bb4720992ac2a515adc09b645e1e3cfda47e9d69/src/inferrsi/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md",
"commit": "bb4720992ac2a515adc09b645e1e3cfda47e9d69",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-graph-mode/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-graph-mode/metadata.json",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library/hardware/ascend/ascend-model-adaptation/SKILL.md",
"commit": "7aa3344bf84c42467f43b3a927e1c7d13642bd55",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/bb4720992ac2a515adc09b645e1e3cfda47e9d69/src/inferrsi/data/skill_library/hardware/ascend/ascend-model-adaptation/SKILL.md",
"commit": "bb4720992ac2a515adc09b645e1e3cfda47e9d69",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-model-adaptation/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-model-adaptation/metadata.json",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md",
"commit": "7aa3344bf84c42467f43b3a927e1c7d13642bd55",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/bb4720992ac2a515adc09b645e1e3cfda47e9d69/src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md",
"commit": "bb4720992ac2a515adc09b645e1e3cfda47e9d69",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-profiling/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-profiling/metadata.json",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@
"sources": [
{
"type": "git",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/7aa3344bf84c42467f43b3a927e1c7d13642bd55/src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-tuning/SKILL.md",
"commit": "7aa3344bf84c42467f43b3a927e1c7d13642bd55",
"url": "https://github.com/InferMatrixLab/InferRSI/blob/bb4720992ac2a515adc09b645e1e3cfda47e9d69/src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-tuning/SKILL.md",
"commit": "bb4720992ac2a515adc09b645e1e3cfda47e9d69",
"paths": [
"src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-tuning/SKILL.md",
"src/inferrsi/data/skill_library/hardware/ascend/ascend-operator-tuning/metadata.json",
Expand Down
Loading
Loading