diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/README.md b/evidence/h3-ref2va-910c-4logical-20261101-297/README.md new file mode 100644 index 000000000..54b3f6c4d --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/README.md @@ -0,0 +1,50 @@ +# MiniMax-H3 Ref2VA: 910C four-logical-device campaign + +Governing acceptance issue: [InferRSI #297](https://github.com/InferMatrixLab/InferRSI/issues/297). +Accountable owner: the requester's Codex H3 session, operating under Dong1017. +Claim and plan: [owner receipt](https://github.com/InferMatrixLab/InferRSI/issues/297#issuecomment-6555458151). +Final campaign acceptance remains with the requester. + +**Status: Draft tracking snapshot; campaign verdict INCONCLUSIVE.** +No accepted 50% complete-HTTP latency reduction exists. All four final tracks +remain incomplete. C1 passed 439 strict checks on each fixed 5 s/15 s request; +one clean 5 s screening pair measured 291.248→286.667 s, a 1.5729% reduction, +below the unchanged 50% bar. This is not final confirmation. + +C0–C23 cover 24 charged direction families. The tested fast lossy profiles have +observed frozen quality failures. Most recently, C21/C22/C23 completed 5s-00 +but failed seven/nine/eight quality checks. C23 V2 passed 44 native tests and +reduced a static 50-call attention cycle by 66.46%; its full 5 s diagnostic HTTP +was 199.065 s. These observations do not qualify model quality or HTTP gains. +No 15 s or clean timing extension follows for those rejected profiles. + +Start with the [experiment report and 24-direction ledger](results/report.md), +[four-track execution snapshot](plan/execution-checkpoint.json), and +[replay/availability notes](plan/reproduction.md). The inventory's publication +hash map verifies available files; it does not qualify device or model behavior. + +Downstream code is tracked in [vLLM-Omni Draft PR #9745](https://github.com/vllm-project/vllm-omni/pull/9745). +It ports only C1 to main, preserving newer piecewise/query-range dispatch, and +passed [12 source-bound native tests](results/c1-upstream-main-native.json). +The historical full-model C1 results remain tied to `be319692`; full-model +quality and clean HTTP have not been rerun on the new upstream base. + +The scope is 5s/15s × lossless/lossy, 1344×768 at 24 FPS, 124/362 frames, +original BF16 dense Ref2VA, 50 steps, four logical 910C devices. This is separate +from the eight-device 910B1 campaign in #294 and the original eight-device requirement. + +- `plan/`: execution identity, original budget, protocol and frozen contract bytes. +- `results/`: scoped numerical results, rejected operator variants and automatic quality comparisons. +- `patch/`: tested private C1 and C10–C23 identities and their exact engine patches. +- `validator/`: byte-preserved validator/scorer snapshots and checksums. + +The original B0 is retained. Migration and candidate sources are separate. +Diagnostic HTTP values, operator improvements, unit tests and numerical equality +are distinct conclusions. Missing quality samples remain incomplete; failures are +not removed because other samples pass. The heldout clips have not been generated. +See [execution checkpoint](plan/execution-checkpoint.json) and [results](results/README.md). + +Private connection details and original raw captures stay outside this repository. +This Draft tracking snapshot contains sanitized projections and engine patch +artifacts. Independent review, four-track acceptance and production qualification +remain incomplete. The inventory registration checks layout/bytes only. diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/README.md b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/README.md new file mode 100644 index 000000000..9e7ae15bc --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/README.md @@ -0,0 +1,70 @@ +# Patch status + +[C1 patch](c1-h3-bf16-prefix.patch) changes exactly three H3 source/test files. +The [identity record](c1-source-identity.json) binds the tested private commit, +parent, Git tree and file hashes. The engine commit remains private/unmerged; +its diff is published here as an experiment artifact. Native focused +tests passed (10 tests). Two complete fixed requests per duration passed strict +comparison for both 5 s and 15 s, 439 checks each. The 15 s audit also recorded +2,500 actual prefix-attention calls per rank per request. Clean performance +screening measured a 1.5729% reduction at5s and failed the50% performance bar. +Final qualification remains pending; no50% HTTP gain is accepted. + +The H3-owned override handles the explicit long-query BF16 fused selection with +independent BNSD head batches and valid K/V-prefix slicing. Default and unsupported +paths retain their parent behavior. It adds no generic backend API, MindIE binary +or common environment change. Other failed candidates are retained in results; +their existence does not establish upstream support. Any eventual engine PR +needs current author checks and the upstream human-review/AI-disclosure gates. + +[C10 patch](c10-h3-audio-cache-guard.patch) changes four H3 source/test files. +Its [identity](c10-source-identity.json) binds private commit `0af99184` to the +native-tested bytes. Fifteen focused tests passed, including native NPU execution. +The first full5s-01 request had no failures among39 observed gates. Independent +5s-00 failed CLAP and audio-caption gates; the original C10 configuration is +rejected for5s. Diagnostic HTTP was236.9/243.0s, with no clean performance claim. +The second audit directly recorded50 actual wrapper forwards per rank and +42checks/27vetoes agreeing across all4ranks. The first audit's missing runtime +counters remain a historical evidence gap. No public API, dependency or frozen +quality-gate change is included. + +## C12–C23 source artifacts + +Each C12/C13/C15–C23 `*-source-identity.json` records its tested engine head, +parent, tree, raw file hashes and `patch_base`. The matching `*-candidate.patch` +is cumulative over C1 `be319692b96413918973b5bd399cdda3f320c574`; apply C1 +first, then one selected cumulative patch. These branches are alternatives. +`published: false` in an archived identity describes the engine commit's status +at capture, not this evidence PR's publication state. + +C13 adapts [vLLM-Omni PR #9351](https://github.com/vllm-project/vllm-omni/pull/9351) +ordering to the pinned chunked DLO implementation, retaining current manifests +and double staging slots. C15–C23 are local experiments; their native tests and +primitive results do not establish upstream support or complete model quality. +C14 is a prerequisite probe and has no adopted engine patch. + +[The experiment report](../results/report.md) records dispositions and limits. +In particular, C21/C22/C23 failed frozen quality and are not promoted for use. +The separate [vLLM-Omni Draft PR #9745](https://github.com/vllm-project/vllm-omni/pull/9745) +ports C1 to main; [its native checks](../results/c1-upstream-main-native.json) +passed 12 tests. New-base full-model/HTTP qualification remains pending. These +historical patch files are unchanged and retain their original tested revisions. +No MindIE-SD code contribution is included. +MindIE-SD source, installed provider binaries and shared environments were not +modified. Future upstream engine contributions require their own focused checks +and review. [Source reconstruction notes](../plan/reproduction.md) state the +external artifacts needed for replay. + + +[C11 dual-sigma patch](c11-modality-sigma-residual.patch) changes five H3 paths. +Its [identity](c11-source-identity.json) binds61b8efa and20 native-focused tests. +Fn1/Bn0 failed5s-00 audio adherence; Fn1/Bn1 failed its frozen audio-level gate. +These are rejected configurations, not production acceleration claims. + +[FP32 residual revision](c11-sigma-fp32-residual.patch) preserves the same +five-path overall boundary and adds a two-file delta over61b8efa. Its +[identity](c11-fp32-source-identity.json) bindsd4d25b4 and23 passed tests, +including three executed NPU tests. The native attempt projection retains two +test-reference failures and the later correction. The production bytes stayed +unchanged across these native attempts. Full5s-00 and15s-00 still failed frozen +quality guards; the patch is not promoted. No frozen scorer/threshold changed. diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c1-h3-bf16-prefix.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c1-h3-bf16-prefix.patch new file mode 100644 index 000000000..2d89028ec --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c1-h3-bf16-prefix.patch @@ -0,0 +1,200 @@ +diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_flash_attention.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_flash_attention.py +new file mode 100644 +index 0000000..b805402 +--- /dev/null ++++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_flash_attention.py +@@ -0,0 +1,91 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++import pytest ++import torch ++ ++from tests.helpers.mark import hardware_marks ++from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata ++from vllm_omni.diffusion.attention.backends.flash_attn import FlashAttentionImpl ++from vllm_omni.diffusion.models.minimax_h3.attention import flash_h3 ++from vllm_omni.platforms import current_omni_platform ++ ++pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=1)] ++ ++ ++@pytest.mark.parametrize( ++ "mode", ["fused", "default", "laser", "causal", "masked", "compact", "gqa", "unresolved", "short"] ++) ++def test_h3_prefix_dispatch_preserves_unselected_paths(monkeypatch, mode): ++ query = torch.arange(16384, dtype=torch.float32).to(torch.bfloat16).reshape(1, 4096, 2, 2) ++ key, value = query + 10, query + 20 ++ calls = [] ++ ++ def parent(self, q, k, v, attn_metadata=None): ++ calls.append("parent") ++ return torch.zeros_like(q) ++ ++ def prefix(q, k, v, scale): ++ calls.append("prefix") ++ assert torch.equal(q, query) ++ assert torch.equal(k, key[:, :4093]) and torch.equal(v, value[:, :4093]) ++ assert scale == 0.17 ++ return torch.ones_like(q) ++ ++ monkeypatch.setattr(FlashAttentionImpl, "forward_fa_npu", parent) ++ monkeypatch.setattr(flash_h3, "_prefix_bnsd", prefix) ++ impl = object.__new__(flash_h3.MiniMaxH3FlashAttentionImpl) ++ impl.causal, impl.softmax_scale = mode == "causal", 0.17 ++ result = None if mode == "unresolved" else ([4093, 4096], [4093, 4096]) ++ monkeypatch.setattr(impl, "_resolve_packed_seq_npu", lambda q, k, extra: result) ++ extra = {"npu_attn_varlen": True} ++ if mode == "compact": ++ extra["minimax_h3_compact_reference_rows"] = 2 ++ mask = torch.ones(1, 4096, dtype=torch.bool) if mode == "masked" else None ++ metadata = AttentionMetadata(attn_mask=mask, extra=extra) ++ if mode == "default": ++ monkeypatch.delenv("MINDIE_SD_FA_TYPE", raising=False) ++ else: ++ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "ascend_laser_attention" if mode == "laser" else "fused_attn_score") ++ actual_query, actual_key, actual_value = query, key[:, :, :1] if mode == "gqa" else key, value ++ if mode == "short": ++ actual_query, actual_key, actual_value = (tensor[:, :128] for tensor in (query, key, value)) ++ output = impl.forward_fa_npu(actual_query, actual_key, actual_value, metadata) ++ assert calls == ["prefix" if mode == "fused" else "parent"] ++ assert torch.equal(output, torch.ones_like(actual_query) if mode == "fused" else torch.zeros_like(actual_query)) ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native Ascend fused attention") ++def test_native_prefix_bnsd_excludes_padding_and_preserves_inputs(monkeypatch): ++ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") ++ torch.manual_seed(19) ++ device = torch.device(current_omni_platform.device_type) ++ query, key, value = [torch.randn(1, 4096, 2, 64, dtype=torch.bfloat16).to(device) for _ in range(3)] ++ original = [tensor.clone() for tensor in (query, key, value)] ++ used, scale = 4093, 0.17 ++ cu = torch.tensor([0, used, 4096], dtype=torch.int32, device=device) ++ metadata = AttentionMetadata( ++ extra={ ++ "npu_attn_varlen": True, ++ "cu_seqlens_q": cu, ++ "cu_seqlens_k": cu, ++ "max_seqlen_q": used, ++ "max_seqlen_k": used, ++ "laser_input_scale": 256.0, ++ } ++ ) ++ impl = flash_h3.MiniMaxH3FlashAttentionImpl(num_heads=2, head_size=64, softmax_scale=scale, qkv_layout="BSND") ++ with torch.inference_mode(): ++ output = impl.forward_fa_npu(query, key, value, metadata) ++ dirty_key, dirty_value = key.clone(), value.clone() ++ dirty_key[:, used:] = 1000 ++ dirty_value[:, used:] = -1000 ++ repeated = impl.forward_fa_npu(query, dirty_key, dirty_value, metadata) ++ q, k, v = query[:, :used].float().cpu(), key[:, :used].float().cpu(), value[:, :used].float().cpu() ++ scores = torch.einsum("bqhd,bkhd->bhqk", q, k) * scale ++ expected = torch.einsum("bhqk,bkhd->bqhd", scores.softmax(dim=-1), v) ++ assert output.is_contiguous() and output.dtype == torch.bfloat16 ++ assert output.shape == query.shape and torch.isfinite(output).all() ++ torch.testing.assert_close(output[:, :used].float().cpu(), expected, rtol=1e-2, atol=2e-3) ++ assert torch.equal(output[:, :used], repeated[:, :used]) ++ assert all(torch.equal(before, after) for before, after in zip(original, (query, key, value))) +diff --git a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py +new file mode 100644 +index 0000000..6ebfb10 +--- /dev/null ++++ b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py +@@ -0,0 +1,76 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++"""Model-owned BF16 prefix attention for H3's explicit NPU fused selection.""" ++ ++from __future__ import annotations ++ ++import os ++ ++import torch ++ ++from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata ++from vllm_omni.diffusion.attention.backends.flash_attn import FlashAttentionImpl ++ ++ ++def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, scale: float) -> torch.Tensor: ++ from mindiesd import attention_forward ++ ++ batch, rows, heads, dim = query.shape ++ ++ def fold(tensor: torch.Tensor) -> torch.Tensor: ++ return tensor.transpose(1, 2).contiguous().reshape(batch * heads, 1, tensor.shape[1], dim) ++ ++ output = attention_forward( ++ fold(query), ++ fold(key), ++ fold(value), ++ attn_mask=None, ++ scale=scale, ++ head_first=True, ++ opt_mode="manual", ++ op_type="fused_attn_score", ++ layout="BNSD", ++ ) ++ # Match the original packed varlen output layout before shared SP restores ++ # local sequence shards. Padding query rows are not consumed by H3. ++ return output.reshape(batch, heads, rows, dim).transpose(1, 2).contiguous() ++ ++ ++class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): ++ """Use independent head batches for explicit long BF16 prefix attention. ++ ++ Default and other operator selections preserve their parent behavior. ++ Mathematical equivalence alone does not promise bit-exact model outputs; ++ each numerical qualification must cover the actual workload. ++ """ ++ ++ def forward_fa_npu( ++ self, ++ query: torch.Tensor, ++ key: torch.Tensor, ++ value: torch.Tensor, ++ attn_metadata: AttentionMetadata | None = None, ++ ) -> torch.Tensor: ++ if os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" or attn_metadata is None: ++ return super().forward_fa_npu(query, key, value, attn_metadata) ++ extra = attn_metadata.extra ++ if ( ++ self.causal ++ # Keep the short text refiners on their original varlen kernels. ++ or query.shape[1] < 4096 ++ or attn_metadata.attn_mask is not None ++ or not extra.get("npu_attn_varlen", False) ++ or extra.get("minimax_h3_compact_reference_rows", 0) ++ or query.dtype != torch.bfloat16 ++ or query.dtype != key.dtype ++ or query.dtype != value.dtype ++ or query.shape[2:] != key.shape[2:] ++ or query.shape[2:] != value.shape[2:] ++ or key.shape[1] != value.shape[1] ++ ): ++ return super().forward_fa_npu(query, key, value, attn_metadata) ++ resolved = self._resolve_packed_seq_npu(query, key, extra) ++ if resolved is None: ++ return super().forward_fa_npu(query, key, value, attn_metadata) ++ used_k = resolved[1][0] ++ return _prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale) +diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py +index 28931c9..09f2e7d 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py ++++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py +@@ -208,6 +208,7 @@ class MiniMaxH3Attention(nn.Module): + self.vdn: VDNH3HybridAttention | None = None + self._prefix = prefix + from .attention.fastvideo_h3 import MiniMaxH3VSAImpl ++ from .attention.flash_h3 import MiniMaxH3FlashAttentionImpl + + self.attention = Attention( + num_heads=self.num_heads, +@@ -221,7 +222,7 @@ class MiniMaxH3Attention(nn.Module): + role_category=role_category, + skip_sequence_parallel=skip_sequence_parallel, + prefix=prefix, +- impl_overrides={"FASTVIDEO_VSA": MiniMaxH3VSAImpl}, ++ impl_overrides={"FASTVIDEO_VSA": MiniMaxH3VSAImpl, "FLASH_ATTN": MiniMaxH3FlashAttentionImpl}, + ) + + def enable_vsa_gate(self) -> None: diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c1-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c1-source-identity.json new file mode 100644 index 000000000..f3d71d989 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c1-source-identity.json @@ -0,0 +1,12 @@ +{ + "head": "be319692b96413918973b5bd399cdda3f320c574", + "tree": "8dfb3c6edbac7ca3a92edfa9659cc7a7eb58bbc3", + "base": "922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb", + "branch": "rsi/h3-297-bf16-prefix", + "files": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "6e175317c8db76b1e4e1b094a4c43923c8fbe3a5f039405f1cba070772497fe6", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "e7e26ad0b2a9120b99281bf59a67eced7f7207f5679a8f02f2ede5859695be1a", + "tests/diffusion/models/minimax_h3/test_minimax_h3_flash_attention.py": "34fb26fb848556d8a67af91f7d3ee6443e77018e149672aed0450fd970be1968" + }, + "published": false +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c10-h3-audio-cache-guard.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c10-h3-audio-cache-guard.patch new file mode 100644 index 000000000..a92ab501c --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c10-h3-audio-cache-guard.patch @@ -0,0 +1,412 @@ +diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_audio_cache_guard.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_audio_cache_guard.py +new file mode 100644 +index 0000000..c973180 +--- /dev/null ++++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_audio_cache_guard.py +@@ -0,0 +1,185 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++from types import SimpleNamespace ++ ++import pytest ++import torch ++import torch.nn as nn ++ ++from tests.helpers.mark import hardware_marks ++from vllm_omni.diffusion.models.minimax_h3 import cache_audio_guard as guard ++from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache_dit import ( ++ MiniMaxH3CachedAdapter, ++ MiniMaxH3CachedBlocks, ++) ++from vllm_omni.platforms import current_omni_platform ++ ++pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=1)] ++ ++ ++def decision(previous, current, indices, **kwargs): ++ return guard._modality_changes_allow_reuse( ++ previous, ++ current, ++ indices, ++ max_seqlen=kwargs.pop("max_seqlen", current.shape[0]), ++ packed_total=kwargs.pop("packed_total", current.shape[0]), ++ video_limit=kwargs.pop("video_limit", 0.08), ++ parallelized=kwargs.pop("parallelized", False), ++ **kwargs, ++ ) ++ ++ ++def test_audio_change_hidden_by_global_mean_vetoes(): ++ before = torch.ones(128, 8) ++ after = before.clone() ++ after[-2:] = 1.3 ++ indices = torch.zeros(128, dtype=torch.long) ++ indices[-2:] = 17 # timestep-index5, audio-modality2 ++ assert ((before - after).abs().mean() / before.abs().mean()).item() < 0.08 ++ assert not decision(before, after, indices) ++ after[-2:] = 1.02 ++ assert decision(before, after, indices) ++ assert not decision(before, after, indices, video_limit=0.01) ++ ++ ++def test_video_change_and_structural_padding(): ++ before = torch.ones(8, 4) ++ after = before.clone() ++ indices = torch.tensor([0, 0, 2, 2, 0, 0, 0, 0]) ++ after[4:] = float("nan") ++ assert decision(before, after, indices, max_seqlen=4) ++ after[:2] = 1.2 ++ assert not decision(before, after, indices, max_seqlen=4) ++ ++ ++def test_zero_denominator_and_invalid_layout(): ++ before = torch.zeros(4, 2) ++ after = before.clone() ++ indices = torch.full((4,), 2) ++ assert decision(before, after, indices) ++ after[0] = 1 ++ assert not decision(before, after, indices) ++ with pytest.raises(ValueError, match="valid packed prefix"): ++ decision(before, after, indices, packed_total=5) ++ with pytest.raises(ValueError, match="modality index"): ++ decision(before, after, indices[:2]) ++ ++ ++def test_remote_audio_participates_when_local_rank_has_none(monkeypatch): ++ from vllm_omni.diffusion.distributed import parallel_state ++ ++ monkeypatch.setattr(guard.dist, "is_initialized", lambda: True) ++ monkeypatch.setattr(parallel_state, "get_sequence_parallel_rank", lambda: 0) ++ monkeypatch.setattr(parallel_state, "get_sequence_parallel_world_size", lambda: 2) ++ calls = [] ++ ++ def reduce(values, op): ++ assert op == guard.dist.ReduceOp.SUM ++ calls.append(values.clone()) ++ values.add_(torch.tensor([0.0, 2.0, 1.0, 0.8, 2.0, 1.0])) ++ ++ monkeypatch.setattr(guard.dist, "all_reduce", reduce) ++ before = torch.ones(2, 8) ++ assert not decision(before, before, torch.zeros(2, dtype=torch.long), packed_total=4, parallelized=True) ++ assert len(calls) == 1 and calls[0][5].item() == 0 ++ ++ ++@pytest.mark.parametrize("change", ["default", "task", "cache", "refkv", "dp", "cfg"]) ++def test_adapter_selection_preserves_other_modes(change): ++ config = SimpleNamespace( ++ task_type="ref2va", ++ cache_backend="cache_dit", ++ kv_offload_config=None, ++ parallel_config=SimpleNamespace(data_parallel_size=1, cfg_parallel_size=1), ++ ) ++ if change == "task": ++ config.task_type = "t2va" ++ elif change == "cache": ++ config.cache_backend = "none" ++ elif change == "refkv": ++ config.kv_offload_config = {"mode": "on"} ++ elif change == "dp": ++ config.parallel_config.data_parallel_size = 2 ++ elif change == "cfg": ++ config.parallel_config.cfg_parallel_size = 2 ++ assert guard.audio_cache_guard_enabled(config) == (change == "default") ++ assert MiniMaxH3CachedAdapter.block_class is MiniMaxH3CachedBlocks ++ assert guard.MiniMaxH3AudioGuardCachedAdapter.block_class is guard.MiniMaxH3AudioGuardCachedBlocks ++ ++ ++def test_parent_denial_does_not_run_guard(): ++ class Owner: ++ _audio_guard_bypass = False ++ _audio_guard_layout = None ++ _audio_guard_checks = 0 ++ _audio_guard_vetoes = 0 ++ ++ owner = Owner() ++ proxy = guard._AudioCacheManager(SimpleNamespace(can_cache=lambda *args, **kwargs: False), owner) ++ assert proxy.can_cache(torch.ones(4, 2)) is False ++ assert owner._audio_guard_checks == 0 ++ ++ ++def test_allowed_parent_hit_is_vetoed_without_changing_buffers(): ++ class Owner: ++ _audio_guard_bypass = False ++ _audio_guard_checks = 0 ++ _audio_guard_vetoes = 0 ++ ++ previous = torch.ones(128, 8) ++ current = previous.clone() ++ current[-2:] = 1.3 ++ indices = torch.zeros(128, dtype=torch.long) ++ indices[-2:] = 2 ++ owner = Owner() ++ owner._audio_guard_layout = (indices, 128, 128) ++ manager = SimpleNamespace( ++ can_cache=lambda *args, **kwargs: True, ++ get_Fn_buffer=lambda prefix: previous, ++ get_residual_diff_threshold=lambda: 0.08, ++ ) ++ proxy = guard._AudioCacheManager(manager, owner) ++ assert proxy.can_cache(current, prefix="blocks_Fn_residual", parallelized=False) is False ++ assert (owner._audio_guard_checks, owner._audio_guard_vetoes) == (1, 1) ++ current[-2:] = 1.01 ++ assert proxy.can_cache(current, prefix="blocks_Fn_residual", parallelized=False) is True ++ assert (owner._audio_guard_checks, owner._audio_guard_vetoes) == (2, 1) ++ assert torch.equal(previous, torch.ones_like(previous)) ++ owner._audio_guard_bypass, owner._audio_guard_layout = True, None ++ assert proxy.can_cache(current) is True ++ assert owner._audio_guard_checks == 2 ++ ++ ++@pytest.mark.parametrize("failure", ["metadata", "parent"]) ++def test_forward_releases_layout_on_failure(monkeypatch, failure): ++ wrapper = object.__new__(guard.MiniMaxH3AudioGuardCachedBlocks) ++ nn.Module.__init__(wrapper) ++ wrapper._audio_guard_layout, wrapper._audio_guard_bypass = None, False ++ ++ def broken(self, *args, **kwargs): ++ raise RuntimeError("parent failed") ++ ++ monkeypatch.setattr(MiniMaxH3CachedBlocks, "forward", broken) ++ kwargs = ( ++ {} ++ if failure == "metadata" ++ else {"combined_indices": torch.zeros(4, dtype=torch.long), "max_seqlen": 4, "packed_total": 4} ++ ) ++ with pytest.raises((KeyError, RuntimeError)): ++ wrapper.forward(torch.ones(4, 2), **kwargs) ++ assert wrapper._audio_guard_layout is None and wrapper._audio_guard_bypass is False ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native Ascend reductions") ++def test_native_audio_guard_preserves_input_values(): ++ device = torch.device(current_omni_platform.device_type) ++ before = torch.ones(128, 64, dtype=torch.bfloat16, device=device) ++ after = before.clone() ++ indices = torch.zeros(128, dtype=torch.long, device=device) ++ indices[-2:] = 2 ++ after[-2:] = 1.3 ++ saved = [tensor.clone() for tensor in (before, after, indices)] ++ assert not decision(before, after, indices) ++ assert all(torch.equal(x, y) for x, y in zip(saved, (before, after, indices))) +diff --git a/vllm_omni/diffusion/models/minimax_h3/cache_audio_guard.py b/vllm_omni/diffusion/models/minimax_h3/cache_audio_guard.py +new file mode 100644 +index 0000000..7472404 +--- /dev/null ++++ b/vllm_omni/diffusion/models/minimax_h3/cache_audio_guard.py +@@ -0,0 +1,174 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++"""Ref2VA-owned veto for cache hits with a changing audio modality.""" ++ ++from __future__ import annotations ++ ++import math ++from typing import Any ++from weakref import ref ++ ++import torch ++import torch.distributed as dist ++ ++from .reference_kv_cache_dit import MiniMaxH3CachedAdapter, MiniMaxH3CachedBlocks ++ ++_AUDIO_REUSE_LIMIT = 0.04 ++ ++ ++def audio_cache_guard_enabled(config: Any) -> bool: ++ parallel = getattr(config, "parallel_config", None) ++ return ( ++ getattr(config, "task_type", None) == "ref2va" ++ and getattr(config, "cache_backend", None) == "cache_dit" ++ and getattr(config, "kv_offload_config", None) is None ++ and (getattr(parallel, "data_parallel_size", 1) or 1) == 1 ++ and (getattr(parallel, "cfg_parallel_size", 1) or 1) == 1 ++ ) ++ ++ ++def _modality_changes_allow_reuse( ++ previous: torch.Tensor, ++ current: torch.Tensor, ++ combined_indices: torch.Tensor, ++ *, ++ max_seqlen: int, ++ packed_total: int, ++ video_limit: float, ++ parallelized: bool, ++) -> bool: ++ if previous.shape != current.shape or current.ndim != 2: ++ raise ValueError("H3 audio cache guard requires matching [local_rows, hidden] states") ++ if combined_indices.ndim != 1 or combined_indices.shape[0] != current.shape[0]: ++ raise ValueError("H3 audio cache guard requires one modality index per local row") ++ if previous.device != current.device or combined_indices.device != current.device: ++ raise ValueError("H3 audio cache guard state and modality devices must match") ++ if not math.isfinite(video_limit) or video_limit <= 0: ++ return False ++ ++ sp_rank, sp_size = 0, 1 ++ if dist.is_initialized(): ++ from vllm_omni.diffusion.distributed.parallel_state import ( ++ get_sequence_parallel_rank, ++ get_sequence_parallel_world_size, ++ ) ++ ++ sp_rank, sp_size = get_sequence_parallel_rank(), get_sequence_parallel_world_size() ++ rows = current.shape[0] ++ if packed_total != rows * sp_size or not 0 < max_seqlen <= packed_total: ++ raise ValueError("H3 audio cache guard requires equal SP shards and a valid packed prefix") ++ ++ # Padding tags were clamped before combined_indices was built. Exclude the ++ # structural suffix using positions rather than interpreting those tags. ++ valid = torch.arange(rows, device=current.device) + sp_rank * rows < max_seqlen ++ modalities = combined_indices.remainder(3) ++ before, after = previous.float(), current.float() ++ delta = (before - after).abs().mean(dim=-1) ++ magnitude = before.abs().mean(dim=-1) ++ scalars = [] ++ for modality in (0, 2): # Video and audio; the parent still checks all tokens. ++ mask = valid & (modalities == modality) ++ scalars.extend( ++ [ ++ torch.where(mask, delta, 0.0).sum(), ++ torch.where(mask, magnitude, 0.0).sum(), ++ mask.sum(dtype=torch.float32), ++ ] ++ ) ++ totals = torch.stack(scalars) ++ if parallelized: ++ if not dist.is_initialized(): ++ raise RuntimeError("Parallel H3 audio cache guard requires an initialized process group") ++ # Every rank participates, including ranks with no local audio rows. ++ # TP replicas duplicate both sums equally, so their ratio is preserved. ++ dist.all_reduce(totals, op=dist.ReduceOp.SUM) ++ values = totals.cpu().tolist() ++ for start, limit in ((0, video_limit), (3, min(video_limit, _AUDIO_REUSE_LIMIT))): ++ numerator, denominator, count = values[start : start + 3] ++ if count == 0: ++ continue ++ if not math.isfinite(numerator) or not math.isfinite(denominator): ++ return False ++ if denominator == 0: ++ if numerator != 0: ++ return False ++ elif numerator / denominator > limit: ++ return False ++ return True ++ ++ ++class _AudioCacheManager: ++ def __init__(self, manager: Any, owner: MiniMaxH3AudioGuardCachedBlocks) -> None: ++ self._manager = manager ++ self._owner = ref(owner) ++ ++ def __getattr__(self, name: str) -> Any: ++ return getattr(self._manager, name) ++ ++ def can_cache(self, states_tensor: torch.Tensor, *args, **kwargs) -> bool: ++ allowed = self._manager.can_cache(states_tensor, *args, **kwargs) ++ owner = self._owner() ++ if not allowed or owner is None or owner._audio_guard_bypass: ++ return allowed ++ if args: ++ raise TypeError("H3 audio cache guard expects named cache-decision options") ++ layout = owner._audio_guard_layout ++ if layout is None: ++ raise RuntimeError("H3 audio cache guard has no active block layout") ++ prefix = kwargs.get("prefix", "Fn") ++ previous = self._manager.get_Fn_buffer(prefix) ++ if previous is None: ++ return False ++ threshold = kwargs.get("threshold") ++ if threshold is None or threshold <= 0: ++ threshold = self._manager.get_residual_diff_threshold() ++ owner._audio_guard_checks += 1 ++ accepted = _modality_changes_allow_reuse( ++ previous, ++ states_tensor, ++ layout[0], ++ max_seqlen=layout[1], ++ packed_total=layout[2], ++ video_limit=float(threshold), ++ parallelized=kwargs.get("parallelized", False), ++ ) ++ owner._audio_guard_vetoes += int(not accepted) ++ return accepted ++ ++ ++class MiniMaxH3AudioGuardCachedBlocks(MiniMaxH3CachedBlocks): ++ def __init__(self, *args, **kwargs) -> None: ++ super().__init__(*args, **kwargs) ++ self._audio_guard_layout = None ++ self._audio_guard_bypass = False ++ self._audio_guard_checks = 0 ++ self._audio_guard_vetoes = 0 ++ self.context_manager = _AudioCacheManager(self.context_manager, self) ++ ++ def audio_cache_guard_stats(self) -> dict[str, int | float]: ++ """Diagnostic counters for this wrapper's lifetime, not quality metrics.""" ++ return { ++ "checks": self._audio_guard_checks, ++ "vetoes": self._audio_guard_vetoes, ++ "audio_reuse_limit": _AUDIO_REUSE_LIMIT, ++ } ++ ++ def forward(self, hidden_states: torch.Tensor, *args, **kwargs): ++ old_layout, old_bypass = self._audio_guard_layout, self._audio_guard_bypass ++ try: ++ self._audio_guard_bypass = ( ++ kwargs.get("num_requests", 1) != 1 or kwargs.get("reference_kv_state") is not None ++ ) ++ self._audio_guard_layout = ( ++ None ++ if self._audio_guard_bypass ++ else (kwargs["combined_indices"], kwargs["max_seqlen"], kwargs["packed_total"]) ++ ) ++ return super().forward(hidden_states, *args, **kwargs) ++ finally: ++ # Do not retain request metadata in a persistent cached model. ++ self._audio_guard_layout, self._audio_guard_bypass = old_layout, old_bypass ++ ++ ++class MiniMaxH3AudioGuardCachedAdapter(MiniMaxH3CachedAdapter): ++ block_class = MiniMaxH3AudioGuardCachedBlocks +diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py +index 9518828..1669757 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py ++++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py +@@ -784,6 +784,14 @@ class MiniMaxH3DiTModel(nn.Module): + self._cache_dit_adapter_config, + cached_adapter_cls=MiniMaxH3CachedAdapter, + ) ++ elif getattr(od_config, "cache_backend", None) == "cache_dit": ++ from .cache_audio_guard import MiniMaxH3AudioGuardCachedAdapter, audio_cache_guard_enabled ++ ++ if audio_cache_guard_enabled(od_config): ++ self._cache_dit_adapter_config = replace( ++ self._cache_dit_adapter_config, ++ cached_adapter_cls=MiniMaxH3AudioGuardCachedAdapter, ++ ) + self.parallel_config = od_config.parallel_config + self.hidden_size = arch.hidden_size + self.num_attention_heads = arch.num_attention_heads +diff --git a/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py b/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py +index f14dcc9..f78598b 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py ++++ b/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py +@@ -125,6 +125,8 @@ class MiniMaxH3CachedBlocks(CachedBlocks_Pattern_3_4_5): + + + class MiniMaxH3CachedAdapter(CachedAdapter): ++ block_class = MiniMaxH3CachedBlocks ++ + @classmethod + def collect_unified_blocks( + cls, block_adapter: BlockAdapter, contexts_kwargs: list[dict] +@@ -139,7 +141,7 @@ class MiniMaxH3CachedAdapter(CachedAdapter): + context_index += 1 + contexts[block_adapter.unique_blocks_name[i][j]] = nn.ModuleList( + [ +- MiniMaxH3CachedBlocks( ++ cls.block_class( + blocks, + transformer=transformer, + forward_pattern=block_adapter.forward_pattern[i][j], diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c10-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c10-source-identity.json new file mode 100644 index 000000000..2fa4f956f --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c10-source-identity.json @@ -0,0 +1,13 @@ +{ + "head": "0af99184bc860cffa8d1e5343c2bff4664f667c5", + "tree": "3e3971e6a1c63e70116cef7dbcb5da2678ff514f", + "base": "922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb", + "branch": "rsi/h3-297-audio-cache-guard", + "files": { + "vllm_omni/diffusion/models/minimax_h3/cache_audio_guard.py": "2844e749c9c876e52bffeef565f2c69904693e2d62c359da6abe66844930daed", + "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "d3381079ff012f32176a917bb0825d12288c058d3a058f5169ecc97d1e2c675c", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "1d5e806489de94cec2f9224f7d31ff96d9ace0a3fb80c23d8f8ba25f262d275d", + "tests/diffusion/models/minimax_h3/test_minimax_h3_audio_cache_guard.py": "c13e1ec677531457e92853d3a97e1e3fdcccb9547883ef0088f75852473d523d" + }, + "published": false +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-fp32-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-fp32-source-identity.json new file mode 100644 index 000000000..54eb936e1 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-fp32-source-identity.json @@ -0,0 +1,24 @@ +{ + "parent": "61b8efa06e88372bc5ab87b9e2668a133eaac282", + "cumulative_patch_base": "922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb", + "head": "d4d25b420889965ab42459c96ab4a82389c9fa64", + "tree": "a1aa28529bd71b733914d82a80aa76e12f5a76f7", + "files": { + "vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py": "fb27fdebdf0cb45e678243c0180d56cf3e7a7470ff424bdf2d3bf144c8e2aee3", + "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "d3381079ff012f32176a917bb0825d12288c058d3a058f5169ecc97d1e2c675c", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a649a09e5b59256b5887c12fe508d8e14e3dedd9019bdb82c65db2d59f11de91", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "a9524734a4f3410a0c6f766fa64fa9e5bd5a4d41bd7481738e9024cac54ce7f9", + "tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py": "16d1e9329c0088eabf4bb17214fc364ec0c6176d92f871786456a2b92aaa418f" + }, + "native_tests_passed": 23, + "native_test_names": [ + "test_native_two_clock_forecast_and_input_immutability", + "test_native_real_cache_manager_computes_then_forecasts", + "test_native_bf16_blocks_use_fp32_anchors_and_keep_output_dtype" + ], + "cleanup_passed": true, + "source_unchanged": true, + "published": false, + "full_model_quality": "5s-00 FAIL adherence;15s-00 FAIL five visual/audio guards", + "performance_acceptance": "PENDING" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-modality-sigma-residual.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-modality-sigma-residual.patch new file mode 100644 index 000000000..18c25cfdb --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-modality-sigma-residual.patch @@ -0,0 +1,620 @@ +diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py +new file mode 100644 +index 0000000..c5a91cc +--- /dev/null ++++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py +@@ -0,0 +1,273 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++from types import SimpleNamespace ++ ++import pytest ++import torch ++import torch.nn as nn ++ ++from tests.helpers.mark import hardware_marks ++from vllm_omni.diffusion.models.minimax_h3 import cache_sigma_prediction as sigma ++from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache_dit import MiniMaxH3CachedBlocks ++from vllm_omni.platforms import current_omni_platform ++ ++pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=1)] ++ ++ ++def config(**changes): ++ values = dict( ++ cache_config={"minimax_h3_sigma_cache": True, "enable_taylorseer": False}, ++ task_type="ref2va", ++ cache_backend="cache_dit", ++ kv_offload_config=None, ++ parallel_config=SimpleNamespace(data_parallel_size=1, cfg_parallel_size=1), ++ ) ++ values.update(changes) ++ return SimpleNamespace(**values) ++ ++ ++def sample(video, audio, *, device="cpu", dtype=torch.float32): ++ return torch.tensor([[1 + 2 * video + video**2], [4 - audio + 3 * audio**2]], device=device, dtype=dtype) ++ ++ ++def anchored(*, device="cpu", dtype=torch.float32): ++ state = sigma.MiniMaxH3SigmaCacheState() ++ originals = [] ++ for step, (video, audio) in enumerate([(1.0, 1.0), (0.9, 0.8), (0.7, 0.5)]): ++ state.begin_step(step, video, audio) ++ residual = sample(video, audio, device=device, dtype=dtype) ++ state.record(residual) ++ originals.append(residual) ++ state.begin_step(3, 0.5, 0.3) ++ return state, originals ++ ++ ++def test_distinct_nonuniform_clocks_recover_quadratic_and_own_anchors(): ++ state, originals = anchored() ++ indices = torch.tensor([0, 17]) ++ expected = sample(0.5, 0.3) ++ assert state.can_predict(expected) ++ torch.testing.assert_close(state.predict(indices), expected) ++ saved = [value.clone() for _, value in state.history] ++ originals[-1].fill_(99) ++ assert all(torch.equal(old, value) for old, (_, value) in zip(saved, state.history)) ++ torch.testing.assert_close(state.predict(indices), expected) ++ # One shared video clock gives the wrong audio result on this fixture. ++ video_weights = sigma._quadratic_weights([1.0, 0.9, 0.7], 0.5) ++ wrong_audio = sum(weight * value[1] for weight, (_, value) in zip(video_weights, state.history)) ++ assert not torch.allclose(wrong_audio, expected[1]) ++ assert state.stats() == {"computed": 3, "predicted": 2, "history_vetoes": 0} ++ state.close() ++ assert state.closed and not state.history and not state.can_predict(expected) ++ ++ ++@pytest.mark.parametrize( ++ "anchors,target", ++ [([1.0, 0.8], 0.7), ([1.0, 0.8, 0.8], 0.7), ([1.0, 0.9, 0.8], 0.1), ([1.0, 0.9, 0.8], float("nan"))], ++) ++def test_unstable_or_insufficient_history_vetoes(anchors, target): ++ assert sigma._quadratic_weights(anchors, target) is None ++ ++ ++@pytest.mark.parametrize( ++ "changes", ++ [ ++ {"task_type": "t2va"}, ++ {"cache_backend": "none"}, ++ {"kv_offload_config": {}}, ++ {"parallel_config": SimpleNamespace(data_parallel_size=2)}, ++ {"cache_config": {"minimax_h3_sigma_cache": 1}}, ++ {"cache_config": {"minimax_h3_sigma_cache": True, "enable_taylorseer": True}}, ++ ], ++) ++def test_explicit_unsupported_selection_fails(changes): ++ with pytest.raises(ValueError): ++ sigma.sigma_cache_enabled(config(**changes)) ++ ++ ++def test_default_and_dataclass_extra_selection(): ++ from vllm_omni.diffusion.data import DiffusionCacheConfig ++ ++ assert not sigma.sigma_cache_enabled(config(cache_config={})) ++ cache = DiffusionCacheConfig.from_dict({"minimax_h3_sigma_cache": True}) ++ assert sigma.sigma_cache_enabled(config(cache_config=cache)) ++ assert sigma.MiniMaxH3SigmaCachedAdapter.block_class is sigma.MiniMaxH3SigmaCachedBlocks ++ ++ ++def test_parent_denial_and_request_prediction_do_not_mutate_inputs(): ++ state, _ = anchored() ++ owner = SimpleOwner(state, torch.tensor([0, 2])) ++ calls = [] ++ manager = SimpleNamespace( ++ can_cache=lambda *args, **kwargs: False, ++ is_cache_residual=lambda: True, ++ set_Bn_buffer=lambda *args, **kwargs: calls.append((args, kwargs)), ++ ) ++ proxy = sigma._SigmaCacheManager(manager, owner) ++ hidden = torch.ones(2, 1) ++ assert not proxy.can_cache(hidden) ++ assert state.history_vetoes == 0 ++ manager.can_cache = lambda *args, **kwargs: True ++ assert proxy.can_cache(hidden) ++ actual, encoder = proxy.apply_cache(hidden, prefix="blocks_Bn_residual") ++ torch.testing.assert_close(actual, sample(0.5, 0.3) + 1) ++ assert encoder is None and torch.equal(hidden, torch.ones_like(hidden)) ++ owner._sigma_state = sigma.MiniMaxH3SigmaCacheState() ++ assert not proxy.can_cache(hidden) ++ assert owner._sigma_state.history_vetoes == 1 ++ ++ ++class SimpleOwner: ++ def __init__(self, state, indices): ++ self._sigma_state, self._sigma_indices = state, indices ++ ++ ++@pytest.mark.parametrize("failure", [False, True]) ++def test_wrapper_releases_transient_request_metadata(monkeypatch, failure): ++ wrapper = object.__new__(sigma.MiniMaxH3SigmaCachedBlocks) ++ nn.Module.__init__(wrapper) ++ wrapper._sigma_state, wrapper._sigma_indices = None, None ++ state = sigma.MiniMaxH3SigmaCacheState() ++ ++ def parent(self, hidden, *args, **kwargs): ++ assert "sigma_cache_state" not in kwargs ++ assert self._sigma_state is state ++ if failure: ++ raise RuntimeError("block failed") ++ return hidden ++ ++ monkeypatch.setattr(MiniMaxH3CachedBlocks, "forward", parent) ++ args = dict(sigma_cache_state=state, combined_indices=torch.tensor([0, 2])) ++ if failure: ++ with pytest.raises(RuntimeError, match="block failed"): ++ wrapper.forward(torch.ones(2, 1), **args) ++ else: ++ wrapper.forward(torch.ones(2, 1), **args) ++ assert wrapper._sigma_state is None and wrapper._sigma_indices is None ++ ++ ++@pytest.mark.parametrize("failure", ["success", "model", "cancel"]) ++def test_denoise_request_closes_history_on_all_exit_paths(monkeypatch, failure): ++ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop ++ ++ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) ++ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) ++ state = sigma.MiniMaxH3SigmaCacheState() ++ ++ class Model: ++ def create_sigma_cache_state(self): ++ return state ++ ++ def __call__(self, **kwargs): ++ assert kwargs["sigma_cache_state"] is state ++ state.record(torch.ones(2, 1)) ++ if failure == "model": ++ raise RuntimeError("model failed") ++ return torch.zeros(1, 96), torch.zeros(1, 32) ++ ++ class Branch: ++ update_mask = torch.ones(1, dtype=torch.bool) ++ update_mask_dev = update_mask ++ audio_update_mask_dev = update_mask ++ locked_audio_rows = None ++ ++ def forward_kwargs(self, **kwargs): ++ return {} ++ ++ monkeypatch.setattr( ++ loop, ++ "minimax_h3_prepare_denoise_rows", ++ lambda **kwargs: (kwargs["initial_video_rows"], kwargs["initial_audio_rows"], None, None), ++ ) ++ ++ def cancellation(**kwargs): ++ if failure == "cancel": ++ raise RuntimeError("request cancelled") ++ ++ monkeypatch.setattr(loop, "check_request_cancellation", cancellation) ++ model = Model() ++ kwargs = dict( ++ model=model, ++ positive=Branch(), ++ initial_video_rows=torch.zeros(1, 96), ++ initial_audio_rows=torch.zeros(1, 32), ++ keyframe_cond_rows=None, ++ sigmas_video=[1.0, 0.0], ++ sigmas_audio=[1.0, 0.0], ++ device=torch.device("cpu"), ++ ) ++ if failure == "success": ++ loop.minimax_h3_denoise_loop(**kwargs) ++ else: ++ with pytest.raises(RuntimeError): ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert state.closed and not state.history ++ assert model._sigma_cache_stats["computed"] == (0 if failure == "cancel" else 1) ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native Ascend forecasting") ++def test_native_two_clock_forecast_and_input_immutability(): ++ device = torch.device(current_omni_platform.device_type) ++ state, originals = anchored(device=device, dtype=torch.bfloat16) ++ indices = torch.tensor([0, 2], device=device) ++ saved = [value.clone() for value in originals] ++ expected = sample(0.5, 0.3, device=device, dtype=torch.bfloat16) ++ assert state.can_predict(expected) ++ actual = state.predict(indices) ++ # BF16 anchor quantization is separate from exact polynomial algebra. ++ torch.testing.assert_close(actual.float(), expected.float(), atol=0.0625, rtol=0) ++ assert all(torch.equal(value, old) for value, old in zip(originals, saved)) ++ assert state.stats()["predicted"] == 1 ++ state.close() ++ assert not state.history ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native Ascend cached block execution") ++def test_native_real_cache_manager_computes_then_forecasts(): ++ from cache_dit import DBCacheConfig, ForwardPattern ++ from cache_dit.caching.cache_contexts.cache_manager import CachedContextManager ++ ++ class Block(nn.Module): ++ def __init__(self, prefix=False): ++ super().__init__() ++ self.prefix = prefix ++ self.calls = 0 ++ ++ def forward(self, hidden_states, *, test_sigmas, **kwargs): ++ self.calls += 1 ++ return hidden_states + (0.125 if self.prefix else sample(*test_sigmas, device=hidden_states.device)) ++ ++ manager = CachedContextManager() ++ context = manager.new_context( ++ name="h3-sigma-native", ++ cache_config=DBCacheConfig( ++ Fn_compute_blocks=1, ++ Bn_compute_blocks=0, ++ max_warmup_steps=4, ++ num_inference_steps=5, ++ residual_diff_threshold=0.08, ++ max_continuous_cached_steps=3, ++ ), ++ ) ++ blocks = nn.ModuleList([Block(prefix=True), Block(), Block()]) ++ wrapper = sigma.MiniMaxH3SigmaCachedBlocks( ++ blocks, ++ forward_pattern=ForwardPattern.Pattern_3, ++ check_forward_pattern=False, ++ cache_prefix="blocks", ++ cache_context=context, ++ context_manager=manager, ++ ) ++ device = torch.device(current_omni_platform.device_type) ++ indices = torch.tensor([0, 2], device=device) ++ state = sigma.MiniMaxH3SigmaCacheState() ++ for step, clocks in enumerate([(1.0, 1.0), (0.95, 0.9), (0.85, 0.75), (0.7, 0.55), (0.5, 0.35)]): ++ state.begin_step(step, *clocks) ++ hidden = torch.ones(2, 1, device=device) ++ actual = wrapper(hidden, sigma_cache_state=state, combined_indices=indices, test_sigmas=clocks) ++ torch.testing.assert_close(actual, hidden + 0.125 + 2 * sample(*clocks, device=device), atol=1e-5, rtol=1e-5) ++ assert wrapper._sigma_state is None and wrapper._sigma_indices is None ++ assert blocks[0].calls == 5 and blocks[1].calls == blocks[2].calls == 4 ++ assert state.stats()["computed"] == 4 and state.stats()["predicted"] == 1 ++ state.close() +diff --git a/vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py b/vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py +new file mode 100644 +index 0000000..50626a9 +--- /dev/null ++++ b/vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py +@@ -0,0 +1,194 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++"""Opt-in, request-owned residual forecasting on H3's two sigma clocks.""" ++ ++from __future__ import annotations ++ ++import math ++from collections.abc import Mapping ++from typing import Any ++from weakref import ref ++ ++import torch ++ ++from .reference_kv_cache_dit import MiniMaxH3CachedAdapter, MiniMaxH3CachedBlocks ++ ++ ++def sigma_cache_enabled(config: Any) -> bool: ++ cache = getattr(config, "cache_config", {}) ++ enabled = ( ++ cache.get("minimax_h3_sigma_cache", False) ++ if isinstance(cache, Mapping) ++ else getattr(cache, "minimax_h3_sigma_cache", False) ++ ) ++ if type(enabled) is not bool: ++ raise ValueError("minimax_h3_sigma_cache must be a boolean") ++ if not enabled: ++ return False ++ parallel = getattr(config, "parallel_config", None) ++ if ( ++ getattr(config, "task_type", None) != "ref2va" ++ or getattr(config, "cache_backend", None) != "cache_dit" ++ or getattr(config, "kv_offload_config", None) is not None ++ or (getattr(parallel, "data_parallel_size", 1) or 1) != 1 ++ or (getattr(parallel, "cfg_parallel_size", 1) or 1) != 1 ++ ): ++ raise ValueError("H3 sigma cache requires Ref2VA Cache-DiT, DP1/CFG1 and no reference-KV cache") ++ taylor = cache.get("enable_taylorseer", False) if isinstance(cache, Mapping) else cache.enable_taylorseer ++ if taylor: ++ raise ValueError("H3 sigma cache replaces TaylorSeer; disable enable_taylorseer") ++ return True ++ ++ ++def _quadratic_weights(anchors: list[float], target: float) -> tuple[float, ...] | None: ++ if len(anchors) != 3 or not all(math.isfinite(x) for x in [*anchors, target]): ++ return None ++ if not anchors[0] > anchors[1] > anchors[2] > target: ++ return None ++ weights = [] ++ for i, anchor in enumerate(anchors): ++ weight = 1.0 ++ for j, other in enumerate(anchors): ++ if i != j: ++ weight *= (target - other) / (anchor - other) ++ weights.append(weight) ++ # A host-only veto: all SP ranks have the same clocks and anchor steps. ++ if not all(math.isfinite(x) for x in weights) or sum(abs(x) for x in weights) > 64.0: ++ return None ++ return tuple(weights) ++ ++ ++class MiniMaxH3SigmaCacheState: ++ """Three real middle-residual anchors, owned by one denoise request.""" ++ ++ def __init__(self) -> None: ++ self.step = -1 ++ self.sigmas = (1.0, 1.0) ++ self.history: list[tuple[tuple[float, float], torch.Tensor]] = [] ++ self.closed = False ++ self.computed = 0 ++ self.predicted = 0 ++ self.history_vetoes = 0 ++ ++ def begin_step(self, step: int, video_sigma: float, audio_sigma: float) -> None: ++ if self.closed or step != self.step + 1: ++ raise RuntimeError("H3 sigma cache requires consecutive steps in an open request") ++ sigmas = (float(video_sigma), float(audio_sigma)) ++ if not all(math.isfinite(x) for x in sigmas): ++ raise ValueError("H3 sigma cache requires finite sigma clocks") ++ if self.step >= 0 and not all(x < old for x, old in zip(sigmas, self.sigmas)): ++ raise ValueError("H3 sigma cache requires descending sigma clocks") ++ self.step, self.sigmas = step, sigmas ++ ++ def _weights(self) -> tuple[tuple[float, ...], tuple[float, ...]] | None: ++ if self.closed or len(self.history) != 3: ++ return None ++ video = _quadratic_weights([clock[0] for clock, _ in self.history], self.sigmas[0]) ++ audio = _quadratic_weights([clock[1] for clock, _ in self.history], self.sigmas[1]) ++ return None if video is None or audio is None else (video, audio) ++ ++ def can_predict(self, current: torch.Tensor) -> bool: ++ if self._weights() is None: ++ self.history_vetoes += 1 ++ return False ++ if any( ++ tensor.shape != current.shape or tensor.device != current.device or tensor.dtype != current.dtype ++ for _, tensor in self.history ++ ): ++ self.history.clear() ++ self.history_vetoes += 1 ++ return False ++ return True ++ ++ def record(self, residual: torch.Tensor) -> None: ++ if self.closed or self.step < 0 or residual.ndim != 2: ++ raise RuntimeError("H3 sigma cache requires an open step and a two-dimensional residual") ++ if self.history and self.history[-1][0] == self.sigmas: ++ raise RuntimeError("H3 sigma cache received duplicate middle-block residuals") ++ # Own the anchor: neither Cache-DiT nor the model may mutate our history. ++ self.history.append((self.sigmas, residual.detach().clone())) ++ del self.history[:-3] ++ self.computed += 1 ++ ++ def predict(self, combined_indices: torch.Tensor) -> torch.Tensor: ++ weights = self._weights() ++ if weights is None: ++ raise RuntimeError("H3 sigma cache cannot forecast without three stable anchors") ++ last = self.history[-1][1] ++ if combined_indices.ndim != 1 or combined_indices.shape[0] != last.shape[0]: ++ raise ValueError("H3 sigma cache requires one modality index per local residual row") ++ if combined_indices.device != last.device: ++ raise ValueError("H3 sigma cache modality and residual devices must match") ++ audio_rows = combined_indices.remainder(3).eq(2).unsqueeze(-1) ++ output = torch.zeros_like(last, dtype=torch.float32) ++ for i, (_, residual) in enumerate(self.history): ++ coefficient = torch.where(audio_rows, weights[1][i], weights[0][i]) ++ output.add_(residual.float() * coefficient) ++ self.predicted += 1 ++ return output.to(last.dtype) ++ ++ def stats(self) -> dict[str, int]: ++ return {"computed": self.computed, "predicted": self.predicted, "history_vetoes": self.history_vetoes} ++ ++ def close(self) -> None: ++ self.history.clear() ++ self.closed = True ++ ++ ++class _SigmaCacheManager: ++ def __init__(self, manager: Any, owner: MiniMaxH3SigmaCachedBlocks) -> None: ++ self._manager = manager ++ self._owner = ref(owner) ++ ++ def __getattr__(self, name: str) -> Any: ++ return getattr(self._manager, name) ++ ++ def _state(self) -> MiniMaxH3SigmaCacheState: ++ owner = self._owner() ++ if owner is None or owner._sigma_state is None: ++ raise RuntimeError("H3 sigma cache manager has no active request") ++ return owner._sigma_state ++ ++ def can_cache(self, states_tensor: torch.Tensor, *args, **kwargs) -> bool: ++ allowed = self._manager.can_cache(states_tensor, *args, **kwargs) ++ return allowed and self._state().can_predict(states_tensor) ++ ++ def set_Bn_buffer(self, buffer: torch.Tensor, prefix: str = "Bn") -> None: ++ if not self._manager.is_cache_residual(): ++ raise ValueError("H3 sigma cache requires middle-block residual caching") ++ self._state().record(buffer) ++ self._manager.set_Bn_buffer(buffer, prefix=prefix) ++ ++ def apply_cache(self, hidden_states: torch.Tensor, encoder_hidden_states=None, **kwargs): ++ if encoder_hidden_states is not None or not self._manager.is_cache_residual(): ++ raise ValueError("H3 sigma cache supports only the packed H3 residual stream") ++ owner = self._owner() ++ if owner is None or owner._sigma_indices is None: ++ raise RuntimeError("H3 sigma cache has no active modality layout") ++ prediction = self._state().predict(owner._sigma_indices) ++ return (prediction + hidden_states).contiguous(), None ++ ++ ++class MiniMaxH3SigmaCachedBlocks(MiniMaxH3CachedBlocks): ++ def __init__(self, *args, **kwargs) -> None: ++ super().__init__(*args, **kwargs) ++ self._sigma_state = None ++ self._sigma_indices = None ++ self.context_manager = _SigmaCacheManager(self.context_manager, self) ++ ++ def forward(self, hidden_states: torch.Tensor, *args, sigma_cache_state=None, **kwargs): ++ if not isinstance(sigma_cache_state, MiniMaxH3SigmaCacheState) or sigma_cache_state.closed: ++ raise RuntimeError("H3 sigma cache requires a live denoise request state") ++ if kwargs.get("num_requests", 1) != 1 or kwargs.get("reference_kv_state") is not None: ++ raise ValueError("H3 sigma cache requires a single request without reference-KV reuse") ++ old_state, old_indices = self._sigma_state, self._sigma_indices ++ try: ++ self._sigma_state = sigma_cache_state ++ self._sigma_indices = kwargs["combined_indices"] ++ return super().forward(hidden_states, *args, **kwargs) ++ finally: ++ self._sigma_state, self._sigma_indices = old_state, old_indices ++ ++ ++class MiniMaxH3SigmaCachedAdapter(MiniMaxH3CachedAdapter): ++ block_class = MiniMaxH3SigmaCachedBlocks +diff --git a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py +index 8cca219..ddbdace 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py ++++ b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py +@@ -379,13 +379,21 @@ def minimax_h3_denoise_loop( + if reference_kv_state is not None: + positive.static_kwargs["reference_kv_state"] = reference_kv_state + ++ sigma_cache_state = None + try: ++ create_sigma_cache_state = getattr(model, "create_sigma_cache_state", None) ++ if callable(create_sigma_cache_state): ++ sigma_cache_state = create_sigma_cache_state() ++ if sigma_cache_state is not None and (video_edit is not None or audio_edit is not None): ++ raise ValueError("H3 sigma cache does not support latent edits") + for step in range(num_steps): + check_request_cancellation() + step_cm = step_profiler(step) if step_profiler is not None else nullcontext() + with step_cm: + s_v = sigmas_video[step] + s_a = sigmas_audio[step] ++ if sigma_cache_state is not None: ++ sigma_cache_state.begin_step(step, s_v, s_a) + if reference_kv_state is not None: + reference_kv_state.begin_step(step, sigma=float(s_v)) + # Publish where we are so step-gated attention features (the dense +@@ -426,6 +434,8 @@ def minimax_h3_denoise_loop( + ) + if getattr(getattr(model, "arch", None), "pdd_steps", None) is not None: + fk["pdd_step"] = step ++ if sigma_cache_state is not None: ++ fk["sigma_cache_state"] = sigma_cache_state + with torch.inference_mode(): + v_video, v_audio = model(**fk) + if reference_kv_state is not None: +@@ -475,6 +485,9 @@ def minimax_h3_denoise_loop( + check_request_cancellation(synchronize=True) + + finally: ++ if sigma_cache_state is not None: ++ model._sigma_cache_stats = sigma_cache_state.stats() ++ sigma_cache_state.close() + minimax_h3_publish_denoise_progress(None, None, None) + if reference_kv_state is not None: + positive.static_kwargs.pop("reference_kv_state", None) +diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py +index 9518828..ca727c6 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py ++++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py +@@ -245,6 +245,7 @@ _FORWARD_SUPPORTED_KWARGS = frozenset( + "control_rows", + "control_context_scale", + "reference_kv_state", ++ "sigma_cache_state", + } + ) + +@@ -777,6 +778,23 @@ class MiniMaxH3DiTModel(nn.Module): + max_bytes=256 * 1024**2 if enabled else 0, offload_weights=offload_adaln + ) + self.od_config = od_config ++ sigma_enabled = ( ++ cache_config.get("minimax_h3_sigma_cache", False) ++ if isinstance(cache_config, Mapping) ++ else getattr(cache_config, "minimax_h3_sigma_cache", False) ++ ) ++ if type(sigma_enabled) is not bool: ++ raise ValueError("minimax_h3_sigma_cache must be a boolean") ++ self._sigma_cache_enabled = False ++ if sigma_enabled: ++ from .cache_sigma_prediction import MiniMaxH3SigmaCachedAdapter, sigma_cache_enabled ++ ++ self._sigma_cache_enabled = sigma_cache_enabled(od_config) ++ if not current_omni_platform.is_npu(): ++ raise ValueError("H3 sigma cache is currently restricted to the Ascend Ref2VA experiment") ++ self._cache_dit_adapter_config = replace( ++ self._cache_dit_adapter_config, cached_adapter_cls=MiniMaxH3SigmaCachedAdapter ++ ) + if getattr(od_config, "kv_offload_config", None) is not None: + from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache_dit import MiniMaxH3CachedAdapter + +@@ -1257,6 +1275,14 @@ class MiniMaxH3DiTModel(nn.Module): + device=device, + ) + ++ def create_sigma_cache_state(self): ++ if not getattr(self, "_sigma_cache_enabled", False): ++ return None ++ from .cache_sigma_prediction import MiniMaxH3SigmaCacheState ++ ++ self._sigma_cache_stats = None ++ return MiniMaxH3SigmaCacheState() ++ + def forward(self, *, pdd_step: int | None = None, **kwargs: Any) -> tuple[torch.Tensor, torch.Tensor]: + """Packed inference forward. + +@@ -1490,6 +1516,14 @@ class MiniMaxH3DiTModel(nn.Module): + if getattr(self, "adaln_cache", None) is not None: + self.adaln_cache.prepare(t_emb) + ++ sigma_state = kwargs.get("sigma_cache_state") ++ sigma_options = {} ++ if sigma_state is not None: ++ if not getattr(self, "_sigma_cache_enabled", False): ++ raise ValueError("H3 sigma cache state was supplied while the feature is disabled") ++ if num_requests != 1 or reference_kv_state is not None or control_rows is not None: ++ raise ValueError("H3 sigma cache does not support packed batches, control or reference-KV reuse") ++ sigma_options["sigma_cache_state"] = sigma_state + for block_index, block in enumerate(self.blocks): + hidden = block( + hidden, +@@ -1505,6 +1539,7 @@ class MiniMaxH3DiTModel(nn.Module): + reference_kv_state=reference_kv_state, + reference_kv_compact=compact_reference, + vdn_window=vdn_window, ++ **sigma_options, + ) + if block_index in hints: + hidden = hidden + hints.pop(block_index) * control_scale +diff --git a/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py b/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py +index f14dcc9..f78598b 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py ++++ b/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py +@@ -125,6 +125,8 @@ class MiniMaxH3CachedBlocks(CachedBlocks_Pattern_3_4_5): + + + class MiniMaxH3CachedAdapter(CachedAdapter): ++ block_class = MiniMaxH3CachedBlocks ++ + @classmethod + def collect_unified_blocks( + cls, block_adapter: BlockAdapter, contexts_kwargs: list[dict] +@@ -139,7 +141,7 @@ class MiniMaxH3CachedAdapter(CachedAdapter): + context_index += 1 + contexts[block_adapter.unique_blocks_name[i][j]] = nn.ModuleList( + [ +- MiniMaxH3CachedBlocks( ++ cls.block_class( + blocks, + transformer=transformer, + forward_pattern=block_adapter.forward_pattern[i][j], diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-sigma-fp32-residual.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-sigma-fp32-residual.patch new file mode 100644 index 000000000..fdbb14d3c --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-sigma-fp32-residual.patch @@ -0,0 +1,685 @@ +diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py +new file mode 100644 +index 0000000..c2d577f +--- /dev/null ++++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py +@@ -0,0 +1,333 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++from types import SimpleNamespace ++ ++import pytest ++import torch ++import torch.nn as nn ++ ++from tests.helpers.mark import hardware_marks ++from vllm_omni.diffusion.models.minimax_h3 import cache_sigma_prediction as sigma ++from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache_dit import MiniMaxH3CachedBlocks ++from vllm_omni.platforms import current_omni_platform ++ ++pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=1)] ++ ++ ++def config(**changes): ++ values = dict( ++ cache_config={"minimax_h3_sigma_cache": True, "enable_taylorseer": False}, ++ task_type="ref2va", ++ cache_backend="cache_dit", ++ kv_offload_config=None, ++ parallel_config=SimpleNamespace(data_parallel_size=1, cfg_parallel_size=1), ++ ) ++ values.update(changes) ++ return SimpleNamespace(**values) ++ ++ ++def sample(video, audio, *, device="cpu", dtype=torch.float32): ++ return torch.tensor([[1 + 2 * video + video**2], [4 - audio + 3 * audio**2]], device=device, dtype=dtype) ++ ++ ++def anchored(*, device="cpu", dtype=torch.float32): ++ state = sigma.MiniMaxH3SigmaCacheState() ++ originals = [] ++ for step, (video, audio) in enumerate([(1.0, 1.0), (0.9, 0.8), (0.7, 0.5)]): ++ state.begin_step(step, video, audio) ++ residual = sample(video, audio, device=device, dtype=dtype) ++ state.record(residual) ++ originals.append(residual) ++ state.begin_step(3, 0.5, 0.3) ++ return state, originals ++ ++ ++def test_distinct_nonuniform_clocks_recover_quadratic_and_own_anchors(): ++ state, originals = anchored() ++ indices = torch.tensor([0, 17]) ++ expected = sample(0.5, 0.3) ++ assert state.can_predict(expected) ++ torch.testing.assert_close(state.predict(indices), expected) ++ saved = [value.clone() for _, value in state.history] ++ originals[-1].fill_(99) ++ assert all(torch.equal(old, value) for old, (_, value) in zip(saved, state.history)) ++ torch.testing.assert_close(state.predict(indices), expected) ++ # One shared video clock gives the wrong audio result on this fixture. ++ video_weights = sigma._quadratic_weights([1.0, 0.9, 0.7], 0.5) ++ wrong_audio = sum(weight * value[1] for weight, (_, value) in zip(video_weights, state.history)) ++ assert not torch.allclose(wrong_audio, expected[1]) ++ assert state.stats() == {"computed": 3, "predicted": 2, "history_vetoes": 0} ++ state.close() ++ assert state.closed and not state.history and not state.can_predict(expected) ++ ++ ++@pytest.mark.parametrize( ++ "anchors,target", ++ [([1.0, 0.8], 0.7), ([1.0, 0.8, 0.8], 0.7), ([1.0, 0.9, 0.8], 0.1), ([1.0, 0.9, 0.8], float("nan"))], ++) ++def test_unstable_or_insufficient_history_vetoes(anchors, target): ++ assert sigma._quadratic_weights(anchors, target) is None ++ ++ ++@pytest.mark.parametrize( ++ "changes", ++ [ ++ {"task_type": "t2va"}, ++ {"cache_backend": "none"}, ++ {"kv_offload_config": {}}, ++ {"parallel_config": SimpleNamespace(data_parallel_size=2)}, ++ {"cache_config": {"minimax_h3_sigma_cache": 1}}, ++ {"cache_config": {"minimax_h3_sigma_cache": True, "enable_taylorseer": True}}, ++ ], ++) ++def test_explicit_unsupported_selection_fails(changes): ++ with pytest.raises(ValueError): ++ sigma.sigma_cache_enabled(config(**changes)) ++ ++ ++def test_default_and_dataclass_extra_selection(): ++ from vllm_omni.diffusion.data import DiffusionCacheConfig ++ ++ assert not sigma.sigma_cache_enabled(config(cache_config={})) ++ cache = DiffusionCacheConfig.from_dict({"minimax_h3_sigma_cache": True}) ++ assert sigma.sigma_cache_enabled(config(cache_config=cache)) ++ assert sigma.MiniMaxH3SigmaCachedAdapter.block_class is sigma.MiniMaxH3SigmaCachedBlocks ++ ++ ++def test_parent_denial_and_request_prediction_do_not_mutate_inputs(): ++ state, _ = anchored() ++ owner = SimpleOwner(state, torch.tensor([0, 2])) ++ calls = [] ++ manager = SimpleNamespace( ++ can_cache=lambda *args, **kwargs: False, ++ is_cache_residual=lambda: True, ++ set_Bn_buffer=lambda *args, **kwargs: calls.append((args, kwargs)), ++ ) ++ proxy = sigma._SigmaCacheManager(manager, owner) ++ hidden = torch.ones(2, 1) ++ assert not proxy.can_cache(hidden) ++ assert state.history_vetoes == 0 ++ manager.can_cache = lambda *args, **kwargs: True ++ assert proxy.can_cache(hidden) ++ actual, encoder = proxy.apply_cache(hidden, prefix="blocks_Bn_residual") ++ torch.testing.assert_close(actual, sample(0.5, 0.3) + 1) ++ assert encoder is None and torch.equal(hidden, torch.ones_like(hidden)) ++ owner._sigma_state = sigma.MiniMaxH3SigmaCacheState() ++ assert not proxy.can_cache(hidden) ++ assert owner._sigma_state.history_vetoes == 1 ++ ++ ++class SimpleOwner: ++ def __init__(self, state, indices): ++ self._sigma_state, self._sigma_indices = state, indices ++ ++ ++def test_fp32_history_accepts_bf16_prefix_and_preserves_output_dtype(): ++ state, _ = anchored() ++ hidden = torch.ones(2, 1, dtype=torch.bfloat16) ++ assert state.can_predict(hidden) ++ owner = SimpleOwner(state, torch.tensor([0, 2])) ++ proxy = sigma._SigmaCacheManager(SimpleNamespace(is_cache_residual=lambda: True), owner) ++ actual, _ = proxy.apply_cache(hidden) ++ assert actual.dtype == hidden.dtype ++ torch.testing.assert_close(actual, (sample(0.5, 0.3) + hidden.float()).to(hidden.dtype)) ++ assert all(value.dtype == torch.float32 for _, value in state.history) ++ ++ ++def test_middle_anchor_retains_unrounded_endpoint_difference(monkeypatch): ++ before = torch.tensor([[0.0712890625]], dtype=torch.bfloat16) ++ computed = torch.tensor([[7.0]], dtype=torch.bfloat16) ++ ++ def parent(self, hidden, *args, **kwargs): ++ return computed, None, computed - hidden ++ ++ monkeypatch.setattr(MiniMaxH3CachedBlocks, "call_Mn_blocks", parent) ++ wrapper = object.__new__(sigma.MiniMaxH3SigmaCachedBlocks) ++ nn.Module.__init__(wrapper) ++ end, encoder, residual = wrapper.call_Mn_blocks(before) ++ assert end is computed and encoder is None and residual.dtype == torch.float32 ++ assert torch.equal(residual, computed.float() - before.float()) ++ assert not torch.equal((computed - before).float(), residual) ++ assert torch.equal((residual + before.float()).to(before.dtype), computed) ++ ++ ++@pytest.mark.parametrize("failure", [False, True]) ++def test_wrapper_releases_transient_request_metadata(monkeypatch, failure): ++ wrapper = object.__new__(sigma.MiniMaxH3SigmaCachedBlocks) ++ nn.Module.__init__(wrapper) ++ wrapper._sigma_state, wrapper._sigma_indices = None, None ++ state = sigma.MiniMaxH3SigmaCacheState() ++ ++ def parent(self, hidden, *args, **kwargs): ++ assert "sigma_cache_state" not in kwargs ++ assert self._sigma_state is state ++ if failure: ++ raise RuntimeError("block failed") ++ return hidden ++ ++ monkeypatch.setattr(MiniMaxH3CachedBlocks, "forward", parent) ++ args = dict(sigma_cache_state=state, combined_indices=torch.tensor([0, 2])) ++ if failure: ++ with pytest.raises(RuntimeError, match="block failed"): ++ wrapper.forward(torch.ones(2, 1), **args) ++ else: ++ wrapper.forward(torch.ones(2, 1), **args) ++ assert wrapper._sigma_state is None and wrapper._sigma_indices is None ++ ++ ++@pytest.mark.parametrize("failure", ["success", "model", "cancel"]) ++def test_denoise_request_closes_history_on_all_exit_paths(monkeypatch, failure): ++ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop ++ ++ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) ++ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) ++ state = sigma.MiniMaxH3SigmaCacheState() ++ ++ class Model: ++ def create_sigma_cache_state(self): ++ return state ++ ++ def __call__(self, **kwargs): ++ assert kwargs["sigma_cache_state"] is state ++ state.record(torch.ones(2, 1)) ++ if failure == "model": ++ raise RuntimeError("model failed") ++ return torch.zeros(1, 96), torch.zeros(1, 32) ++ ++ class Branch: ++ update_mask = torch.ones(1, dtype=torch.bool) ++ update_mask_dev = update_mask ++ audio_update_mask_dev = update_mask ++ locked_audio_rows = None ++ ++ def forward_kwargs(self, **kwargs): ++ return {} ++ ++ monkeypatch.setattr( ++ loop, ++ "minimax_h3_prepare_denoise_rows", ++ lambda **kwargs: (kwargs["initial_video_rows"], kwargs["initial_audio_rows"], None, None), ++ ) ++ ++ def cancellation(**kwargs): ++ if failure == "cancel": ++ raise RuntimeError("request cancelled") ++ ++ monkeypatch.setattr(loop, "check_request_cancellation", cancellation) ++ model = Model() ++ kwargs = dict( ++ model=model, ++ positive=Branch(), ++ initial_video_rows=torch.zeros(1, 96), ++ initial_audio_rows=torch.zeros(1, 32), ++ keyframe_cond_rows=None, ++ sigmas_video=[1.0, 0.0], ++ sigmas_audio=[1.0, 0.0], ++ device=torch.device("cpu"), ++ ) ++ if failure == "success": ++ loop.minimax_h3_denoise_loop(**kwargs) ++ else: ++ with pytest.raises(RuntimeError): ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert state.closed and not state.history ++ assert model._sigma_cache_stats["computed"] == (0 if failure == "cancel" else 1) ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native Ascend forecasting") ++def test_native_two_clock_forecast_and_input_immutability(): ++ device = torch.device(current_omni_platform.device_type) ++ state, originals = anchored(device=device, dtype=torch.bfloat16) ++ indices = torch.tensor([0, 2], device=device) ++ saved = [value.clone() for value in originals] ++ expected = sample(0.5, 0.3, device=device, dtype=torch.bfloat16) ++ assert state.can_predict(expected) ++ actual = state.predict(indices) ++ # BF16 anchor quantization is separate from exact polynomial algebra. ++ torch.testing.assert_close(actual.float(), expected.float(), atol=0.0625, rtol=0) ++ assert all(torch.equal(value, old) for value, old in zip(originals, saved)) ++ assert state.stats()["predicted"] == 1 ++ state.close() ++ assert not state.history ++ ++ ++def _native_real_cache_manager(dtype): ++ from cache_dit import DBCacheConfig, ForwardPattern ++ from cache_dit.caching.cache_contexts.cache_manager import CachedContextManager ++ ++ class Block(nn.Module): ++ def __init__(self, prefix=False): ++ super().__init__() ++ self.prefix = prefix ++ self.calls = 0 ++ ++ def forward(self, hidden_states, *, test_sigmas, **kwargs): ++ self.calls += 1 ++ return hidden_states + ( ++ 0.125 if self.prefix else sample(*test_sigmas, device=hidden_states.device, dtype=hidden_states.dtype) ++ ) ++ ++ manager = CachedContextManager() ++ context = manager.new_context( ++ name="h3-sigma-native", ++ cache_config=DBCacheConfig( ++ Fn_compute_blocks=1, ++ Bn_compute_blocks=0, ++ max_warmup_steps=4, ++ num_inference_steps=5, ++ residual_diff_threshold=0.08, ++ max_continuous_cached_steps=3, ++ ), ++ ) ++ blocks = nn.ModuleList([Block(prefix=True), Block(), Block()]) ++ wrapper = sigma.MiniMaxH3SigmaCachedBlocks( ++ blocks, ++ forward_pattern=ForwardPattern.Pattern_3, ++ check_forward_pattern=False, ++ cache_prefix="blocks", ++ cache_context=context, ++ context_manager=manager, ++ ) ++ device = torch.device(current_omni_platform.device_type) ++ indices = torch.tensor([0, 2], device=device) ++ state = sigma.MiniMaxH3SigmaCacheState() ++ for step, clocks in enumerate([(1.0, 1.0), (0.95, 0.9), (0.85, 0.75), (0.7, 0.55), (0.5, 0.35)]): ++ state.begin_step(step, *clocks) ++ hidden = torch.ones(2, 1, device=device, dtype=dtype) ++ actual = wrapper(hidden, sigma_cache_state=state, combined_indices=indices, test_sigmas=clocks) ++ assert actual.dtype == hidden.dtype ++ assert all(value.dtype == torch.float32 for _, value in state.history) ++ if step < 4: ++ delta = sample(*clocks, device=device, dtype=dtype) ++ expected = hidden + 0.125 + delta + delta ++ torch.testing.assert_close(actual, expected, atol=0, rtol=0) ++ else: ++ # BF16 full-compute endpoints are quantized, so they do not lie ++ # exactly on the analytic polynomial. Fit the real anchors with an ++ # independent FP64 Vandermonde solve rather than ignoring that error. ++ forecast = torch.empty(2, 1, dtype=torch.float64) ++ for row, modality in enumerate((0, 1)): ++ anchors = [clock[modality] for clock, _ in state.history] ++ matrix = torch.tensor([[1, x, x * x] for x in anchors], dtype=torch.float64) ++ values = torch.stack([value[row].cpu().double() for _, value in state.history]) ++ coefficients = torch.linalg.solve(matrix, values) ++ target = clocks[modality] ++ forecast[row] = torch.tensor([1, target, target * target], dtype=torch.float64) @ coefficients ++ expected = (hidden.cpu().double() + 0.125 + forecast).to(device=device, dtype=dtype) ++ tolerance = 0.03125 if dtype == torch.bfloat16 else 1e-5 ++ torch.testing.assert_close(actual, expected, atol=tolerance, rtol=1e-5) ++ assert wrapper._sigma_state is None and wrapper._sigma_indices is None ++ assert blocks[0].calls == 5 and blocks[1].calls == blocks[2].calls == 4 ++ assert state.stats()["computed"] == 4 and state.stats()["predicted"] == 1 ++ state.close() ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native Ascend cached block execution") ++def test_native_real_cache_manager_computes_then_forecasts(): ++ _native_real_cache_manager(torch.float32) ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native BF16 blocks with FP32 residual history") ++def test_native_bf16_blocks_use_fp32_anchors_and_keep_output_dtype(): ++ _native_real_cache_manager(torch.bfloat16) +diff --git a/vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py b/vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py +new file mode 100644 +index 0000000..cfea2a8 +--- /dev/null ++++ b/vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py +@@ -0,0 +1,199 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++"""Opt-in, request-owned residual forecasting on H3's two sigma clocks.""" ++ ++from __future__ import annotations ++ ++import math ++from collections.abc import Mapping ++from typing import Any ++from weakref import ref ++ ++import torch ++ ++from .reference_kv_cache_dit import MiniMaxH3CachedAdapter, MiniMaxH3CachedBlocks ++ ++ ++def sigma_cache_enabled(config: Any) -> bool: ++ cache = getattr(config, "cache_config", {}) ++ enabled = ( ++ cache.get("minimax_h3_sigma_cache", False) ++ if isinstance(cache, Mapping) ++ else getattr(cache, "minimax_h3_sigma_cache", False) ++ ) ++ if type(enabled) is not bool: ++ raise ValueError("minimax_h3_sigma_cache must be a boolean") ++ if not enabled: ++ return False ++ parallel = getattr(config, "parallel_config", None) ++ if ( ++ getattr(config, "task_type", None) != "ref2va" ++ or getattr(config, "cache_backend", None) != "cache_dit" ++ or getattr(config, "kv_offload_config", None) is not None ++ or (getattr(parallel, "data_parallel_size", 1) or 1) != 1 ++ or (getattr(parallel, "cfg_parallel_size", 1) or 1) != 1 ++ ): ++ raise ValueError("H3 sigma cache requires Ref2VA Cache-DiT, DP1/CFG1 and no reference-KV cache") ++ taylor = cache.get("enable_taylorseer", False) if isinstance(cache, Mapping) else cache.enable_taylorseer ++ if taylor: ++ raise ValueError("H3 sigma cache replaces TaylorSeer; disable enable_taylorseer") ++ return True ++ ++ ++def _quadratic_weights(anchors: list[float], target: float) -> tuple[float, ...] | None: ++ if len(anchors) != 3 or not all(math.isfinite(x) for x in [*anchors, target]): ++ return None ++ if not anchors[0] > anchors[1] > anchors[2] > target: ++ return None ++ weights = [] ++ for i, anchor in enumerate(anchors): ++ weight = 1.0 ++ for j, other in enumerate(anchors): ++ if i != j: ++ weight *= (target - other) / (anchor - other) ++ weights.append(weight) ++ # A host-only veto: all SP ranks have the same clocks and anchor steps. ++ if not all(math.isfinite(x) for x in weights) or sum(abs(x) for x in weights) > 64.0: ++ return None ++ return tuple(weights) ++ ++ ++class MiniMaxH3SigmaCacheState: ++ """Three real middle-residual anchors, owned by one denoise request.""" ++ ++ def __init__(self) -> None: ++ self.step = -1 ++ self.sigmas = (1.0, 1.0) ++ self.history: list[tuple[tuple[float, float], torch.Tensor]] = [] ++ self.closed = False ++ self.computed = 0 ++ self.predicted = 0 ++ self.history_vetoes = 0 ++ ++ def begin_step(self, step: int, video_sigma: float, audio_sigma: float) -> None: ++ if self.closed or step != self.step + 1: ++ raise RuntimeError("H3 sigma cache requires consecutive steps in an open request") ++ sigmas = (float(video_sigma), float(audio_sigma)) ++ if not all(math.isfinite(x) for x in sigmas): ++ raise ValueError("H3 sigma cache requires finite sigma clocks") ++ if self.step >= 0 and not all(x < old for x, old in zip(sigmas, self.sigmas)): ++ raise ValueError("H3 sigma cache requires descending sigma clocks") ++ self.step, self.sigmas = step, sigmas ++ ++ def _weights(self) -> tuple[tuple[float, ...], tuple[float, ...]] | None: ++ if self.closed or len(self.history) != 3: ++ return None ++ video = _quadratic_weights([clock[0] for clock, _ in self.history], self.sigmas[0]) ++ audio = _quadratic_weights([clock[1] for clock, _ in self.history], self.sigmas[1]) ++ return None if video is None or audio is None else (video, audio) ++ ++ def can_predict(self, current: torch.Tensor) -> bool: ++ if self._weights() is None: ++ self.history_vetoes += 1 ++ return False ++ if any(tensor.shape != current.shape or tensor.device != current.device for _, tensor in self.history): ++ self.history.clear() ++ self.history_vetoes += 1 ++ return False ++ return True ++ ++ def record(self, residual: torch.Tensor) -> None: ++ if self.closed or self.step < 0 or residual.ndim != 2: ++ raise RuntimeError("H3 sigma cache requires an open step and a two-dimensional residual") ++ if self.history and self.history[-1][0] == self.sigmas: ++ raise RuntimeError("H3 sigma cache received duplicate middle-block residuals") ++ # Own the anchor: neither Cache-DiT nor the model may mutate our history. ++ self.history.append((self.sigmas, residual.detach().clone())) ++ del self.history[:-3] ++ self.computed += 1 ++ ++ def predict(self, combined_indices: torch.Tensor) -> torch.Tensor: ++ weights = self._weights() ++ if weights is None: ++ raise RuntimeError("H3 sigma cache cannot forecast without three stable anchors") ++ last = self.history[-1][1] ++ if combined_indices.ndim != 1 or combined_indices.shape[0] != last.shape[0]: ++ raise ValueError("H3 sigma cache requires one modality index per local residual row") ++ if combined_indices.device != last.device: ++ raise ValueError("H3 sigma cache modality and residual devices must match") ++ audio_rows = combined_indices.remainder(3).eq(2).unsqueeze(-1) ++ output = torch.zeros_like(last, dtype=torch.float32) ++ for i, (_, residual) in enumerate(self.history): ++ coefficient = torch.where(audio_rows, weights[1][i], weights[0][i]) ++ output.add_(residual.float() * coefficient) ++ self.predicted += 1 ++ return output ++ ++ def stats(self) -> dict[str, int]: ++ return {"computed": self.computed, "predicted": self.predicted, "history_vetoes": self.history_vetoes} ++ ++ def close(self) -> None: ++ self.history.clear() ++ self.closed = True ++ ++ ++class _SigmaCacheManager: ++ def __init__(self, manager: Any, owner: MiniMaxH3SigmaCachedBlocks) -> None: ++ self._manager = manager ++ self._owner = ref(owner) ++ ++ def __getattr__(self, name: str) -> Any: ++ return getattr(self._manager, name) ++ ++ def _state(self) -> MiniMaxH3SigmaCacheState: ++ owner = self._owner() ++ if owner is None or owner._sigma_state is None: ++ raise RuntimeError("H3 sigma cache manager has no active request") ++ return owner._sigma_state ++ ++ def can_cache(self, states_tensor: torch.Tensor, *args, **kwargs) -> bool: ++ allowed = self._manager.can_cache(states_tensor, *args, **kwargs) ++ return allowed and self._state().can_predict(states_tensor) ++ ++ def set_Bn_buffer(self, buffer: torch.Tensor, prefix: str = "Bn") -> None: ++ if not self._manager.is_cache_residual(): ++ raise ValueError("H3 sigma cache requires middle-block residual caching") ++ self._state().record(buffer) ++ self._manager.set_Bn_buffer(buffer, prefix=prefix) ++ ++ def apply_cache(self, hidden_states: torch.Tensor, encoder_hidden_states=None, **kwargs): ++ if encoder_hidden_states is not None or not self._manager.is_cache_residual(): ++ raise ValueError("H3 sigma cache supports only the packed H3 residual stream") ++ owner = self._owner() ++ if owner is None or owner._sigma_indices is None: ++ raise RuntimeError("H3 sigma cache has no active modality layout") ++ prediction = self._state().predict(owner._sigma_indices) ++ output = (prediction + hidden_states.float()).to(hidden_states.dtype) ++ return output.contiguous(), None ++ ++ ++class MiniMaxH3SigmaCachedBlocks(MiniMaxH3CachedBlocks): ++ def __init__(self, *args, **kwargs) -> None: ++ super().__init__(*args, **kwargs) ++ self._sigma_state = None ++ self._sigma_indices = None ++ self.context_manager = _SigmaCacheManager(self.context_manager, self) ++ ++ def call_Mn_blocks(self, hidden_states: torch.Tensor, *args, **kwargs): ++ computed, encoder, _ = super().call_Mn_blocks(hidden_states, *args, **kwargs) ++ # Fit the exact difference of the computed BF16 endpoints. Reconstruct ++ # in FP32 and cast once so downstream blocks retain their input dtype. ++ residual = computed.float() - hidden_states.float() ++ return computed, encoder, residual ++ ++ def forward(self, hidden_states: torch.Tensor, *args, sigma_cache_state=None, **kwargs): ++ if not isinstance(sigma_cache_state, MiniMaxH3SigmaCacheState) or sigma_cache_state.closed: ++ raise RuntimeError("H3 sigma cache requires a live denoise request state") ++ if kwargs.get("num_requests", 1) != 1 or kwargs.get("reference_kv_state") is not None: ++ raise ValueError("H3 sigma cache requires a single request without reference-KV reuse") ++ old_state, old_indices = self._sigma_state, self._sigma_indices ++ try: ++ self._sigma_state = sigma_cache_state ++ self._sigma_indices = kwargs["combined_indices"] ++ return super().forward(hidden_states, *args, **kwargs) ++ finally: ++ self._sigma_state, self._sigma_indices = old_state, old_indices ++ ++ ++class MiniMaxH3SigmaCachedAdapter(MiniMaxH3CachedAdapter): ++ block_class = MiniMaxH3SigmaCachedBlocks +diff --git a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py +index 8cca219..ddbdace 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py ++++ b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py +@@ -379,13 +379,21 @@ def minimax_h3_denoise_loop( + if reference_kv_state is not None: + positive.static_kwargs["reference_kv_state"] = reference_kv_state + ++ sigma_cache_state = None + try: ++ create_sigma_cache_state = getattr(model, "create_sigma_cache_state", None) ++ if callable(create_sigma_cache_state): ++ sigma_cache_state = create_sigma_cache_state() ++ if sigma_cache_state is not None and (video_edit is not None or audio_edit is not None): ++ raise ValueError("H3 sigma cache does not support latent edits") + for step in range(num_steps): + check_request_cancellation() + step_cm = step_profiler(step) if step_profiler is not None else nullcontext() + with step_cm: + s_v = sigmas_video[step] + s_a = sigmas_audio[step] ++ if sigma_cache_state is not None: ++ sigma_cache_state.begin_step(step, s_v, s_a) + if reference_kv_state is not None: + reference_kv_state.begin_step(step, sigma=float(s_v)) + # Publish where we are so step-gated attention features (the dense +@@ -426,6 +434,8 @@ def minimax_h3_denoise_loop( + ) + if getattr(getattr(model, "arch", None), "pdd_steps", None) is not None: + fk["pdd_step"] = step ++ if sigma_cache_state is not None: ++ fk["sigma_cache_state"] = sigma_cache_state + with torch.inference_mode(): + v_video, v_audio = model(**fk) + if reference_kv_state is not None: +@@ -475,6 +485,9 @@ def minimax_h3_denoise_loop( + check_request_cancellation(synchronize=True) + + finally: ++ if sigma_cache_state is not None: ++ model._sigma_cache_stats = sigma_cache_state.stats() ++ sigma_cache_state.close() + minimax_h3_publish_denoise_progress(None, None, None) + if reference_kv_state is not None: + positive.static_kwargs.pop("reference_kv_state", None) +diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py +index 9518828..ca727c6 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py ++++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py +@@ -245,6 +245,7 @@ _FORWARD_SUPPORTED_KWARGS = frozenset( + "control_rows", + "control_context_scale", + "reference_kv_state", ++ "sigma_cache_state", + } + ) + +@@ -777,6 +778,23 @@ class MiniMaxH3DiTModel(nn.Module): + max_bytes=256 * 1024**2 if enabled else 0, offload_weights=offload_adaln + ) + self.od_config = od_config ++ sigma_enabled = ( ++ cache_config.get("minimax_h3_sigma_cache", False) ++ if isinstance(cache_config, Mapping) ++ else getattr(cache_config, "minimax_h3_sigma_cache", False) ++ ) ++ if type(sigma_enabled) is not bool: ++ raise ValueError("minimax_h3_sigma_cache must be a boolean") ++ self._sigma_cache_enabled = False ++ if sigma_enabled: ++ from .cache_sigma_prediction import MiniMaxH3SigmaCachedAdapter, sigma_cache_enabled ++ ++ self._sigma_cache_enabled = sigma_cache_enabled(od_config) ++ if not current_omni_platform.is_npu(): ++ raise ValueError("H3 sigma cache is currently restricted to the Ascend Ref2VA experiment") ++ self._cache_dit_adapter_config = replace( ++ self._cache_dit_adapter_config, cached_adapter_cls=MiniMaxH3SigmaCachedAdapter ++ ) + if getattr(od_config, "kv_offload_config", None) is not None: + from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache_dit import MiniMaxH3CachedAdapter + +@@ -1257,6 +1275,14 @@ class MiniMaxH3DiTModel(nn.Module): + device=device, + ) + ++ def create_sigma_cache_state(self): ++ if not getattr(self, "_sigma_cache_enabled", False): ++ return None ++ from .cache_sigma_prediction import MiniMaxH3SigmaCacheState ++ ++ self._sigma_cache_stats = None ++ return MiniMaxH3SigmaCacheState() ++ + def forward(self, *, pdd_step: int | None = None, **kwargs: Any) -> tuple[torch.Tensor, torch.Tensor]: + """Packed inference forward. + +@@ -1490,6 +1516,14 @@ class MiniMaxH3DiTModel(nn.Module): + if getattr(self, "adaln_cache", None) is not None: + self.adaln_cache.prepare(t_emb) + ++ sigma_state = kwargs.get("sigma_cache_state") ++ sigma_options = {} ++ if sigma_state is not None: ++ if not getattr(self, "_sigma_cache_enabled", False): ++ raise ValueError("H3 sigma cache state was supplied while the feature is disabled") ++ if num_requests != 1 or reference_kv_state is not None or control_rows is not None: ++ raise ValueError("H3 sigma cache does not support packed batches, control or reference-KV reuse") ++ sigma_options["sigma_cache_state"] = sigma_state + for block_index, block in enumerate(self.blocks): + hidden = block( + hidden, +@@ -1505,6 +1539,7 @@ class MiniMaxH3DiTModel(nn.Module): + reference_kv_state=reference_kv_state, + reference_kv_compact=compact_reference, + vdn_window=vdn_window, ++ **sigma_options, + ) + if block_index in hints: + hidden = hidden + hints.pop(block_index) * control_scale +diff --git a/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py b/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py +index f14dcc9..f78598b 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py ++++ b/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py +@@ -125,6 +125,8 @@ class MiniMaxH3CachedBlocks(CachedBlocks_Pattern_3_4_5): + + + class MiniMaxH3CachedAdapter(CachedAdapter): ++ block_class = MiniMaxH3CachedBlocks ++ + @classmethod + def collect_unified_blocks( + cls, block_adapter: BlockAdapter, contexts_kwargs: list[dict] +@@ -139,7 +141,7 @@ class MiniMaxH3CachedAdapter(CachedAdapter): + context_index += 1 + contexts[block_adapter.unique_blocks_name[i][j]] = nn.ModuleList( + [ +- MiniMaxH3CachedBlocks( ++ cls.block_class( + blocks, + transformer=transformer, + forward_pattern=block_adapter.forward_pattern[i][j], diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-source-identity.json new file mode 100644 index 000000000..47ec5b291 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c11-source-identity.json @@ -0,0 +1,22 @@ +{ + "base": "922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb", + "head": "61b8efa06e88372bc5ab87b9e2668a133eaac282", + "tree": "b5f29f2e136e17b23744f07fe65664237fdae627", + "files": { + "vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py": "ac29ef338d569529e40f59210c89a57dd2e07c510b5e2952c0da451f183da52a", + "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "d3381079ff012f32176a917bb0825d12288c058d3a058f5169ecc97d1e2c675c", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a649a09e5b59256b5887c12fe508d8e14e3dedd9019bdb82c65db2d59f11de91", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "a9524734a4f3410a0c6f766fa64fa9e5bd5a4d41bd7481738e9024cac54ce7f9", + "tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py": "9ba094c0bc0686e8d044ebe064beaa2606a6cad496449e63ec282411bb2e727a" + }, + "native_tests_passed": 20, + "native_test_names": [ + "test_native_two_clock_forecast_and_input_immutability", + "test_native_real_cache_manager_computes_then_forecasts" + ], + "source_unchanged": true, + "cleanup_passed": true, + "published": false, + "full_model_quality": "original Fn1Bn0 failed5s-00 adherence; Fn1Bn1 failed5s-00 audiolevel", + "performance_acceptance": "PENDING" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c12-candidate.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c12-candidate.patch new file mode 100644 index 000000000..f18f79855 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c12-candidate.patch @@ -0,0 +1,633 @@ +diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_state_cache.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_state_cache.py +new file mode 100644 +index 0000000..6896e2c +--- /dev/null ++++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_state_cache.py +@@ -0,0 +1,252 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++from datetime import timedelta ++from types import SimpleNamespace ++ ++import pytest ++import torch ++import torch.distributed as dist ++import torch.nn as nn ++ ++from tests.helpers.mark import hardware_marks ++from vllm_omni.diffusion.models.minimax_h3 import cache_state_alignment as aligned ++from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache_dit import MiniMaxH3CachedBlocks ++from vllm_omni.platforms import current_omni_platform ++ ++pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=4)] ++ ++ ++def feature(t, device="cpu", dtype=torch.float32): ++ return torch.tensor([[t, t * t, 5.0]], device=device, dtype=dtype) ++ ++ ++def history(device="cpu", group=2): ++ state = aligned.MiniMaxH3AlignedCacheState() ++ state.set_groups(torch.tensor([group], device=device)) ++ for step in range(3): ++ state.begin_step(step) ++ f = feature(step, device) ++ state.record(f, 2 * f + 3) ++ state.begin_step(3) ++ return state ++ ++ ++def test_feature_forecast_uses_current_state_and_owns_anchors(): ++ state = history() ++ current = feature(2.5) ++ assert state.can_predict(current, parallelized=False) ++ torch.testing.assert_close(state.predict(current), 3 * current + 3, atol=0.01, rtol=0) ++ saved = [f.clone() for f, _ in state.history] ++ current.fill_(99) ++ assert all(torch.equal(a, b) for a, (b, _) in zip(saved, state.history)) ++ state.close() ++ assert state.closed and not state.history and state.groups is None and state.coefficients is None ++ ++ ++@pytest.mark.parametrize("kind", ["missing", "out_of_span", "unstable", "nonfinite"]) ++def test_invalid_prediction_is_vetoed(kind): ++ state = history() ++ current = feature(2.5) ++ if kind == "missing": ++ state.history.pop() ++ elif kind == "out_of_span": ++ current[0, 2] = 100 ++ elif kind == "unstable": ++ current = feature(30) ++ else: ++ current[0, 0] = float("nan") ++ assert not state.can_predict(current, parallelized=False) ++ assert state.vetoes == 1 ++ ++ ++def test_structural_groups_preserve_reference_and_exclude_padding(): ++ state = aligned.MiniMaxH3AlignedCacheState() ++ branch = SimpleNamespace( ++ seq_len=7, ++ device="cpu", ++ text_pos_dev=torch.tensor([0]), ++ img_pos_dev=torch.tensor([1, 2]), ++ update_mask_dev=torch.tensor([False, True]), ++ audio_pos_dev=torch.tensor([3, 4]), ++ audio_update_mask_dev=torch.tensor([False, True]), ++ ) ++ state.configure(branch) ++ assert state.groups.tolist() == [0, 1, 2, 3, 4, -1, -1] ++ for step in range(3): ++ state.begin_step(step) ++ f = feature(step).expand(7, -1).clone() ++ state.record(f, 2 * f + 3) ++ state.begin_step(3) ++ current = feature(2.5).expand(7, -1).clone() ++ current[5:] = 1000 ++ assert state.can_predict(current, parallelized=False) ++ torch.testing.assert_close(state.predict(current)[:5], (3 * current + 3)[:5], atol=0.01, rtol=0) ++ ++ ++@pytest.mark.parametrize("failure", [False, True]) ++def test_wrapper_releases_request_and_does_not_forward_private_state(monkeypatch, failure): ++ wrapper = object.__new__(aligned.MiniMaxH3AlignedCachedBlocks) ++ nn.Module.__init__(wrapper) ++ wrapper._alignment_state = wrapper._alignment_prefix = None ++ state = history() ++ ++ def parent(self, hidden, **kwargs): ++ assert "alignment_cache_state" not in kwargs and self._alignment_state is state ++ if failure: ++ raise RuntimeError("test failure") ++ return hidden ++ ++ monkeypatch.setattr(MiniMaxH3CachedBlocks, "forward", parent) ++ if failure: ++ with pytest.raises(RuntimeError, match="test failure"): ++ wrapper.forward(feature(2.5), alignment_cache_state=state) ++ else: ++ wrapper.forward(feature(2.5), alignment_cache_state=state) ++ assert wrapper._alignment_state is None and wrapper._alignment_prefix is None ++ ++ ++def test_default_and_unsupported_configuration(): ++ config = SimpleNamespace(cache_config={}) ++ assert not aligned.alignment_cache_enabled(config) ++ config.cache_config = {"minimax_h3_alignment_cache": True} ++ with pytest.raises(ValueError): ++ aligned.alignment_cache_enabled(config) ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native Ascend execution") ++def test_native_bf16_and_actual_cache_wrapper(): ++ from cache_dit import DBCacheConfig, ForwardPattern ++ from cache_dit.caching.cache_contexts.cache_manager import CachedContextManager ++ ++ class Block(nn.Module): ++ def __init__(self, prefix=False): ++ super().__init__() ++ self.prefix = prefix ++ self.calls = 0 ++ ++ def forward(self, hidden, **kwargs): ++ self.calls += 1 ++ return hidden + (0.125 if self.prefix else 1.0) ++ ++ manager = CachedContextManager() ++ context = manager.new_context( ++ name="h3-alignment-native", ++ cache_config=DBCacheConfig( ++ Fn_compute_blocks=1, ++ Bn_compute_blocks=0, ++ max_warmup_steps=4, ++ num_inference_steps=5, ++ residual_diff_threshold=0.15, ++ max_continuous_cached_steps=4, ++ ), ++ ) ++ blocks = nn.ModuleList([Block(True), Block(), Block()]) ++ wrapper = aligned.MiniMaxH3AlignedCachedBlocks( ++ blocks, ++ forward_pattern=ForwardPattern.Pattern_3, ++ check_forward_pattern=False, ++ cache_prefix="blocks", ++ cache_context=context, ++ context_manager=manager, ++ ) ++ state = aligned.MiniMaxH3AlignedCacheState() ++ state.set_groups(torch.tensor([2], device="npu")) ++ for step in range(5): ++ state.begin_step(step) ++ f = torch.ones(1, 3, dtype=torch.bfloat16, device="npu") ++ output = wrapper(f, alignment_cache_state=state) ++ assert torch.equal(output, f + 2.125) and output.dtype == f.dtype ++ assert wrapper._alignment_state is None and wrapper._alignment_prefix is None ++ assert state.computed == 4 and state.predicted == 1 ++ assert blocks[1].calls == 4 and blocks[0].calls == 5 ++ state.close() ++ ++ ++def _distributed_worker(rank): ++ import torch_npu # noqa: F401 ++ ++ torch.npu.set_device(rank) ++ dist.init_process_group( ++ "hccl", init_method="tcp://127.0.0.1:29597", rank=rank, world_size=4, timeout=timedelta(seconds=120) ++ ) ++ try: ++ # Only rank3 has audio; all ranks have empty reference-audio groups. ++ state = history(device=f"npu:{rank}", group=[0, 1, 2, 4][rank]) ++ current = feature(2.5, device=f"npu:{rank}", dtype=torch.bfloat16) ++ assert state.can_predict(current, parallelized=True) ++ torch.testing.assert_close(state.predict(current).float(), 3 * current.float() + 3, atol=0.0625, rtol=0) ++ state.close() ++ assert state.groups is None and not state.history ++ dist.barrier() ++ finally: ++ dist.destroy_process_group() ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires four native Ascend devices") ++def test_native_four_rank_empty_group_collectives(): ++ torch.multiprocessing.spawn(_distributed_worker, nprocs=4, join=True) ++ ++ ++@pytest.mark.parametrize("failure", ["success", "model", "cancel"]) ++def test_denoise_request_closes_history_on_all_exit_paths(monkeypatch, failure): ++ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop ++ ++ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) ++ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) ++ state = aligned.MiniMaxH3AlignedCacheState() ++ ++ class Model: ++ def create_alignment_cache_state(self): ++ return state ++ ++ def __call__(self, **kwargs): ++ assert kwargs["alignment_cache_state"] is state ++ state.record(torch.ones(2, 1), torch.ones(2, 1)) ++ if failure == "model": ++ raise RuntimeError("model failed") ++ return torch.zeros(1, 96), torch.zeros(1, 32) ++ ++ class Branch: ++ seq_len = 2 ++ device = torch.device("cpu") ++ text_pos_dev = torch.tensor([], dtype=torch.long) ++ img_pos_dev = torch.tensor([0]) ++ audio_pos_dev = torch.tensor([1]) ++ update_mask = torch.ones(1, dtype=torch.bool) ++ update_mask_dev = update_mask ++ audio_update_mask_dev = update_mask ++ locked_audio_rows = None ++ ++ def forward_kwargs(self, **kwargs): ++ return {} ++ ++ monkeypatch.setattr( ++ loop, ++ "minimax_h3_prepare_denoise_rows", ++ lambda **kwargs: (kwargs["initial_video_rows"], kwargs["initial_audio_rows"], None, None), ++ ) ++ ++ def cancellation(**kwargs): ++ if failure == "cancel": ++ raise RuntimeError("request cancelled") ++ ++ monkeypatch.setattr(loop, "check_request_cancellation", cancellation) ++ model = Model() ++ kwargs = dict( ++ model=model, ++ positive=Branch(), ++ initial_video_rows=torch.zeros(1, 96), ++ initial_audio_rows=torch.zeros(1, 32), ++ keyframe_cond_rows=None, ++ sigmas_video=[1.0, 0.0], ++ sigmas_audio=[1.0, 0.0], ++ device=torch.device("cpu"), ++ ) ++ if failure == "success": ++ loop.minimax_h3_denoise_loop(**kwargs) ++ else: ++ with pytest.raises(RuntimeError): ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert state.closed and not state.history ++ assert model._alignment_cache_stats["computed"] == (0 if failure == "cancel" else 1) +diff --git a/vllm_omni/diffusion/models/minimax_h3/cache_state_alignment.py b/vllm_omni/diffusion/models/minimax_h3/cache_state_alignment.py +new file mode 100644 +index 0000000..af1c360 +--- /dev/null ++++ b/vllm_omni/diffusion/models/minimax_h3/cache_state_alignment.py +@@ -0,0 +1,226 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++"""Experimental request-owned residual prediction from current H3 features.""" ++ ++from __future__ import annotations ++ ++import math ++from collections.abc import Mapping ++from typing import Any ++from weakref import ref ++ ++import torch ++import torch.distributed as dist ++ ++from .reference_kv_cache_dit import MiniMaxH3CachedAdapter, MiniMaxH3CachedBlocks ++ ++ ++def alignment_cache_enabled(config: Any) -> bool: ++ cache = getattr(config, "cache_config", {}) ++ enabled = ( ++ cache.get("minimax_h3_alignment_cache", False) ++ if isinstance(cache, Mapping) ++ else getattr(cache, "minimax_h3_alignment_cache", False) ++ ) ++ if type(enabled) is not bool: ++ raise ValueError("minimax_h3_alignment_cache must be boolean") ++ if not enabled: ++ return False ++ parallel = getattr(config, "parallel_config", None) ++ taylor = cache.get("enable_taylorseer", False) if isinstance(cache, Mapping) else cache.enable_taylorseer ++ if ( ++ getattr(config, "task_type", None) != "ref2va" ++ or getattr(config, "cache_backend", None) != "cache_dit" ++ or getattr(config, "kv_offload_config", None) is not None ++ or taylor ++ or (getattr(parallel, "data_parallel_size", 1) or 1) != 1 ++ or (getattr(parallel, "cfg_parallel_size", 1) or 1) != 1 ++ ): ++ raise ValueError("H3 alignment cache requires Ref2VA Cache-DiT, DP1/CFG1, no Taylor/RefKV") ++ return True ++ ++ ++class MiniMaxH3AlignedCacheState: ++ """Own three real prefix/residual pairs and structural row identities.""" ++ ++ def __init__(self) -> None: ++ self.history: list[tuple[torch.Tensor, torch.Tensor]] = [] ++ self.groups: torch.Tensor | None = None ++ self.coefficients: torch.Tensor | None = None ++ self.step = -1 ++ self.closed = False ++ self.computed = self.predicted = self.vetoes = 0 ++ ++ def configure(self, branch) -> None: ++ groups = torch.full((branch.seq_len,), -1, dtype=torch.long, device=branch.device) ++ groups[branch.text_pos_dev] = 0 ++ groups[branch.img_pos_dev] = torch.where(branch.update_mask_dev, 2, 1) ++ groups[branch.audio_pos_dev] = torch.where(branch.audio_update_mask_dev, 4, 3) ++ self.set_groups(groups) ++ ++ def set_groups(self, groups: torch.Tensor) -> None: ++ if self.groups is not None or self.closed or groups.ndim != 1: ++ raise ValueError("H3 alignment groups must be initialized once per request") ++ self.groups = groups.detach().clone() ++ ++ def begin_step(self, step: int, *_clocks) -> None: ++ if self.closed or step != self.step + 1: ++ raise RuntimeError("H3 alignment cache requires consecutive request steps") ++ self.step = step ++ self.coefficients = None ++ ++ def _local_groups(self, rows: int) -> torch.Tensor: ++ if self.groups is None: ++ raise RuntimeError("H3 alignment cache has no structural groups") ++ if self.groups.numel() == rows: ++ return self.groups ++ from vllm_omni.diffusion.distributed.parallel_state import ( ++ get_sequence_parallel_rank, ++ get_sequence_parallel_world_size, ++ ) ++ ++ size, rank = get_sequence_parallel_world_size(), get_sequence_parallel_rank() ++ if self.groups.numel() != rows * size: ++ raise ValueError("H3 alignment cache requires equal pure-Ulysses shards") ++ return self.groups.narrow(0, rows * rank, rows) ++ ++ def record(self, prefix: torch.Tensor, residual: torch.Tensor) -> None: ++ if self.closed or self.step < 0 or prefix.ndim != 2 or prefix.shape != residual.shape: ++ raise RuntimeError("H3 alignment cache requires an open step and matching matrices") ++ self.history.append((prefix.detach().float().clone(), residual.detach().float().clone())) ++ del self.history[:-3] ++ self.computed += 1 ++ ++ def can_predict(self, current: torch.Tensor, *, parallelized: bool) -> bool: ++ if len(self.history) != 3 or self.closed: ++ self.vetoes += 1 ++ return False ++ if any(f.shape != current.shape or f.device != current.device for f, _ in self.history): ++ self.history.clear() ++ self.vetoes += 1 ++ return False ++ groups = self._local_groups(current.shape[0]) ++ f0, f1, f2 = [item[0] for item in self.history] ++ d0, d1, u = f2 - f1, f1 - f0, current.float() - f2 ++ # Reduce feature products per row once; grouping never materializes ++ # a [groups, rows, hidden] tensor and padding contributes no products. ++ products = torch.stack( ++ [ ++ (d0 * d0).sum(-1), ++ (d0 * d1).sum(-1), ++ (d1 * d1).sum(-1), ++ (d0 * u).sum(-1), ++ (d1 * u).sum(-1), ++ (u * u).sum(-1), ++ ], ++ dim=-1, ++ ) ++ totals = torch.stack([torch.where((groups == group)[:, None], products, 0.0).sum(0) for group in range(5)]) ++ if parallelized: ++ if not dist.is_initialized(): ++ raise RuntimeError("H3 alignment requires initialized collectives") ++ # DP1/CFG1, pure USP: WORLD is the four-rank model group. Empty ++ # local audio/reference groups must still enter this collective. ++ dist.all_reduce(totals, op=dist.ReduceOp.SUM) ++ coefficients = [] ++ valid = True ++ for g00, g01, g11, h0, h1, uu in totals.cpu().double().tolist(): ++ if not all(math.isfinite(x) for x in [g00, g01, g11, h0, h1, uu]): ++ valid = False ++ coefficients.append((0.0, 0.0)) ++ continue ++ if uu == 0: ++ coefficients.append((0.0, 0.0)) ++ continue ++ ridge = 1e-4 * (g00 + g11) ++ determinant = (g00 + ridge) * (g11 + ridge) - g01 * g01 ++ if determinant <= 0: ++ valid = False ++ coefficients.append((0.0, 0.0)) ++ continue ++ a0 = ((g11 + ridge) * h0 - g01 * h1) / determinant ++ a1 = ((g00 + ridge) * h1 - g01 * h0) / determinant ++ error = max(0.0, uu - 2 * (a0 * h0 + a1 * h1) + a0 * a0 * g00 + 2 * a0 * a1 * g01 + a1 * a1 * g11) ++ valid &= abs(a0) + abs(a1) <= 8.0 and error <= 0.25**2 * uu ++ coefficients.append((a0, a1)) ++ if not valid: ++ self.vetoes += 1 ++ return False ++ self.coefficients = torch.tensor(coefficients, dtype=torch.float32, device=current.device) ++ return True ++ ++ def predict(self, current: torch.Tensor) -> torch.Tensor: ++ if self.coefficients is None or self.closed: ++ raise RuntimeError("H3 alignment prediction has not been admitted") ++ groups = self._local_groups(current.shape[0]) ++ coefficients = self.coefficients[groups.clamp(min=0)] ++ coefficients = torch.where((groups >= 0)[:, None], coefficients, 0.0) ++ b0, b1, b2 = [item[1] for item in self.history] ++ predicted = b2 + coefficients[:, :1] * (b2 - b1) + coefficients[:, 1:] * (b1 - b0) ++ self.predicted += 1 ++ return (current.float() + predicted).to(current.dtype).contiguous() ++ ++ def stats(self) -> dict[str, int]: ++ return {"computed": self.computed, "predicted": self.predicted, "vetoes": self.vetoes} ++ ++ def close(self) -> None: ++ self.history.clear() ++ self.groups = self.coefficients = None ++ self.closed = True ++ ++ ++class _AlignmentManager: ++ def __init__(self, manager: Any, owner) -> None: ++ self._manager, self._owner = manager, ref(owner) ++ ++ def __getattr__(self, name): ++ return getattr(self._manager, name) ++ ++ def can_cache(self, states, *args, **kwargs): ++ allowed = self._manager.can_cache(states, *args, **kwargs) ++ owner = self._owner() ++ return allowed and owner._alignment_state.can_predict( ++ owner._alignment_prefix, parallelized=kwargs.get("parallelized", False) ++ ) ++ ++ def set_Bn_buffer(self, buffer, prefix="Bn"): ++ owner = self._owner() ++ owner._alignment_state.record(owner._alignment_prefix, buffer) ++ self._manager.set_Bn_buffer(buffer, prefix=prefix) ++ ++ def apply_cache(self, hidden_states, encoder_hidden_states=None, **kwargs): ++ if encoder_hidden_states is not None or not self._manager.is_cache_residual(): ++ raise ValueError("H3 alignment supports only residual-cached packed hidden states") ++ return self._owner()._alignment_state.predict(hidden_states), None ++ ++ ++class MiniMaxH3AlignedCachedBlocks(MiniMaxH3CachedBlocks): ++ def __init__(self, *args, **kwargs) -> None: ++ super().__init__(*args, **kwargs) ++ self._alignment_state = self._alignment_prefix = None ++ self.context_manager = _AlignmentManager(self.context_manager, self) ++ ++ def call_Fn_blocks(self, hidden_states, *args, **kwargs): ++ output = super().call_Fn_blocks(hidden_states, *args, **kwargs) ++ self._alignment_prefix = output[0] ++ return output ++ ++ def call_Mn_blocks(self, hidden_states, *args, **kwargs): ++ computed, encoder, _ = super().call_Mn_blocks(hidden_states, *args, **kwargs) ++ return computed, encoder, computed.float() - hidden_states.float() ++ ++ def forward(self, hidden_states, *args, alignment_cache_state=None, **kwargs): ++ if not isinstance(alignment_cache_state, MiniMaxH3AlignedCacheState) or alignment_cache_state.closed: ++ raise RuntimeError("H3 alignment requires an active request state") ++ if kwargs.get("num_requests", 1) != 1 or kwargs.get("reference_kv_state") is not None: ++ raise ValueError("H3 alignment requires one request without reference-KV reuse") ++ old_state, old_prefix = self._alignment_state, self._alignment_prefix ++ try: ++ self._alignment_state = alignment_cache_state ++ return super().forward(hidden_states, *args, **kwargs) ++ finally: ++ self._alignment_state, self._alignment_prefix = old_state, old_prefix ++ ++ ++class MiniMaxH3AlignedCachedAdapter(MiniMaxH3CachedAdapter): ++ block_class = MiniMaxH3AlignedCachedBlocks +diff --git a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py +index 8cca219..716b931 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py ++++ b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py +@@ -379,13 +379,23 @@ def minimax_h3_denoise_loop( + if reference_kv_state is not None: + positive.static_kwargs["reference_kv_state"] = reference_kv_state + ++ alignment_cache_state = None + try: ++ create_alignment_cache_state = getattr(model, "create_alignment_cache_state", None) ++ if callable(create_alignment_cache_state): ++ alignment_cache_state = create_alignment_cache_state() ++ if alignment_cache_state is not None: ++ alignment_cache_state.configure(positive) ++ if alignment_cache_state is not None and (video_edit is not None or audio_edit is not None): ++ raise ValueError("H3 alignment cache does not support latent edits") + for step in range(num_steps): + check_request_cancellation() + step_cm = step_profiler(step) if step_profiler is not None else nullcontext() + with step_cm: + s_v = sigmas_video[step] + s_a = sigmas_audio[step] ++ if alignment_cache_state is not None: ++ alignment_cache_state.begin_step(step, s_v, s_a) + if reference_kv_state is not None: + reference_kv_state.begin_step(step, sigma=float(s_v)) + # Publish where we are so step-gated attention features (the dense +@@ -426,6 +436,8 @@ def minimax_h3_denoise_loop( + ) + if getattr(getattr(model, "arch", None), "pdd_steps", None) is not None: + fk["pdd_step"] = step ++ if alignment_cache_state is not None: ++ fk["alignment_cache_state"] = alignment_cache_state + with torch.inference_mode(): + v_video, v_audio = model(**fk) + if reference_kv_state is not None: +@@ -475,6 +487,9 @@ def minimax_h3_denoise_loop( + check_request_cancellation(synchronize=True) + + finally: ++ if alignment_cache_state is not None: ++ model._alignment_cache_stats = alignment_cache_state.stats() ++ alignment_cache_state.close() + minimax_h3_publish_denoise_progress(None, None, None) + if reference_kv_state is not None: + positive.static_kwargs.pop("reference_kv_state", None) +diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py +index 9518828..63f1530 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py ++++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py +@@ -245,6 +245,7 @@ _FORWARD_SUPPORTED_KWARGS = frozenset( + "control_rows", + "control_context_scale", + "reference_kv_state", ++ "alignment_cache_state", + } + ) + +@@ -777,6 +778,23 @@ class MiniMaxH3DiTModel(nn.Module): + max_bytes=256 * 1024**2 if enabled else 0, offload_weights=offload_adaln + ) + self.od_config = od_config ++ alignment_enabled = ( ++ cache_config.get("minimax_h3_alignment_cache", False) ++ if isinstance(cache_config, Mapping) ++ else getattr(cache_config, "minimax_h3_alignment_cache", False) ++ ) ++ if type(alignment_enabled) is not bool: ++ raise ValueError("minimax_h3_alignment_cache must be a boolean") ++ self._alignment_cache_enabled = False ++ if alignment_enabled: ++ from .cache_state_alignment import MiniMaxH3AlignedCachedAdapter, alignment_cache_enabled ++ ++ self._alignment_cache_enabled = alignment_cache_enabled(od_config) ++ if not current_omni_platform.is_npu(): ++ raise ValueError("H3 alignment cache is currently restricted to the Ascend Ref2VA experiment") ++ self._cache_dit_adapter_config = replace( ++ self._cache_dit_adapter_config, cached_adapter_cls=MiniMaxH3AlignedCachedAdapter ++ ) + if getattr(od_config, "kv_offload_config", None) is not None: + from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache_dit import MiniMaxH3CachedAdapter + +@@ -1257,6 +1275,14 @@ class MiniMaxH3DiTModel(nn.Module): + device=device, + ) + ++ def create_alignment_cache_state(self): ++ if not getattr(self, "_alignment_cache_enabled", False): ++ return None ++ from .cache_state_alignment import MiniMaxH3AlignedCacheState ++ ++ self._alignment_cache_stats = None ++ return MiniMaxH3AlignedCacheState() ++ + def forward(self, *, pdd_step: int | None = None, **kwargs: Any) -> tuple[torch.Tensor, torch.Tensor]: + """Packed inference forward. + +@@ -1490,6 +1516,14 @@ class MiniMaxH3DiTModel(nn.Module): + if getattr(self, "adaln_cache", None) is not None: + self.adaln_cache.prepare(t_emb) + ++ alignment_state = kwargs.get("alignment_cache_state") ++ alignment_options = {} ++ if alignment_state is not None: ++ if not getattr(self, "_alignment_cache_enabled", False): ++ raise ValueError("H3 alignment cache state was supplied while the feature is disabled") ++ if num_requests != 1 or reference_kv_state is not None or control_rows is not None: ++ raise ValueError("H3 alignment cache does not support packed batches, control or reference-KV reuse") ++ alignment_options["alignment_cache_state"] = alignment_state + for block_index, block in enumerate(self.blocks): + hidden = block( + hidden, +@@ -1505,6 +1539,7 @@ class MiniMaxH3DiTModel(nn.Module): + reference_kv_state=reference_kv_state, + reference_kv_compact=compact_reference, + vdn_window=vdn_window, ++ **alignment_options, + ) + if block_index in hints: + hidden = hidden + hints.pop(block_index) * control_scale +diff --git a/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py b/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py +index f14dcc9..f78598b 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py ++++ b/vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py +@@ -125,6 +125,8 @@ class MiniMaxH3CachedBlocks(CachedBlocks_Pattern_3_4_5): + + + class MiniMaxH3CachedAdapter(CachedAdapter): ++ block_class = MiniMaxH3CachedBlocks ++ + @classmethod + def collect_unified_blocks( + cls, block_adapter: BlockAdapter, contexts_kwargs: list[dict] +@@ -139,7 +141,7 @@ class MiniMaxH3CachedAdapter(CachedAdapter): + context_index += 1 + contexts[block_adapter.unique_blocks_name[i][j]] = nn.ModuleList( + [ +- MiniMaxH3CachedBlocks( ++ cls.block_class( + blocks, + transformer=transformer, + forward_pattern=block_adapter.forward_pattern[i][j], diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c12-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c12-source-identity.json new file mode 100644 index 000000000..331f49363 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c12-source-identity.json @@ -0,0 +1,15 @@ +{ + "head": "a29648e5acd3077b7b22739bc9522017782fe1ac", + "tree": "3490989d041cc8fca66a4fc3a234d71a3366a352", + "base": "be319692b96413918973b5bd399cdda3f320c574", + "branch": "rsi/h3-297-state-aligned-residual", + "files": { + "vllm_omni/diffusion/models/minimax_h3/cache_state_alignment.py": "3340ec21a77aef78531c601be241bcfb1e44107c795e8c8649907d1a18ed377f", + "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "f4cd0e9aa9e824c92ec1acd02c57a8f7d8684c35d4edf92ffdfae3345ff44e99", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "436ec52cd6329cbc64e91d94b450bfa6f773a9a953b8f974db73a5eb986a98bb", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "4dec8f02a04bfe7e8d2ada24fc6d5ae40a0058671d8515c8a266ecf88824b7b0", + "tests/diffusion/models/minimax_h3/test_minimax_h3_state_cache.py": "89cf57482b35d8d8ad8d4978aec87e9ee3ef45330e5b911161adef4379950a65" + }, + "published": false, + "patch_base": "be319692b96413918973b5bd399cdda3f320c574" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c13-candidate.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c13-candidate.patch new file mode 100644 index 000000000..09b2049e6 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c13-candidate.patch @@ -0,0 +1,817 @@ +diff --git a/tests/diffusion/distributed/test_collective_gate.py b/tests/diffusion/distributed/test_collective_gate.py +new file mode 100644 +index 0000000..e66fdae +--- /dev/null ++++ b/tests/diffusion/distributed/test_collective_gate.py +@@ -0,0 +1,205 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++"""Unit tests for the DLO/QKV-AllToAll ordering gate. ++ ++Covers the barrier rendezvous and the hook's deferred-AllGather state machine ++that keeps the background weight gather from taking the links ahead of the ++block's foreground AllToAll. ++""" ++ ++from unittest.mock import Mock ++ ++import pytest ++import torch ++from torch import nn ++ ++import vllm_omni.diffusion.offloader.distributed_layerwise_backend as dist_backend_module ++from tests.diffusion.offloader.helpers import DummyStream, patch_offload_runtime ++from vllm_omni.diffusion.distributed.collective_gate import QKVAllToAllBarrier ++from vllm_omni.diffusion.offloader.distributed_layerwise_backend import ( ++ DistributedLayerwiseOffloadHook, ++ attach_collective_gate, ++) ++ ++pytestmark = [pytest.mark.diffusion, pytest.mark.cpu, pytest.mark.core_model] ++ ++ ++@pytest.fixture ++def patched_offload_runtime(monkeypatch): ++ patch_offload_runtime(monkeypatch, dist_backend_module.current_omni_platform, synchronize=True) ++ ++ ++class TestQKVAllToAllBarrier: ++ def test_signal_without_arm_is_a_noop(self): ++ on_signal = Mock() ++ barrier = QKVAllToAllBarrier(on_signal) ++ ++ barrier.signal() ++ ++ on_signal.assert_not_called() ++ assert not barrier.armed ++ ++ def test_arm_then_signal_fires_once(self): ++ on_signal = Mock() ++ barrier = QKVAllToAllBarrier(on_signal) ++ ++ barrier.arm() ++ assert barrier.armed ++ barrier.signal() ++ ++ on_signal.assert_called_once_with() ++ assert not barrier.armed ++ ++ # A second signal in the same forward must not re-issue the gather. ++ barrier.signal() ++ on_signal.assert_called_once_with() ++ ++ def test_disarm_suppresses_a_pending_signal(self): ++ on_signal = Mock() ++ barrier = QKVAllToAllBarrier(on_signal) ++ ++ barrier.arm() ++ barrier.disarm() ++ barrier.signal() ++ ++ on_signal.assert_not_called() ++ ++ ++def _gated_hook() -> DistributedLayerwiseOffloadHook: ++ """A hook wired for the gated path, with hand-set shard buffers.""" ++ hook = DistributedLayerwiseOffloadHook( ++ next_block=nn.Linear(2, 2, bias=False), ++ device=torch.device("cpu"), ++ dp_group=object(), ++ dp_size=2, ++ rank=0, ++ copy_stream=DummyStream(), ++ comm_stream=DummyStream(), ++ pin_memory=False, ++ order_allgather_after_attn_a2a=True, ++ ) ++ device_buf = {torch.float32: torch.zeros(4)} ++ shard_buf = {torch.float32: torch.zeros(2)} ++ hook.gpu_buffers = [dict(device_buf), dict(device_buf)] ++ hook.gpu_shard_buffers = [dict(shard_buf), dict(shard_buf)] ++ hook.cpu_shards = {torch.float32: torch.zeros(2)} ++ hook._ag_output_sizes = {torch.float32: 4} ++ hook.metadata = {} ++ hook._cached_repoint = () ++ hook.rank_local_mmap = False ++ return hook ++ ++ ++class TestDeferredAllGather: ++ def test_prefetch_defers_the_gather_until_the_barrier_signals(self, patched_offload_runtime, monkeypatch): ++ hook = _gated_hook() ++ gather = Mock() ++ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) ++ ++ hook.prefetch_layer(0, non_blocking=True) ++ ++ # H2D ran; the collective is held back for the block's AllToAll. ++ gather.assert_not_called() ++ assert hook._a2a_barrier.armed ++ assert hook._deferred_allgather is not None ++ # The ready event must still be published for the consumer. ++ assert hook.ready_events[0] is hook._prefetch_done ++ ++ hook._a2a_barrier.signal() ++ ++ gather.assert_called_once() ++ assert hook._deferred_allgather is None ++ assert not hook._a2a_barrier.armed ++ ++ def test_post_forward_flush_issues_an_unsignalled_gather(self, patched_offload_runtime, monkeypatch): ++ """A block whose attention never ran must still get its weights.""" ++ hook = _gated_hook() ++ gather = Mock() ++ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) ++ ++ hook.prefetch_layer(0, non_blocking=True) ++ gather.assert_not_called() ++ ++ # pre_forward ran, attention did not: the pairing flush must fire. ++ hook.post_forward(hook.next_block, None) ++ ++ gather.assert_called_once() ++ assert hook._deferred_allgather is None ++ assert not hook._a2a_barrier.armed ++ ++ def test_sync_prefetch_issues_immediately_without_arming(self, patched_offload_runtime, monkeypatch): ++ """The cache-dit sync fallback has no attention signal to wait for.""" ++ hook = _gated_hook() ++ gather = Mock() ++ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) ++ ++ hook.prefetch_layer(0, non_blocking=False) ++ ++ gather.assert_called_once() ++ assert not hook._a2a_barrier.armed ++ assert hook._deferred_allgather is None ++ ++ def test_disabled_by_default(self, patched_offload_runtime, monkeypatch): ++ hook = DistributedLayerwiseOffloadHook( ++ next_block=nn.Linear(2, 2, bias=False), ++ device=torch.device("cpu"), ++ dp_group=object(), ++ dp_size=2, ++ rank=0, ++ copy_stream=DummyStream(), ++ comm_stream=DummyStream(), ++ pin_memory=False, ++ ) ++ hook.gpu_buffers = [{torch.float32: torch.zeros(4)}] * 2 ++ hook.gpu_shard_buffers = [{torch.float32: torch.zeros(2)}] * 2 ++ hook.cpu_shards = {torch.float32: torch.zeros(2)} ++ hook._ag_output_sizes = {torch.float32: 4} ++ hook.metadata = {} ++ hook._cached_repoint = () ++ gather = Mock() ++ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) ++ ++ hook.prefetch_layer(0, non_blocking=True) ++ ++ assert hook._a2a_barrier is None ++ gather.assert_called_once() ++ ++ ++class TestAttachCollectiveGate: ++ def test_block_without_attention_reports_zero(self): ++ hook = _gated_hook() ++ ++ assert attach_collective_gate(nn.Linear(2, 2), hook) == 0 ++ ++ ++@pytest.mark.parametrize("flush", [False, True]) ++def test_chunked_gate_defers_copies_and_reuses_bounded_slots(patched_offload_runtime, monkeypatch, flush): ++ from types import SimpleNamespace ++ ++ hook = _gated_hook() ++ chunks = tuple( ++ SimpleNamespace(chunk_id=i, local_numel=1, cpu_offset=i, full_offset=2 * i, padded_numel=2) for i in range(3) ++ ) ++ hook.manifest = SimpleNamespace(dtypes=(SimpleNamespace(dtype=torch.float32, chunks=chunks),)) ++ hook.cpu_shards = {torch.float32: torch.tensor([11.0, 22.0, 33.0])} ++ hook.gpu_buffers = [{torch.float32: torch.zeros(6)} for _ in range(2)] ++ calls = [] ++ ++ def gather(output, local, **kwargs): ++ calls.append(local.clone()) ++ output.copy_(local.repeat(2)) ++ ++ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) ++ hook.prefetch_layer(0, non_blocking=True) ++ assert not calls and hook._deferred_chunked_allgather is not None ++ assert torch.equal(hook.gpu_shard_buffers[0][torch.float32], torch.zeros(2)) ++ if flush: ++ hook._flush_deferred_allgather() ++ else: ++ hook._a2a_barrier.signal() ++ assert len(calls) == 3 and hook._deferred_chunked_allgather is None ++ assert not hook._a2a_barrier.armed ++ assert torch.equal(hook.gpu_buffers[0][torch.float32], torch.tensor([11.0, 11.0, 22.0, 22.0, 33.0, 33.0])) ++ hook._a2a_barrier.signal() ++ assert len(calls) == 3 +diff --git a/vllm_omni/config/omni_config.py b/vllm_omni/config/omni_config.py +index 0a803cf..44fc48c 100644 +--- a/vllm_omni/config/omni_config.py ++++ b/vllm_omni/config/omni_config.py +@@ -863,6 +863,7 @@ class _DiffusionConfigProjection(_TrackExplicitConfigFields): + dlo_use_allgather: bool = True + dlo_chunk_size_mb: int = Field(default=64, gt=0, strict=True) + dlo_resident_layers: int = Field(default=0, ge=0) ++ dlo_order_allgather_after_attn_a2a: bool = False + host_weight_runtime_mode: Literal["disabled", "preferred", "required"] = "disabled" + host_weight_runtime_root: str | None = None + dlo_host_registration_limit_gib: float = Field(default=0.0, ge=0) +diff --git a/vllm_omni/config/stage_config.py b/vllm_omni/config/stage_config.py +index 76a58a9..3c69986 100644 +--- a/vllm_omni/config/stage_config.py ++++ b/vllm_omni/config/stage_config.py +@@ -672,6 +672,7 @@ class StageDeployConfig: + dlo_use_allgather: bool | None = None + dlo_chunk_size_mb: int | None = None + dlo_resident_layers: int | None = None ++ dlo_order_allgather_after_attn_a2a: bool | None = None + host_weight_runtime_mode: str | None = None + host_weight_runtime_root: str | None = None + dlo_host_registration_limit_gib: float | None = None +diff --git a/vllm_omni/diffusion/attention/layer.py b/vllm_omni/diffusion/attention/layer.py +index 5520390..574eafa 100644 +--- a/vllm_omni/diffusion/attention/layer.py ++++ b/vllm_omni/diffusion/attention/layer.py +@@ -9,7 +9,7 @@ + + from collections.abc import Mapping + from dataclasses import replace +-from typing import cast ++from typing import Any, cast + + import torch + import torch.nn as nn +@@ -333,6 +333,11 @@ class Attention(nn.Module): + # Local strategy when SP is intentionally inactive outside sharded regions. + self._no_parallel_strategy = NoParallelAttention() + ++ # Optional cross-module rendezvous (see distributed/collective_gate.py). ++ # Installed by the DLO backend when it must keep its background weight ++ # AllGather from taking the links ahead of this layer's QKV AllToAll. ++ self._collective_gate: Any | None = None ++ + self.layer_idx: int | None = _try_extract_layer_index(prefix) + + self._kv_cache_dtype: str | None = None +@@ -506,10 +511,12 @@ class Attention(nn.Module): + reason = self._cross_backend_limitation(attn_metadata) + if reason is not None: + blocked = self._cross_fallback_blocker(attn_metadata) +- result = self.sdpa_fallback.resolve_execution_path(resolved_context, query, key, value, attn_metadata) ++ result = cast(SDPAImpl, self.sdpa_fallback).resolve_execution_path( ++ resolved_context, query, key, value, attn_metadata ++ ) + return replace( + result, +- backend=self.attn_backend.get_name() if blocked else "SDPA", ++ backend=cast(type[AttentionBackend], self.attn_backend).get_name() if blocked else "SDPA", + path="cross_unsupported" if blocked else "cross_sdpa_fallback", + support=CapabilityResult.unsupported(blocked) if blocked else CapabilityResult.unmigrated(reason), + ) +@@ -591,6 +598,12 @@ class Attention(nn.Module): + # For Ring: Concat joint_q + query, key, value, attn_metadata, ctx = strategy.pre_attention(query, key, value, attn_metadata) + ++ # The QKV AllToAll is now enqueued on this stream. Release any gather ++ # that was deferred to give this collective the links first. ++ gate = self._collective_gate ++ if gate is not None: ++ gate.signal() ++ + # MiniMax-H3 Tier2 stores reference K/V in the balanced layout produced + # by Ulysses. Keeping this hook at the generic post-parallel boundary + # avoids a model import in the parallel strategy and leaves other models +@@ -760,7 +773,7 @@ class Attention(nn.Module): + def _cross_fallback_blocker(self, attn_metadata: AttentionMetadata | None) -> str | None: + if self.backend_explicit: + return ( +- f"Explicit backend {self.attn_backend.get_name()} does not support " ++ f"Explicit backend {cast(type[AttentionBackend], self.attn_backend).get_name()} does not support " + f"{self._cross_backend_limitation(attn_metadata)}" + ) + if attn_metadata is not None and attn_metadata.extra.get("kv_cache_dtype") not in (None, "float"): +@@ -793,7 +806,7 @@ class Attention(nn.Module): + blocked = self._cross_fallback_blocker(attn_metadata) + if blocked is not None: + raise NotImplementedError(blocked) +- return self.sdpa_fallback.forward(query, key, value, attn_metadata) ++ return cast(SDPAImpl, self.sdpa_fallback).forward(query, key, value, attn_metadata) + dense_mask = attn_metadata.attn_mask if attn_metadata is not None else None + # Piecewise backends honor full_attn_spans ahead of attn_mask and never read the + # dense mask (e.g. HunyuanImage3, which sends both), so there is nothing to divert. +diff --git a/vllm_omni/diffusion/data.py b/vllm_omni/diffusion/data.py +index 8528574..fcbeec0 100644 +--- a/vllm_omni/diffusion/data.py ++++ b/vllm_omni/diffusion/data.py +@@ -962,6 +962,10 @@ class OmniDiffusionConfig: + dlo_chunk_size_mb: int = 64 + # Leading main-DiT blocks kept resident by distributed layerwise offload. + dlo_resident_layers: int = 0 ++ # Hold each DiT block's weight AllGather until that block's QKV AllToAll has ++ # been enqueued, so the background collective does not take the links ahead ++ # of the critical-path all-to-all. ++ dlo_order_allgather_after_attn_a2a: bool = False + # Final-layout Host Weight Runtime policy. The loader only activates this + # for eligible no-AllGather DLO; all other configurations preserve their + # existing loader/storage path. +diff --git a/vllm_omni/diffusion/distributed/collective_gate.py b/vllm_omni/diffusion/distributed/collective_gate.py +new file mode 100644 +index 0000000..1386d89 +--- /dev/null ++++ b/vllm_omni/diffusion/distributed/collective_gate.py +@@ -0,0 +1,72 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++"""Cross-module rendezvous that orders background collectives after the ++foreground all-to-all of a transformer block. ++ ++Distributed layerwise offload (DLO) issues the *next* block's weight AllGather ++on a dedicated ``comm_stream`` while the *current* block computes. Both that ++AllGather and the block's QKV sequence-parallel AllToAll are collectives over ++the same ranks, so whichever reaches the links first wins: the AllGather is ++submitted at block entry, well before the block reaches its attention, and the ++critical-path AllToAll ends up queued behind it. ++ ++A plain event wait cannot fix this by itself. ``wait_event`` snapshots the ++records enqueued *before* the call, so "enqueue the wait at block entry, record ++the event later in the block" is a no-op -- the wait resolves against the ++previous iteration's record. The gather therefore has to be *issued* after the ++AllToAll: the hook arms this barrier at block entry and defers the gather, the ++attention signals it right after ``strategy.pre_attention``, and the signal ++records the event and lets the hook submit the gather gated on it. ++ ++The gather still overlaps the rest of the block (attention output, MLP, the ++next block's projections); it simply cannot start before the AllToAll finished. ++""" ++ ++from __future__ import annotations ++ ++from collections.abc import Callable ++ ++__all__ = ["QKVAllToAllBarrier"] ++ ++ ++class QKVAllToAllBarrier: ++ """One-block rendezvous between a DLO hook and the block's attention. ++ ++ Owned by a single :class:`DistributedLayerwiseOffloadHook` (one per ++ streaming block) and shared with the attention layer inside that block. ++ Both live on the same block, so the arming and the signalling always come ++ from the same module forward. ++ ++ Args: ++ on_signal: Invoked at most once per :meth:`arm` when the block's ++ attention reports its QKV AllToAll. It must submit the deferred ++ weight AllGather. ++ """ ++ ++ def __init__(self, on_signal: Callable[[], None]) -> None: ++ self._on_signal = on_signal ++ self._armed = False ++ ++ @property ++ def armed(self) -> bool: ++ """True between :meth:`arm` and the matching :meth:`signal`.""" ++ return self._armed ++ ++ def arm(self) -> None: ++ """Mark the next attention QKV AllToAll as the gather's gate.""" ++ self._armed = True ++ ++ def disarm(self) -> None: ++ """Drop a pending arm without firing it (the caller flushes instead).""" ++ self._armed = False ++ ++ def signal(self) -> None: ++ """Report that the QKV AllToAll is now enqueued on the compute stream. ++ ++ A no-op unless armed, so it is cheap to call unconditionally from the ++ attention hot path. ++ """ ++ if not self._armed: ++ return ++ self._armed = False ++ self._on_signal() +diff --git a/vllm_omni/diffusion/offloader/base.py b/vllm_omni/diffusion/offloader/base.py +index 2505d48..50b73e5 100644 +--- a/vllm_omni/diffusion/offloader/base.py ++++ b/vllm_omni/diffusion/offloader/base.py +@@ -71,6 +71,10 @@ class OffloadConfig: + dlo_use_allgather: bool = True + dlo_resident_layers: int = 0 # leading DiT layers kept on device + chunk_size_bytes: int = 64 * 1024 * 1024 ++ # Hold each DiT block's weight AllGather until that block's QKV AllToAll has ++ # been enqueued, so the background collective does not take the links ahead ++ # of the critical-path all-to-all. See ``collective_gate.py``. ++ dlo_order_allgather_after_attn_a2a: bool = False + # Optional per-worker ceiling for registering an HWR mmap. Zero means no + # additional ceiling; pin_cpu_memory controls whether registration is tried. + dlo_host_registration_limit_gib: float = 0.0 +@@ -216,6 +220,7 @@ class OffloadConfig: + dlo_use_allgather=dit_uses_allgather, + dlo_resident_layers=dlo_resident_layers, + chunk_size_bytes=chunk_size_bytes, ++ dlo_order_allgather_after_attn_a2a=bool(getattr(od_config, "dlo_order_allgather_after_attn_a2a", False)), + dlo_host_registration_limit_gib=dlo_host_registration_limit_gib, + components=components, + dlo_transfers=dlo_transfers, +diff --git a/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py b/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py +index 7f0f081..8c4f87f 100644 +--- a/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py ++++ b/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py +@@ -31,6 +31,7 @@ import torch.distributed + from torch import nn + from vllm.logger import init_logger + ++from vllm_omni.diffusion.distributed.collective_gate import QKVAllToAllBarrier + from vllm_omni.diffusion.hooks import HookRegistry, ModelHook + from vllm_omni.diffusion.model_loader.host_weight_plan import ( + HostWeightPlan, +@@ -142,6 +143,7 @@ class DistributedLayerwiseOffloadHook(ModelHook): + tensor_transforms: dict[int, Any] | None = None, + materialization_probe_tensor: torch.Tensor | None = None, + chunk_size_bytes: int = 64 * 1024 * 1024, ++ order_allgather_after_attn_a2a: bool = False, + ): + assert isinstance(next_block, nn.Module), "transformer block must be type `torch.nn.Module`" + if type(dp_size) is not int or dp_size < 1: +@@ -167,6 +169,7 @@ class DistributedLayerwiseOffloadHook(ModelHook): + + self.chunk_size_bytes = chunk_size_bytes + self.manifest: PartManifest | None = None ++ self._deferred_chunked_allgather = None + self._h2d_done_events = [current_omni_platform.Event(), current_omni_platform.Event()] + self._ag_done_events = [current_omni_platform.Event(), current_omni_platform.Event()] + # Hook-local events protect chunk-input reuse within this hook. Across +@@ -239,6 +242,19 @@ class DistributedLayerwiseOffloadHook(ModelHook): + # Per-block synchronization primitive: set after H2D copy completes. + self._prefetch_done: Any | None = None + ++ # Optional ordering: hold the background weight AllGather until the ++ # block's QKV AllToAll is on the compute stream. See ++ # distributed/collective_gate.py. Off by default; the attention layer ++ # inside this block is handed the same barrier at install time. ++ self._a2a_barrier: QKVAllToAllBarrier | None = None ++ self._a2a_event: Any | None = None ++ # ``(slot, gpu_shards, gpu_weights, evt)`` of a gather armed but not yet ++ # issued, so ``post_forward`` can flush it if the attention never ran. ++ self._deferred_allgather: tuple[int, dict, dict, Any] | None = None ++ if order_allgather_after_attn_a2a: ++ self._a2a_event = current_omni_platform.Event() ++ self._a2a_barrier = QKVAllToAllBarrier(self._allgather_after_attn_a2a) ++ + # Shared shard (AllGather input) buffers — assigned by backend. + self.gpu_shard_buffers: list[dict[torch.dtype, torch.Tensor] | None] = [None, None] + +@@ -653,49 +669,23 @@ class DistributedLayerwiseOffloadHook(ModelHook): + gpu_shard = shard_bufs[dtype][: cpu_shard.numel()] + gpu_shard.copy_(cpu_shard, non_blocking=non_blocking) + gpu_shards[dtype] = gpu_shard +- self.comm_stream.wait_stream(self.copy_stream) +- with current_omni_platform.stream(self.comm_stream): +- for dtype, local_shard in gpu_shards.items(): +- gw = gpu_weights[dtype][: self._ag_output_sizes[dtype]] +- torch.distributed.all_gather_into_tensor( +- gw, +- local_shard, +- group=self.dp_group, +- ) +- evt.record(self.comm_stream) ++ ++ if self._a2a_barrier is not None and non_blocking and getattr(self, "_nz_cast_fn", None) is None: ++ # Hold the collective until the block's attention reports its ++ # QKV AllToAll, so the background gather cannot take the links ++ # ahead of the critical-path all-to-all. The H2D above has ++ # already been enqueued; only the gather moves, so it still ++ # overlaps the rest of the block. Deferral needs a repoint that ++ # emits no stream work, hence the NZ-cast guard. ++ self._deferred_allgather = (slot, gpu_shards, gpu_weights, evt) ++ self._a2a_barrier.arm() ++ else: ++ self._issue_allgather(slot, gpu_shards, gpu_weights, evt) ++ elif self._a2a_barrier is not None and non_blocking: ++ self._deferred_chunked_allgather = (slot, gpu_weights, evt) ++ self._a2a_barrier.arm() + else: +- for dtype_manifest in self.manifest.dtypes: +- cpu_shard = self.cpu_shards[dtype_manifest.dtype] +- buf = shard_bufs[dtype_manifest.dtype] +- for chunk in dtype_manifest.chunks: +- if buf.numel() >= 2 * chunk.local_numel: +- width = buf.numel() // 2 +- input_slot = chunk.chunk_id % 2 +- else: +- width = buf.numel() +- input_slot = 0 +- reuse = self._chunk_reuse_events[input_slot] +- if reuse is not None: +- self.copy_stream.wait_event(reuse) +- local_input = buf[input_slot * width : input_slot * width + chunk.local_numel] +- source = cpu_shard[chunk.cpu_offset : chunk.cpu_offset + chunk.local_numel] +- with current_omni_platform.stream(self.copy_stream): +- local_input.copy_(source, non_blocking=non_blocking) +- self._h2d_done_events[input_slot].record(self.copy_stream) +- self.comm_stream.wait_event(self._h2d_done_events[input_slot]) +- with current_omni_platform.stream(self.comm_stream): +- full_out = gpu_weights[dtype_manifest.dtype][ +- chunk.full_offset : chunk.full_offset + chunk.padded_numel +- ] +- torch.distributed.all_gather_into_tensor( +- full_out, +- local_input, +- group=self.dp_group, +- ) +- self._ag_done_events[input_slot].record(self.comm_stream) +- self._chunk_reuse_events[input_slot] = self._ag_done_events[input_slot] +- with current_omni_platform.stream(self.comm_stream): +- evt.record(self.comm_stream) ++ self._issue_chunked_allgather(slot, gpu_weights, evt, non_blocking=non_blocking) + + self.ready_events[slot] = evt + self._prefetch_done = evt +@@ -719,6 +709,131 @@ class DistributedLayerwiseOffloadHook(ModelHook): + ), + ) + ++ def _issue_chunked_allgather(self, slot, gpu_weights, evt, *, gated=False, non_blocking=True) -> None: ++ """Submit chunk copies and gathers together after the foreground gate. ++ ++ Chunk staging slots are bounded and reused. Deferring only gathers ++ would overwrite an input slot before its collective consumes it. ++ """ ++ shard_bufs = self.gpu_shard_buffers[slot] ++ assert shard_bufs is not None and self.manifest is not None ++ if gated: ++ self.comm_stream.wait_event(self._a2a_event) ++ for dtype_manifest in self.manifest.dtypes: ++ cpu_shard = self.cpu_shards[dtype_manifest.dtype] ++ buf = shard_bufs[dtype_manifest.dtype] ++ for chunk in dtype_manifest.chunks: ++ if buf.numel() >= 2 * chunk.local_numel: ++ width = buf.numel() // 2 ++ input_slot = chunk.chunk_id % 2 ++ else: ++ width = buf.numel() ++ input_slot = 0 ++ reuse = self._chunk_reuse_events[input_slot] ++ if reuse is not None: ++ self.copy_stream.wait_event(reuse) ++ local_input = buf[input_slot * width : input_slot * width + chunk.local_numel] ++ source = cpu_shard[chunk.cpu_offset : chunk.cpu_offset + chunk.local_numel] ++ with current_omni_platform.stream(self.copy_stream): ++ local_input.copy_(source, non_blocking=non_blocking) ++ self._h2d_done_events[input_slot].record(self.copy_stream) ++ self.comm_stream.wait_event(self._h2d_done_events[input_slot]) ++ with current_omni_platform.stream(self.comm_stream): ++ full_out = gpu_weights[dtype_manifest.dtype][ ++ chunk.full_offset : chunk.full_offset + chunk.padded_numel ++ ] ++ torch.distributed.all_gather_into_tensor( ++ full_out, ++ local_input, ++ group=self.dp_group, ++ ) ++ self._ag_done_events[input_slot].record(self.comm_stream) ++ self._chunk_reuse_events[input_slot] = self._ag_done_events[input_slot] ++ with current_omni_platform.stream(self.comm_stream): ++ evt.record(self.comm_stream) ++ ++ def _issue_allgather( ++ self, ++ slot: int, ++ gpu_shards: dict[torch.dtype, torch.Tensor], ++ gpu_weights: dict[torch.dtype, torch.Tensor], ++ evt: Any, ++ *, ++ gated: bool = False, ++ ) -> None: ++ """Submit this block's weight AllGather on ``comm_stream``. ++ ++ ``gated`` additionally waits on the QKV AllToAll barrier event recorded ++ by the block's attention, so the foreground collective gets the links ++ first and this gather queues behind it. ++ """ ++ self.comm_stream.wait_stream(self.copy_stream) ++ if gated: ++ self.comm_stream.wait_event(self._a2a_event) ++ with current_omni_platform.stream(self.comm_stream): ++ for dtype, local_shard in gpu_shards.items(): ++ # Slice the shared (max-sized) output buffer down to this ++ # block's actual AllGather output size. The buffers are ++ # sized to the *largest* block across all groups, so for any ++ # smaller block the full buffer would violate the ++ # all_gather_into_tensor contract ++ # (output.numel() == world_size * input.numel()). ++ # Repoint offsets are relative to the block's flattened ++ # buffer, so a prefix slice is safe. ++ gw = gpu_weights[dtype][: self._ag_output_sizes[dtype]] ++ torch.distributed.all_gather_into_tensor( ++ gw, ++ local_shard, ++ group=self.dp_group, ++ ) ++ evt.record(self.comm_stream) ++ ++ def _allgather_after_attn_a2a(self) -> None: ++ """Barrier callback: the block's QKV AllToAll is now enqueued. ++ ++ Recording on the compute stream here (rather than at block entry) is ++ what makes the gate effective: an event wait snapshots the records ++ enqueued before it, so the wait must be enqueued *after* the all-to-all. ++ """ ++ chunked = self._deferred_chunked_allgather ++ if chunked is not None: ++ self._deferred_chunked_allgather = None ++ assert self._a2a_event is not None ++ self._a2a_event.record(current_omni_platform.current_stream()) ++ self._issue_chunked_allgather(*chunked, gated=True) ++ return ++ deferred = self._deferred_allgather ++ if deferred is None: ++ return ++ self._deferred_allgather = None ++ assert self._a2a_event is not None ++ self._a2a_event.record(current_omni_platform.current_stream()) ++ slot, gpu_shards, gpu_weights, evt = deferred ++ self._issue_allgather(slot, gpu_shards, gpu_weights, evt, gated=True) ++ ++ def _flush_deferred_allgather(self) -> None: ++ """Issue a deferred gather whose attention never signalled. ++ ++ Keeps the double buffer live: a block whose attention is skipped ++ (cache-dit, an all-resident stage) still needs its weights, so the ++ gather is submitted ungated at the end of that block's forward. ++ """ ++ chunked = self._deferred_chunked_allgather ++ if chunked is not None: ++ self._deferred_chunked_allgather = None ++ if self._a2a_barrier is not None: ++ self._a2a_barrier.disarm() ++ self._issue_chunked_allgather(*chunked) ++ return ++ deferred = self._deferred_allgather ++ if deferred is None: ++ return ++ self._deferred_allgather = None ++ if self._a2a_barrier is not None: ++ self._a2a_barrier.disarm() ++ slot, gpu_shards, gpu_weights, evt = deferred ++ self._issue_allgather(slot, gpu_shards, gpu_weights, evt) ++ + def _wait_for_weights(self, slot: int) -> None: + """Wait until the slot holding this block is ready for compute. + +@@ -840,6 +955,10 @@ class DistributedLayerwiseOffloadHook(ModelHook): + return args, kwargs + + def post_forward(self, module: nn.Module, output: Any) -> Any: ++ # Pair with pre_forward: if the block's attention never signalled the ++ # gate, the deferred gather must still go out before this block's ++ # storage is released (offload_layer waits on _prefetch_done). ++ self._flush_deferred_allgather() + self.offload_layer() + return output + +@@ -864,6 +983,7 @@ def apply_distributed_block_hook( + tensor_transforms: dict[int, Any] | None = None, + materialization_probe_tensor: torch.Tensor | None = None, + chunk_size_bytes: int = 64 * 1024 * 1024, ++ order_allgather_after_attn_a2a: bool = False, + ) -> DistributedLayerwiseOffloadHook: + """Register a DistributedLayerwiseOffloadHook on *module*.""" + registry = HookRegistry.get_or_create(module) +@@ -881,11 +1001,32 @@ def apply_distributed_block_hook( + tensor_transforms=tensor_transforms, + materialization_probe_tensor=materialization_probe_tensor, + chunk_size_bytes=chunk_size_bytes, ++ order_allgather_after_attn_a2a=order_allgather_after_attn_a2a, + ) + registry.register_hook(DistributedLayerwiseOffloadHook._HOOK_NAME, hook) + return hook + + ++def attach_collective_gate(block: nn.Module, hook: DistributedLayerwiseOffloadHook) -> int: ++ """Hand ``hook``'s barrier to every Attention layer inside *block*. ++ ++ The attention signals the barrier right after its QKV AllToAll. The import ++ is local to keep this module independent of the attention package at import ++ time. ++ ++ Returns the number of attention layers wired (0 means the hook will rely on ++ the ``post_forward`` flush instead). ++ """ ++ from vllm_omni.diffusion.attention.layer import Attention ++ ++ attached = 0 ++ for module in block.modules(): ++ if isinstance(module, Attention): ++ module._collective_gate = hook._a2a_barrier ++ attached += 1 ++ return attached ++ ++ + def remove_distributed_block_hook(module: nn.Module) -> None: + """Remove the distributed layerwise offload hook from *module*.""" + registry: HookRegistry | None = getattr(module, "_hook_registry", None) +@@ -1566,29 +1707,45 @@ class DistributedLayerwiseOffloadBackend(OffloadBackend): + self._all_hook_groups.append(hooks) + self._blocks.append(block_list) + probes = {id(block): module_materialization_probe(block) for block in block_list} ++ # Only the DiT blocks issue a QKV AllToAll the gather can queue behind. ++ order_gate = bool(getattr(self.config, "dlo_order_allgather_after_attn_a2a", False)) and ( ++ component == DIT_COMPONENT ++ ) ++ if order_gate and not getattr(self, "_collective_gate_logged", False): ++ self._collective_gate_logged = True ++ logger.info( ++ "Distributed layerwise offload will hold each DiT block's weight AllGather " ++ "until that block's QKV AllToAll is on the compute stream" ++ ) + for block, next_block in zip( + chain((block_list[-1],), block_list[:-1]), + block_list, + strict=True, + ): +- hooks.append( +- apply_distributed_block_hook( +- block, +- next_block, +- self.device, +- group, +- group_size, +- group_rank, +- self.copy_stream, +- self.comm_stream, +- self.config.pin_cpu_memory, +- shared_buffers=[None, None], +- rank_local_mmap=self._using_rank_local_mmap if use_dit_mmap else False, +- tensor_transforms=self._mmap_transforms_by_tensor_id if use_dit_mmap else None, +- materialization_probe_tensor=probes[id(block)], +- chunk_size_bytes=self.config.chunk_size_bytes, +- ) ++ hook = apply_distributed_block_hook( ++ block, ++ next_block, ++ self.device, ++ group, ++ group_size, ++ group_rank, ++ self.copy_stream, ++ self.comm_stream, ++ self.config.pin_cpu_memory, ++ shared_buffers=[None, None], ++ rank_local_mmap=self._using_rank_local_mmap if use_dit_mmap else False, ++ tensor_transforms=self._mmap_transforms_by_tensor_id if use_dit_mmap else None, ++ materialization_probe_tensor=probes[id(block)], ++ chunk_size_bytes=self.config.chunk_size_bytes, ++ order_allgather_after_attn_a2a=order_gate, + ) ++ if hook._a2a_barrier is not None and attach_collective_gate(block, hook) == 0: ++ logger.warning( ++ "DLO AllGather ordering is enabled but block %s exposes no Attention layer; " ++ "its gather will fall back to the ungated path", ++ block.__class__.__name__, ++ ) ++ hooks.append(hook) + + # hooks = [last -> first, block0 -> block1, ...]. Alternating slots + # keep a prefetch from overwriting the current block for any ring size. +diff --git a/vllm_omni/engine/arg_utils.py b/vllm_omni/engine/arg_utils.py +index 0a876b9..bf8dcdc 100644 +--- a/vllm_omni/engine/arg_utils.py ++++ b/vllm_omni/engine/arg_utils.py +@@ -807,6 +807,7 @@ class OrchestratorArgs: + dlo_use_allgather: bool = True + dlo_chunk_size_mb: int = 64 + dlo_resident_layers: int = 0 ++ dlo_order_allgather_after_attn_a2a: bool = False + host_weight_runtime_mode: str = "disabled" + host_weight_runtime_root: str | None = None + dlo_host_registration_limit_gib: float = 0.0 +diff --git a/vllm_omni/entrypoints/cli/serve.py b/vllm_omni/entrypoints/cli/serve.py +index f2b55e7..aff71b5 100644 +--- a/vllm_omni/entrypoints/cli/serve.py ++++ b/vllm_omni/entrypoints/cli/serve.py +@@ -1041,6 +1041,17 @@ class OmniServeCommand(CLISubcommand): + help="Full-chunk target size in MiB for distributed layerwise offload " + "(each chunk is one host-to-device copy + AllGather unit).", + ) ++ omni_config_group.add_argument( ++ "--dlo-order-allgather-after-attn-a2a", ++ action="store_true", ++ default=False, ++ help=( ++ "Hold each DiT block's weight AllGather until that block's QKV AllToAll " ++ "has been enqueued, so the background gather does not take the collective " ++ "links ahead of the critical-path all-to-all. Requires distributed " ++ "layerwise offload with AllGather." ++ ), ++ ) + omni_config_group.add_argument( + "--host-weight-runtime-mode", + choices=("disabled", "preferred", "required"), diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c13-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c13-source-identity.json new file mode 100644 index 000000000..939545aaf --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c13-source-identity.json @@ -0,0 +1,21 @@ +{ + "head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "tree": "cc5924c4270350d34c8d3f2213bce95d88fa2210", + "base": "be319692b96413918973b5bd399cdda3f320c574", + "branch": "rsi/h3-297-c1-dlo-stack", + "files": { + "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", + "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" + }, + "published": false, + "patch_base": "be319692b96413918973b5bd399cdda3f320c574" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c15-candidate.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c15-candidate.patch new file mode 100644 index 000000000..5410afa5b --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c15-candidate.patch @@ -0,0 +1,890 @@ +diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py +new file mode 100644 +index 0000000..ce8464c +--- /dev/null ++++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py +@@ -0,0 +1,380 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++from datetime import timedelta ++from types import SimpleNamespace ++ ++import pytest ++import torch ++import torch.distributed as dist ++import torch.nn as nn ++ ++from tests.helpers.mark import hardware_marks ++from vllm_omni.diffusion.models.minimax_h3 import minimax_h3_blocks as blocks ++from vllm_omni.diffusion.models.minimax_h3.attention.flash_h3 import _prefix_bnsd, _subset_prefix_bnsd ++from vllm_omni.diffusion.models.minimax_h3.video_row_cache import MiniMaxH3VideoRowCache ++from vllm_omni.platforms import current_omni_platform ++ ++pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=4)] ++ ++ ++def state_for_rows(device="cpu", start=0, length=4): ++ state = MiniMaxH3VideoRowCache(torch.tensor([1, 2], device=device), seq_len=4, used_len=3, num_layers=1) ++ state.prepare_local_span(start, length) ++ return state ++ ++ ++def anchor(state, x): ++ for step in range(5): ++ state.begin_step(step) ++ state.capture(0, x, x + 0.125) ++ state.end_step() ++ state.begin_step(5) ++ ++ ++def test_fixed_schedule_and_row_ownership_excludes_padding(): ++ assert [i for i in range(50) if MiniMaxH3VideoRowCache.is_full_step(i)] == [ ++ 0, ++ 1, ++ 2, ++ 3, ++ 4, ++ 9, ++ 14, ++ 19, ++ 24, ++ 29, ++ 34, ++ 39, ++ 44, ++ 46, ++ 47, ++ 48, ++ 49, ++ ] ++ state = state_for_rows() ++ assert state.global_dense_positions.tolist() == [0] ++ assert state.local_dense_positions.tolist() == [0] ++ assert state.local_video_positions.tolist() == [1, 2] ++ with pytest.raises(RuntimeError, match="ownership"): ++ state.prepare_local_span(0, 3) ++ ++ ++@pytest.mark.parametrize("positions", [[1, 1], [2, 1], [-1], [3]]) ++def test_invalid_target_positions_are_rejected(positions): ++ with pytest.raises(ValueError): ++ MiniMaxH3VideoRowCache(torch.tensor(positions), seq_len=4, used_len=3, num_layers=1) ++ ++ ++def test_delta_is_fp16_and_restore_owns_output(): ++ state = state_for_rows() ++ x = torch.tensor([[1, 2], [3, 4], [5, 6], [7, 8]], dtype=torch.bfloat16) ++ anchor(state, x) ++ assert state._deltas[0].dtype == torch.float16 ++ before = x.clone() ++ output = state.restore(0, x) ++ assert torch.equal(output[1:3], x[1:3] + 0.125) ++ assert torch.equal(output[[0, 3]], x[[0, 3]]) ++ output.zero_() ++ assert torch.equal(x, before) ++ state.end_step() ++ state.close() ++ assert state.closed and not state._deltas and state.global_dense_positions.numel() == 0 ++ ++ ++class CountMLP(nn.Module): ++ def __init__(self): ++ super().__init__() ++ self.rows = [] ++ ++ def forward(self, x): ++ self.rows.append(x.shape[0]) ++ return x + 0.25 ++ ++ ++def tiny_block(): ++ block = object.__new__(blocks.MiniMaxH3DiTBlock) ++ nn.Module.__init__(block) ++ block.layer_index = 0 ++ block.norm1 = SimpleNamespace(weight=torch.ones(2), variance_epsilon=1e-6) ++ block.norm2 = SimpleNamespace(weight=torch.ones(2), variance_epsilon=1e-6) ++ block.adaln_proj = lambda t: ( ++ torch.zeros(3, 2), ++ torch.zeros(3, 2), ++ torch.ones(3, 2), ++ torch.zeros(3, 2), ++ torch.zeros(3, 2), ++ torch.ones(3, 2), ++ ) ++ block.attn = lambda x, **kwargs: x * 0.5 ++ block.mlp = CountMLP() ++ return block ++ ++ ++@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) ++def test_real_block_dense_path_and_partial_mlp_empty_rows(start, length): ++ state = state_for_rows(start=start, length=length) ++ block = tiny_block() ++ x = torch.arange(length * 2).reshape(length, 2).to(torch.bfloat16) ++ before = x.clone() ++ kwargs = dict( ++ t_emb=torch.zeros(1), ++ combined_indices=torch.zeros(length, dtype=torch.long), ++ rope_table=None, ++ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), ++ max_seqlen=3, ++ packed_total=4, ++ ) ++ expected = block(x, **kwargs) ++ for step in range(5): ++ state.begin_step(step) ++ actual = block(x, video_row_cache=state, **kwargs) ++ assert torch.equal(actual.view(torch.uint8), expected.view(torch.uint8)) ++ state.end_step() ++ block.mlp.rows.clear() ++ state.begin_step(5) ++ actual = block(x, video_row_cache=state, **kwargs) ++ state.end_step() ++ rows = state.local_dense_positions ++ assert block.mlp.rows == ([rows.numel()] if rows.numel() else []) ++ assert torch.equal(actual.index_select(0, rows), expected.index_select(0, rows)) ++ assert torch.equal(x, before) ++ ++ ++@pytest.mark.parametrize("failure", ["success", "model", "cancel"]) ++def test_request_state_closed_and_removed_on_every_exit(monkeypatch, failure): ++ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop ++ ++ monkeypatch.setenv("H3_VIDEO_ROW_CACHE", "1") ++ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) ++ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) ++ state = MiniMaxH3VideoRowCache(torch.tensor([1]), seq_len=4, used_len=3, num_layers=1) ++ ++ class Model: ++ def create_video_row_cache(self, *args, **kwargs): ++ return state ++ ++ def __call__(self, **kwargs): ++ assert kwargs["video_row_cache"] is state ++ if failure == "model": ++ raise RuntimeError("model failed") ++ state.prepare_local_span(0, 4) ++ x = torch.ones(4, 2, dtype=torch.bfloat16) ++ if state.partial: ++ state.restore(0, x) ++ else: ++ state.capture(0, x, x + 0.125) ++ return torch.zeros(2, 96), torch.zeros(1, 32) ++ ++ class Branch: ++ task = "ref2va" ++ seq_len = 4 ++ used_len = 3 ++ img_pos = torch.tensor([0, 1]) ++ update_mask = torch.tensor([False, True]) ++ update_mask_dev = update_mask ++ audio_update_mask_dev = torch.tensor([True]) ++ locked_audio_rows = None ++ static_kwargs = {"video_token_layout": SimpleNamespace(video_spans=[SimpleNamespace(role="target", start=1)])} ++ ++ def forward_kwargs(self, **kwargs): ++ return dict(self.static_kwargs) ++ ++ monkeypatch.setattr( ++ loop, ++ "minimax_h3_prepare_denoise_rows", ++ lambda **kwargs: (kwargs["initial_video_rows"], kwargs["initial_audio_rows"], torch.zeros(1, 96), None), ++ ) ++ ++ def cancellation(**kwargs): ++ if failure == "cancel": ++ raise RuntimeError("request cancelled") ++ ++ monkeypatch.setattr(loop, "check_request_cancellation", cancellation) ++ branch, model = Branch(), Model() ++ kwargs = dict( ++ model=model, ++ positive=branch, ++ initial_video_rows=torch.zeros(2, 96), ++ initial_audio_rows=torch.zeros(1, 32), ++ keyframe_cond_rows=torch.zeros(1, 96), ++ sigmas_video=[1 - i / 50 for i in range(51)], ++ sigmas_audio=[1 - i / 50 for i in range(51)], ++ device=torch.device("cpu"), ++ ) ++ if failure == "success": ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert state.full_steps == 17 and state.partial_steps == 33 ++ else: ++ with pytest.raises(RuntimeError): ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert state.closed and not state._deltas and "video_row_cache" not in branch.static_kwargs ++ assert model._video_row_cache_stats["closed"] ++ ++ ++def test_default_factory_does_not_create_state(monkeypatch): ++ from vllm_omni.diffusion.models.minimax_h3.minimax_h3_transformer import MiniMaxH3DiTModel ++ ++ monkeypatch.delenv("H3_VIDEO_ROW_CACHE", raising=False) ++ assert ( ++ MiniMaxH3DiTModel.create_video_row_cache(None, torch.tensor([1]), seq_len=4, used_len=3, num_steps=50) is None ++ ) ++ ++ ++@pytest.mark.parametrize("ref2va", [False, True]) ++def test_real_packing_task_selects_ref2va_and_rejects_fl2va(monkeypatch, ref2va): ++ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop ++ from vllm_omni.diffusion.models.minimax_h3.packed_sequence import ( ++ minimax_h3_packed_sequence, ++ minimax_h3_packed_sequence_ref2va_blocks, ++ ) ++ ++ common = dict(text_len=2, latent_t=2, latent_h=4, latent_w=4, audio_t=2) ++ packed = ( ++ minimax_h3_packed_sequence_ref2va_blocks(**common, ref_blocks=[{"kind": "image", "latent_h": 4, "latent_w": 4}]) ++ if ref2va ++ else minimax_h3_packed_sequence(**common, include_keyframe_cond=True, keyframe_frame_indices=[0], frame_count=5) ++ ) ++ branch = loop.MiniMaxH3DenoiseBranch( ++ packed=packed, ++ text_embeddings=torch.zeros(2, 5120), ++ token_tags=packed["token_tags"], ++ device=torch.device("cpu"), ++ task="ref2va" if ref2va else "fl2va", ++ ) ++ layout = branch.static_kwargs["video_token_layout"] ++ assert not any(span.role == "reference" for span in layout.video_spans) ++ monkeypatch.setenv("H3_VIDEO_ROW_CACHE", "1") ++ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) ++ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) ++ monkeypatch.setattr(loop, "check_request_cancellation", lambda **kwargs: None) ++ states = [] ++ ++ class Model: ++ def create_video_row_cache(self, positions, **kwargs): ++ state = MiniMaxH3VideoRowCache( ++ positions, seq_len=kwargs["seq_len"], used_len=kwargs["used_len"], num_layers=1 ++ ) ++ state.prepare_local_span(0, branch.seq_len) ++ states.append(state) ++ return state ++ ++ def __call__(self, **kwargs): ++ state = kwargs["video_row_cache"] ++ x = torch.ones(branch.seq_len, 2, dtype=torch.bfloat16) ++ if state.partial: ++ state.restore(0, x) ++ else: ++ state.capture(0, x, x + 0.125) ++ return torch.zeros(branch.img_pos.numel(), 96), torch.zeros(branch.audio_pos.numel(), 32) ++ ++ kwargs = dict( ++ model=Model(), ++ positive=branch, ++ initial_video_rows=torch.zeros(branch.img_pos.numel(), 96), ++ initial_audio_rows=torch.zeros(branch.audio_pos.numel(), 32), ++ keyframe_cond_rows=torch.zeros(int((~branch.update_mask).sum()), 96), ++ sigmas_video=[1 - i / 50 for i in range(51)], ++ sigmas_audio=[1 - i / 50 for i in range(51)], ++ device=torch.device("cpu"), ++ ) ++ if ref2va: ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert len(states) == 1 and states[0].closed ++ assert states[0].full_steps == 17 and states[0].partial_steps == 33 ++ else: ++ with pytest.raises(ValueError, match="requires Ref2VA"): ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert not states ++ ++ ++@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) ++def test_attention_keeps_collective_path_and_projects_only_dense_local_rows(start, length): ++ state = state_for_rows(start=start, length=length) ++ x = torch.ones(length, 2, dtype=torch.bfloat16) ++ anchor(state, x) ++ attention = object.__new__(blocks.MiniMaxH3Attention) ++ nn.Module.__init__(attention) ++ attention.num_heads = 1 ++ attention.head_dim = 2 ++ attention.vdn = None ++ attention.to_gate_compress = None ++ calls, projected_rows = [], [] ++ attention._project_qkv = lambda *args: (x.reshape(length, 1, 2),) * 5 ++ ++ def packed(q, k, v, **kwargs): ++ calls.append(q.shape[0]) ++ assert kwargs["video_row_cache"] is state ++ return q ++ ++ def project(rows): ++ projected_rows.append(rows.shape[0]) ++ return rows * 2, None ++ ++ attention._run_packed_attention = packed ++ attention.out_proj = project ++ output = attention( ++ x, ++ rope_table=None, ++ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), ++ max_seqlen=3, ++ packed_total=4, ++ video_row_cache=state, ++ ) ++ rows = state.local_dense_positions ++ assert calls == [length] ++ assert projected_rows == ([rows.numel()] if rows.numel() else []) ++ assert torch.equal(output.index_select(0, rows), (x * 2).index_select(0, rows)) ++ assert torch.count_nonzero(output.index_select(0, state.local_video_positions)) == 0 ++ ++ ++def _native_worker(rank): ++ import torch_npu # noqa: F401 ++ ++ torch.npu.set_device(rank) ++ dist.init_process_group( ++ "hccl", init_method="tcp://127.0.0.1:29597", rank=rank, world_size=4, timeout=timedelta(seconds=120) ++ ) ++ try: ++ torch.manual_seed(19) ++ full = [torch.randn(32, 4, 128, dtype=torch.bfloat16).npu() for _ in range(3)] ++ exchanged = [] ++ for tensor in full: ++ local = tensor[rank * 8 : (rank + 1) * 8] ++ send = local.reshape(8, 4, 1, 128).permute(1, 0, 2, 3).contiguous() ++ receive = torch.empty_like(send) ++ dist.all_to_all_single(receive, send) ++ exchanged.append(receive.reshape(1, 32, 1, 128)) ++ state = MiniMaxH3VideoRowCache(torch.arange(8, 24, device="npu"), seq_len=32, used_len=30, num_layers=1) ++ state.prepare_local_span(rank * 8, 8) ++ x = torch.ones(8, 2, dtype=torch.bfloat16, device="npu") ++ anchor(state, x) ++ output = _subset_prefix_bnsd(exchanged[0], exchanged[1][:, :30], exchanged[2][:, :30], 128**-0.5, state) ++ send = output.reshape(4, 8, 1, 128).contiguous() ++ receive = torch.empty_like(send) ++ dist.all_to_all_single(receive, send) ++ local_output = receive.permute(1, 0, 2, 3).reshape(8, 4, 128) ++ expected = _prefix_bnsd(full[0].unsqueeze(0), full[1][:30].unsqueeze(0), full[2][:30].unsqueeze(0), 128**-0.5)[ ++ 0 ++ ] ++ rows = state.local_dense_positions ++ torch.testing.assert_close( ++ local_output.index_select(0, rows).float(), ++ expected[rank * 8 : (rank + 1) * 8].index_select(0, rows).float(), ++ atol=0.03125, ++ rtol=0.02, ++ ) ++ assert state.subset_attention_calls == 1 and state.subset_query_rows == 14 ++ assert rows.numel() == [8, 0, 0, 6][rank] ++ state.restore(0, x) ++ state.end_step() ++ state.close() ++ torch.npu.synchronize() ++ dist.barrier() ++ finally: ++ dist.destroy_process_group() ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires four native Ascend devices") ++def test_native_four_rank_subset_attention_with_empty_local_dense_rows(): ++ torch.multiprocessing.spawn(_native_worker, nprocs=4, join=True) +diff --git a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py +index 6ebfb10..044c1f0 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py ++++ b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py +@@ -36,6 +36,16 @@ def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, sc + return output.reshape(batch, heads, rows, dim).transpose(1, 2).contiguous() + + ++def _subset_prefix_bnsd(query, key, value, scale, state): ++ """Compute only fresh conditioning queries; preserve the shared SP shape.""" ++ rows = state.global_dense_positions ++ dense = _prefix_bnsd(query.index_select(1, rows), key, value, scale) ++ output = torch.zeros_like(query) ++ output.index_copy_(1, rows, dense) ++ state.record_subset_attention(rows.numel()) ++ return output ++ ++ + class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + """Use independent head batches for explicit long BF16 prefix attention. + +@@ -51,7 +61,10 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + value: torch.Tensor, + attn_metadata: AttentionMetadata | None = None, + ) -> torch.Tensor: ++ state = attn_metadata.extra.get("minimax_h3_video_row_cache") if attn_metadata is not None else None + if os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" or attn_metadata is None: ++ if state is not None: ++ raise RuntimeError("H3 video-row cache requires the explicit fused attention path") + return super().forward_fa_npu(query, key, value, attn_metadata) + extra = attn_metadata.extra + if ( +@@ -68,9 +81,17 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + or query.shape[2:] != value.shape[2:] + or key.shape[1] != value.shape[1] + ): ++ if state is not None: ++ raise RuntimeError("H3 video-row cache received unsupported fused attention inputs") + return super().forward_fa_npu(query, key, value, attn_metadata) + resolved = self._resolve_packed_seq_npu(query, key, extra) + if resolved is None: ++ if state is not None: ++ raise RuntimeError("H3 video-row cache requires a resolved packed prefix") + return super().forward_fa_npu(query, key, value, attn_metadata) + used_k = resolved[1][0] ++ if state is not None: ++ if not state.partial or used_k != state.used_len or query.shape[1] != state.seq_len: ++ raise RuntimeError("H3 video-row cache attention layout changed") ++ return _subset_prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale, state) + return _prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale) +diff --git a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py +index 8cca219..07fb372 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py ++++ b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py +@@ -9,6 +9,7 @@ rows stay pinned to their noised step-0 anchors. + + from __future__ import annotations + ++import os + from collections.abc import Callable + from contextlib import AbstractContextManager, nullcontext + from typing import Any +@@ -69,7 +70,9 @@ class MiniMaxH3DenoiseBranch: + text_embeddings: torch.Tensor, + token_tags: torch.Tensor, + device: torch.device, ++ task: str | None = None, + ) -> None: ++ self.task = task + seq_len = int(packed["seq_len"]) + self.seq_len = seq_len + self.img_pos = packed["img_pos"].view(-1).to(torch.long) +@@ -380,6 +383,24 @@ def minimax_h3_denoise_loop( + positive.static_kwargs["reference_kv_state"] = reference_kv_state + + try: ++ video_row_cache = None ++ create_video_cache = getattr(model, "create_video_row_cache", None) ++ if os.environ.get("H3_VIDEO_ROW_CACHE") == "1": ++ if ( ++ not callable(create_video_cache) ++ or video_edit is not None ++ or audio_edit is not None ++ or keyframe_cond_rows is None ++ or positive.task != "ref2va" ++ ): ++ raise ValueError("H3 video-row cache requires Ref2VA image references without editing") ++ video_row_cache = create_video_cache( ++ positive.img_pos[positive.update_mask].to(device), ++ seq_len=positive.seq_len, ++ used_len=positive.used_len, ++ num_steps=num_steps, ++ ) ++ positive.static_kwargs["video_row_cache"] = video_row_cache + for step in range(num_steps): + check_request_cancellation() + step_cm = step_profiler(step) if step_profiler is not None else nullcontext() +@@ -388,6 +409,8 @@ def minimax_h3_denoise_loop( + s_a = sigmas_audio[step] + if reference_kv_state is not None: + reference_kv_state.begin_step(step, sigma=float(s_v)) ++ if video_row_cache is not None: ++ video_row_cache.begin_step(step) + # Publish where we are so step-gated attention features (the dense + # warmup of RAINFUSION_ATTN, the timestep gate of TRTLLM_ATTN) can + # see it. Gates use the scheduler-style descending timestep, which +@@ -430,6 +453,8 @@ def minimax_h3_denoise_loop( + v_video, v_audio = model(**fk) + if reference_kv_state is not None: + reference_kv_state.end_step() ++ if video_row_cache is not None: ++ video_row_cache.end_step() + mv_video_t = v_video.float()[update] + mv_audio_t = v_audio.float()[audio_update] + +@@ -476,6 +501,10 @@ def minimax_h3_denoise_loop( + + finally: + minimax_h3_publish_denoise_progress(None, None, None) ++ if video_row_cache is not None: ++ positive.static_kwargs.pop("video_row_cache", None) ++ video_row_cache.close() ++ model._video_row_cache_stats = video_row_cache.stats() + if reference_kv_state is not None: + positive.static_kwargs.pop("reference_kv_state", None) + reference_kv_state.close() +diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py +index 09f2e7d..94ef291 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py ++++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py +@@ -45,6 +45,7 @@ if TYPE_CHECKING: + from vllm.model_executor.layers.quantization.base_config import QuantizationConfig + + from .reference_kv_cache import MiniMaxH3ReferenceKVTier2State ++ from .video_row_cache import MiniMaxH3VideoRowCache + + _BF16_DTYPE = torch.bfloat16 + _FP32_DTYPE = torch.float32 +@@ -291,6 +292,7 @@ class MiniMaxH3Attention(nn.Module): + reference_kv_layer_index: int | None = None, + reference_kv_compact: bool = False, + vdn_window: VDNLayout | None = None, ++ video_row_cache: MiniMaxH3VideoRowCache | None = None, + ) -> torch.Tensor: + """Run packed attention as a small eager island. + +@@ -368,6 +370,8 @@ class MiniMaxH3Attention(nn.Module): + "vsa_h3_ref_policy": self.vsa_ref_policy, + "vsa_h3_ref_keep_rate": self.vsa_ref_keep_rate, + } ++ if video_row_cache is not None and video_row_cache.partial: ++ extra["minimax_h3_video_row_cache"] = video_row_cache + if gate_compress is not None: + extra["gate_compress"] = gate_compress.unsqueeze(0) + if video_layout is not None and video_layout.video_spans: +@@ -455,6 +459,7 @@ class MiniMaxH3Attention(nn.Module): + reference_kv_layer_index: int | None = None, + reference_kv_compact: bool = False, + vdn_window: VDNLayout | None = None, ++ video_row_cache: MiniMaxH3VideoRowCache | None = None, + ) -> torch.Tensor: + """x: [T, hidden] packed thd rows -> [T, hidden]. + +@@ -508,10 +513,19 @@ class MiniMaxH3Attention(nn.Module): + reference_kv_layer_index=reference_kv_layer_index, + reference_kv_compact=reference_kv_compact, + vdn_window=vdn_window, ++ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), + ) + if self.vdn is not None: + out = self.vdn.gate_softmax(out, x) + out = out.reshape(total, self.num_heads * self.head_dim) ++ if video_row_cache is not None and video_row_cache.partial: ++ rows = video_row_cache.local_dense_positions ++ assert rows is not None ++ projected = x.new_zeros((total, x.shape[-1])) ++ if rows.numel(): ++ dense, _ = self.out_proj(out.index_select(0, rows)) ++ projected.index_copy_(0, rows, dense) ++ return projected + out, _ = self.out_proj(out) + if self.vdn is not None: + out = self.vdn.add_linear(out, x, q_raw, k_raw, v, vdn_window) +@@ -742,6 +756,7 @@ class MiniMaxH3DiTBlock(nn.Module): + reference_kv_state: MiniMaxH3ReferenceKVTier2State | None = None, + reference_kv_compact: bool = False, + vdn_window: VDNLayout | None = None, ++ video_row_cache: MiniMaxH3VideoRowCache | None = None, + ) -> torch.Tensor: + """x: [T, H]; t_emb: [M, t_dim]; combined_indices: [T] + (= inverse_indices * modality_num + token_tags.clamp(min=0)). +@@ -759,6 +774,7 @@ class MiniMaxH3DiTBlock(nn.Module): + gate_mlp, + ) = self.adaln_proj(t_emb) + ++ block_input = x + residual = x + h = rms_norm_indexed_scale_shift( + x, +@@ -782,6 +798,7 @@ class MiniMaxH3DiTBlock(nn.Module): + reference_kv_layer_index=self.layer_index, + reference_kv_compact=reference_kv_compact, + vdn_window=vdn_window, ++ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), + ) + x, h = indexed_gate_rms_norm_scale_shift( + residual, +@@ -794,5 +811,22 @@ class MiniMaxH3DiTBlock(nn.Module): + self.norm2.variance_epsilon, + ) + residual = x ++ if video_row_cache is not None and video_row_cache.partial: ++ rows = video_row_cache.local_dense_positions ++ assert rows is not None ++ output = video_row_cache.restore(self.layer_index, block_input) ++ if rows.numel(): ++ dense = indexed_gate( ++ residual.index_select(0, rows), ++ gate_mlp, ++ self.mlp(h.index_select(0, rows)), ++ combined_indices.index_select(0, rows), ++ ) ++ output.index_copy_(0, rows, dense) ++ video_row_cache.partial_mlp_rows += rows.numel() ++ return output + h = self.mlp(h) +- return indexed_gate(residual, gate_mlp, h, combined_indices) ++ output = indexed_gate(residual, gate_mlp, h, combined_indices) ++ if video_row_cache is not None: ++ video_row_cache.capture(self.layer_index, block_input, output) ++ return output +diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py +index 9518828..2cf7aae 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py ++++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py +@@ -10,6 +10,7 @@ layout. + from __future__ import annotations + + import math ++import os + from bisect import bisect_left + from collections.abc import Iterable, Mapping + from dataclasses import replace +@@ -66,6 +67,8 @@ if TYPE_CHECKING: + from vllm_omni.diffusion.data import OmniDiffusionConfig + from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache import MiniMaxH3ReferenceKVTier2State + ++ from .video_row_cache import MiniMaxH3VideoRowCache ++ + logger = init_logger(__name__) + + +@@ -245,6 +248,7 @@ _FORWARD_SUPPORTED_KWARGS = frozenset( + "control_rows", + "control_context_scale", + "reference_kv_state", ++ "video_row_cache", + } + ) + +@@ -1232,6 +1236,35 @@ class MiniMaxH3DiTModel(nn.Module): + t_emb = self.time_embedder(unique_timesteps) + return embeddings, t_emb + ++ def create_video_row_cache( ++ self, target_positions: torch.Tensor, *, seq_len: int, used_len: int, num_steps: int ++ ) -> MiniMaxH3VideoRowCache | None: ++ if os.environ.get("H3_VIDEO_ROW_CACHE") != "1": ++ return None ++ p = self.parallel_config ++ if ( ++ not current_omni_platform.is_npu() ++ or num_steps != 50 ++ or p.tensor_parallel_size != 1 ++ or p.data_parallel_size not in (None, 1) ++ or p.cfg_parallel_size != 1 ++ or p.pipeline_parallel_size != 1 ++ or p.ulysses_degree != 4 ++ or p.ring_degree != 1 ++ or p.allgather_degree != 1 ++ or p.ulysses_mode != "strict" ++ or self.od_config.cache_backend not in (None, "none") ++ or self.od_config.kv_offload_config is not None ++ or self.vdn_config is not None ++ or os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" ++ or any(block.attn.attention.attn_backend.get_name() != "FLASH_ATTN" for block in self.blocks) ++ ): ++ raise ValueError("H3 video-row cache requires dense NPU BF16 Ref2VA with strict Ulysses4, TP/DP/CFG/PP1") ++ from .video_row_cache import MiniMaxH3VideoRowCache ++ ++ self._video_row_cache_stats = None ++ return MiniMaxH3VideoRowCache(target_positions, seq_len=seq_len, used_len=used_len, num_layers=len(self.blocks)) ++ + def create_reference_kv_state( + self, + *, +@@ -1308,6 +1341,14 @@ class MiniMaxH3DiTModel(nn.Module): + refiner_max = int(self._psp_field(refiner_psp, "refiner_packed_seq_params", "max_seqlen_q")) + video_layout = kwargs.get("video_token_layout") + reference_kv_state = kwargs.get("reference_kv_state") ++ video_row_cache = kwargs.get("video_row_cache") ++ if video_row_cache is not None and ( ++ num_requests != 1 ++ or reference_kv_state is not None ++ or self.vdn_config is not None ++ or kwargs.get("control_rows") is not None ++ ): ++ raise ValueError("H3 video-row cache cannot combine with packed requests, reference KV, VDN or control") + vdn_window = None + if self.vdn_config is not None: + # Text rows lead the packed document. +@@ -1442,6 +1483,11 @@ class MiniMaxH3DiTModel(nn.Module): + if reference_kv_state is not None: + reference_kv_state.set_local_sequence_rows(hidden.shape[0]) + ++ if video_row_cache is not None: ++ if hidden.shape[0] != local_len: ++ raise RuntimeError("H3 video-row cache requires the strict local sequence shard") ++ video_row_cache.prepare_local_span(local_start, local_len) ++ + block_video_layout = video_layout + if compact_reference: + block_video_layout = None +@@ -1505,6 +1551,7 @@ class MiniMaxH3DiTModel(nn.Module): + reference_kv_state=reference_kv_state, + reference_kv_compact=compact_reference, + vdn_window=vdn_window, ++ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), + ) + if block_index in hints: + hidden = hidden + hints.pop(block_index) * control_scale +diff --git a/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py b/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py +index 476c3cc..ebce8fc 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py ++++ b/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py +@@ -2115,6 +2115,7 @@ class MiniMaxH3Pipeline( + text_embeddings=text_embeddings, + token_tags=tags, + device=self.device, ++ task=task, + ) + + if control_rows is not None: +diff --git a/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py b/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py +new file mode 100644 +index 0000000..f8437f7 +--- /dev/null ++++ b/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py +@@ -0,0 +1,155 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++"""Request-owned, experimental target-video block residuals for H3 Ref2VA.""" ++ ++from __future__ import annotations ++ ++import torch ++ ++ ++class MiniMaxH3VideoRowCache: ++ """Keep non-video computation dense while reusing video block updates. ++ ++ The first experimental profile is deliberately fixed. No state is shared ++ across requests, and every rank retains all attention collectives. ++ """ ++ ++ def __init__(self, target_positions: torch.Tensor, *, seq_len: int, used_len: int, num_layers: int) -> None: ++ positions = target_positions.detach().cpu().tolist() ++ if not 0 < used_len <= seq_len or not positions or num_layers < 1: ++ raise ValueError("video-row cache requires a nonempty valid packed document") ++ if positions != sorted(set(positions)) or positions[0] < 0 or positions[-1] >= used_len: ++ raise ValueError("target-video positions must be sorted unique valid rows") ++ target_set = set(positions) ++ dense = [i for i in range(used_len) if i not in target_set] ++ if not dense: ++ raise ValueError("video-row cache requires non-video conditioning rows") ++ self.seq_len = seq_len ++ self.used_len = used_len ++ self.num_layers = num_layers ++ self._target_positions = positions ++ self.global_dense_positions = torch.tensor(dense, dtype=torch.long, device=target_positions.device) ++ self.local_dense_positions: torch.Tensor | None = None ++ self.local_video_positions: torch.Tensor | None = None ++ self._local_span: tuple[int, int] | None = None ++ self._deltas: dict[int, torch.Tensor] = {} ++ self._step = -1 ++ self._blocks_this_step = 0 ++ self.partial = False ++ self.closed = False ++ self.full_steps = 0 ++ self.partial_steps = 0 ++ self.full_blocks = 0 ++ self.partial_blocks = 0 ++ self.subset_attention_calls = 0 ++ self.subset_query_rows = 0 ++ self.omitted_query_rows = 0 ++ self.partial_mlp_rows = 0 ++ self.cache_bytes = 0 ++ ++ @staticmethod ++ def is_full_step(step: int) -> bool: ++ if not 0 <= step < 50: ++ raise ValueError("the video-row cache profile requires 50 steps") ++ return step < 4 or step >= 46 or (step - 4) % 5 == 0 ++ ++ def begin_step(self, step: int) -> None: ++ if self.closed or step != self._step + 1: ++ raise RuntimeError("video-row cache step order changed") ++ if self._step >= 0 and self._blocks_this_step != self.num_layers: ++ raise RuntimeError("previous video-row cache step is incomplete") ++ self.partial = not self.is_full_step(step) ++ if self.partial and len(self._deltas) != self.num_layers: ++ raise RuntimeError("partial video step has no complete dense anchor") ++ self._step = step ++ self._blocks_this_step = 0 ++ if self.partial: ++ self.partial_steps += 1 ++ else: ++ self.full_steps += 1 ++ ++ def prepare_local_span(self, start: int, length: int) -> None: ++ if self.closed or start < 0 or length < 1 or start + length > self.seq_len: ++ raise ValueError("invalid local packed span") ++ if self._local_span is not None: ++ if self._local_span != (start, length): ++ raise RuntimeError("video-row cache local ownership changed") ++ return ++ device = self.global_dense_positions.device ++ target = set(self._target_positions) ++ self.local_video_positions = torch.tensor( ++ [i - start for i in self._target_positions if start <= i < start + length], dtype=torch.long, device=device ++ ) ++ self.local_dense_positions = torch.tensor( ++ [i - start for i in range(start, min(start + length, self.used_len)) if i not in target], ++ dtype=torch.long, ++ device=device, ++ ) ++ self._local_span = (start, length) ++ ++ def _visit(self, layer: int, x: torch.Tensor) -> None: ++ if self.closed or self._step < 0 or layer != self._blocks_this_step: ++ raise RuntimeError("video-row cache block order changed") ++ if self._local_span is None or x.shape[0] != self._local_span[1] or x.dtype != torch.bfloat16: ++ raise ValueError("video-row cache requires the prepared BF16 row shard") ++ self._blocks_this_step += 1 ++ ++ def capture(self, layer: int, x: torch.Tensor, output: torch.Tensor) -> None: ++ if self.partial: ++ raise RuntimeError("cannot replace the dense anchor during a partial step") ++ self._visit(layer, x) ++ rows = self.local_video_positions ++ assert rows is not None ++ self._deltas[layer] = (output.index_select(0, rows).float() - x.index_select(0, rows).float()).to(torch.float16) ++ self.cache_bytes = sum(t.numel() * t.element_size() for t in self._deltas.values()) ++ self.full_blocks += 1 ++ ++ def restore(self, layer: int, x: torch.Tensor) -> torch.Tensor: ++ if not self.partial: ++ raise RuntimeError("cannot reuse a video residual on a full step") ++ self._visit(layer, x) ++ rows = self.local_video_positions ++ assert rows is not None ++ # This copy owns the returned full shard, preserving caller input and ++ # unused padding while fresh non-video rows are filled by the block. ++ output = x.clone() ++ if rows.numel(): ++ output.index_copy_(0, rows, (x.index_select(0, rows).float() + self._deltas[layer].float()).to(x.dtype)) ++ self.partial_blocks += 1 ++ return output ++ ++ def record_subset_attention(self, rows: int) -> None: ++ if not self.partial or rows != self.global_dense_positions.numel(): ++ raise RuntimeError("unexpected subset attention query count") ++ self.subset_attention_calls += 1 ++ self.subset_query_rows += rows ++ self.omitted_query_rows += len(self._target_positions) ++ ++ def end_step(self) -> None: ++ if self._blocks_this_step != self.num_layers: ++ raise RuntimeError("video-row cache step did not visit every main block") ++ ++ def stats(self) -> dict[str, int | bool]: ++ return { ++ name: getattr(self, name) ++ for name in [ ++ "full_steps", ++ "partial_steps", ++ "full_blocks", ++ "partial_blocks", ++ "subset_attention_calls", ++ "subset_query_rows", ++ "omitted_query_rows", ++ "partial_mlp_rows", ++ "cache_bytes", ++ "closed", ++ ] ++ } ++ ++ def close(self) -> None: ++ self._deltas.clear() ++ self._target_positions.clear() ++ self.global_dense_positions = torch.empty(0, dtype=torch.long) ++ self.local_dense_positions = None ++ self.local_video_positions = None ++ self.closed = True diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c15-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c15-source-identity.json new file mode 100644 index 000000000..80157f1a1 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c15-source-identity.json @@ -0,0 +1,17 @@ +{ + "head": "81c1beb6e47ecfa7602e4309a7342a38108aabc8", + "tree": "590f9798d83a2e11cf001c4b7ac1043bae1c5476", + "base": "be319692b96413918973b5bd399cdda3f320c574", + "branch": "rsi/h3-297-video-row-cache", + "files": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "8ec1b3ae4cd252f5d95ab23e0232b983900939e8cb15d1e64f4740f7204abf39", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" + }, + "published": false, + "patch_base": "be319692b96413918973b5bd399cdda3f320c574" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c16-candidate.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c16-candidate.patch new file mode 100644 index 000000000..36082d6ac --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c16-candidate.patch @@ -0,0 +1,924 @@ +diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py +new file mode 100644 +index 0000000..94ff958 +--- /dev/null ++++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py +@@ -0,0 +1,406 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++from datetime import timedelta ++from types import SimpleNamespace ++ ++import pytest ++import torch ++import torch.distributed as dist ++import torch.nn as nn ++ ++from tests.helpers.mark import hardware_marks ++from vllm_omni.diffusion.models.minimax_h3 import minimax_h3_blocks as blocks ++from vllm_omni.diffusion.models.minimax_h3.attention.flash_h3 import _prefix_bnsd, _subset_prefix_bnsd ++from vllm_omni.diffusion.models.minimax_h3.video_row_cache import MiniMaxH3VideoRowCache ++from vllm_omni.platforms import current_omni_platform ++ ++pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=4)] ++ ++ ++def state_for_rows(device="cpu", start=0, length=4): ++ state = MiniMaxH3VideoRowCache(torch.tensor([1, 2], device=device), seq_len=4, used_len=3, num_layers=1) ++ state.prepare_local_span(start, length) ++ return state ++ ++ ++def anchor(state, x): ++ for step in range(5): ++ state.begin_step(step) ++ state.capture(0, x, x + 0.125) ++ state.end_step() ++ state.begin_step(5) ++ ++ ++def test_fixed_schedule_and_row_ownership_excludes_padding(): ++ assert [i for i in range(50) if MiniMaxH3VideoRowCache.is_full_step(i)] == [ ++ 0, ++ 1, ++ 2, ++ 3, ++ 4, ++ 9, ++ 14, ++ 19, ++ 24, ++ 29, ++ 34, ++ 39, ++ 44, ++ 46, ++ 47, ++ 48, ++ 49, ++ ] ++ state = state_for_rows() ++ assert state.global_dense_positions.tolist() == [0] ++ assert state.local_dense_positions.tolist() == [0] ++ assert state.local_video_positions.tolist() == [1, 2] ++ with pytest.raises(RuntimeError, match="ownership"): ++ state.prepare_local_span(0, 3) ++ ++ ++@pytest.mark.parametrize("positions", [[1, 1], [2, 1], [-1], [3]]) ++def test_invalid_target_positions_are_rejected(positions): ++ with pytest.raises(ValueError): ++ MiniMaxH3VideoRowCache(torch.tensor(positions), seq_len=4, used_len=3, num_layers=1) ++ ++ ++def test_delta_is_fp32_and_restore_owns_output(): ++ state = state_for_rows() ++ x = torch.tensor([[1, 2], [3, 4], [5, 6], [7, 8]], dtype=torch.bfloat16) ++ anchor(state, x) ++ assert state._deltas[0].dtype == torch.float32 ++ before = x.clone() ++ output = state.restore(0, x) ++ assert torch.equal(output[1:3], x[1:3] + 0.125) ++ assert torch.equal(output[[0, 3]], x[[0, 3]]) ++ output.zero_() ++ assert torch.equal(x, before) ++ state.end_step() ++ state.close() ++ assert state.closed and not state._deltas and state.global_dense_positions.numel() == 0 ++ ++ ++def large_delta_roundtrip(device="cpu"): ++ state = state_for_rows(device=device) ++ x = torch.full((4, 2), 100000, dtype=torch.bfloat16, device=device) ++ expected = -x ++ delta = expected.float() - x.float() ++ assert torch.isfinite(delta).all() ++ assert not torch.isfinite(delta.to(torch.float16)).all() ++ for step in range(5): ++ state.begin_step(step) ++ state.capture(0, x, expected) ++ state.end_step() ++ state.begin_step(5) ++ actual = state.restore(0, x) ++ assert torch.isfinite(actual).all() ++ assert torch.equal(actual[1:3].cpu().view(torch.uint8), expected[1:3].cpu().view(torch.uint8)) ++ assert torch.equal(actual[[0, 3]], x[[0, 3]]) ++ state.end_step() ++ state.close() ++ assert state.stats()["live_cache_bytes"] == 0 ++ ++ ++def test_large_finite_bf16_delta_roundtrip_exceeds_fp16_storage_range(): ++ large_delta_roundtrip() ++ ++ ++class CountMLP(nn.Module): ++ def __init__(self): ++ super().__init__() ++ self.rows = [] ++ ++ def forward(self, x): ++ self.rows.append(x.shape[0]) ++ return x + 0.25 ++ ++ ++def tiny_block(): ++ block = object.__new__(blocks.MiniMaxH3DiTBlock) ++ nn.Module.__init__(block) ++ block.layer_index = 0 ++ block.norm1 = SimpleNamespace(weight=torch.ones(2), variance_epsilon=1e-6) ++ block.norm2 = SimpleNamespace(weight=torch.ones(2), variance_epsilon=1e-6) ++ block.adaln_proj = lambda t: ( ++ torch.zeros(3, 2), ++ torch.zeros(3, 2), ++ torch.ones(3, 2), ++ torch.zeros(3, 2), ++ torch.zeros(3, 2), ++ torch.ones(3, 2), ++ ) ++ block.attn = lambda x, **kwargs: x * 0.5 ++ block.mlp = CountMLP() ++ return block ++ ++ ++@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) ++def test_real_block_dense_path_and_partial_mlp_empty_rows(start, length): ++ state = state_for_rows(start=start, length=length) ++ block = tiny_block() ++ x = torch.arange(length * 2).reshape(length, 2).to(torch.bfloat16) ++ before = x.clone() ++ kwargs = dict( ++ t_emb=torch.zeros(1), ++ combined_indices=torch.zeros(length, dtype=torch.long), ++ rope_table=None, ++ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), ++ max_seqlen=3, ++ packed_total=4, ++ ) ++ expected = block(x, **kwargs) ++ for step in range(5): ++ state.begin_step(step) ++ actual = block(x, video_row_cache=state, **kwargs) ++ assert torch.equal(actual.view(torch.uint8), expected.view(torch.uint8)) ++ state.end_step() ++ block.mlp.rows.clear() ++ state.begin_step(5) ++ actual = block(x, video_row_cache=state, **kwargs) ++ state.end_step() ++ rows = state.local_dense_positions ++ assert block.mlp.rows == ([rows.numel()] if rows.numel() else []) ++ assert torch.equal(actual.index_select(0, rows), expected.index_select(0, rows)) ++ assert torch.equal(x, before) ++ ++ ++@pytest.mark.parametrize("failure", ["success", "model", "cancel"]) ++def test_request_state_closed_and_removed_on_every_exit(monkeypatch, failure): ++ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop ++ ++ monkeypatch.setenv("H3_VIDEO_ROW_CACHE", "1") ++ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) ++ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) ++ state = MiniMaxH3VideoRowCache(torch.tensor([1]), seq_len=4, used_len=3, num_layers=1) ++ ++ class Model: ++ def create_video_row_cache(self, *args, **kwargs): ++ return state ++ ++ def __call__(self, **kwargs): ++ assert kwargs["video_row_cache"] is state ++ if failure == "model": ++ raise RuntimeError("model failed") ++ state.prepare_local_span(0, 4) ++ x = torch.ones(4, 2, dtype=torch.bfloat16) ++ if state.partial: ++ state.restore(0, x) ++ else: ++ state.capture(0, x, x + 0.125) ++ return torch.zeros(2, 96), torch.zeros(1, 32) ++ ++ class Branch: ++ task = "ref2va" ++ seq_len = 4 ++ used_len = 3 ++ img_pos = torch.tensor([0, 1]) ++ update_mask = torch.tensor([False, True]) ++ update_mask_dev = update_mask ++ audio_update_mask_dev = torch.tensor([True]) ++ locked_audio_rows = None ++ static_kwargs = {"video_token_layout": SimpleNamespace(video_spans=[SimpleNamespace(role="target", start=1)])} ++ ++ def forward_kwargs(self, **kwargs): ++ return dict(self.static_kwargs) ++ ++ monkeypatch.setattr( ++ loop, ++ "minimax_h3_prepare_denoise_rows", ++ lambda **kwargs: (kwargs["initial_video_rows"], kwargs["initial_audio_rows"], torch.zeros(1, 96), None), ++ ) ++ ++ def cancellation(**kwargs): ++ if failure == "cancel": ++ raise RuntimeError("request cancelled") ++ ++ monkeypatch.setattr(loop, "check_request_cancellation", cancellation) ++ branch, model = Branch(), Model() ++ kwargs = dict( ++ model=model, ++ positive=branch, ++ initial_video_rows=torch.zeros(2, 96), ++ initial_audio_rows=torch.zeros(1, 32), ++ keyframe_cond_rows=torch.zeros(1, 96), ++ sigmas_video=[1 - i / 50 for i in range(51)], ++ sigmas_audio=[1 - i / 50 for i in range(51)], ++ device=torch.device("cpu"), ++ ) ++ if failure == "success": ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert state.full_steps == 17 and state.partial_steps == 33 ++ else: ++ with pytest.raises(RuntimeError): ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert state.closed and not state._deltas and "video_row_cache" not in branch.static_kwargs ++ assert model._video_row_cache_stats["closed"] ++ ++ ++def test_default_factory_does_not_create_state(monkeypatch): ++ from vllm_omni.diffusion.models.minimax_h3.minimax_h3_transformer import MiniMaxH3DiTModel ++ ++ monkeypatch.delenv("H3_VIDEO_ROW_CACHE", raising=False) ++ assert ( ++ MiniMaxH3DiTModel.create_video_row_cache(None, torch.tensor([1]), seq_len=4, used_len=3, num_steps=50) is None ++ ) ++ ++ ++@pytest.mark.parametrize("ref2va", [False, True]) ++def test_real_packing_task_selects_ref2va_and_rejects_fl2va(monkeypatch, ref2va): ++ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop ++ from vllm_omni.diffusion.models.minimax_h3.packed_sequence import ( ++ minimax_h3_packed_sequence, ++ minimax_h3_packed_sequence_ref2va_blocks, ++ ) ++ ++ common = dict(text_len=2, latent_t=2, latent_h=4, latent_w=4, audio_t=2) ++ packed = ( ++ minimax_h3_packed_sequence_ref2va_blocks(**common, ref_blocks=[{"kind": "image", "latent_h": 4, "latent_w": 4}]) ++ if ref2va ++ else minimax_h3_packed_sequence(**common, include_keyframe_cond=True, keyframe_frame_indices=[0], frame_count=5) ++ ) ++ branch = loop.MiniMaxH3DenoiseBranch( ++ packed=packed, ++ text_embeddings=torch.zeros(2, 5120), ++ token_tags=packed["token_tags"], ++ device=torch.device("cpu"), ++ task="ref2va" if ref2va else "fl2va", ++ ) ++ layout = branch.static_kwargs["video_token_layout"] ++ assert not any(span.role == "reference" for span in layout.video_spans) ++ monkeypatch.setenv("H3_VIDEO_ROW_CACHE", "1") ++ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) ++ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) ++ monkeypatch.setattr(loop, "check_request_cancellation", lambda **kwargs: None) ++ states = [] ++ ++ class Model: ++ def create_video_row_cache(self, positions, **kwargs): ++ state = MiniMaxH3VideoRowCache( ++ positions, seq_len=kwargs["seq_len"], used_len=kwargs["used_len"], num_layers=1 ++ ) ++ state.prepare_local_span(0, branch.seq_len) ++ states.append(state) ++ return state ++ ++ def __call__(self, **kwargs): ++ state = kwargs["video_row_cache"] ++ x = torch.ones(branch.seq_len, 2, dtype=torch.bfloat16) ++ if state.partial: ++ state.restore(0, x) ++ else: ++ state.capture(0, x, x + 0.125) ++ return torch.zeros(branch.img_pos.numel(), 96), torch.zeros(branch.audio_pos.numel(), 32) ++ ++ kwargs = dict( ++ model=Model(), ++ positive=branch, ++ initial_video_rows=torch.zeros(branch.img_pos.numel(), 96), ++ initial_audio_rows=torch.zeros(branch.audio_pos.numel(), 32), ++ keyframe_cond_rows=torch.zeros(int((~branch.update_mask).sum()), 96), ++ sigmas_video=[1 - i / 50 for i in range(51)], ++ sigmas_audio=[1 - i / 50 for i in range(51)], ++ device=torch.device("cpu"), ++ ) ++ if ref2va: ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert len(states) == 1 and states[0].closed ++ assert states[0].full_steps == 17 and states[0].partial_steps == 33 ++ else: ++ with pytest.raises(ValueError, match="requires Ref2VA"): ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert not states ++ ++ ++@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) ++def test_attention_keeps_collective_path_and_projects_only_dense_local_rows(start, length): ++ state = state_for_rows(start=start, length=length) ++ x = torch.ones(length, 2, dtype=torch.bfloat16) ++ anchor(state, x) ++ attention = object.__new__(blocks.MiniMaxH3Attention) ++ nn.Module.__init__(attention) ++ attention.num_heads = 1 ++ attention.head_dim = 2 ++ attention.vdn = None ++ attention.to_gate_compress = None ++ calls, projected_rows = [], [] ++ attention._project_qkv = lambda *args: (x.reshape(length, 1, 2),) * 5 ++ ++ def packed(q, k, v, **kwargs): ++ calls.append(q.shape[0]) ++ assert kwargs["video_row_cache"] is state ++ return q ++ ++ def project(rows): ++ projected_rows.append(rows.shape[0]) ++ return rows * 2, None ++ ++ attention._run_packed_attention = packed ++ attention.out_proj = project ++ output = attention( ++ x, ++ rope_table=None, ++ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), ++ max_seqlen=3, ++ packed_total=4, ++ video_row_cache=state, ++ ) ++ rows = state.local_dense_positions ++ assert calls == [length] ++ assert projected_rows == ([rows.numel()] if rows.numel() else []) ++ assert torch.equal(output.index_select(0, rows), (x * 2).index_select(0, rows)) ++ assert torch.count_nonzero(output.index_select(0, state.local_video_positions)) == 0 ++ ++ ++def _native_worker(rank): ++ import torch_npu # noqa: F401 ++ ++ torch.npu.set_device(rank) ++ dist.init_process_group( ++ "hccl", init_method="tcp://127.0.0.1:29597", rank=rank, world_size=4, timeout=timedelta(seconds=120) ++ ) ++ try: ++ large_delta_roundtrip(device=f"npu:{rank}") ++ torch.manual_seed(19) ++ full = [torch.randn(32, 4, 128, dtype=torch.bfloat16).npu() for _ in range(3)] ++ exchanged = [] ++ for tensor in full: ++ local = tensor[rank * 8 : (rank + 1) * 8] ++ send = local.reshape(8, 4, 1, 128).permute(1, 0, 2, 3).contiguous() ++ receive = torch.empty_like(send) ++ dist.all_to_all_single(receive, send) ++ exchanged.append(receive.reshape(1, 32, 1, 128)) ++ state = MiniMaxH3VideoRowCache(torch.arange(8, 24, device="npu"), seq_len=32, used_len=30, num_layers=1) ++ state.prepare_local_span(rank * 8, 8) ++ x = torch.ones(8, 2, dtype=torch.bfloat16, device="npu") ++ anchor(state, x) ++ output = _subset_prefix_bnsd(exchanged[0], exchanged[1][:, :30], exchanged[2][:, :30], 128**-0.5, state) ++ send = output.reshape(4, 8, 1, 128).contiguous() ++ receive = torch.empty_like(send) ++ dist.all_to_all_single(receive, send) ++ local_output = receive.permute(1, 0, 2, 3).reshape(8, 4, 128) ++ expected = _prefix_bnsd(full[0].unsqueeze(0), full[1][:30].unsqueeze(0), full[2][:30].unsqueeze(0), 128**-0.5)[ ++ 0 ++ ] ++ rows = state.local_dense_positions ++ torch.testing.assert_close( ++ local_output.index_select(0, rows).float(), ++ expected[rank * 8 : (rank + 1) * 8].index_select(0, rows).float(), ++ atol=0.03125, ++ rtol=0.02, ++ ) ++ assert state.subset_attention_calls == 1 and state.subset_query_rows == 14 ++ assert rows.numel() == [8, 0, 0, 6][rank] ++ state.restore(0, x) ++ state.end_step() ++ state.close() ++ torch.npu.synchronize() ++ dist.barrier() ++ finally: ++ dist.destroy_process_group() ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires four native Ascend devices") ++def test_native_four_rank_subset_attention_with_empty_local_dense_rows(): ++ torch.multiprocessing.spawn(_native_worker, nprocs=4, join=True) +diff --git a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py +index 6ebfb10..044c1f0 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py ++++ b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py +@@ -36,6 +36,16 @@ def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, sc + return output.reshape(batch, heads, rows, dim).transpose(1, 2).contiguous() + + ++def _subset_prefix_bnsd(query, key, value, scale, state): ++ """Compute only fresh conditioning queries; preserve the shared SP shape.""" ++ rows = state.global_dense_positions ++ dense = _prefix_bnsd(query.index_select(1, rows), key, value, scale) ++ output = torch.zeros_like(query) ++ output.index_copy_(1, rows, dense) ++ state.record_subset_attention(rows.numel()) ++ return output ++ ++ + class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + """Use independent head batches for explicit long BF16 prefix attention. + +@@ -51,7 +61,10 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + value: torch.Tensor, + attn_metadata: AttentionMetadata | None = None, + ) -> torch.Tensor: ++ state = attn_metadata.extra.get("minimax_h3_video_row_cache") if attn_metadata is not None else None + if os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" or attn_metadata is None: ++ if state is not None: ++ raise RuntimeError("H3 video-row cache requires the explicit fused attention path") + return super().forward_fa_npu(query, key, value, attn_metadata) + extra = attn_metadata.extra + if ( +@@ -68,9 +81,17 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + or query.shape[2:] != value.shape[2:] + or key.shape[1] != value.shape[1] + ): ++ if state is not None: ++ raise RuntimeError("H3 video-row cache received unsupported fused attention inputs") + return super().forward_fa_npu(query, key, value, attn_metadata) + resolved = self._resolve_packed_seq_npu(query, key, extra) + if resolved is None: ++ if state is not None: ++ raise RuntimeError("H3 video-row cache requires a resolved packed prefix") + return super().forward_fa_npu(query, key, value, attn_metadata) + used_k = resolved[1][0] ++ if state is not None: ++ if not state.partial or used_k != state.used_len or query.shape[1] != state.seq_len: ++ raise RuntimeError("H3 video-row cache attention layout changed") ++ return _subset_prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale, state) + return _prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale) +diff --git a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py +index 8cca219..07fb372 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py ++++ b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py +@@ -9,6 +9,7 @@ rows stay pinned to their noised step-0 anchors. + + from __future__ import annotations + ++import os + from collections.abc import Callable + from contextlib import AbstractContextManager, nullcontext + from typing import Any +@@ -69,7 +70,9 @@ class MiniMaxH3DenoiseBranch: + text_embeddings: torch.Tensor, + token_tags: torch.Tensor, + device: torch.device, ++ task: str | None = None, + ) -> None: ++ self.task = task + seq_len = int(packed["seq_len"]) + self.seq_len = seq_len + self.img_pos = packed["img_pos"].view(-1).to(torch.long) +@@ -380,6 +383,24 @@ def minimax_h3_denoise_loop( + positive.static_kwargs["reference_kv_state"] = reference_kv_state + + try: ++ video_row_cache = None ++ create_video_cache = getattr(model, "create_video_row_cache", None) ++ if os.environ.get("H3_VIDEO_ROW_CACHE") == "1": ++ if ( ++ not callable(create_video_cache) ++ or video_edit is not None ++ or audio_edit is not None ++ or keyframe_cond_rows is None ++ or positive.task != "ref2va" ++ ): ++ raise ValueError("H3 video-row cache requires Ref2VA image references without editing") ++ video_row_cache = create_video_cache( ++ positive.img_pos[positive.update_mask].to(device), ++ seq_len=positive.seq_len, ++ used_len=positive.used_len, ++ num_steps=num_steps, ++ ) ++ positive.static_kwargs["video_row_cache"] = video_row_cache + for step in range(num_steps): + check_request_cancellation() + step_cm = step_profiler(step) if step_profiler is not None else nullcontext() +@@ -388,6 +409,8 @@ def minimax_h3_denoise_loop( + s_a = sigmas_audio[step] + if reference_kv_state is not None: + reference_kv_state.begin_step(step, sigma=float(s_v)) ++ if video_row_cache is not None: ++ video_row_cache.begin_step(step) + # Publish where we are so step-gated attention features (the dense + # warmup of RAINFUSION_ATTN, the timestep gate of TRTLLM_ATTN) can + # see it. Gates use the scheduler-style descending timestep, which +@@ -430,6 +453,8 @@ def minimax_h3_denoise_loop( + v_video, v_audio = model(**fk) + if reference_kv_state is not None: + reference_kv_state.end_step() ++ if video_row_cache is not None: ++ video_row_cache.end_step() + mv_video_t = v_video.float()[update] + mv_audio_t = v_audio.float()[audio_update] + +@@ -476,6 +501,10 @@ def minimax_h3_denoise_loop( + + finally: + minimax_h3_publish_denoise_progress(None, None, None) ++ if video_row_cache is not None: ++ positive.static_kwargs.pop("video_row_cache", None) ++ video_row_cache.close() ++ model._video_row_cache_stats = video_row_cache.stats() + if reference_kv_state is not None: + positive.static_kwargs.pop("reference_kv_state", None) + reference_kv_state.close() +diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py +index 09f2e7d..94ef291 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py ++++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py +@@ -45,6 +45,7 @@ if TYPE_CHECKING: + from vllm.model_executor.layers.quantization.base_config import QuantizationConfig + + from .reference_kv_cache import MiniMaxH3ReferenceKVTier2State ++ from .video_row_cache import MiniMaxH3VideoRowCache + + _BF16_DTYPE = torch.bfloat16 + _FP32_DTYPE = torch.float32 +@@ -291,6 +292,7 @@ class MiniMaxH3Attention(nn.Module): + reference_kv_layer_index: int | None = None, + reference_kv_compact: bool = False, + vdn_window: VDNLayout | None = None, ++ video_row_cache: MiniMaxH3VideoRowCache | None = None, + ) -> torch.Tensor: + """Run packed attention as a small eager island. + +@@ -368,6 +370,8 @@ class MiniMaxH3Attention(nn.Module): + "vsa_h3_ref_policy": self.vsa_ref_policy, + "vsa_h3_ref_keep_rate": self.vsa_ref_keep_rate, + } ++ if video_row_cache is not None and video_row_cache.partial: ++ extra["minimax_h3_video_row_cache"] = video_row_cache + if gate_compress is not None: + extra["gate_compress"] = gate_compress.unsqueeze(0) + if video_layout is not None and video_layout.video_spans: +@@ -455,6 +459,7 @@ class MiniMaxH3Attention(nn.Module): + reference_kv_layer_index: int | None = None, + reference_kv_compact: bool = False, + vdn_window: VDNLayout | None = None, ++ video_row_cache: MiniMaxH3VideoRowCache | None = None, + ) -> torch.Tensor: + """x: [T, hidden] packed thd rows -> [T, hidden]. + +@@ -508,10 +513,19 @@ class MiniMaxH3Attention(nn.Module): + reference_kv_layer_index=reference_kv_layer_index, + reference_kv_compact=reference_kv_compact, + vdn_window=vdn_window, ++ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), + ) + if self.vdn is not None: + out = self.vdn.gate_softmax(out, x) + out = out.reshape(total, self.num_heads * self.head_dim) ++ if video_row_cache is not None and video_row_cache.partial: ++ rows = video_row_cache.local_dense_positions ++ assert rows is not None ++ projected = x.new_zeros((total, x.shape[-1])) ++ if rows.numel(): ++ dense, _ = self.out_proj(out.index_select(0, rows)) ++ projected.index_copy_(0, rows, dense) ++ return projected + out, _ = self.out_proj(out) + if self.vdn is not None: + out = self.vdn.add_linear(out, x, q_raw, k_raw, v, vdn_window) +@@ -742,6 +756,7 @@ class MiniMaxH3DiTBlock(nn.Module): + reference_kv_state: MiniMaxH3ReferenceKVTier2State | None = None, + reference_kv_compact: bool = False, + vdn_window: VDNLayout | None = None, ++ video_row_cache: MiniMaxH3VideoRowCache | None = None, + ) -> torch.Tensor: + """x: [T, H]; t_emb: [M, t_dim]; combined_indices: [T] + (= inverse_indices * modality_num + token_tags.clamp(min=0)). +@@ -759,6 +774,7 @@ class MiniMaxH3DiTBlock(nn.Module): + gate_mlp, + ) = self.adaln_proj(t_emb) + ++ block_input = x + residual = x + h = rms_norm_indexed_scale_shift( + x, +@@ -782,6 +798,7 @@ class MiniMaxH3DiTBlock(nn.Module): + reference_kv_layer_index=self.layer_index, + reference_kv_compact=reference_kv_compact, + vdn_window=vdn_window, ++ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), + ) + x, h = indexed_gate_rms_norm_scale_shift( + residual, +@@ -794,5 +811,22 @@ class MiniMaxH3DiTBlock(nn.Module): + self.norm2.variance_epsilon, + ) + residual = x ++ if video_row_cache is not None and video_row_cache.partial: ++ rows = video_row_cache.local_dense_positions ++ assert rows is not None ++ output = video_row_cache.restore(self.layer_index, block_input) ++ if rows.numel(): ++ dense = indexed_gate( ++ residual.index_select(0, rows), ++ gate_mlp, ++ self.mlp(h.index_select(0, rows)), ++ combined_indices.index_select(0, rows), ++ ) ++ output.index_copy_(0, rows, dense) ++ video_row_cache.partial_mlp_rows += rows.numel() ++ return output + h = self.mlp(h) +- return indexed_gate(residual, gate_mlp, h, combined_indices) ++ output = indexed_gate(residual, gate_mlp, h, combined_indices) ++ if video_row_cache is not None: ++ video_row_cache.capture(self.layer_index, block_input, output) ++ return output +diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py +index 9518828..2cf7aae 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py ++++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py +@@ -10,6 +10,7 @@ layout. + from __future__ import annotations + + import math ++import os + from bisect import bisect_left + from collections.abc import Iterable, Mapping + from dataclasses import replace +@@ -66,6 +67,8 @@ if TYPE_CHECKING: + from vllm_omni.diffusion.data import OmniDiffusionConfig + from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache import MiniMaxH3ReferenceKVTier2State + ++ from .video_row_cache import MiniMaxH3VideoRowCache ++ + logger = init_logger(__name__) + + +@@ -245,6 +248,7 @@ _FORWARD_SUPPORTED_KWARGS = frozenset( + "control_rows", + "control_context_scale", + "reference_kv_state", ++ "video_row_cache", + } + ) + +@@ -1232,6 +1236,35 @@ class MiniMaxH3DiTModel(nn.Module): + t_emb = self.time_embedder(unique_timesteps) + return embeddings, t_emb + ++ def create_video_row_cache( ++ self, target_positions: torch.Tensor, *, seq_len: int, used_len: int, num_steps: int ++ ) -> MiniMaxH3VideoRowCache | None: ++ if os.environ.get("H3_VIDEO_ROW_CACHE") != "1": ++ return None ++ p = self.parallel_config ++ if ( ++ not current_omni_platform.is_npu() ++ or num_steps != 50 ++ or p.tensor_parallel_size != 1 ++ or p.data_parallel_size not in (None, 1) ++ or p.cfg_parallel_size != 1 ++ or p.pipeline_parallel_size != 1 ++ or p.ulysses_degree != 4 ++ or p.ring_degree != 1 ++ or p.allgather_degree != 1 ++ or p.ulysses_mode != "strict" ++ or self.od_config.cache_backend not in (None, "none") ++ or self.od_config.kv_offload_config is not None ++ or self.vdn_config is not None ++ or os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" ++ or any(block.attn.attention.attn_backend.get_name() != "FLASH_ATTN" for block in self.blocks) ++ ): ++ raise ValueError("H3 video-row cache requires dense NPU BF16 Ref2VA with strict Ulysses4, TP/DP/CFG/PP1") ++ from .video_row_cache import MiniMaxH3VideoRowCache ++ ++ self._video_row_cache_stats = None ++ return MiniMaxH3VideoRowCache(target_positions, seq_len=seq_len, used_len=used_len, num_layers=len(self.blocks)) ++ + def create_reference_kv_state( + self, + *, +@@ -1308,6 +1341,14 @@ class MiniMaxH3DiTModel(nn.Module): + refiner_max = int(self._psp_field(refiner_psp, "refiner_packed_seq_params", "max_seqlen_q")) + video_layout = kwargs.get("video_token_layout") + reference_kv_state = kwargs.get("reference_kv_state") ++ video_row_cache = kwargs.get("video_row_cache") ++ if video_row_cache is not None and ( ++ num_requests != 1 ++ or reference_kv_state is not None ++ or self.vdn_config is not None ++ or kwargs.get("control_rows") is not None ++ ): ++ raise ValueError("H3 video-row cache cannot combine with packed requests, reference KV, VDN or control") + vdn_window = None + if self.vdn_config is not None: + # Text rows lead the packed document. +@@ -1442,6 +1483,11 @@ class MiniMaxH3DiTModel(nn.Module): + if reference_kv_state is not None: + reference_kv_state.set_local_sequence_rows(hidden.shape[0]) + ++ if video_row_cache is not None: ++ if hidden.shape[0] != local_len: ++ raise RuntimeError("H3 video-row cache requires the strict local sequence shard") ++ video_row_cache.prepare_local_span(local_start, local_len) ++ + block_video_layout = video_layout + if compact_reference: + block_video_layout = None +@@ -1505,6 +1551,7 @@ class MiniMaxH3DiTModel(nn.Module): + reference_kv_state=reference_kv_state, + reference_kv_compact=compact_reference, + vdn_window=vdn_window, ++ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), + ) + if block_index in hints: + hidden = hidden + hints.pop(block_index) * control_scale +diff --git a/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py b/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py +index 476c3cc..ebce8fc 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py ++++ b/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py +@@ -2115,6 +2115,7 @@ class MiniMaxH3Pipeline( + text_embeddings=text_embeddings, + token_tags=tags, + device=self.device, ++ task=task, + ) + + if control_rows is not None: +diff --git a/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py b/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py +new file mode 100644 +index 0000000..5ccfa35 +--- /dev/null ++++ b/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py +@@ -0,0 +1,163 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++"""Request-owned, experimental target-video block residuals for H3 Ref2VA.""" ++ ++from __future__ import annotations ++ ++import torch ++ ++ ++class MiniMaxH3VideoRowCache: ++ """Keep non-video computation dense while reusing video block updates. ++ ++ The first experimental profile is deliberately fixed. No state is shared ++ across requests, and every rank retains all attention collectives. ++ """ ++ ++ def __init__(self, target_positions: torch.Tensor, *, seq_len: int, used_len: int, num_layers: int) -> None: ++ positions = target_positions.detach().cpu().tolist() ++ if not 0 < used_len <= seq_len or not positions or num_layers < 1: ++ raise ValueError("video-row cache requires a nonempty valid packed document") ++ if positions != sorted(set(positions)) or positions[0] < 0 or positions[-1] >= used_len: ++ raise ValueError("target-video positions must be sorted unique valid rows") ++ target_set = set(positions) ++ dense = [i for i in range(used_len) if i not in target_set] ++ if not dense: ++ raise ValueError("video-row cache requires non-video conditioning rows") ++ self.seq_len = seq_len ++ self.used_len = used_len ++ self.num_layers = num_layers ++ self._target_positions = positions ++ self.global_dense_positions = torch.tensor(dense, dtype=torch.long, device=target_positions.device) ++ self.local_dense_positions: torch.Tensor | None = None ++ self.local_video_positions: torch.Tensor | None = None ++ self._local_span: tuple[int, int] | None = None ++ self._deltas: dict[int, torch.Tensor] = {} ++ self._step = -1 ++ self._blocks_this_step = 0 ++ self.partial = False ++ self.closed = False ++ self.full_steps = 0 ++ self.partial_steps = 0 ++ self.full_blocks = 0 ++ self.partial_blocks = 0 ++ self.subset_attention_calls = 0 ++ self.subset_query_rows = 0 ++ self.omitted_query_rows = 0 ++ self.partial_mlp_rows = 0 ++ self.peak_cache_bytes = 0 ++ ++ @staticmethod ++ def is_full_step(step: int) -> bool: ++ if not 0 <= step < 50: ++ raise ValueError("the video-row cache profile requires 50 steps") ++ return step < 4 or step >= 46 or (step - 4) % 5 == 0 ++ ++ def begin_step(self, step: int) -> None: ++ if self.closed or step != self._step + 1: ++ raise RuntimeError("video-row cache step order changed") ++ if self._step >= 0 and self._blocks_this_step != self.num_layers: ++ raise RuntimeError("previous video-row cache step is incomplete") ++ self.partial = not self.is_full_step(step) ++ if self.partial and len(self._deltas) != self.num_layers: ++ raise RuntimeError("partial video step has no complete dense anchor") ++ self._step = step ++ self._blocks_this_step = 0 ++ if self.partial: ++ self.partial_steps += 1 ++ else: ++ self.full_steps += 1 ++ ++ def prepare_local_span(self, start: int, length: int) -> None: ++ if self.closed or start < 0 or length < 1 or start + length > self.seq_len: ++ raise ValueError("invalid local packed span") ++ if self._local_span is not None: ++ if self._local_span != (start, length): ++ raise RuntimeError("video-row cache local ownership changed") ++ return ++ device = self.global_dense_positions.device ++ target = set(self._target_positions) ++ self.local_video_positions = torch.tensor( ++ [i - start for i in self._target_positions if start <= i < start + length], dtype=torch.long, device=device ++ ) ++ self.local_dense_positions = torch.tensor( ++ [i - start for i in range(start, min(start + length, self.used_len)) if i not in target], ++ dtype=torch.long, ++ device=device, ++ ) ++ self._local_span = (start, length) ++ ++ def _visit(self, layer: int, x: torch.Tensor) -> None: ++ if self.closed or self._step < 0 or layer != self._blocks_this_step: ++ raise RuntimeError("video-row cache block order changed") ++ if self._local_span is None or x.shape[0] != self._local_span[1] or x.dtype != torch.bfloat16: ++ raise ValueError("video-row cache requires the prepared BF16 row shard") ++ self._blocks_this_step += 1 ++ ++ def capture(self, layer: int, x: torch.Tensor, output: torch.Tensor) -> None: ++ if self.partial: ++ raise RuntimeError("cannot replace the dense anchor during a partial step") ++ self._visit(layer, x) ++ rows = self.local_video_positions ++ assert rows is not None ++ # BF16 endpoints may have a finite difference outside FP16's range. ++ # Keep that difference in FP32; only the reconstructed block output is ++ # rounded to the original BF16 activation dtype. ++ self._deltas[layer] = output.index_select(0, rows).float() - x.index_select(0, rows).float() ++ self.peak_cache_bytes = max( ++ self.peak_cache_bytes, sum(t.numel() * t.element_size() for t in self._deltas.values()) ++ ) ++ self.full_blocks += 1 ++ ++ def restore(self, layer: int, x: torch.Tensor) -> torch.Tensor: ++ if not self.partial: ++ raise RuntimeError("cannot reuse a video residual on a full step") ++ self._visit(layer, x) ++ rows = self.local_video_positions ++ assert rows is not None ++ # This copy owns the returned full shard, preserving caller input and ++ # unused padding while fresh non-video rows are filled by the block. ++ output = x.clone() ++ if rows.numel(): ++ output.index_copy_(0, rows, (x.index_select(0, rows).float() + self._deltas[layer].float()).to(x.dtype)) ++ self.partial_blocks += 1 ++ return output ++ ++ def record_subset_attention(self, rows: int) -> None: ++ if not self.partial or rows != self.global_dense_positions.numel(): ++ raise RuntimeError("unexpected subset attention query count") ++ self.subset_attention_calls += 1 ++ self.subset_query_rows += rows ++ self.omitted_query_rows += len(self._target_positions) ++ ++ def end_step(self) -> None: ++ if self._blocks_this_step != self.num_layers: ++ raise RuntimeError("video-row cache step did not visit every main block") ++ ++ def stats(self) -> dict[str, int | bool | str]: ++ counters: dict[str, int | bool | str] = { ++ name: getattr(self, name) ++ for name in [ ++ "full_steps", ++ "partial_steps", ++ "full_blocks", ++ "partial_blocks", ++ "subset_attention_calls", ++ "subset_query_rows", ++ "omitted_query_rows", ++ "partial_mlp_rows", ++ "peak_cache_bytes", ++ "closed", ++ ] ++ } ++ counters["storage_dtype"] = "torch.float32" ++ counters["live_cache_bytes"] = sum(t.numel() * t.element_size() for t in self._deltas.values()) ++ return counters ++ ++ def close(self) -> None: ++ self._deltas.clear() ++ self._target_positions.clear() ++ self.global_dense_positions = torch.empty(0, dtype=torch.long) ++ self.local_dense_positions = None ++ self.local_video_positions = None ++ self.closed = True diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c16-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c16-source-identity.json new file mode 100644 index 000000000..e94e5aa8f --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c16-source-identity.json @@ -0,0 +1,17 @@ +{ + "head": "e0a3462fcc44de698b5dedcf8ff2c99f9802f757", + "tree": "830fab783d4750193c321611272cdd087634fb39", + "base": "81c1beb6e47ecfa7602e4309a7342a38108aabc8", + "branch": "rsi/h3-297-video-row-fp32", + "files": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "c555d76f7bd6457a81238fca56100d11cb7ab043a4a649d55e23b6203d09bde2", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" + }, + "published": false, + "patch_base": "be319692b96413918973b5bd399cdda3f320c574" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c17-candidate.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c17-candidate.patch new file mode 100644 index 000000000..4a7c562dd --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c17-candidate.patch @@ -0,0 +1,282 @@ +diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py +new file mode 100644 +index 0000000..b29edf8 +--- /dev/null ++++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py +@@ -0,0 +1,164 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++from types import SimpleNamespace ++ ++import pytest ++import torch ++ ++from tests.helpers.mark import hardware_marks ++from vllm_omni.diffusion.models.minimax_h3.attention import flash_h3 as flash ++from vllm_omni.platforms import current_omni_platform ++ ++pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=1)] ++ ++ ++def reference(query, key, value, scale): ++ q, k, v = [tensor.transpose(1, 2).float() for tensor in (query, key, value)] ++ return (torch.softmax(q @ k.transpose(-2, -1) * scale, dim=-1) @ v).transpose(1, 2).to(query.dtype) ++ ++ ++def fixture(frames=5, prefix=4, rows=3, padding=2, heads=2, dim=8, device="cpu"): ++ torch.manual_seed(19) ++ used = prefix + frames * rows ++ q = torch.randn(1, used + padding, heads, dim, dtype=torch.bfloat16).to(device) ++ k, v = [torch.randn(1, used, heads, dim, dtype=torch.bfloat16).to(device) for _ in range(2)] ++ return q, k, v ++ ++ ++@pytest.mark.parametrize("frames,prefix", [(3, 0), (3, 4), (5, 4), (7, 2)]) ++def test_window_matches_explicit_per_frame_key_sets(monkeypatch, frames, prefix): ++ q, k, v = fixture(frames=frames, prefix=prefix) ++ before = [tensor.clone() for tensor in (q, k, v)] ++ calls = [] ++ ++ def primitive(query, key, value, scale): ++ calls.append((query.shape, key.shape)) ++ return reference(query, key, value, scale) ++ ++ monkeypatch.setattr(flash, "_prefix_bnsd", primitive) ++ actual = flash._video_temporal_window_bnsd(q, k, v, 0.25, target_start=prefix, frames=frames, rows_per_frame=3) ++ if prefix: ++ torch.testing.assert_close(actual[:, :prefix], reference(q[:, :prefix], k, v, 0.25), rtol=0, atol=0) ++ for t in range(frames): ++ start = min(max(t - 1, 0), frames - 3) ++ keys = torch.cat((k[:, :prefix], k[:, prefix + start * 3 : prefix + (start + 3) * 3]), dim=1) ++ values = torch.cat((v[:, :prefix], v[:, prefix + start * 3 : prefix + (start + 3) * 3]), dim=1) ++ expected = reference(q[:, prefix + t * 3 : prefix + (t + 1) * 3], keys, values, 0.25) ++ torch.testing.assert_close(actual[:, prefix + t * 3 : prefix + (t + 1) * 3], expected, rtol=0, atol=0) ++ assert torch.count_nonzero(actual[:, k.shape[1] :]) == 0 ++ assert all(torch.equal(a, b) for a, b in zip(before, (q, k, v))) ++ assert len(calls) == (2 if prefix else 1) ++ assert calls[-1][0] == (frames, 3, 2, 8) ++ assert calls[-1][1] == (frames, prefix + 9, 2, 8) ++ ++ ++@pytest.mark.parametrize("kind", ["short", "tail", "batch", "query"]) ++def test_requested_unsupported_geometry_raises(kind): ++ q, k, v = fixture() ++ kwargs = dict(target_start=4, frames=5, rows_per_frame=3) ++ if kind == "short": ++ kwargs["frames"] = 2 ++ elif kind == "tail": ++ kwargs["target_start"] = 3 ++ elif kind == "batch": ++ q, k, v = [tensor.expand(2, -1, -1, -1) for tensor in (q, k, v)] ++ else: ++ q = q[:, :2] ++ with pytest.raises(ValueError, match="temporal window"): ++ flash._video_temporal_window_bnsd(q, k, v, 0.25, **kwargs) ++ ++ ++def test_fixed_step_profile_and_invalid_progress(monkeypatch): ++ monkeypatch.setenv("H3_VIDEO_TEMPORAL_WINDOW", "3") ++ for step in range(50): ++ monkeypatch.setattr( ++ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=step, total_denoise_steps=50) ++ ) ++ assert flash._use_video_temporal_window() == (step not in (0, 49)) ++ monkeypatch.setattr( ++ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=None, total_denoise_steps=50) ++ ) ++ with pytest.raises(ValueError, match="published"): ++ flash._use_video_temporal_window() ++ monkeypatch.setenv("H3_VIDEO_TEMPORAL_WINDOW", "5") ++ with pytest.raises(ValueError, match="window=3"): ++ flash._use_video_temporal_window() ++ ++ ++def test_default_long_prefix_path_does_not_read_window_progress(monkeypatch): ++ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata ++ ++ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") ++ monkeypatch.delenv("H3_VIDEO_TEMPORAL_WINDOW", raising=False) ++ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) ++ impl.causal, impl.softmax_scale = False, 0.25 ++ impl._resolve_packed_seq_npu = lambda *args: ([4096], [4093]) ++ monkeypatch.setattr(flash, "get_forward_context", lambda: pytest.fail("default path read window progress")) ++ calls = [] ++ monkeypatch.setattr(flash, "_prefix_bnsd", lambda q, k, v, scale: calls.append(k.shape[1]) or q) ++ q = torch.ones(1, 4096, 1, 8, dtype=torch.bfloat16) ++ metadata = AttentionMetadata(extra={"npu_attn_varlen": True}) ++ assert impl.forward_fa_npu(q, q, q, metadata) is q ++ assert calls == [4093] ++ ++ ++@pytest.mark.parametrize("step", [0, 1, 49]) ++def test_configured_long_path_routes_published_geometry(monkeypatch, step): ++ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata, VideoTokenLayout, VideoTokenSpan ++ ++ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") ++ monkeypatch.setenv("H3_VIDEO_TEMPORAL_WINDOW", "3") ++ monkeypatch.setattr( ++ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=step, total_denoise_steps=50) ++ ) ++ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) ++ impl.causal, impl.softmax_scale = False, 0.25 ++ impl._resolve_packed_seq_npu = lambda *args: ([4160], [4160]) ++ q = torch.ones(1, 4192, 1, 8, dtype=torch.bfloat16) ++ layout = VideoTokenLayout( ++ used_len=4160, video_spans=(VideoTokenSpan(start=64, latent_grid=(4, 32, 32), role="target"),) ++ ) ++ metadata = AttentionMetadata(extra={"npu_attn_varlen": True}, video_layout=layout) ++ calls = [] ++ monkeypatch.setattr(flash, "_prefix_bnsd", lambda *args: calls.append("dense") or q) ++ ++ def window(query, key, value, scale, **kwargs): ++ assert key.shape[1] == value.shape[1] == 4160 ++ assert kwargs == {"target_start": 64, "frames": 4, "rows_per_frame": 1024} ++ calls.append("window") ++ return q ++ ++ monkeypatch.setattr(flash, "_video_temporal_window_bnsd", window) ++ assert impl.forward_fa_npu(q, q, q, metadata) is q ++ assert calls == (["dense"] if step in (0, 49) else ["window"]) ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native Ascend FAS") ++def test_native_bf16_window_fas_executes_real_batched_geometry(): ++ import torch_npu # noqa: F401 ++ ++ q, k, v = fixture(rows=8, heads=14, dim=128, device="npu") ++ output = flash._video_temporal_window_bnsd(q, k, v, 128**-0.5, target_start=4, frames=5, rows_per_frame=8) ++ torch.npu.synchronize() ++ assert output.dtype == torch.bfloat16 and torch.isfinite(output).all() ++ for t in range(5): ++ start = min(max(t - 1, 0), 2) ++ keys = torch.cat((k[:, :4], k[:, 4 + start * 8 : 4 + (start + 3) * 8]), dim=1) ++ values = torch.cat((v[:, :4], v[:, 4 + start * 8 : 4 + (start + 3) * 8]), dim=1) ++ expected = reference(q[:, 4 + t * 8 : 4 + (t + 1) * 8].cpu(), keys.cpu(), values.cpu(), 128**-0.5) ++ torch.testing.assert_close( ++ output[:, 4 + t * 8 : 4 + (t + 1) * 8].cpu().float(), expected.float(), atol=0.03125, rtol=0.02 ++ ) ++ ++ ++def test_requested_main_geometry_does_not_silently_fallback(monkeypatch): ++ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata ++ ++ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") ++ monkeypatch.setenv("H3_VIDEO_TEMPORAL_WINDOW", "3") ++ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) ++ impl.causal = True ++ q = torch.ones(1, 4096, 1, 8, dtype=torch.bfloat16) ++ with pytest.raises(ValueError, match="unsupported"): ++ impl.forward_fa_npu(q, q, q, AttentionMetadata(extra={"npu_attn_varlen": True})) +diff --git a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py +index 6ebfb10..0f0024b 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py ++++ b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py +@@ -10,6 +10,7 @@ import torch + + from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata + from vllm_omni.diffusion.attention.backends.flash_attn import FlashAttentionImpl ++from vllm_omni.diffusion.forward_context import get_forward_context + + + def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, scale: float) -> torch.Tensor: +@@ -36,6 +37,58 @@ def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, sc + return output.reshape(batch, heads, rows, dim).transpose(1, 2).contiguous() + + ++def _video_temporal_window_bnsd( ++ query: torch.Tensor, ++ key: torch.Tensor, ++ value: torch.Tensor, ++ scale: float, ++ *, ++ target_start: int, ++ frames: int, ++ rows_per_frame: int, ++) -> torch.Tensor: ++ """Fresh global-prefix queries and local video queries, in original row order.""" ++ if ( ++ query.shape[0] != 1 ++ or key.shape[0] != 1 ++ or value.shape != key.shape ++ or query.shape[2:] != key.shape[2:] ++ or target_start < 0 ++ or frames < 3 ++ or rows_per_frame < 1 ++ or target_start + frames * rows_per_frame != key.shape[1] ++ or query.shape[1] < key.shape[1] ++ ): ++ raise ValueError("H3 temporal window requires one final contiguous target-video span and a valid KV prefix") ++ heads, dim = query.shape[2:] ++ starts = (torch.arange(frames, device=key.device) - 1).clamp(0, frames - 3) ++ ++ def pack_windows(tensor: torch.Tensor) -> torch.Tensor: ++ video = tensor[:, target_start:].reshape(frames, rows_per_frame, heads, dim) ++ windows = video.unfold(0, 3, 1).permute(0, 4, 1, 2, 3).reshape(frames - 2, 3 * rows_per_frame, heads, dim) ++ selected = windows.index_select(0, starts) ++ prefix = tensor[:, :target_start].expand(frames, -1, -1, -1) ++ return torch.cat((prefix, selected), dim=1) ++ ++ output = torch.zeros_like(query) ++ if target_start: ++ output[:, :target_start] = _prefix_bnsd(query[:, :target_start], key, value, scale) ++ video_query = query[:, target_start : key.shape[1]].reshape(frames, rows_per_frame, heads, dim) ++ video_output = _prefix_bnsd(video_query, pack_windows(key), pack_windows(value), scale) ++ output[:, target_start : key.shape[1]] = video_output.reshape(1, frames * rows_per_frame, heads, dim) ++ return output ++ ++ ++def _use_video_temporal_window() -> bool: ++ if os.environ.get("H3_VIDEO_TEMPORAL_WINDOW") != "3": ++ raise ValueError("The experimental H3 temporal-window profile requires window=3") ++ context = get_forward_context() ++ step, total = context.denoise_step_idx, context.total_denoise_steps ++ if total != 50 or step is None or not 0 <= step < 50: ++ raise ValueError("The experimental H3 temporal-window profile requires published 50-step progress") ++ return step not in (0, 49) ++ ++ + class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + """Use independent head batches for explicit long BF16 prefix attention. + +@@ -51,7 +104,10 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + value: torch.Tensor, + attn_metadata: AttentionMetadata | None = None, + ) -> torch.Tensor: ++ window_requested = os.environ.get("H3_VIDEO_TEMPORAL_WINDOW") is not None and query.shape[1] >= 4096 + if os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" or attn_metadata is None: ++ if window_requested: ++ raise ValueError("H3 temporal window requires explicit fused attention and packed metadata") + return super().forward_fa_npu(query, key, value, attn_metadata) + extra = attn_metadata.extra + if ( +@@ -68,9 +124,29 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + or query.shape[2:] != value.shape[2:] + or key.shape[1] != value.shape[1] + ): ++ if window_requested: ++ raise ValueError("H3 temporal window received unsupported packed fused-attention inputs") + return super().forward_fa_npu(query, key, value, attn_metadata) + resolved = self._resolve_packed_seq_npu(query, key, extra) + if resolved is None: ++ if window_requested: ++ raise ValueError("H3 temporal window requires a resolved packed prefix") + return super().forward_fa_npu(query, key, value, attn_metadata) + used_k = resolved[1][0] ++ if os.environ.get("H3_VIDEO_TEMPORAL_WINDOW") is not None and _use_video_temporal_window(): ++ layout = attn_metadata.video_layout ++ targets = [] if layout is None else [span for span in layout.video_spans if span.role == "target"] ++ if len(targets) != 1: ++ raise ValueError("H3 temporal window requires exactly one target-video span") ++ target = targets[0] ++ frames, height, width = target.latent_grid ++ return _video_temporal_window_bnsd( ++ query, ++ key[:, :used_k], ++ value[:, :used_k], ++ self.softmax_scale, ++ target_start=target.start, ++ frames=frames, ++ rows_per_frame=height * width, ++ ) + return _prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale) diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c17-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c17-source-identity.json new file mode 100644 index 000000000..ff1baa78f --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c17-source-identity.json @@ -0,0 +1,12 @@ +{ + "head": "c1589a2b5e4ada617e7cca38ef7c300fe9495c58", + "tree": "47307747212d03fc20f4739ea9e0f3e3ec08f2ee", + "base": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "branch": "rsi/h3-297-video-temporal-window", + "files": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" + }, + "published": false, + "patch_base": "e89b1d40d37f7f72767892a56fa7980ea572ba8e" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c18-candidate.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c18-candidate.patch new file mode 100644 index 000000000..0bf4eade1 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c18-candidate.patch @@ -0,0 +1,328 @@ +diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py +new file mode 100644 +index 0000000..5cffb88 +--- /dev/null ++++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py +@@ -0,0 +1,183 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++import math ++from types import SimpleNamespace ++ ++import pytest ++import torch ++ ++from tests.helpers.mark import hardware_marks ++from vllm_omni.diffusion.models.minimax_h3.attention import flash_h3 as flash ++from vllm_omni.platforms import current_omni_platform ++ ++pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=1)] ++ ++ ++def dense(query, key, value, scale): ++ q, k, v = [x.transpose(1, 2).float() for x in (query, key, value)] ++ return (torch.softmax(q @ k.transpose(-2, -1) * scale, dim=-1) @ v).transpose(1, 2).to(query.dtype) ++ ++ ++def expected_video(q, k, v, scale, frames, height, width, prefix): ++ rows = height * width ++ result = torch.zeros_like(q) ++ result[:, :prefix] = dense(q[:, :prefix], k, v, scale) ++ for t in range(frames): ++ key_parts, value_parts = ( ++ [k[:, :prefix], k[:, prefix + t * rows : prefix + (t + 1) * rows]], ++ [v[:, :prefix], v[:, prefix + t * rows : prefix + (t + 1) * rows]], ++ ) ++ for f in range(frames): ++ if f == t: ++ continue ++ for y in range(0, height, 3): ++ for x in range(0, width, 3): ++ indices = [prefix + f * rows + (y + dy) * width + x + dx for dy in range(3) for dx in range(3)] ++ key_parts.append(k[:, indices].float().mean(1, keepdim=True).to(k.dtype)) ++ value_parts.append(v[:, indices].float().mean(1, keepdim=True).to(v.dtype)) ++ keys, values = torch.cat(key_parts, 1), torch.cat(value_parts, 1) ++ queries = q[:, prefix + t * rows : prefix + (t + 1) * rows] ++ logits = queries.transpose(1, 2).float() @ keys.transpose(1, 2).float().transpose(-2, -1) * scale ++ logits[..., prefix + rows :] += math.log(9) ++ output = torch.softmax(logits, -1) @ values.transpose(1, 2).float() ++ result[:, prefix + t * rows : prefix + (t + 1) * rows] = output.transpose(1, 2).to(q.dtype) ++ return result ++ ++ ++def fixture(frames=3, height=3, width=6, prefix=4, heads=2, dim=8, device="cpu"): ++ torch.manual_seed(217) ++ used = prefix + frames * height * width ++ q = torch.randn(1, used + 2, heads, dim, dtype=torch.bfloat16).to(device) ++ k, v = [torch.randn(1, used, heads, dim, dtype=torch.bfloat16).to(device) for _ in range(2)] ++ return q, k, v ++ ++ ++@pytest.mark.parametrize("frames,height,width,prefix", [(2, 3, 3, 0), (3, 3, 6, 4), (5, 6, 3, 2)]) ++def test_pooling_matches_independent_weighted_key_reference(monkeypatch, frames, height, width, prefix): ++ q, k, v = fixture(frames, height, width, prefix) ++ saved = [x.clone() for x in (q, k, v)] ++ monkeypatch.setattr(flash, "_prefix_bnsd", dense) ++ actual = flash._video_pooled_context_bnsd( ++ q, k, v, 0.25, target_start=prefix, frames=frames, height=height, width=width ++ ) ++ expected = expected_video(q, k, v, 0.25, frames, height, width, prefix) ++ torch.testing.assert_close(actual.float(), expected.float(), atol=0.0078125, rtol=0.02) ++ assert torch.count_nonzero(actual[:, k.shape[1] :]) == 0 ++ assert all(torch.equal(a, b) for a, b in zip(saved, (q, k, v))) ++ ++ ++def test_nine_identical_keys_retain_original_softmax_mass(monkeypatch): ++ q, k, v = fixture(height=3, width=3) ++ for tensor in (k, v): ++ for f in range(3): ++ tensor[:, 4 + f * 9 : 4 + (f + 1) * 9] = tensor[:, 4 + f * 9 : 5 + f * 9].clone() ++ monkeypatch.setattr(flash, "_prefix_bnsd", dense) ++ actual = flash._video_pooled_context_bnsd(q, k, v, 0.25, target_start=4, frames=3, height=3, width=3) ++ expected = dense(q[:, : k.shape[1]], k, v, 0.25) ++ torch.testing.assert_close(actual[:, : k.shape[1]].float(), expected.float(), atol=0.0078125, rtol=0.02) ++ ++ ++@pytest.mark.parametrize("kind", ["height", "width", "tail", "frames", "batch", "query", "scale"]) ++def test_invalid_requested_geometry_raises(kind): ++ q, k, v = fixture() ++ kwargs = dict(target_start=4, frames=3, height=3, width=6, scale=0.25) ++ if kind in ("height", "width"): ++ kwargs[kind] += 1 ++ elif kind == "tail": ++ kwargs["target_start"] += 1 ++ elif kind == "frames": ++ kwargs["frames"] = 1 ++ elif kind == "batch": ++ q, k, v = [tensor.expand(2, -1, -1, -1) for tensor in (q, k, v)] ++ elif kind == "query": ++ q = q[:, :2] ++ else: ++ kwargs["scale"] = 0 ++ with pytest.raises(ValueError, match="pooled context"): ++ flash._video_pooled_context_bnsd(q, k, v, **kwargs) ++ ++ ++def test_fixed_progress_and_default_off(monkeypatch): ++ monkeypatch.setenv("H3_VIDEO_POOLED_CONTEXT", "3x3-local1") ++ for step in range(50): ++ monkeypatch.setattr( ++ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=step, total_denoise_steps=50) ++ ) ++ assert flash._use_video_pooled_context() == (step not in (0, 49)) ++ monkeypatch.setattr( ++ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=None, total_denoise_steps=50) ++ ) ++ with pytest.raises(ValueError, match="published"): ++ flash._use_video_pooled_context() ++ monkeypatch.setenv("H3_VIDEO_POOLED_CONTEXT", "other") ++ with pytest.raises(ValueError, match="3x3-local1"): ++ flash._use_video_pooled_context() ++ monkeypatch.delenv("H3_VIDEO_POOLED_CONTEXT") ++ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") ++ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) ++ impl.causal, impl.softmax_scale = False, 0.25 ++ impl._resolve_packed_seq_npu = lambda *args: ([4096], [4093]) ++ monkeypatch.setattr(flash, "get_forward_context", lambda: pytest.fail("default path read experimental progress")) ++ monkeypatch.setattr(flash, "_prefix_bnsd", lambda q, k, v, scale: q) ++ q = torch.ones(1, 4096, 1, 8, dtype=torch.bfloat16) ++ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata ++ ++ assert impl.forward_fa_npu(q, q, q, AttentionMetadata(extra={"npu_attn_varlen": True})) is q ++ ++ ++@pytest.mark.parametrize("step", [0, 1, 49]) ++def test_requested_dispatch_uses_published_grid(monkeypatch, step): ++ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata, VideoTokenLayout, VideoTokenSpan ++ ++ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") ++ monkeypatch.setenv("H3_VIDEO_POOLED_CONTEXT", "3x3-local1") ++ monkeypatch.setattr( ++ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=step, total_denoise_steps=50) ++ ) ++ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) ++ impl.causal, impl.softmax_scale = False, 0.25 ++ impl._resolve_packed_seq_npu = lambda *args: ([4096], [4096]) ++ q = torch.ones(1, 4192, 1, 8, dtype=torch.bfloat16) ++ metadata = AttentionMetadata( ++ extra={"npu_attn_varlen": True}, ++ video_layout=VideoTokenLayout( ++ used_len=4096, video_spans=(VideoTokenSpan(start=64, latent_grid=(4, 24, 42), role="target"),) ++ ), ++ ) ++ calls = [] ++ monkeypatch.setattr(flash, "_prefix_bnsd", lambda *args: calls.append("dense") or q) ++ ++ def pooled(query, key, value, scale, **kwargs): ++ assert key.shape[1] == value.shape[1] == 4096 ++ assert kwargs == {"target_start": 64, "frames": 4, "height": 24, "width": 42} ++ calls.append("pooled") ++ return q ++ ++ monkeypatch.setattr(flash, "_video_pooled_context_bnsd", pooled) ++ assert impl.forward_fa_npu(q, q, q, metadata) is q ++ assert calls == (["dense"] if step in (0, 49) else ["pooled"]) ++ ++ ++def test_requested_unsupported_main_geometry_does_not_fallback(monkeypatch): ++ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata ++ ++ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") ++ monkeypatch.setenv("H3_VIDEO_POOLED_CONTEXT", "3x3-local1") ++ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) ++ impl.causal = True ++ q = torch.ones(1, 4096, 1, 8, dtype=torch.bfloat16) ++ with pytest.raises(ValueError, match="unsupported"): ++ impl.forward_fa_npu(q, q, q, AttentionMetadata(extra={"npu_attn_varlen": True})) ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native Ascend FAS") ++def test_native_pooled_fas_executes_padded_head_with_mass_compensation(): ++ import torch_npu # noqa: F401 ++ ++ q, k, v = fixture(height=6, width=6, heads=14, dim=128, device="npu") ++ actual = flash._video_pooled_context_bnsd(q, k, v, 128**-0.5, target_start=4, frames=3, height=6, width=6) ++ torch.npu.synchronize() ++ assert actual.dtype == torch.bfloat16 and torch.isfinite(actual).all() ++ expected = expected_video(q.cpu(), k.cpu(), v.cpu(), 128**-0.5, 3, 6, 6, 4) ++ torch.testing.assert_close(actual.cpu().float(), expected.float(), atol=0.03125, rtol=0.025) +diff --git a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py +index 6ebfb10..e2b46bc 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py ++++ b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py +@@ -4,12 +4,14 @@ + + from __future__ import annotations + ++import math + import os + + import torch + + from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata + from vllm_omni.diffusion.attention.backends.flash_attn import FlashAttentionImpl ++from vllm_omni.diffusion.forward_context import get_forward_context + + + def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, scale: float) -> torch.Tensor: +@@ -36,6 +38,77 @@ def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, sc + return output.reshape(batch, heads, rows, dim).transpose(1, 2).contiguous() + + ++def _video_pooled_context_bnsd( ++ query: torch.Tensor, ++ key: torch.Tensor, ++ value: torch.Tensor, ++ scale: float, ++ *, ++ target_start: int, ++ frames: int, ++ height: int, ++ width: int, ++) -> torch.Tensor: ++ """Keep current-frame detail and mass-compensated global spatial summaries.""" ++ rows = height * width ++ if ( ++ query.shape[0] != 1 ++ or key.shape[0] != 1 ++ or value.shape != key.shape ++ or query.shape[2:] != key.shape[2:] ++ or target_start < 0 ++ or frames < 2 ++ or height < 3 ++ or width < 3 ++ or height % 3 ++ or width % 3 ++ or target_start + frames * rows != key.shape[1] ++ or query.shape[1] < key.shape[1] ++ or scale <= 0 ++ ): ++ raise ValueError("H3 pooled context requires a final video grid divisible by three") ++ heads, dim = query.shape[2:] ++ indices = torch.arange(frames, device=key.device).expand(frames, -1) ++ others = indices[~torch.eye(frames, dtype=torch.bool, device=key.device)].reshape(frames, frames - 1) ++ ++ def pack(tensor: torch.Tensor) -> torch.Tensor: ++ video = tensor[:, target_start:].reshape(frames, rows, heads, dim) ++ pooled = ( ++ video.reshape(frames, height // 3, 3, width // 3, 3, heads, dim) ++ .float() ++ .mean(dim=(2, 4)) ++ .to(tensor.dtype) ++ .reshape(frames, rows // 9, heads, dim) ++ ) ++ global_context = pooled.index_select(0, others.flatten()).reshape(frames, -1, heads, dim) ++ return torch.cat((tensor[:, :target_start].expand(frames, -1, -1, -1), video, global_context), dim=1) ++ ++ output = torch.zeros_like(query) ++ if target_start: ++ output[:, :target_start] = _prefix_bnsd(query[:, :target_start], key, value, scale) ++ video_query = query[:, target_start : key.shape[1]].reshape(frames, rows, heads, dim) ++ # An extra dot-product coordinate adds log(9) to each nine-token summary. ++ # Keep the original scale; changing the padded head size must not rescale QK. ++ padded_q = torch.nn.functional.pad(video_query, (0, 16)) ++ padded_k = torch.nn.functional.pad(pack(key), (0, 16)) ++ padded_v = torch.nn.functional.pad(pack(value), (0, 16)) ++ padded_q[..., dim] = 1 ++ padded_k[:, target_start + rows :, :, dim] = math.log(9) / scale ++ attended = _prefix_bnsd(padded_q, padded_k, padded_v, scale)[..., :dim] ++ output[:, target_start : key.shape[1]] = attended.reshape(1, frames * rows, heads, dim) ++ return output ++ ++ ++def _use_video_pooled_context() -> bool: ++ if os.environ.get("H3_VIDEO_POOLED_CONTEXT") != "3x3-local1": ++ raise ValueError("The experimental H3 pooled-context profile requires 3x3-local1") ++ context = get_forward_context() ++ step, total = context.denoise_step_idx, context.total_denoise_steps ++ if total != 50 or step is None or not 0 <= step < 50: ++ raise ValueError("The experimental H3 pooled-context profile requires published 50-step progress") ++ return step not in (0, 49) ++ ++ + class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + """Use independent head batches for explicit long BF16 prefix attention. + +@@ -51,7 +124,10 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + value: torch.Tensor, + attn_metadata: AttentionMetadata | None = None, + ) -> torch.Tensor: ++ requested = os.environ.get("H3_VIDEO_POOLED_CONTEXT") is not None and query.shape[1] >= 4096 + if os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" or attn_metadata is None: ++ if requested: ++ raise ValueError("H3 pooled context requires explicit fused attention and packed metadata") + return super().forward_fa_npu(query, key, value, attn_metadata) + extra = attn_metadata.extra + if ( +@@ -68,9 +144,30 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + or query.shape[2:] != value.shape[2:] + or key.shape[1] != value.shape[1] + ): ++ if requested: ++ raise ValueError("H3 pooled context received unsupported packed fused-attention inputs") + return super().forward_fa_npu(query, key, value, attn_metadata) + resolved = self._resolve_packed_seq_npu(query, key, extra) + if resolved is None: ++ if requested: ++ raise ValueError("H3 pooled context requires a resolved packed prefix") + return super().forward_fa_npu(query, key, value, attn_metadata) + used_k = resolved[1][0] ++ if os.environ.get("H3_VIDEO_POOLED_CONTEXT") is not None and _use_video_pooled_context(): ++ layout = attn_metadata.video_layout ++ targets = [] if layout is None else [span for span in layout.video_spans if span.role == "target"] ++ if len(targets) != 1: ++ raise ValueError("H3 pooled context requires exactly one target-video span") ++ target = targets[0] ++ frames, height, width = target.latent_grid ++ return _video_pooled_context_bnsd( ++ query, ++ key[:, :used_k], ++ value[:, :used_k], ++ self.softmax_scale, ++ target_start=target.start, ++ frames=frames, ++ height=height, ++ width=width, ++ ) + return _prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale) diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c18-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c18-source-identity.json new file mode 100644 index 000000000..cdd19ca7f --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c18-source-identity.json @@ -0,0 +1,12 @@ +{ + "head": "f278cb2bf4647aeb852d61bcc82d9fcc1420d64d", + "tree": "626aacaf0da39c527e1c20c09008e6e942119c5f", + "base": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "branch": "rsi/h3-297-video-pooled-context", + "files": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" + }, + "published": false, + "patch_base": "e89b1d40d37f7f72767892a56fa7980ea572ba8e" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c19-candidate.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c19-candidate.patch new file mode 100644 index 000000000..9d511c4e9 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c19-candidate.patch @@ -0,0 +1,317 @@ +diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py +new file mode 100644 +index 0000000..b0c13de +--- /dev/null ++++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py +@@ -0,0 +1,177 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++import math ++from types import SimpleNamespace ++ ++import pytest ++import torch ++ ++from tests.helpers.mark import hardware_marks ++from vllm_omni.diffusion.models.minimax_h3.attention import flash_h3 as flash ++from vllm_omni.platforms import current_omni_platform ++ ++pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=1)] ++ ++ ++def dense(query, key, value, scale): ++ q, k, v = [x.transpose(1, 2).float() for x in (query, key, value)] ++ return (torch.softmax(q @ k.transpose(-2, -1) * scale, dim=-1) @ v).transpose(1, 2).to(query.dtype) ++ ++ ++def expected_video(q, k, v, scale, frames, height, width, prefix): ++ rows = height * width ++ result = torch.zeros_like(q) ++ result[:, :prefix] = dense(q[:, :prefix], k, v, scale) ++ key_parts, value_parts = [k[:, :prefix]], [v[:, :prefix]] ++ for f in range(frames): ++ for y in range(0, height, 3): ++ for x in range(0, width, 3): ++ indices = [prefix + f * rows + (y + dy) * width + x + dx for dy in range(3) for dx in range(3)] ++ key_parts.append(k[:, indices].float().mean(1, keepdim=True).to(k.dtype)) ++ value_parts.append(v[:, indices].float().mean(1, keepdim=True).to(v.dtype)) ++ keys, values = torch.cat(key_parts, 1), torch.cat(value_parts, 1) ++ queries = q[:, prefix : k.shape[1]] ++ logits = queries.transpose(1, 2).float() @ keys.transpose(1, 2).float().transpose(-2, -1) * scale ++ logits[..., prefix:] += math.log(9) ++ output = torch.softmax(logits, -1) @ values.transpose(1, 2).float() ++ result[:, prefix : k.shape[1]] = output.transpose(1, 2).to(q.dtype) ++ return result ++ ++ ++def fixture(frames=3, height=3, width=6, prefix=4, heads=2, dim=8, device="cpu"): ++ torch.manual_seed(217) ++ used = prefix + frames * height * width ++ q = torch.randn(1, used + 2, heads, dim, dtype=torch.bfloat16).to(device) ++ k, v = [torch.randn(1, used, heads, dim, dtype=torch.bfloat16).to(device) for _ in range(2)] ++ return q, k, v ++ ++ ++@pytest.mark.parametrize("frames,height,width,prefix", [(2, 3, 3, 0), (3, 3, 6, 4), (5, 6, 3, 2)]) ++def test_pooling_matches_independent_weighted_key_reference(monkeypatch, frames, height, width, prefix): ++ q, k, v = fixture(frames, height, width, prefix) ++ saved = [x.clone() for x in (q, k, v)] ++ monkeypatch.setattr(flash, "_prefix_bnsd", dense) ++ actual = flash._video_global_pool_bnsd( ++ q, k, v, 0.25, target_start=prefix, frames=frames, height=height, width=width ++ ) ++ expected = expected_video(q, k, v, 0.25, frames, height, width, prefix) ++ torch.testing.assert_close(actual.float(), expected.float(), atol=0.0078125, rtol=0.02) ++ assert torch.count_nonzero(actual[:, k.shape[1] :]) == 0 ++ assert all(torch.equal(a, b) for a, b in zip(saved, (q, k, v))) ++ ++ ++def test_nine_identical_keys_retain_original_softmax_mass(monkeypatch): ++ q, k, v = fixture(height=3, width=3) ++ for tensor in (k, v): ++ for f in range(3): ++ tensor[:, 4 + f * 9 : 4 + (f + 1) * 9] = tensor[:, 4 + f * 9 : 5 + f * 9].clone() ++ monkeypatch.setattr(flash, "_prefix_bnsd", dense) ++ actual = flash._video_global_pool_bnsd(q, k, v, 0.25, target_start=4, frames=3, height=3, width=3) ++ expected = dense(q[:, : k.shape[1]], k, v, 0.25) ++ torch.testing.assert_close(actual[:, : k.shape[1]].float(), expected.float(), atol=0.0078125, rtol=0.02) ++ ++ ++@pytest.mark.parametrize("kind", ["height", "width", "tail", "frames", "batch", "query", "scale"]) ++def test_invalid_requested_geometry_raises(kind): ++ q, k, v = fixture() ++ kwargs = dict(target_start=4, frames=3, height=3, width=6, scale=0.25) ++ if kind in ("height", "width"): ++ kwargs[kind] += 1 ++ elif kind == "tail": ++ kwargs["target_start"] += 1 ++ elif kind == "frames": ++ kwargs["frames"] = 1 ++ elif kind == "batch": ++ q, k, v = [tensor.expand(2, -1, -1, -1) for tensor in (q, k, v)] ++ elif kind == "query": ++ q = q[:, :2] ++ else: ++ kwargs["scale"] = 0 ++ with pytest.raises(ValueError, match="pooled context"): ++ flash._video_global_pool_bnsd(q, k, v, **kwargs) ++ ++ ++def test_fixed_progress_and_default_off(monkeypatch): ++ monkeypatch.setenv("H3_VIDEO_GLOBAL_POOL", "3x3-global") ++ for step in range(50): ++ monkeypatch.setattr( ++ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=step, total_denoise_steps=50) ++ ) ++ assert flash._use_video_global_pool() == (step not in (0, 49)) ++ monkeypatch.setattr( ++ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=None, total_denoise_steps=50) ++ ) ++ with pytest.raises(ValueError, match="published"): ++ flash._use_video_global_pool() ++ monkeypatch.setenv("H3_VIDEO_GLOBAL_POOL", "other") ++ with pytest.raises(ValueError, match="3x3-global"): ++ flash._use_video_global_pool() ++ monkeypatch.delenv("H3_VIDEO_GLOBAL_POOL") ++ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") ++ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) ++ impl.causal, impl.softmax_scale = False, 0.25 ++ impl._resolve_packed_seq_npu = lambda *args: ([4096], [4093]) ++ monkeypatch.setattr(flash, "get_forward_context", lambda: pytest.fail("default path read experimental progress")) ++ monkeypatch.setattr(flash, "_prefix_bnsd", lambda q, k, v, scale: q) ++ q = torch.ones(1, 4096, 1, 8, dtype=torch.bfloat16) ++ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata ++ ++ assert impl.forward_fa_npu(q, q, q, AttentionMetadata(extra={"npu_attn_varlen": True})) is q ++ ++ ++@pytest.mark.parametrize("step", [0, 1, 49]) ++def test_requested_dispatch_uses_published_grid(monkeypatch, step): ++ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata, VideoTokenLayout, VideoTokenSpan ++ ++ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") ++ monkeypatch.setenv("H3_VIDEO_GLOBAL_POOL", "3x3-global") ++ monkeypatch.setattr( ++ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=step, total_denoise_steps=50) ++ ) ++ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) ++ impl.causal, impl.softmax_scale = False, 0.25 ++ impl._resolve_packed_seq_npu = lambda *args: ([4096], [4096]) ++ q = torch.ones(1, 4192, 1, 8, dtype=torch.bfloat16) ++ metadata = AttentionMetadata( ++ extra={"npu_attn_varlen": True}, ++ video_layout=VideoTokenLayout( ++ used_len=4096, video_spans=(VideoTokenSpan(start=64, latent_grid=(4, 24, 42), role="target"),) ++ ), ++ ) ++ calls = [] ++ monkeypatch.setattr(flash, "_prefix_bnsd", lambda *args: calls.append("dense") or q) ++ ++ def pooled(query, key, value, scale, **kwargs): ++ assert key.shape[1] == value.shape[1] == 4096 ++ assert kwargs == {"target_start": 64, "frames": 4, "height": 24, "width": 42} ++ calls.append("pooled") ++ return q ++ ++ monkeypatch.setattr(flash, "_video_global_pool_bnsd", pooled) ++ assert impl.forward_fa_npu(q, q, q, metadata) is q ++ assert calls == (["dense"] if step in (0, 49) else ["pooled"]) ++ ++ ++def test_requested_unsupported_main_geometry_does_not_fallback(monkeypatch): ++ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata ++ ++ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") ++ monkeypatch.setenv("H3_VIDEO_GLOBAL_POOL", "3x3-global") ++ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) ++ impl.causal = True ++ q = torch.ones(1, 4096, 1, 8, dtype=torch.bfloat16) ++ with pytest.raises(ValueError, match="unsupported"): ++ impl.forward_fa_npu(q, q, q, AttentionMetadata(extra={"npu_attn_varlen": True})) ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native Ascend FAS") ++def test_native_global_pool_fas_executes_shared_kv_with_mass_compensation(): ++ import torch_npu # noqa: F401 ++ ++ q, k, v = fixture(height=6, width=6, heads=14, dim=128, device="npu") ++ actual = flash._video_global_pool_bnsd(q, k, v, 128**-0.5, target_start=4, frames=3, height=6, width=6) ++ torch.npu.synchronize() ++ assert actual.dtype == torch.bfloat16 and torch.isfinite(actual).all() ++ expected = expected_video(q.cpu(), k.cpu(), v.cpu(), 128**-0.5, 3, 6, 6, 4) ++ torch.testing.assert_close(actual.cpu().float(), expected.float(), atol=0.03125, rtol=0.025) +diff --git a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py +index 6ebfb10..4932926 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py ++++ b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py +@@ -4,12 +4,14 @@ + + from __future__ import annotations + ++import math + import os + + import torch + + from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata + from vllm_omni.diffusion.attention.backends.flash_attn import FlashAttentionImpl ++from vllm_omni.diffusion.forward_context import get_forward_context + + + def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, scale: float) -> torch.Tensor: +@@ -36,6 +38,72 @@ def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, sc + return output.reshape(batch, heads, rows, dim).transpose(1, 2).contiguous() + + ++def _video_global_pool_bnsd( ++ query: torch.Tensor, ++ key: torch.Tensor, ++ value: torch.Tensor, ++ scale: float, ++ *, ++ target_start: int, ++ frames: int, ++ height: int, ++ width: int, ++) -> torch.Tensor: ++ """Share mass-compensated global spatial summaries across all video queries.""" ++ rows = height * width ++ if ( ++ query.shape[0] != 1 ++ or key.shape[0] != 1 ++ or value.shape != key.shape ++ or query.shape[2:] != key.shape[2:] ++ or target_start < 0 ++ or frames < 2 ++ or height < 3 ++ or width < 3 ++ or height % 3 ++ or width % 3 ++ or target_start + frames * rows != key.shape[1] ++ or query.shape[1] < key.shape[1] ++ or scale <= 0 ++ ): ++ raise ValueError("H3 pooled context requires a final video grid divisible by three") ++ heads, dim = query.shape[2:] ++ ++ def pack(tensor: torch.Tensor) -> torch.Tensor: ++ pooled = ( ++ tensor[:, target_start:] ++ .reshape(frames, height // 3, 3, width // 3, 3, heads, dim) ++ .float() ++ .mean(dim=(2, 4)) ++ .to(tensor.dtype) ++ .reshape(1, frames * rows // 9, heads, dim) ++ ) ++ return torch.cat((tensor[:, :target_start], pooled), dim=1) ++ ++ output = torch.zeros_like(query) ++ if target_start: ++ output[:, :target_start] = _prefix_bnsd(query[:, :target_start], key, value, scale) ++ video_query = query[:, target_start : key.shape[1]] ++ padded_q = torch.nn.functional.pad(video_query, (0, 16)) ++ padded_k = torch.nn.functional.pad(pack(key), (0, 16)) ++ padded_v = torch.nn.functional.pad(pack(value), (0, 16)) ++ padded_q[..., dim] = 1 ++ padded_k[:, target_start:, :, dim] = math.log(9) / scale ++ attended = _prefix_bnsd(padded_q, padded_k, padded_v, scale)[..., :dim] ++ output[:, target_start : key.shape[1]] = attended ++ return output ++ ++ ++def _use_video_global_pool() -> bool: ++ if os.environ.get("H3_VIDEO_GLOBAL_POOL") != "3x3-global": ++ raise ValueError("The experimental H3 pooled-context profile requires 3x3-global") ++ context = get_forward_context() ++ step, total = context.denoise_step_idx, context.total_denoise_steps ++ if total != 50 or step is None or not 0 <= step < 50: ++ raise ValueError("The experimental H3 pooled-context profile requires published 50-step progress") ++ return step not in (0, 49) ++ ++ + class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + """Use independent head batches for explicit long BF16 prefix attention. + +@@ -51,7 +119,10 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + value: torch.Tensor, + attn_metadata: AttentionMetadata | None = None, + ) -> torch.Tensor: ++ requested = os.environ.get("H3_VIDEO_GLOBAL_POOL") is not None and query.shape[1] >= 4096 + if os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" or attn_metadata is None: ++ if requested: ++ raise ValueError("H3 pooled context requires explicit fused attention and packed metadata") + return super().forward_fa_npu(query, key, value, attn_metadata) + extra = attn_metadata.extra + if ( +@@ -68,9 +139,30 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + or query.shape[2:] != value.shape[2:] + or key.shape[1] != value.shape[1] + ): ++ if requested: ++ raise ValueError("H3 pooled context received unsupported packed fused-attention inputs") + return super().forward_fa_npu(query, key, value, attn_metadata) + resolved = self._resolve_packed_seq_npu(query, key, extra) + if resolved is None: ++ if requested: ++ raise ValueError("H3 pooled context requires a resolved packed prefix") + return super().forward_fa_npu(query, key, value, attn_metadata) + used_k = resolved[1][0] ++ if os.environ.get("H3_VIDEO_GLOBAL_POOL") is not None and _use_video_global_pool(): ++ layout = attn_metadata.video_layout ++ targets = [] if layout is None else [span for span in layout.video_spans if span.role == "target"] ++ if len(targets) != 1: ++ raise ValueError("H3 pooled context requires exactly one target-video span") ++ target = targets[0] ++ frames, height, width = target.latent_grid ++ return _video_global_pool_bnsd( ++ query, ++ key[:, :used_k], ++ value[:, :used_k], ++ self.softmax_scale, ++ target_start=target.start, ++ frames=frames, ++ height=height, ++ width=width, ++ ) + return _prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale) diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c19-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c19-source-identity.json new file mode 100644 index 000000000..bde8516a5 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c19-source-identity.json @@ -0,0 +1,12 @@ +{ + "head": "c896a8c98493d50e51ed6fa167e4b8d8c59b3ecb", + "tree": "367e7f05c2d640211ad4713cc0b62742b0b25a0a", + "base": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "branch": "rsi/h3-297-video-global-pool", + "files": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ddfd6d6822cc6a5528fc95539f7f98dfdeed25d5489da3ed5cf76e6076a5ec94", + "tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py": "5457db550b59e3373be8f5f21873c1b78ac357a6ac170c2603ee531b50e91467" + }, + "published": false, + "patch_base": "e89b1d40d37f7f72767892a56fa7980ea572ba8e" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c20-candidate.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c20-candidate.patch new file mode 100644 index 000000000..34957ab40 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c20-candidate.patch @@ -0,0 +1,328 @@ +diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py +new file mode 100644 +index 0000000..cda0c4a +--- /dev/null ++++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py +@@ -0,0 +1,182 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++import math ++from types import SimpleNamespace ++ ++import pytest ++import torch ++ ++from tests.helpers.mark import hardware_marks ++from vllm_omni.diffusion.models.minimax_h3.attention import flash_h3 as flash ++from vllm_omni.platforms import current_omni_platform ++ ++pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=1)] ++ ++ ++def dense(query, key, value, scale): ++ q, k, v = [x.transpose(1, 2).float() for x in (query, key, value)] ++ return (torch.softmax(q @ k.transpose(-2, -1) * scale, dim=-1) @ v).transpose(1, 2).to(query.dtype) ++ ++ ++def expected_video(q, k, v, scale, frames, height, width, prefix): ++ rows = height * width ++ result = torch.zeros_like(q) ++ result[:, :prefix] = dense(q[:, :prefix], k, v, scale) ++ key_parts, value_parts = [k[:, :prefix]], [v[:, :prefix]] ++ variances = [torch.zeros(1, prefix, k.shape[2])] ++ for f in range(frames): ++ for y in range(0, height, 3): ++ for x in range(0, width, 3): ++ indices = [prefix + f * rows + (y + dy) * width + x + dx for dy in range(3) for dx in range(3)] ++ key_parts.append(k[:, indices].float().mean(1, keepdim=True).to(k.dtype)) ++ value_parts.append(v[:, indices].float().mean(1, keepdim=True).to(v.dtype)) ++ variances.append(k[:, indices].float().var(dim=1, unbiased=False).mean(-1).unsqueeze(1)) ++ keys, values = torch.cat(key_parts, 1), torch.cat(value_parts, 1) ++ queries = q[:, prefix : k.shape[1]] ++ logits = queries.transpose(1, 2).float() @ keys.transpose(1, 2).float().transpose(-2, -1) * scale ++ logits[..., prefix:] += math.log(9) ++ norm_sq = queries.transpose(1, 2).float().square().sum(-1, keepdim=True) ++ variance = torch.cat(variances, 1).transpose(1, 2).unsqueeze(2) ++ logits += 0.5 * scale**2 * norm_sq * variance ++ output = torch.softmax(logits, -1) @ values.transpose(1, 2).float() ++ result[:, prefix : k.shape[1]] = output.transpose(1, 2).to(q.dtype) ++ return result ++ ++ ++def fixture(frames=3, height=3, width=6, prefix=4, heads=2, dim=8, device="cpu"): ++ torch.manual_seed(217) ++ used = prefix + frames * height * width ++ q = torch.randn(1, used + 2, heads, dim, dtype=torch.bfloat16).to(device) ++ k, v = [torch.randn(1, used, heads, dim, dtype=torch.bfloat16).to(device) for _ in range(2)] ++ return q, k, v ++ ++ ++@pytest.mark.parametrize("frames,height,width,prefix", [(2, 3, 3, 0), (3, 3, 6, 4), (5, 6, 3, 2)]) ++def test_pooling_matches_independent_weighted_key_reference(monkeypatch, frames, height, width, prefix): ++ q, k, v = fixture(frames, height, width, prefix) ++ saved = [x.clone() for x in (q, k, v)] ++ monkeypatch.setattr(flash, "_prefix_bnsd", dense) ++ actual = flash._video_variance_pool_bnsd( ++ q, k, v, 0.25, target_start=prefix, frames=frames, height=height, width=width ++ ) ++ expected = expected_video(q, k, v, 0.25, frames, height, width, prefix) ++ torch.testing.assert_close(actual.float(), expected.float(), atol=0.0078125, rtol=0.02) ++ assert torch.count_nonzero(actual[:, k.shape[1] :]) == 0 ++ assert all(torch.equal(a, b) for a, b in zip(saved, (q, k, v))) ++ ++ ++def test_nine_identical_keys_retain_original_softmax_mass(monkeypatch): ++ q, k, v = fixture(height=3, width=3) ++ for tensor in (k, v): ++ for f in range(3): ++ tensor[:, 4 + f * 9 : 4 + (f + 1) * 9] = tensor[:, 4 + f * 9 : 5 + f * 9].clone() ++ monkeypatch.setattr(flash, "_prefix_bnsd", dense) ++ actual = flash._video_variance_pool_bnsd(q, k, v, 0.25, target_start=4, frames=3, height=3, width=3) ++ expected = dense(q[:, : k.shape[1]], k, v, 0.25) ++ torch.testing.assert_close(actual[:, : k.shape[1]].float(), expected.float(), atol=0.0078125, rtol=0.02) ++ ++ ++@pytest.mark.parametrize("kind", ["height", "width", "tail", "frames", "batch", "query", "scale"]) ++def test_invalid_requested_geometry_raises(kind): ++ q, k, v = fixture() ++ kwargs = dict(target_start=4, frames=3, height=3, width=6, scale=0.25) ++ if kind in ("height", "width"): ++ kwargs[kind] += 1 ++ elif kind == "tail": ++ kwargs["target_start"] += 1 ++ elif kind == "frames": ++ kwargs["frames"] = 1 ++ elif kind == "batch": ++ q, k, v = [tensor.expand(2, -1, -1, -1) for tensor in (q, k, v)] ++ elif kind == "query": ++ q = q[:, :2] ++ else: ++ kwargs["scale"] = 0 ++ with pytest.raises(ValueError, match="pooled context"): ++ flash._video_variance_pool_bnsd(q, k, v, **kwargs) ++ ++ ++def test_fixed_progress_and_default_off(monkeypatch): ++ monkeypatch.setenv("H3_VIDEO_VARIANCE_POOL", "3x3-variance") ++ for step in range(50): ++ monkeypatch.setattr( ++ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=step, total_denoise_steps=50) ++ ) ++ assert flash._use_video_variance_pool() == (step not in (0, 49)) ++ monkeypatch.setattr( ++ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=None, total_denoise_steps=50) ++ ) ++ with pytest.raises(ValueError, match="published"): ++ flash._use_video_variance_pool() ++ monkeypatch.setenv("H3_VIDEO_VARIANCE_POOL", "other") ++ with pytest.raises(ValueError, match="3x3-variance"): ++ flash._use_video_variance_pool() ++ monkeypatch.delenv("H3_VIDEO_VARIANCE_POOL") ++ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") ++ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) ++ impl.causal, impl.softmax_scale = False, 0.25 ++ impl._resolve_packed_seq_npu = lambda *args: ([4096], [4093]) ++ monkeypatch.setattr(flash, "get_forward_context", lambda: pytest.fail("default path read experimental progress")) ++ monkeypatch.setattr(flash, "_prefix_bnsd", lambda q, k, v, scale: q) ++ q = torch.ones(1, 4096, 1, 8, dtype=torch.bfloat16) ++ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata ++ ++ assert impl.forward_fa_npu(q, q, q, AttentionMetadata(extra={"npu_attn_varlen": True})) is q ++ ++ ++@pytest.mark.parametrize("step", [0, 1, 49]) ++def test_requested_dispatch_uses_published_grid(monkeypatch, step): ++ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata, VideoTokenLayout, VideoTokenSpan ++ ++ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") ++ monkeypatch.setenv("H3_VIDEO_VARIANCE_POOL", "3x3-variance") ++ monkeypatch.setattr( ++ flash, "get_forward_context", lambda: SimpleNamespace(denoise_step_idx=step, total_denoise_steps=50) ++ ) ++ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) ++ impl.causal, impl.softmax_scale = False, 0.25 ++ impl._resolve_packed_seq_npu = lambda *args: ([4096], [4096]) ++ q = torch.ones(1, 4192, 1, 8, dtype=torch.bfloat16) ++ metadata = AttentionMetadata( ++ extra={"npu_attn_varlen": True}, ++ video_layout=VideoTokenLayout( ++ used_len=4096, video_spans=(VideoTokenSpan(start=64, latent_grid=(4, 24, 42), role="target"),) ++ ), ++ ) ++ calls = [] ++ monkeypatch.setattr(flash, "_prefix_bnsd", lambda *args: calls.append("dense") or q) ++ ++ def pooled(query, key, value, scale, **kwargs): ++ assert key.shape[1] == value.shape[1] == 4096 ++ assert kwargs == {"target_start": 64, "frames": 4, "height": 24, "width": 42} ++ calls.append("pooled") ++ return q ++ ++ monkeypatch.setattr(flash, "_video_variance_pool_bnsd", pooled) ++ assert impl.forward_fa_npu(q, q, q, metadata) is q ++ assert calls == (["dense"] if step in (0, 49) else ["pooled"]) ++ ++ ++def test_requested_unsupported_main_geometry_does_not_fallback(monkeypatch): ++ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata ++ ++ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") ++ monkeypatch.setenv("H3_VIDEO_VARIANCE_POOL", "3x3-variance") ++ impl = object.__new__(flash.MiniMaxH3FlashAttentionImpl) ++ impl.causal = True ++ q = torch.ones(1, 4096, 1, 8, dtype=torch.bfloat16) ++ with pytest.raises(ValueError, match="unsupported"): ++ impl.forward_fa_npu(q, q, q, AttentionMetadata(extra={"npu_attn_varlen": True})) ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native Ascend FAS") ++def test_native_variance_pool_fas_executes_second_moment_compensation(): ++ import torch_npu # noqa: F401 ++ ++ q, k, v = fixture(height=6, width=6, heads=14, dim=128, device="npu") ++ actual = flash._video_variance_pool_bnsd(q, k, v, 128**-0.5, target_start=4, frames=3, height=6, width=6) ++ torch.npu.synchronize() ++ assert actual.dtype == torch.bfloat16 and torch.isfinite(actual).all() ++ expected = expected_video(q.cpu(), k.cpu(), v.cpu(), 128**-0.5, 3, 6, 6, 4) ++ torch.testing.assert_close(actual.cpu().float(), expected.float(), atol=0.03125, rtol=0.025) +diff --git a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py +index 6ebfb10..9dce8b9 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py ++++ b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py +@@ -4,12 +4,14 @@ + + from __future__ import annotations + ++import math + import os + + import torch + + from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata + from vllm_omni.diffusion.attention.backends.flash_attn import FlashAttentionImpl ++from vllm_omni.diffusion.forward_context import get_forward_context + + + def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, scale: float) -> torch.Tensor: +@@ -36,6 +38,78 @@ def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, sc + return output.reshape(batch, heads, rows, dim).transpose(1, 2).contiguous() + + ++def _video_variance_pool_bnsd( ++ query: torch.Tensor, ++ key: torch.Tensor, ++ value: torch.Tensor, ++ scale: float, ++ *, ++ target_start: int, ++ frames: int, ++ height: int, ++ width: int, ++) -> torch.Tensor: ++ """Share global summaries with isotropic key-variance mass compensation.""" ++ rows = height * width ++ if ( ++ query.shape[0] != 1 ++ or key.shape[0] != 1 ++ or value.shape != key.shape ++ or query.shape[2:] != key.shape[2:] ++ or target_start < 0 ++ or frames < 2 ++ or height < 3 ++ or width < 3 ++ or height % 3 ++ or width % 3 ++ or target_start + frames * rows != key.shape[1] ++ or query.shape[1] < key.shape[1] ++ or scale <= 0 ++ ): ++ raise ValueError("H3 pooled context requires a final video grid divisible by three") ++ heads, dim = query.shape[2:] ++ ++ def pack(tensor: torch.Tensor) -> torch.Tensor: ++ pooled = ( ++ tensor[:, target_start:] ++ .reshape(frames, height // 3, 3, width // 3, 3, heads, dim) ++ .float() ++ .mean(dim=(2, 4)) ++ .to(tensor.dtype) ++ .reshape(1, frames * rows // 9, heads, dim) ++ ) ++ return torch.cat((tensor[:, :target_start], pooled), dim=1) ++ ++ output = torch.zeros_like(query) ++ if target_start: ++ output[:, :target_start] = _prefix_bnsd(query[:, :target_start], key, value, scale) ++ video_query = query[:, target_start : key.shape[1]] ++ grouped_key = key[:, target_start:].reshape(frames, height // 3, 3, width // 3, 3, heads, dim).float() ++ variance = grouped_key.var(dim=(2, 4), unbiased=False).mean(dim=-1).reshape(1, frames * rows // 9, heads) ++ query_norm_sq = video_query.float().square().sum(dim=-1) ++ padded_q = torch.nn.functional.pad(video_query, (0, 16)) ++ padded_k = torch.nn.functional.pad(pack(key), (0, 16)) ++ padded_v = torch.nn.functional.pad(pack(value), (0, 16)) ++ padded_q[..., dim] = 1 ++ padded_k[:, target_start:, :, dim] = math.log(9) / scale ++ # Isotropic second-moment approximation to the pooled log-partition. ++ padded_q[..., dim + 1] = 0.5 * scale * query_norm_sq ++ padded_k[:, target_start:, :, dim + 1] = variance ++ attended = _prefix_bnsd(padded_q, padded_k, padded_v, scale)[..., :dim] ++ output[:, target_start : key.shape[1]] = attended ++ return output ++ ++ ++def _use_video_variance_pool() -> bool: ++ if os.environ.get("H3_VIDEO_VARIANCE_POOL") != "3x3-variance": ++ raise ValueError("The experimental H3 pooled-context profile requires 3x3-variance") ++ context = get_forward_context() ++ step, total = context.denoise_step_idx, context.total_denoise_steps ++ if total != 50 or step is None or not 0 <= step < 50: ++ raise ValueError("The experimental H3 pooled-context profile requires published 50-step progress") ++ return step not in (0, 49) ++ ++ + class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + """Use independent head batches for explicit long BF16 prefix attention. + +@@ -51,7 +125,10 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + value: torch.Tensor, + attn_metadata: AttentionMetadata | None = None, + ) -> torch.Tensor: ++ requested = os.environ.get("H3_VIDEO_VARIANCE_POOL") is not None and query.shape[1] >= 4096 + if os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" or attn_metadata is None: ++ if requested: ++ raise ValueError("H3 pooled context requires explicit fused attention and packed metadata") + return super().forward_fa_npu(query, key, value, attn_metadata) + extra = attn_metadata.extra + if ( +@@ -68,9 +145,30 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + or query.shape[2:] != value.shape[2:] + or key.shape[1] != value.shape[1] + ): ++ if requested: ++ raise ValueError("H3 pooled context received unsupported packed fused-attention inputs") + return super().forward_fa_npu(query, key, value, attn_metadata) + resolved = self._resolve_packed_seq_npu(query, key, extra) + if resolved is None: ++ if requested: ++ raise ValueError("H3 pooled context requires a resolved packed prefix") + return super().forward_fa_npu(query, key, value, attn_metadata) + used_k = resolved[1][0] ++ if os.environ.get("H3_VIDEO_VARIANCE_POOL") is not None and _use_video_variance_pool(): ++ layout = attn_metadata.video_layout ++ targets = [] if layout is None else [span for span in layout.video_spans if span.role == "target"] ++ if len(targets) != 1: ++ raise ValueError("H3 pooled context requires exactly one target-video span") ++ target = targets[0] ++ frames, height, width = target.latent_grid ++ return _video_variance_pool_bnsd( ++ query, ++ key[:, :used_k], ++ value[:, :used_k], ++ self.softmax_scale, ++ target_start=target.start, ++ frames=frames, ++ height=height, ++ width=width, ++ ) + return _prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale) diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c20-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c20-source-identity.json new file mode 100644 index 000000000..19b78b6bc --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c20-source-identity.json @@ -0,0 +1,12 @@ +{ + "head": "bba4f7e8802497772f46628b6f408391add84679", + "tree": "5a7ed678aa86cf2555a80f64a5f21ee9bda540fe", + "base": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "branch": "rsi/h3-297-video-variance-pool", + "files": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "4390450bc5324f7028125101cd4bb32e9de237b04bc12fa879b48110d7122e3c", + "tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py": "e871fb4b222d61d10b6c442a333a52c976cceaa0d406233502ecbcaddfeba8c9" + }, + "published": false, + "patch_base": "e89b1d40d37f7f72767892a56fa7980ea572ba8e" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c21-candidate.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c21-candidate.patch new file mode 100644 index 000000000..017fbf116 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c21-candidate.patch @@ -0,0 +1,2041 @@ +diff --git a/tests/diffusion/distributed/test_collective_gate.py b/tests/diffusion/distributed/test_collective_gate.py +new file mode 100644 +index 0000000..e66fdae +--- /dev/null ++++ b/tests/diffusion/distributed/test_collective_gate.py +@@ -0,0 +1,205 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++"""Unit tests for the DLO/QKV-AllToAll ordering gate. ++ ++Covers the barrier rendezvous and the hook's deferred-AllGather state machine ++that keeps the background weight gather from taking the links ahead of the ++block's foreground AllToAll. ++""" ++ ++from unittest.mock import Mock ++ ++import pytest ++import torch ++from torch import nn ++ ++import vllm_omni.diffusion.offloader.distributed_layerwise_backend as dist_backend_module ++from tests.diffusion.offloader.helpers import DummyStream, patch_offload_runtime ++from vllm_omni.diffusion.distributed.collective_gate import QKVAllToAllBarrier ++from vllm_omni.diffusion.offloader.distributed_layerwise_backend import ( ++ DistributedLayerwiseOffloadHook, ++ attach_collective_gate, ++) ++ ++pytestmark = [pytest.mark.diffusion, pytest.mark.cpu, pytest.mark.core_model] ++ ++ ++@pytest.fixture ++def patched_offload_runtime(monkeypatch): ++ patch_offload_runtime(monkeypatch, dist_backend_module.current_omni_platform, synchronize=True) ++ ++ ++class TestQKVAllToAllBarrier: ++ def test_signal_without_arm_is_a_noop(self): ++ on_signal = Mock() ++ barrier = QKVAllToAllBarrier(on_signal) ++ ++ barrier.signal() ++ ++ on_signal.assert_not_called() ++ assert not barrier.armed ++ ++ def test_arm_then_signal_fires_once(self): ++ on_signal = Mock() ++ barrier = QKVAllToAllBarrier(on_signal) ++ ++ barrier.arm() ++ assert barrier.armed ++ barrier.signal() ++ ++ on_signal.assert_called_once_with() ++ assert not barrier.armed ++ ++ # A second signal in the same forward must not re-issue the gather. ++ barrier.signal() ++ on_signal.assert_called_once_with() ++ ++ def test_disarm_suppresses_a_pending_signal(self): ++ on_signal = Mock() ++ barrier = QKVAllToAllBarrier(on_signal) ++ ++ barrier.arm() ++ barrier.disarm() ++ barrier.signal() ++ ++ on_signal.assert_not_called() ++ ++ ++def _gated_hook() -> DistributedLayerwiseOffloadHook: ++ """A hook wired for the gated path, with hand-set shard buffers.""" ++ hook = DistributedLayerwiseOffloadHook( ++ next_block=nn.Linear(2, 2, bias=False), ++ device=torch.device("cpu"), ++ dp_group=object(), ++ dp_size=2, ++ rank=0, ++ copy_stream=DummyStream(), ++ comm_stream=DummyStream(), ++ pin_memory=False, ++ order_allgather_after_attn_a2a=True, ++ ) ++ device_buf = {torch.float32: torch.zeros(4)} ++ shard_buf = {torch.float32: torch.zeros(2)} ++ hook.gpu_buffers = [dict(device_buf), dict(device_buf)] ++ hook.gpu_shard_buffers = [dict(shard_buf), dict(shard_buf)] ++ hook.cpu_shards = {torch.float32: torch.zeros(2)} ++ hook._ag_output_sizes = {torch.float32: 4} ++ hook.metadata = {} ++ hook._cached_repoint = () ++ hook.rank_local_mmap = False ++ return hook ++ ++ ++class TestDeferredAllGather: ++ def test_prefetch_defers_the_gather_until_the_barrier_signals(self, patched_offload_runtime, monkeypatch): ++ hook = _gated_hook() ++ gather = Mock() ++ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) ++ ++ hook.prefetch_layer(0, non_blocking=True) ++ ++ # H2D ran; the collective is held back for the block's AllToAll. ++ gather.assert_not_called() ++ assert hook._a2a_barrier.armed ++ assert hook._deferred_allgather is not None ++ # The ready event must still be published for the consumer. ++ assert hook.ready_events[0] is hook._prefetch_done ++ ++ hook._a2a_barrier.signal() ++ ++ gather.assert_called_once() ++ assert hook._deferred_allgather is None ++ assert not hook._a2a_barrier.armed ++ ++ def test_post_forward_flush_issues_an_unsignalled_gather(self, patched_offload_runtime, monkeypatch): ++ """A block whose attention never ran must still get its weights.""" ++ hook = _gated_hook() ++ gather = Mock() ++ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) ++ ++ hook.prefetch_layer(0, non_blocking=True) ++ gather.assert_not_called() ++ ++ # pre_forward ran, attention did not: the pairing flush must fire. ++ hook.post_forward(hook.next_block, None) ++ ++ gather.assert_called_once() ++ assert hook._deferred_allgather is None ++ assert not hook._a2a_barrier.armed ++ ++ def test_sync_prefetch_issues_immediately_without_arming(self, patched_offload_runtime, monkeypatch): ++ """The cache-dit sync fallback has no attention signal to wait for.""" ++ hook = _gated_hook() ++ gather = Mock() ++ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) ++ ++ hook.prefetch_layer(0, non_blocking=False) ++ ++ gather.assert_called_once() ++ assert not hook._a2a_barrier.armed ++ assert hook._deferred_allgather is None ++ ++ def test_disabled_by_default(self, patched_offload_runtime, monkeypatch): ++ hook = DistributedLayerwiseOffloadHook( ++ next_block=nn.Linear(2, 2, bias=False), ++ device=torch.device("cpu"), ++ dp_group=object(), ++ dp_size=2, ++ rank=0, ++ copy_stream=DummyStream(), ++ comm_stream=DummyStream(), ++ pin_memory=False, ++ ) ++ hook.gpu_buffers = [{torch.float32: torch.zeros(4)}] * 2 ++ hook.gpu_shard_buffers = [{torch.float32: torch.zeros(2)}] * 2 ++ hook.cpu_shards = {torch.float32: torch.zeros(2)} ++ hook._ag_output_sizes = {torch.float32: 4} ++ hook.metadata = {} ++ hook._cached_repoint = () ++ gather = Mock() ++ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) ++ ++ hook.prefetch_layer(0, non_blocking=True) ++ ++ assert hook._a2a_barrier is None ++ gather.assert_called_once() ++ ++ ++class TestAttachCollectiveGate: ++ def test_block_without_attention_reports_zero(self): ++ hook = _gated_hook() ++ ++ assert attach_collective_gate(nn.Linear(2, 2), hook) == 0 ++ ++ ++@pytest.mark.parametrize("flush", [False, True]) ++def test_chunked_gate_defers_copies_and_reuses_bounded_slots(patched_offload_runtime, monkeypatch, flush): ++ from types import SimpleNamespace ++ ++ hook = _gated_hook() ++ chunks = tuple( ++ SimpleNamespace(chunk_id=i, local_numel=1, cpu_offset=i, full_offset=2 * i, padded_numel=2) for i in range(3) ++ ) ++ hook.manifest = SimpleNamespace(dtypes=(SimpleNamespace(dtype=torch.float32, chunks=chunks),)) ++ hook.cpu_shards = {torch.float32: torch.tensor([11.0, 22.0, 33.0])} ++ hook.gpu_buffers = [{torch.float32: torch.zeros(6)} for _ in range(2)] ++ calls = [] ++ ++ def gather(output, local, **kwargs): ++ calls.append(local.clone()) ++ output.copy_(local.repeat(2)) ++ ++ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) ++ hook.prefetch_layer(0, non_blocking=True) ++ assert not calls and hook._deferred_chunked_allgather is not None ++ assert torch.equal(hook.gpu_shard_buffers[0][torch.float32], torch.zeros(2)) ++ if flush: ++ hook._flush_deferred_allgather() ++ else: ++ hook._a2a_barrier.signal() ++ assert len(calls) == 3 and hook._deferred_chunked_allgather is None ++ assert not hook._a2a_barrier.armed ++ assert torch.equal(hook.gpu_buffers[0][torch.float32], torch.tensor([11.0, 11.0, 22.0, 22.0, 33.0, 33.0])) ++ hook._a2a_barrier.signal() ++ assert len(calls) == 3 +diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py +new file mode 100644 +index 0000000..4e9979e +--- /dev/null ++++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py +@@ -0,0 +1,198 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++from datetime import timedelta ++ ++import pytest ++import torch ++import torch.distributed as dist ++import torch.nn as nn ++ ++from tests.diffusion.models.minimax_h3.test_minimax_h3_video_row_cache import tiny_block ++from tests.helpers.mark import hardware_marks ++from vllm_omni.diffusion.layers.indexed_modulation import indexed_gate ++from vllm_omni.diffusion.models.minimax_h3.attention.flash_h3 import _prefix_bnsd, _subset_prefix_bnsd ++from vllm_omni.diffusion.models.minimax_h3.video_row_cache import MiniMaxH3VideoRowCache ++from vllm_omni.platforms import current_omni_platform ++ ++pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=4)] ++ ++ ++def make_state(start=0, length=4): ++ state = MiniMaxH3VideoRowCache(torch.tensor([1, 2]), seq_len=4, used_len=3, num_layers=1, pre_gate=True) ++ state.prepare_local_span(start, length) ++ return state ++ ++ ++def test_fixed_pre_gate_schedule_storage_and_request_cleanup(): ++ state = make_state() ++ x = torch.ones(4, 2, dtype=torch.bfloat16) ++ full = [] ++ for step in range(50): ++ state.begin_step(step) ++ if not state.partial: ++ full.append(step) ++ for stage in ("attn", "mlp"): ++ out = state.pre_gate_output(0, x, stage) ++ assert out.dtype == torch.bfloat16 ++ if state.partial: ++ state.record_fresh_gate(stage) ++ state.end_step() ++ assert full == [0, 1, 4, 9, 19, 29, 39, 49] ++ assert state.full_blocks == 8 and state.partial_blocks == 42 ++ assert state.fresh_attention_gates == state.fresh_mlp_gates == 42 ++ assert all(t.dtype == torch.bfloat16 for t in [*state._attention_outputs.values(), *state._mlp_outputs.values()]) ++ assert state.stats()["live_cache_bytes"] == 16 ++ state.close() ++ assert state.stats()["live_cache_bytes"] == 0 and state.closed ++ ++ ++@pytest.mark.parametrize("failure", ["missing_mlp", "mlp_first", "gated_capture", "gated_restore"]) ++def test_pre_gate_stage_order_cannot_silently_use_block_residual(failure): ++ state = make_state() ++ x = torch.ones(4, 2, dtype=torch.bfloat16) ++ state.begin_step(0) ++ with pytest.raises(RuntimeError): ++ if failure == "missing_mlp": ++ state.pre_gate_output(0, x, "attn") ++ state.end_step() ++ elif failure == "mlp_first": ++ state.pre_gate_output(0, x, "mlp") ++ elif failure == "gated_capture": ++ state.capture(0, x, x) ++ else: ++ state.restore(0, x) ++ state.close() ++ ++ ++class ConstantMLP(nn.Module): ++ def __init__(self): ++ super().__init__() ++ self.rows = [] ++ ++ def forward(self, x): ++ self.rows.append(x.shape[0]) ++ return torch.full_like(x, 3) ++ ++ ++@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) ++def test_real_block_reapplies_changed_gates_and_preserves_empty_dense_shards(start, length): ++ state = make_state(start, length) ++ block = tiny_block() ++ gates = [1.0, 1.0] ++ block.adaln_proj = lambda t: ( ++ torch.zeros(3, 2), ++ torch.zeros(3, 2), ++ torch.full((3, 2), gates[0]), ++ torch.zeros(3, 2), ++ torch.zeros(3, 2), ++ torch.full((3, 2), gates[1]), ++ ) ++ block.attn = lambda x, **kwargs: torch.full_like(x, 2) ++ block.mlp = ConstantMLP() ++ x = torch.ones(length, 2, dtype=torch.bfloat16) ++ kwargs = dict( ++ t_emb=torch.zeros(1), ++ combined_indices=torch.zeros(length, dtype=torch.long), ++ rope_table=None, ++ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), ++ max_seqlen=3, ++ packed_total=4, ++ ) ++ for step in (0, 1): ++ state.begin_step(step) ++ expected = block(x, **kwargs) ++ actual = block(x, video_row_cache=state, **kwargs) ++ assert torch.equal(actual.view(torch.uint8), expected.view(torch.uint8)) ++ state.end_step() ++ gates[:] = [0.5, -0.25] ++ current = x * 2 ++ saved = current.clone() ++ expected = block(current, **kwargs) ++ block.mlp.rows.clear() ++ state.begin_step(2) ++ actual = block(current, video_row_cache=state, **kwargs) ++ state.end_step() ++ rows = state.local_dense_positions ++ assert block.mlp.rows == ([rows.numel()] if rows.numel() else []) ++ assert torch.equal( ++ actual[: min(length, 3 - start)].view(torch.uint8), expected[: min(length, 3 - start)].view(torch.uint8) ++ ) ++ assert torch.equal( ++ actual.index_select(0, state.local_video_positions), torch.full((state.local_video_positions.numel(), 2), 2.25) ++ ) ++ assert not torch.equal( ++ actual.index_select(0, state.local_video_positions), current.index_select(0, state.local_video_positions) + 5 ++ ) ++ assert torch.equal(current, saved) ++ assert state.fresh_attention_gates == state.fresh_mlp_gates == 1 ++ state.close() ++ ++ ++def _native_pre_gate_worker(rank): ++ import torch_npu # noqa: F401 ++ ++ torch.npu.set_device(rank) ++ dist.init_process_group( ++ "hccl", init_method="tcp://127.0.0.1:29597", rank=rank, world_size=4, timeout=timedelta(seconds=120) ++ ) ++ try: ++ torch.manual_seed(219) ++ full = [torch.randn(32, 4, 128, dtype=torch.bfloat16).npu() for _ in range(3)] ++ exchanged = [] ++ for tensor in full: ++ local = tensor[rank * 8 : (rank + 1) * 8] ++ send = local.reshape(8, 4, 1, 128).permute(1, 0, 2, 3).contiguous() ++ receive = torch.empty_like(send) ++ dist.all_to_all_single(receive, send) ++ exchanged.append(receive.reshape(1, 32, 1, 128)) ++ state = MiniMaxH3VideoRowCache( ++ torch.arange(8, 24, device="npu"), seq_len=32, used_len=30, num_layers=1, pre_gate=True ++ ) ++ state.prepare_local_span(rank * 8, 8) ++ x = torch.full((8, 512), 2, dtype=torch.bfloat16, device="npu") ++ for step in (0, 1): ++ state.begin_step(step) ++ state.pre_gate_output(0, torch.full_like(x, 0.5), "attn") ++ state.pre_gate_output(0, torch.full_like(x, 0.25), "mlp") ++ state.end_step() ++ state.begin_step(2) ++ out = _subset_prefix_bnsd(exchanged[0], exchanged[1][:, :30], exchanged[2][:, :30], 128**-0.5, state) ++ send, receive = out.reshape(4, 8, 1, 128).contiguous(), torch.empty_like(out.reshape(4, 8, 1, 128)) ++ dist.all_to_all_single(receive, send) ++ local_out = receive.permute(1, 0, 2, 3).reshape(8, 512) ++ expected = _prefix_bnsd(full[0].unsqueeze(0), full[1][:30].unsqueeze(0), full[2][:30].unsqueeze(0), 128**-0.5)[ ++ 0 ++ ].reshape(32, 512) ++ dense = state.local_dense_positions ++ torch.testing.assert_close( ++ local_out.index_select(0, dense).float(), ++ expected[rank * 8 : (rank + 1) * 8].index_select(0, dense).float(), ++ atol=0.03125, ++ rtol=0.02, ++ ) ++ attn = state.pre_gate_output(0, local_out, "attn") ++ indices = torch.zeros(8, dtype=torch.long, device="npu") ++ gate = torch.full((1, 512), 0.5, dtype=torch.float32, device="npu") ++ after_attn = indexed_gate(x, gate, attn, indices) ++ state.record_fresh_gate("attn") ++ mlp = state.pre_gate_output(0, torch.zeros_like(x), "mlp") ++ result = indexed_gate(after_attn, gate, mlp, indices) ++ state.record_fresh_gate("mlp") ++ video = state.local_video_positions ++ assert torch.equal( ++ result.index_select(0, video), torch.full((video.numel(), 512), 2.375, dtype=torch.bfloat16, device="npu") ++ ) ++ assert dense.numel() == [8, 0, 0, 6][rank] ++ state.end_step() ++ state.close() ++ assert state.stats()["live_cache_bytes"] == 0 ++ torch.npu.synchronize() ++ dist.barrier() ++ finally: ++ dist.destroy_process_group() ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires four native Ascend devices") ++def test_native_four_rank_pre_gate_subset_attention_and_current_gates(): ++ torch.multiprocessing.spawn(_native_pre_gate_worker, nprocs=4, join=True) +diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py +new file mode 100644 +index 0000000..94ff958 +--- /dev/null ++++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py +@@ -0,0 +1,406 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++from datetime import timedelta ++from types import SimpleNamespace ++ ++import pytest ++import torch ++import torch.distributed as dist ++import torch.nn as nn ++ ++from tests.helpers.mark import hardware_marks ++from vllm_omni.diffusion.models.minimax_h3 import minimax_h3_blocks as blocks ++from vllm_omni.diffusion.models.minimax_h3.attention.flash_h3 import _prefix_bnsd, _subset_prefix_bnsd ++from vllm_omni.diffusion.models.minimax_h3.video_row_cache import MiniMaxH3VideoRowCache ++from vllm_omni.platforms import current_omni_platform ++ ++pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=4)] ++ ++ ++def state_for_rows(device="cpu", start=0, length=4): ++ state = MiniMaxH3VideoRowCache(torch.tensor([1, 2], device=device), seq_len=4, used_len=3, num_layers=1) ++ state.prepare_local_span(start, length) ++ return state ++ ++ ++def anchor(state, x): ++ for step in range(5): ++ state.begin_step(step) ++ state.capture(0, x, x + 0.125) ++ state.end_step() ++ state.begin_step(5) ++ ++ ++def test_fixed_schedule_and_row_ownership_excludes_padding(): ++ assert [i for i in range(50) if MiniMaxH3VideoRowCache.is_full_step(i)] == [ ++ 0, ++ 1, ++ 2, ++ 3, ++ 4, ++ 9, ++ 14, ++ 19, ++ 24, ++ 29, ++ 34, ++ 39, ++ 44, ++ 46, ++ 47, ++ 48, ++ 49, ++ ] ++ state = state_for_rows() ++ assert state.global_dense_positions.tolist() == [0] ++ assert state.local_dense_positions.tolist() == [0] ++ assert state.local_video_positions.tolist() == [1, 2] ++ with pytest.raises(RuntimeError, match="ownership"): ++ state.prepare_local_span(0, 3) ++ ++ ++@pytest.mark.parametrize("positions", [[1, 1], [2, 1], [-1], [3]]) ++def test_invalid_target_positions_are_rejected(positions): ++ with pytest.raises(ValueError): ++ MiniMaxH3VideoRowCache(torch.tensor(positions), seq_len=4, used_len=3, num_layers=1) ++ ++ ++def test_delta_is_fp32_and_restore_owns_output(): ++ state = state_for_rows() ++ x = torch.tensor([[1, 2], [3, 4], [5, 6], [7, 8]], dtype=torch.bfloat16) ++ anchor(state, x) ++ assert state._deltas[0].dtype == torch.float32 ++ before = x.clone() ++ output = state.restore(0, x) ++ assert torch.equal(output[1:3], x[1:3] + 0.125) ++ assert torch.equal(output[[0, 3]], x[[0, 3]]) ++ output.zero_() ++ assert torch.equal(x, before) ++ state.end_step() ++ state.close() ++ assert state.closed and not state._deltas and state.global_dense_positions.numel() == 0 ++ ++ ++def large_delta_roundtrip(device="cpu"): ++ state = state_for_rows(device=device) ++ x = torch.full((4, 2), 100000, dtype=torch.bfloat16, device=device) ++ expected = -x ++ delta = expected.float() - x.float() ++ assert torch.isfinite(delta).all() ++ assert not torch.isfinite(delta.to(torch.float16)).all() ++ for step in range(5): ++ state.begin_step(step) ++ state.capture(0, x, expected) ++ state.end_step() ++ state.begin_step(5) ++ actual = state.restore(0, x) ++ assert torch.isfinite(actual).all() ++ assert torch.equal(actual[1:3].cpu().view(torch.uint8), expected[1:3].cpu().view(torch.uint8)) ++ assert torch.equal(actual[[0, 3]], x[[0, 3]]) ++ state.end_step() ++ state.close() ++ assert state.stats()["live_cache_bytes"] == 0 ++ ++ ++def test_large_finite_bf16_delta_roundtrip_exceeds_fp16_storage_range(): ++ large_delta_roundtrip() ++ ++ ++class CountMLP(nn.Module): ++ def __init__(self): ++ super().__init__() ++ self.rows = [] ++ ++ def forward(self, x): ++ self.rows.append(x.shape[0]) ++ return x + 0.25 ++ ++ ++def tiny_block(): ++ block = object.__new__(blocks.MiniMaxH3DiTBlock) ++ nn.Module.__init__(block) ++ block.layer_index = 0 ++ block.norm1 = SimpleNamespace(weight=torch.ones(2), variance_epsilon=1e-6) ++ block.norm2 = SimpleNamespace(weight=torch.ones(2), variance_epsilon=1e-6) ++ block.adaln_proj = lambda t: ( ++ torch.zeros(3, 2), ++ torch.zeros(3, 2), ++ torch.ones(3, 2), ++ torch.zeros(3, 2), ++ torch.zeros(3, 2), ++ torch.ones(3, 2), ++ ) ++ block.attn = lambda x, **kwargs: x * 0.5 ++ block.mlp = CountMLP() ++ return block ++ ++ ++@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) ++def test_real_block_dense_path_and_partial_mlp_empty_rows(start, length): ++ state = state_for_rows(start=start, length=length) ++ block = tiny_block() ++ x = torch.arange(length * 2).reshape(length, 2).to(torch.bfloat16) ++ before = x.clone() ++ kwargs = dict( ++ t_emb=torch.zeros(1), ++ combined_indices=torch.zeros(length, dtype=torch.long), ++ rope_table=None, ++ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), ++ max_seqlen=3, ++ packed_total=4, ++ ) ++ expected = block(x, **kwargs) ++ for step in range(5): ++ state.begin_step(step) ++ actual = block(x, video_row_cache=state, **kwargs) ++ assert torch.equal(actual.view(torch.uint8), expected.view(torch.uint8)) ++ state.end_step() ++ block.mlp.rows.clear() ++ state.begin_step(5) ++ actual = block(x, video_row_cache=state, **kwargs) ++ state.end_step() ++ rows = state.local_dense_positions ++ assert block.mlp.rows == ([rows.numel()] if rows.numel() else []) ++ assert torch.equal(actual.index_select(0, rows), expected.index_select(0, rows)) ++ assert torch.equal(x, before) ++ ++ ++@pytest.mark.parametrize("failure", ["success", "model", "cancel"]) ++def test_request_state_closed_and_removed_on_every_exit(monkeypatch, failure): ++ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop ++ ++ monkeypatch.setenv("H3_VIDEO_ROW_CACHE", "1") ++ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) ++ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) ++ state = MiniMaxH3VideoRowCache(torch.tensor([1]), seq_len=4, used_len=3, num_layers=1) ++ ++ class Model: ++ def create_video_row_cache(self, *args, **kwargs): ++ return state ++ ++ def __call__(self, **kwargs): ++ assert kwargs["video_row_cache"] is state ++ if failure == "model": ++ raise RuntimeError("model failed") ++ state.prepare_local_span(0, 4) ++ x = torch.ones(4, 2, dtype=torch.bfloat16) ++ if state.partial: ++ state.restore(0, x) ++ else: ++ state.capture(0, x, x + 0.125) ++ return torch.zeros(2, 96), torch.zeros(1, 32) ++ ++ class Branch: ++ task = "ref2va" ++ seq_len = 4 ++ used_len = 3 ++ img_pos = torch.tensor([0, 1]) ++ update_mask = torch.tensor([False, True]) ++ update_mask_dev = update_mask ++ audio_update_mask_dev = torch.tensor([True]) ++ locked_audio_rows = None ++ static_kwargs = {"video_token_layout": SimpleNamespace(video_spans=[SimpleNamespace(role="target", start=1)])} ++ ++ def forward_kwargs(self, **kwargs): ++ return dict(self.static_kwargs) ++ ++ monkeypatch.setattr( ++ loop, ++ "minimax_h3_prepare_denoise_rows", ++ lambda **kwargs: (kwargs["initial_video_rows"], kwargs["initial_audio_rows"], torch.zeros(1, 96), None), ++ ) ++ ++ def cancellation(**kwargs): ++ if failure == "cancel": ++ raise RuntimeError("request cancelled") ++ ++ monkeypatch.setattr(loop, "check_request_cancellation", cancellation) ++ branch, model = Branch(), Model() ++ kwargs = dict( ++ model=model, ++ positive=branch, ++ initial_video_rows=torch.zeros(2, 96), ++ initial_audio_rows=torch.zeros(1, 32), ++ keyframe_cond_rows=torch.zeros(1, 96), ++ sigmas_video=[1 - i / 50 for i in range(51)], ++ sigmas_audio=[1 - i / 50 for i in range(51)], ++ device=torch.device("cpu"), ++ ) ++ if failure == "success": ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert state.full_steps == 17 and state.partial_steps == 33 ++ else: ++ with pytest.raises(RuntimeError): ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert state.closed and not state._deltas and "video_row_cache" not in branch.static_kwargs ++ assert model._video_row_cache_stats["closed"] ++ ++ ++def test_default_factory_does_not_create_state(monkeypatch): ++ from vllm_omni.diffusion.models.minimax_h3.minimax_h3_transformer import MiniMaxH3DiTModel ++ ++ monkeypatch.delenv("H3_VIDEO_ROW_CACHE", raising=False) ++ assert ( ++ MiniMaxH3DiTModel.create_video_row_cache(None, torch.tensor([1]), seq_len=4, used_len=3, num_steps=50) is None ++ ) ++ ++ ++@pytest.mark.parametrize("ref2va", [False, True]) ++def test_real_packing_task_selects_ref2va_and_rejects_fl2va(monkeypatch, ref2va): ++ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop ++ from vllm_omni.diffusion.models.minimax_h3.packed_sequence import ( ++ minimax_h3_packed_sequence, ++ minimax_h3_packed_sequence_ref2va_blocks, ++ ) ++ ++ common = dict(text_len=2, latent_t=2, latent_h=4, latent_w=4, audio_t=2) ++ packed = ( ++ minimax_h3_packed_sequence_ref2va_blocks(**common, ref_blocks=[{"kind": "image", "latent_h": 4, "latent_w": 4}]) ++ if ref2va ++ else minimax_h3_packed_sequence(**common, include_keyframe_cond=True, keyframe_frame_indices=[0], frame_count=5) ++ ) ++ branch = loop.MiniMaxH3DenoiseBranch( ++ packed=packed, ++ text_embeddings=torch.zeros(2, 5120), ++ token_tags=packed["token_tags"], ++ device=torch.device("cpu"), ++ task="ref2va" if ref2va else "fl2va", ++ ) ++ layout = branch.static_kwargs["video_token_layout"] ++ assert not any(span.role == "reference" for span in layout.video_spans) ++ monkeypatch.setenv("H3_VIDEO_ROW_CACHE", "1") ++ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) ++ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) ++ monkeypatch.setattr(loop, "check_request_cancellation", lambda **kwargs: None) ++ states = [] ++ ++ class Model: ++ def create_video_row_cache(self, positions, **kwargs): ++ state = MiniMaxH3VideoRowCache( ++ positions, seq_len=kwargs["seq_len"], used_len=kwargs["used_len"], num_layers=1 ++ ) ++ state.prepare_local_span(0, branch.seq_len) ++ states.append(state) ++ return state ++ ++ def __call__(self, **kwargs): ++ state = kwargs["video_row_cache"] ++ x = torch.ones(branch.seq_len, 2, dtype=torch.bfloat16) ++ if state.partial: ++ state.restore(0, x) ++ else: ++ state.capture(0, x, x + 0.125) ++ return torch.zeros(branch.img_pos.numel(), 96), torch.zeros(branch.audio_pos.numel(), 32) ++ ++ kwargs = dict( ++ model=Model(), ++ positive=branch, ++ initial_video_rows=torch.zeros(branch.img_pos.numel(), 96), ++ initial_audio_rows=torch.zeros(branch.audio_pos.numel(), 32), ++ keyframe_cond_rows=torch.zeros(int((~branch.update_mask).sum()), 96), ++ sigmas_video=[1 - i / 50 for i in range(51)], ++ sigmas_audio=[1 - i / 50 for i in range(51)], ++ device=torch.device("cpu"), ++ ) ++ if ref2va: ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert len(states) == 1 and states[0].closed ++ assert states[0].full_steps == 17 and states[0].partial_steps == 33 ++ else: ++ with pytest.raises(ValueError, match="requires Ref2VA"): ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert not states ++ ++ ++@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) ++def test_attention_keeps_collective_path_and_projects_only_dense_local_rows(start, length): ++ state = state_for_rows(start=start, length=length) ++ x = torch.ones(length, 2, dtype=torch.bfloat16) ++ anchor(state, x) ++ attention = object.__new__(blocks.MiniMaxH3Attention) ++ nn.Module.__init__(attention) ++ attention.num_heads = 1 ++ attention.head_dim = 2 ++ attention.vdn = None ++ attention.to_gate_compress = None ++ calls, projected_rows = [], [] ++ attention._project_qkv = lambda *args: (x.reshape(length, 1, 2),) * 5 ++ ++ def packed(q, k, v, **kwargs): ++ calls.append(q.shape[0]) ++ assert kwargs["video_row_cache"] is state ++ return q ++ ++ def project(rows): ++ projected_rows.append(rows.shape[0]) ++ return rows * 2, None ++ ++ attention._run_packed_attention = packed ++ attention.out_proj = project ++ output = attention( ++ x, ++ rope_table=None, ++ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), ++ max_seqlen=3, ++ packed_total=4, ++ video_row_cache=state, ++ ) ++ rows = state.local_dense_positions ++ assert calls == [length] ++ assert projected_rows == ([rows.numel()] if rows.numel() else []) ++ assert torch.equal(output.index_select(0, rows), (x * 2).index_select(0, rows)) ++ assert torch.count_nonzero(output.index_select(0, state.local_video_positions)) == 0 ++ ++ ++def _native_worker(rank): ++ import torch_npu # noqa: F401 ++ ++ torch.npu.set_device(rank) ++ dist.init_process_group( ++ "hccl", init_method="tcp://127.0.0.1:29597", rank=rank, world_size=4, timeout=timedelta(seconds=120) ++ ) ++ try: ++ large_delta_roundtrip(device=f"npu:{rank}") ++ torch.manual_seed(19) ++ full = [torch.randn(32, 4, 128, dtype=torch.bfloat16).npu() for _ in range(3)] ++ exchanged = [] ++ for tensor in full: ++ local = tensor[rank * 8 : (rank + 1) * 8] ++ send = local.reshape(8, 4, 1, 128).permute(1, 0, 2, 3).contiguous() ++ receive = torch.empty_like(send) ++ dist.all_to_all_single(receive, send) ++ exchanged.append(receive.reshape(1, 32, 1, 128)) ++ state = MiniMaxH3VideoRowCache(torch.arange(8, 24, device="npu"), seq_len=32, used_len=30, num_layers=1) ++ state.prepare_local_span(rank * 8, 8) ++ x = torch.ones(8, 2, dtype=torch.bfloat16, device="npu") ++ anchor(state, x) ++ output = _subset_prefix_bnsd(exchanged[0], exchanged[1][:, :30], exchanged[2][:, :30], 128**-0.5, state) ++ send = output.reshape(4, 8, 1, 128).contiguous() ++ receive = torch.empty_like(send) ++ dist.all_to_all_single(receive, send) ++ local_output = receive.permute(1, 0, 2, 3).reshape(8, 4, 128) ++ expected = _prefix_bnsd(full[0].unsqueeze(0), full[1][:30].unsqueeze(0), full[2][:30].unsqueeze(0), 128**-0.5)[ ++ 0 ++ ] ++ rows = state.local_dense_positions ++ torch.testing.assert_close( ++ local_output.index_select(0, rows).float(), ++ expected[rank * 8 : (rank + 1) * 8].index_select(0, rows).float(), ++ atol=0.03125, ++ rtol=0.02, ++ ) ++ assert state.subset_attention_calls == 1 and state.subset_query_rows == 14 ++ assert rows.numel() == [8, 0, 0, 6][rank] ++ state.restore(0, x) ++ state.end_step() ++ state.close() ++ torch.npu.synchronize() ++ dist.barrier() ++ finally: ++ dist.destroy_process_group() ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires four native Ascend devices") ++def test_native_four_rank_subset_attention_with_empty_local_dense_rows(): ++ torch.multiprocessing.spawn(_native_worker, nprocs=4, join=True) +diff --git a/vllm_omni/config/omni_config.py b/vllm_omni/config/omni_config.py +index 0a803cf..44fc48c 100644 +--- a/vllm_omni/config/omni_config.py ++++ b/vllm_omni/config/omni_config.py +@@ -863,6 +863,7 @@ class _DiffusionConfigProjection(_TrackExplicitConfigFields): + dlo_use_allgather: bool = True + dlo_chunk_size_mb: int = Field(default=64, gt=0, strict=True) + dlo_resident_layers: int = Field(default=0, ge=0) ++ dlo_order_allgather_after_attn_a2a: bool = False + host_weight_runtime_mode: Literal["disabled", "preferred", "required"] = "disabled" + host_weight_runtime_root: str | None = None + dlo_host_registration_limit_gib: float = Field(default=0.0, ge=0) +diff --git a/vllm_omni/config/stage_config.py b/vllm_omni/config/stage_config.py +index 76a58a9..3c69986 100644 +--- a/vllm_omni/config/stage_config.py ++++ b/vllm_omni/config/stage_config.py +@@ -672,6 +672,7 @@ class StageDeployConfig: + dlo_use_allgather: bool | None = None + dlo_chunk_size_mb: int | None = None + dlo_resident_layers: int | None = None ++ dlo_order_allgather_after_attn_a2a: bool | None = None + host_weight_runtime_mode: str | None = None + host_weight_runtime_root: str | None = None + dlo_host_registration_limit_gib: float | None = None +diff --git a/vllm_omni/diffusion/attention/layer.py b/vllm_omni/diffusion/attention/layer.py +index 5520390..574eafa 100644 +--- a/vllm_omni/diffusion/attention/layer.py ++++ b/vllm_omni/diffusion/attention/layer.py +@@ -9,7 +9,7 @@ + + from collections.abc import Mapping + from dataclasses import replace +-from typing import cast ++from typing import Any, cast + + import torch + import torch.nn as nn +@@ -333,6 +333,11 @@ class Attention(nn.Module): + # Local strategy when SP is intentionally inactive outside sharded regions. + self._no_parallel_strategy = NoParallelAttention() + ++ # Optional cross-module rendezvous (see distributed/collective_gate.py). ++ # Installed by the DLO backend when it must keep its background weight ++ # AllGather from taking the links ahead of this layer's QKV AllToAll. ++ self._collective_gate: Any | None = None ++ + self.layer_idx: int | None = _try_extract_layer_index(prefix) + + self._kv_cache_dtype: str | None = None +@@ -506,10 +511,12 @@ class Attention(nn.Module): + reason = self._cross_backend_limitation(attn_metadata) + if reason is not None: + blocked = self._cross_fallback_blocker(attn_metadata) +- result = self.sdpa_fallback.resolve_execution_path(resolved_context, query, key, value, attn_metadata) ++ result = cast(SDPAImpl, self.sdpa_fallback).resolve_execution_path( ++ resolved_context, query, key, value, attn_metadata ++ ) + return replace( + result, +- backend=self.attn_backend.get_name() if blocked else "SDPA", ++ backend=cast(type[AttentionBackend], self.attn_backend).get_name() if blocked else "SDPA", + path="cross_unsupported" if blocked else "cross_sdpa_fallback", + support=CapabilityResult.unsupported(blocked) if blocked else CapabilityResult.unmigrated(reason), + ) +@@ -591,6 +598,12 @@ class Attention(nn.Module): + # For Ring: Concat joint_q + query, key, value, attn_metadata, ctx = strategy.pre_attention(query, key, value, attn_metadata) + ++ # The QKV AllToAll is now enqueued on this stream. Release any gather ++ # that was deferred to give this collective the links first. ++ gate = self._collective_gate ++ if gate is not None: ++ gate.signal() ++ + # MiniMax-H3 Tier2 stores reference K/V in the balanced layout produced + # by Ulysses. Keeping this hook at the generic post-parallel boundary + # avoids a model import in the parallel strategy and leaves other models +@@ -760,7 +773,7 @@ class Attention(nn.Module): + def _cross_fallback_blocker(self, attn_metadata: AttentionMetadata | None) -> str | None: + if self.backend_explicit: + return ( +- f"Explicit backend {self.attn_backend.get_name()} does not support " ++ f"Explicit backend {cast(type[AttentionBackend], self.attn_backend).get_name()} does not support " + f"{self._cross_backend_limitation(attn_metadata)}" + ) + if attn_metadata is not None and attn_metadata.extra.get("kv_cache_dtype") not in (None, "float"): +@@ -793,7 +806,7 @@ class Attention(nn.Module): + blocked = self._cross_fallback_blocker(attn_metadata) + if blocked is not None: + raise NotImplementedError(blocked) +- return self.sdpa_fallback.forward(query, key, value, attn_metadata) ++ return cast(SDPAImpl, self.sdpa_fallback).forward(query, key, value, attn_metadata) + dense_mask = attn_metadata.attn_mask if attn_metadata is not None else None + # Piecewise backends honor full_attn_spans ahead of attn_mask and never read the + # dense mask (e.g. HunyuanImage3, which sends both), so there is nothing to divert. +diff --git a/vllm_omni/diffusion/data.py b/vllm_omni/diffusion/data.py +index 8528574..fcbeec0 100644 +--- a/vllm_omni/diffusion/data.py ++++ b/vllm_omni/diffusion/data.py +@@ -962,6 +962,10 @@ class OmniDiffusionConfig: + dlo_chunk_size_mb: int = 64 + # Leading main-DiT blocks kept resident by distributed layerwise offload. + dlo_resident_layers: int = 0 ++ # Hold each DiT block's weight AllGather until that block's QKV AllToAll has ++ # been enqueued, so the background collective does not take the links ahead ++ # of the critical-path all-to-all. ++ dlo_order_allgather_after_attn_a2a: bool = False + # Final-layout Host Weight Runtime policy. The loader only activates this + # for eligible no-AllGather DLO; all other configurations preserve their + # existing loader/storage path. +diff --git a/vllm_omni/diffusion/distributed/collective_gate.py b/vllm_omni/diffusion/distributed/collective_gate.py +new file mode 100644 +index 0000000..1386d89 +--- /dev/null ++++ b/vllm_omni/diffusion/distributed/collective_gate.py +@@ -0,0 +1,72 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++"""Cross-module rendezvous that orders background collectives after the ++foreground all-to-all of a transformer block. ++ ++Distributed layerwise offload (DLO) issues the *next* block's weight AllGather ++on a dedicated ``comm_stream`` while the *current* block computes. Both that ++AllGather and the block's QKV sequence-parallel AllToAll are collectives over ++the same ranks, so whichever reaches the links first wins: the AllGather is ++submitted at block entry, well before the block reaches its attention, and the ++critical-path AllToAll ends up queued behind it. ++ ++A plain event wait cannot fix this by itself. ``wait_event`` snapshots the ++records enqueued *before* the call, so "enqueue the wait at block entry, record ++the event later in the block" is a no-op -- the wait resolves against the ++previous iteration's record. The gather therefore has to be *issued* after the ++AllToAll: the hook arms this barrier at block entry and defers the gather, the ++attention signals it right after ``strategy.pre_attention``, and the signal ++records the event and lets the hook submit the gather gated on it. ++ ++The gather still overlaps the rest of the block (attention output, MLP, the ++next block's projections); it simply cannot start before the AllToAll finished. ++""" ++ ++from __future__ import annotations ++ ++from collections.abc import Callable ++ ++__all__ = ["QKVAllToAllBarrier"] ++ ++ ++class QKVAllToAllBarrier: ++ """One-block rendezvous between a DLO hook and the block's attention. ++ ++ Owned by a single :class:`DistributedLayerwiseOffloadHook` (one per ++ streaming block) and shared with the attention layer inside that block. ++ Both live on the same block, so the arming and the signalling always come ++ from the same module forward. ++ ++ Args: ++ on_signal: Invoked at most once per :meth:`arm` when the block's ++ attention reports its QKV AllToAll. It must submit the deferred ++ weight AllGather. ++ """ ++ ++ def __init__(self, on_signal: Callable[[], None]) -> None: ++ self._on_signal = on_signal ++ self._armed = False ++ ++ @property ++ def armed(self) -> bool: ++ """True between :meth:`arm` and the matching :meth:`signal`.""" ++ return self._armed ++ ++ def arm(self) -> None: ++ """Mark the next attention QKV AllToAll as the gather's gate.""" ++ self._armed = True ++ ++ def disarm(self) -> None: ++ """Drop a pending arm without firing it (the caller flushes instead).""" ++ self._armed = False ++ ++ def signal(self) -> None: ++ """Report that the QKV AllToAll is now enqueued on the compute stream. ++ ++ A no-op unless armed, so it is cheap to call unconditionally from the ++ attention hot path. ++ """ ++ if not self._armed: ++ return ++ self._armed = False ++ self._on_signal() +diff --git a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py +index 6ebfb10..044c1f0 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py ++++ b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py +@@ -36,6 +36,16 @@ def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, sc + return output.reshape(batch, heads, rows, dim).transpose(1, 2).contiguous() + + ++def _subset_prefix_bnsd(query, key, value, scale, state): ++ """Compute only fresh conditioning queries; preserve the shared SP shape.""" ++ rows = state.global_dense_positions ++ dense = _prefix_bnsd(query.index_select(1, rows), key, value, scale) ++ output = torch.zeros_like(query) ++ output.index_copy_(1, rows, dense) ++ state.record_subset_attention(rows.numel()) ++ return output ++ ++ + class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + """Use independent head batches for explicit long BF16 prefix attention. + +@@ -51,7 +61,10 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + value: torch.Tensor, + attn_metadata: AttentionMetadata | None = None, + ) -> torch.Tensor: ++ state = attn_metadata.extra.get("minimax_h3_video_row_cache") if attn_metadata is not None else None + if os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" or attn_metadata is None: ++ if state is not None: ++ raise RuntimeError("H3 video-row cache requires the explicit fused attention path") + return super().forward_fa_npu(query, key, value, attn_metadata) + extra = attn_metadata.extra + if ( +@@ -68,9 +81,17 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + or query.shape[2:] != value.shape[2:] + or key.shape[1] != value.shape[1] + ): ++ if state is not None: ++ raise RuntimeError("H3 video-row cache received unsupported fused attention inputs") + return super().forward_fa_npu(query, key, value, attn_metadata) + resolved = self._resolve_packed_seq_npu(query, key, extra) + if resolved is None: ++ if state is not None: ++ raise RuntimeError("H3 video-row cache requires a resolved packed prefix") + return super().forward_fa_npu(query, key, value, attn_metadata) + used_k = resolved[1][0] ++ if state is not None: ++ if not state.partial or used_k != state.used_len or query.shape[1] != state.seq_len: ++ raise RuntimeError("H3 video-row cache attention layout changed") ++ return _subset_prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale, state) + return _prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale) +diff --git a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py +index 8cca219..07fb372 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py ++++ b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py +@@ -9,6 +9,7 @@ rows stay pinned to their noised step-0 anchors. + + from __future__ import annotations + ++import os + from collections.abc import Callable + from contextlib import AbstractContextManager, nullcontext + from typing import Any +@@ -69,7 +70,9 @@ class MiniMaxH3DenoiseBranch: + text_embeddings: torch.Tensor, + token_tags: torch.Tensor, + device: torch.device, ++ task: str | None = None, + ) -> None: ++ self.task = task + seq_len = int(packed["seq_len"]) + self.seq_len = seq_len + self.img_pos = packed["img_pos"].view(-1).to(torch.long) +@@ -380,6 +383,24 @@ def minimax_h3_denoise_loop( + positive.static_kwargs["reference_kv_state"] = reference_kv_state + + try: ++ video_row_cache = None ++ create_video_cache = getattr(model, "create_video_row_cache", None) ++ if os.environ.get("H3_VIDEO_ROW_CACHE") == "1": ++ if ( ++ not callable(create_video_cache) ++ or video_edit is not None ++ or audio_edit is not None ++ or keyframe_cond_rows is None ++ or positive.task != "ref2va" ++ ): ++ raise ValueError("H3 video-row cache requires Ref2VA image references without editing") ++ video_row_cache = create_video_cache( ++ positive.img_pos[positive.update_mask].to(device), ++ seq_len=positive.seq_len, ++ used_len=positive.used_len, ++ num_steps=num_steps, ++ ) ++ positive.static_kwargs["video_row_cache"] = video_row_cache + for step in range(num_steps): + check_request_cancellation() + step_cm = step_profiler(step) if step_profiler is not None else nullcontext() +@@ -388,6 +409,8 @@ def minimax_h3_denoise_loop( + s_a = sigmas_audio[step] + if reference_kv_state is not None: + reference_kv_state.begin_step(step, sigma=float(s_v)) ++ if video_row_cache is not None: ++ video_row_cache.begin_step(step) + # Publish where we are so step-gated attention features (the dense + # warmup of RAINFUSION_ATTN, the timestep gate of TRTLLM_ATTN) can + # see it. Gates use the scheduler-style descending timestep, which +@@ -430,6 +453,8 @@ def minimax_h3_denoise_loop( + v_video, v_audio = model(**fk) + if reference_kv_state is not None: + reference_kv_state.end_step() ++ if video_row_cache is not None: ++ video_row_cache.end_step() + mv_video_t = v_video.float()[update] + mv_audio_t = v_audio.float()[audio_update] + +@@ -476,6 +501,10 @@ def minimax_h3_denoise_loop( + + finally: + minimax_h3_publish_denoise_progress(None, None, None) ++ if video_row_cache is not None: ++ positive.static_kwargs.pop("video_row_cache", None) ++ video_row_cache.close() ++ model._video_row_cache_stats = video_row_cache.stats() + if reference_kv_state is not None: + positive.static_kwargs.pop("reference_kv_state", None) + reference_kv_state.close() +diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py +index 09f2e7d..601e2ae 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py ++++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py +@@ -45,6 +45,7 @@ if TYPE_CHECKING: + from vllm.model_executor.layers.quantization.base_config import QuantizationConfig + + from .reference_kv_cache import MiniMaxH3ReferenceKVTier2State ++ from .video_row_cache import MiniMaxH3VideoRowCache + + _BF16_DTYPE = torch.bfloat16 + _FP32_DTYPE = torch.float32 +@@ -291,6 +292,7 @@ class MiniMaxH3Attention(nn.Module): + reference_kv_layer_index: int | None = None, + reference_kv_compact: bool = False, + vdn_window: VDNLayout | None = None, ++ video_row_cache: MiniMaxH3VideoRowCache | None = None, + ) -> torch.Tensor: + """Run packed attention as a small eager island. + +@@ -368,6 +370,8 @@ class MiniMaxH3Attention(nn.Module): + "vsa_h3_ref_policy": self.vsa_ref_policy, + "vsa_h3_ref_keep_rate": self.vsa_ref_keep_rate, + } ++ if video_row_cache is not None and video_row_cache.partial: ++ extra["minimax_h3_video_row_cache"] = video_row_cache + if gate_compress is not None: + extra["gate_compress"] = gate_compress.unsqueeze(0) + if video_layout is not None and video_layout.video_spans: +@@ -455,6 +459,7 @@ class MiniMaxH3Attention(nn.Module): + reference_kv_layer_index: int | None = None, + reference_kv_compact: bool = False, + vdn_window: VDNLayout | None = None, ++ video_row_cache: MiniMaxH3VideoRowCache | None = None, + ) -> torch.Tensor: + """x: [T, hidden] packed thd rows -> [T, hidden]. + +@@ -508,10 +513,19 @@ class MiniMaxH3Attention(nn.Module): + reference_kv_layer_index=reference_kv_layer_index, + reference_kv_compact=reference_kv_compact, + vdn_window=vdn_window, ++ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), + ) + if self.vdn is not None: + out = self.vdn.gate_softmax(out, x) + out = out.reshape(total, self.num_heads * self.head_dim) ++ if video_row_cache is not None and video_row_cache.partial: ++ rows = video_row_cache.local_dense_positions ++ assert rows is not None ++ projected = x.new_zeros((total, x.shape[-1])) ++ if rows.numel(): ++ dense, _ = self.out_proj(out.index_select(0, rows)) ++ projected.index_copy_(0, rows, dense) ++ return projected + out, _ = self.out_proj(out) + if self.vdn is not None: + out = self.vdn.add_linear(out, x, q_raw, k_raw, v, vdn_window) +@@ -742,6 +756,7 @@ class MiniMaxH3DiTBlock(nn.Module): + reference_kv_state: MiniMaxH3ReferenceKVTier2State | None = None, + reference_kv_compact: bool = False, + vdn_window: VDNLayout | None = None, ++ video_row_cache: MiniMaxH3VideoRowCache | None = None, + ) -> torch.Tensor: + """x: [T, H]; t_emb: [M, t_dim]; combined_indices: [T] + (= inverse_indices * modality_num + token_tags.clamp(min=0)). +@@ -759,6 +774,7 @@ class MiniMaxH3DiTBlock(nn.Module): + gate_mlp, + ) = self.adaln_proj(t_emb) + ++ block_input = x + residual = x + h = rms_norm_indexed_scale_shift( + x, +@@ -782,7 +798,10 @@ class MiniMaxH3DiTBlock(nn.Module): + reference_kv_layer_index=self.layer_index, + reference_kv_compact=reference_kv_compact, + vdn_window=vdn_window, ++ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), + ) ++ if video_row_cache is not None and video_row_cache.pre_gate: ++ h = video_row_cache.pre_gate_output(self.layer_index, h, "attn") + x, h = indexed_gate_rms_norm_scale_shift( + residual, + gate_msa, +@@ -793,6 +812,36 @@ class MiniMaxH3DiTBlock(nn.Module): + combined_indices, + self.norm2.variance_epsilon, + ) ++ if video_row_cache is not None and video_row_cache.pre_gate and video_row_cache.partial: ++ video_row_cache.record_fresh_gate("attn") + residual = x ++ if video_row_cache is not None and video_row_cache.partial: ++ rows = video_row_cache.local_dense_positions ++ assert rows is not None ++ if video_row_cache.pre_gate: ++ mlp_output = torch.zeros_like(h) ++ if rows.numel(): ++ mlp_output.index_copy_(0, rows, self.mlp(h.index_select(0, rows))) ++ mlp_output = video_row_cache.pre_gate_output(self.layer_index, mlp_output, "mlp") ++ output = indexed_gate(residual, gate_mlp, mlp_output, combined_indices) ++ video_row_cache.record_fresh_gate("mlp") ++ video_row_cache.partial_mlp_rows += rows.numel() ++ return output ++ output = video_row_cache.restore(self.layer_index, block_input) ++ if rows.numel(): ++ dense = indexed_gate( ++ residual.index_select(0, rows), ++ gate_mlp, ++ self.mlp(h.index_select(0, rows)), ++ combined_indices.index_select(0, rows), ++ ) ++ output.index_copy_(0, rows, dense) ++ video_row_cache.partial_mlp_rows += rows.numel() ++ return output + h = self.mlp(h) +- return indexed_gate(residual, gate_mlp, h, combined_indices) ++ if video_row_cache is not None and video_row_cache.pre_gate: ++ h = video_row_cache.pre_gate_output(self.layer_index, h, "mlp") ++ output = indexed_gate(residual, gate_mlp, h, combined_indices) ++ if video_row_cache is not None and not video_row_cache.pre_gate: ++ video_row_cache.capture(self.layer_index, block_input, output) ++ return output +diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py +index 9518828..b9e07dc 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py ++++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py +@@ -10,6 +10,7 @@ layout. + from __future__ import annotations + + import math ++import os + from bisect import bisect_left + from collections.abc import Iterable, Mapping + from dataclasses import replace +@@ -66,6 +67,8 @@ if TYPE_CHECKING: + from vllm_omni.diffusion.data import OmniDiffusionConfig + from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache import MiniMaxH3ReferenceKVTier2State + ++ from .video_row_cache import MiniMaxH3VideoRowCache ++ + logger = init_logger(__name__) + + +@@ -245,6 +248,7 @@ _FORWARD_SUPPORTED_KWARGS = frozenset( + "control_rows", + "control_context_scale", + "reference_kv_state", ++ "video_row_cache", + } + ) + +@@ -1232,6 +1236,41 @@ class MiniMaxH3DiTModel(nn.Module): + t_emb = self.time_embedder(unique_timesteps) + return embeddings, t_emb + ++ def create_video_row_cache( ++ self, target_positions: torch.Tensor, *, seq_len: int, used_len: int, num_steps: int ++ ) -> MiniMaxH3VideoRowCache | None: ++ if os.environ.get("H3_VIDEO_ROW_CACHE") != "1": ++ return None ++ p = self.parallel_config ++ if ( ++ not current_omni_platform.is_npu() ++ or num_steps != 50 ++ or p.tensor_parallel_size != 1 ++ or p.data_parallel_size not in (None, 1) ++ or p.cfg_parallel_size != 1 ++ or p.pipeline_parallel_size != 1 ++ or p.ulysses_degree != 4 ++ or p.ring_degree != 1 ++ or p.allgather_degree != 1 ++ or p.ulysses_mode != "strict" ++ or self.od_config.cache_backend not in (None, "none") ++ or self.od_config.kv_offload_config is not None ++ or self.vdn_config is not None ++ or os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" ++ or any(block.attn.attention.attn_backend.get_name() != "FLASH_ATTN" for block in self.blocks) ++ ): ++ raise ValueError("H3 video-row cache requires dense NPU BF16 Ref2VA with strict Ulysses4, TP/DP/CFG/PP1") ++ from .video_row_cache import MiniMaxH3VideoRowCache ++ ++ self._video_row_cache_stats = None ++ return MiniMaxH3VideoRowCache( ++ target_positions, ++ seq_len=seq_len, ++ used_len=used_len, ++ num_layers=len(self.blocks), ++ pre_gate=os.environ.get("H3_VIDEO_PRE_GATE_CACHE") == "1", ++ ) ++ + def create_reference_kv_state( + self, + *, +@@ -1308,6 +1347,14 @@ class MiniMaxH3DiTModel(nn.Module): + refiner_max = int(self._psp_field(refiner_psp, "refiner_packed_seq_params", "max_seqlen_q")) + video_layout = kwargs.get("video_token_layout") + reference_kv_state = kwargs.get("reference_kv_state") ++ video_row_cache = kwargs.get("video_row_cache") ++ if video_row_cache is not None and ( ++ num_requests != 1 ++ or reference_kv_state is not None ++ or self.vdn_config is not None ++ or kwargs.get("control_rows") is not None ++ ): ++ raise ValueError("H3 video-row cache cannot combine with packed requests, reference KV, VDN or control") + vdn_window = None + if self.vdn_config is not None: + # Text rows lead the packed document. +@@ -1442,6 +1489,11 @@ class MiniMaxH3DiTModel(nn.Module): + if reference_kv_state is not None: + reference_kv_state.set_local_sequence_rows(hidden.shape[0]) + ++ if video_row_cache is not None: ++ if hidden.shape[0] != local_len: ++ raise RuntimeError("H3 video-row cache requires the strict local sequence shard") ++ video_row_cache.prepare_local_span(local_start, local_len) ++ + block_video_layout = video_layout + if compact_reference: + block_video_layout = None +@@ -1505,6 +1557,7 @@ class MiniMaxH3DiTModel(nn.Module): + reference_kv_state=reference_kv_state, + reference_kv_compact=compact_reference, + vdn_window=vdn_window, ++ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), + ) + if block_index in hints: + hidden = hidden + hints.pop(block_index) * control_scale +diff --git a/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py b/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py +index 476c3cc..ebce8fc 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py ++++ b/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py +@@ -2115,6 +2115,7 @@ class MiniMaxH3Pipeline( + text_embeddings=text_embeddings, + token_tags=tags, + device=self.device, ++ task=task, + ) + + if control_rows is not None: +diff --git a/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py b/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py +new file mode 100644 +index 0000000..68f9501 +--- /dev/null ++++ b/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py +@@ -0,0 +1,236 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++"""Request-owned, experimental target-video block residuals for H3 Ref2VA.""" ++ ++from __future__ import annotations ++ ++import torch ++ ++ ++class MiniMaxH3VideoRowCache: ++ """Keep non-video computation dense while reusing video block updates. ++ ++ The first experimental profile is deliberately fixed. No state is shared ++ across requests, and every rank retains all attention collectives. ++ """ ++ ++ def __init__( ++ self, target_positions: torch.Tensor, *, seq_len: int, used_len: int, num_layers: int, pre_gate: bool = False ++ ) -> None: ++ positions = target_positions.detach().cpu().tolist() ++ if not 0 < used_len <= seq_len or not positions or num_layers < 1: ++ raise ValueError("video-row cache requires a nonempty valid packed document") ++ if positions != sorted(set(positions)) or positions[0] < 0 or positions[-1] >= used_len: ++ raise ValueError("target-video positions must be sorted unique valid rows") ++ target_set = set(positions) ++ dense = [i for i in range(used_len) if i not in target_set] ++ if not dense: ++ raise ValueError("video-row cache requires non-video conditioning rows") ++ self.seq_len = seq_len ++ self.used_len = used_len ++ self.num_layers = num_layers ++ self._target_positions = positions ++ self.global_dense_positions = torch.tensor(dense, dtype=torch.long, device=target_positions.device) ++ self.local_dense_positions: torch.Tensor | None = None ++ self.local_video_positions: torch.Tensor | None = None ++ self._local_span: tuple[int, int] | None = None ++ self._deltas: dict[int, torch.Tensor] = {} ++ self.pre_gate = pre_gate ++ self._attention_outputs: dict[int, torch.Tensor] = {} ++ self._mlp_outputs: dict[int, torch.Tensor] = {} ++ self._awaiting_mlp_layer: int | None = None ++ self.fresh_attention_gates = 0 ++ self.fresh_mlp_gates = 0 ++ self._step = -1 ++ self._blocks_this_step = 0 ++ self.partial = False ++ self.closed = False ++ self.full_steps = 0 ++ self.partial_steps = 0 ++ self.full_blocks = 0 ++ self.partial_blocks = 0 ++ self.subset_attention_calls = 0 ++ self.subset_query_rows = 0 ++ self.omitted_query_rows = 0 ++ self.partial_mlp_rows = 0 ++ self.peak_cache_bytes = 0 ++ ++ @staticmethod ++ def is_full_step(step: int) -> bool: ++ if not 0 <= step < 50: ++ raise ValueError("the video-row cache profile requires 50 steps") ++ return step < 4 or step >= 46 or (step - 4) % 5 == 0 ++ ++ def begin_step(self, step: int) -> None: ++ if self.closed or step != self._step + 1: ++ raise RuntimeError("video-row cache step order changed") ++ if self._step >= 0 and self._blocks_this_step != self.num_layers: ++ raise RuntimeError("previous video-row cache step is incomplete") ++ full = self.is_full_step(step) ++ if self.pre_gate: ++ full = step in (0, 1, 4, 9, 19, 29, 39, 49) ++ self.partial = not full ++ if self._awaiting_mlp_layer is not None: ++ raise RuntimeError("previous video pre-gate sublayer is incomplete") ++ complete = ( ++ len(self._attention_outputs) == len(self._mlp_outputs) == self.num_layers ++ if self.pre_gate ++ else len(self._deltas) == self.num_layers ++ ) ++ if self.partial and not complete: ++ raise RuntimeError("partial video step has no complete dense anchor") ++ self._step = step ++ self._blocks_this_step = 0 ++ if self.partial: ++ self.partial_steps += 1 ++ else: ++ self.full_steps += 1 ++ ++ def prepare_local_span(self, start: int, length: int) -> None: ++ if self.closed or start < 0 or length < 1 or start + length > self.seq_len: ++ raise ValueError("invalid local packed span") ++ if self._local_span is not None: ++ if self._local_span != (start, length): ++ raise RuntimeError("video-row cache local ownership changed") ++ return ++ device = self.global_dense_positions.device ++ target = set(self._target_positions) ++ self.local_video_positions = torch.tensor( ++ [i - start for i in self._target_positions if start <= i < start + length], dtype=torch.long, device=device ++ ) ++ self.local_dense_positions = torch.tensor( ++ [i - start for i in range(start, min(start + length, self.used_len)) if i not in target], ++ dtype=torch.long, ++ device=device, ++ ) ++ self._local_span = (start, length) ++ ++ def _visit(self, layer: int, x: torch.Tensor) -> None: ++ if self.closed or self._step < 0 or layer != self._blocks_this_step: ++ raise RuntimeError("video-row cache block order changed") ++ if self._local_span is None or x.shape[0] != self._local_span[1] or x.dtype != torch.bfloat16: ++ raise ValueError("video-row cache requires the prepared BF16 row shard") ++ self._blocks_this_step += 1 ++ ++ def capture(self, layer: int, x: torch.Tensor, output: torch.Tensor) -> None: ++ if self.pre_gate: ++ raise RuntimeError("pre-gate state cannot capture a gated block residual") ++ if self.partial: ++ raise RuntimeError("cannot replace the dense anchor during a partial step") ++ self._visit(layer, x) ++ rows = self.local_video_positions ++ assert rows is not None ++ # BF16 endpoints may have a finite difference outside FP16's range. ++ # Keep that difference in FP32; only the reconstructed block output is ++ # rounded to the original BF16 activation dtype. ++ self._deltas[layer] = output.index_select(0, rows).float() - x.index_select(0, rows).float() ++ self.peak_cache_bytes = max( ++ self.peak_cache_bytes, sum(t.numel() * t.element_size() for t in self._deltas.values()) ++ ) ++ self.full_blocks += 1 ++ ++ def restore(self, layer: int, x: torch.Tensor) -> torch.Tensor: ++ if self.pre_gate: ++ raise RuntimeError("pre-gate state cannot restore a gated block residual") ++ if not self.partial: ++ raise RuntimeError("cannot reuse a video residual on a full step") ++ self._visit(layer, x) ++ rows = self.local_video_positions ++ assert rows is not None ++ # This copy owns the returned full shard, preserving caller input and ++ # unused padding while fresh non-video rows are filled by the block. ++ output = x.clone() ++ if rows.numel(): ++ output.index_copy_(0, rows, (x.index_select(0, rows).float() + self._deltas[layer].float()).to(x.dtype)) ++ self.partial_blocks += 1 ++ return output ++ ++ def _cache_bytes(self) -> int: ++ return sum( ++ tensor.numel() * tensor.element_size() ++ for cache in (self._deltas, self._attention_outputs, self._mlp_outputs) ++ for tensor in cache.values() ++ ) ++ ++ def pre_gate_output(self, layer: int, output: torch.Tensor, stage: str) -> torch.Tensor: ++ """Capture or fill only target rows before applying the current gate.""" ++ if not self.pre_gate or stage not in ("attn", "mlp"): ++ raise ValueError("pre-gate output requires the selected attention or MLP stage") ++ if stage == "attn": ++ if self._awaiting_mlp_layer is not None: ++ raise RuntimeError("previous video pre-gate sublayer is incomplete") ++ self._visit(layer, output) ++ self._awaiting_mlp_layer = layer ++ if self.partial: ++ self.partial_blocks += 1 ++ else: ++ self.full_blocks += 1 ++ else: ++ if self.closed or self._awaiting_mlp_layer != layer: ++ raise RuntimeError("video pre-gate MLP order changed") ++ if output.dtype != torch.bfloat16 or output.shape[0] != self._local_span[1]: ++ raise ValueError("video pre-gate output must retain the BF16 local shard") ++ self._awaiting_mlp_layer = None ++ rows = self.local_video_positions ++ assert rows is not None ++ cache = self._attention_outputs if stage == "attn" else self._mlp_outputs ++ if self.partial: ++ restored = output.clone() ++ restored.index_copy_(0, rows, cache[layer]) ++ return restored ++ cache[layer] = output.index_select(0, rows) ++ self.peak_cache_bytes = max(self.peak_cache_bytes, self._cache_bytes()) ++ return output ++ ++ def record_fresh_gate(self, stage: str) -> None: ++ if not self.pre_gate or not self.partial or stage not in ("attn", "mlp"): ++ raise RuntimeError("unexpected fresh gate record") ++ if stage == "attn": ++ self.fresh_attention_gates += 1 ++ else: ++ self.fresh_mlp_gates += 1 ++ ++ def record_subset_attention(self, rows: int) -> None: ++ if not self.partial or rows != self.global_dense_positions.numel(): ++ raise RuntimeError("unexpected subset attention query count") ++ self.subset_attention_calls += 1 ++ self.subset_query_rows += rows ++ self.omitted_query_rows += len(self._target_positions) ++ ++ def end_step(self) -> None: ++ if self._blocks_this_step != self.num_layers or self._awaiting_mlp_layer is not None: ++ raise RuntimeError("video-row cache step did not visit every main block") ++ ++ def stats(self) -> dict[str, int | bool | str]: ++ counters: dict[str, int | bool | str] = { ++ name: getattr(self, name) ++ for name in [ ++ "full_steps", ++ "partial_steps", ++ "full_blocks", ++ "partial_blocks", ++ "subset_attention_calls", ++ "subset_query_rows", ++ "omitted_query_rows", ++ "partial_mlp_rows", ++ "peak_cache_bytes", ++ "closed", ++ ] ++ } ++ counters["storage_dtype"] = "torch.bfloat16" if self.pre_gate else "torch.float32" ++ counters["cache_stage"] = "pre_gate" if self.pre_gate else "block_residual" ++ counters["fresh_attention_gates"] = self.fresh_attention_gates ++ counters["fresh_mlp_gates"] = self.fresh_mlp_gates ++ counters["live_cache_bytes"] = self._cache_bytes() ++ return counters ++ ++ def close(self) -> None: ++ self._deltas.clear() ++ self._attention_outputs.clear() ++ self._mlp_outputs.clear() ++ self._awaiting_mlp_layer = None ++ self._target_positions.clear() ++ self.global_dense_positions = torch.empty(0, dtype=torch.long) ++ self.local_dense_positions = None ++ self.local_video_positions = None ++ self.closed = True +diff --git a/vllm_omni/diffusion/offloader/base.py b/vllm_omni/diffusion/offloader/base.py +index 2505d48..50b73e5 100644 +--- a/vllm_omni/diffusion/offloader/base.py ++++ b/vllm_omni/diffusion/offloader/base.py +@@ -71,6 +71,10 @@ class OffloadConfig: + dlo_use_allgather: bool = True + dlo_resident_layers: int = 0 # leading DiT layers kept on device + chunk_size_bytes: int = 64 * 1024 * 1024 ++ # Hold each DiT block's weight AllGather until that block's QKV AllToAll has ++ # been enqueued, so the background collective does not take the links ahead ++ # of the critical-path all-to-all. See ``collective_gate.py``. ++ dlo_order_allgather_after_attn_a2a: bool = False + # Optional per-worker ceiling for registering an HWR mmap. Zero means no + # additional ceiling; pin_cpu_memory controls whether registration is tried. + dlo_host_registration_limit_gib: float = 0.0 +@@ -216,6 +220,7 @@ class OffloadConfig: + dlo_use_allgather=dit_uses_allgather, + dlo_resident_layers=dlo_resident_layers, + chunk_size_bytes=chunk_size_bytes, ++ dlo_order_allgather_after_attn_a2a=bool(getattr(od_config, "dlo_order_allgather_after_attn_a2a", False)), + dlo_host_registration_limit_gib=dlo_host_registration_limit_gib, + components=components, + dlo_transfers=dlo_transfers, +diff --git a/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py b/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py +index 7f0f081..8c4f87f 100644 +--- a/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py ++++ b/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py +@@ -31,6 +31,7 @@ import torch.distributed + from torch import nn + from vllm.logger import init_logger + ++from vllm_omni.diffusion.distributed.collective_gate import QKVAllToAllBarrier + from vllm_omni.diffusion.hooks import HookRegistry, ModelHook + from vllm_omni.diffusion.model_loader.host_weight_plan import ( + HostWeightPlan, +@@ -142,6 +143,7 @@ class DistributedLayerwiseOffloadHook(ModelHook): + tensor_transforms: dict[int, Any] | None = None, + materialization_probe_tensor: torch.Tensor | None = None, + chunk_size_bytes: int = 64 * 1024 * 1024, ++ order_allgather_after_attn_a2a: bool = False, + ): + assert isinstance(next_block, nn.Module), "transformer block must be type `torch.nn.Module`" + if type(dp_size) is not int or dp_size < 1: +@@ -167,6 +169,7 @@ class DistributedLayerwiseOffloadHook(ModelHook): + + self.chunk_size_bytes = chunk_size_bytes + self.manifest: PartManifest | None = None ++ self._deferred_chunked_allgather = None + self._h2d_done_events = [current_omni_platform.Event(), current_omni_platform.Event()] + self._ag_done_events = [current_omni_platform.Event(), current_omni_platform.Event()] + # Hook-local events protect chunk-input reuse within this hook. Across +@@ -239,6 +242,19 @@ class DistributedLayerwiseOffloadHook(ModelHook): + # Per-block synchronization primitive: set after H2D copy completes. + self._prefetch_done: Any | None = None + ++ # Optional ordering: hold the background weight AllGather until the ++ # block's QKV AllToAll is on the compute stream. See ++ # distributed/collective_gate.py. Off by default; the attention layer ++ # inside this block is handed the same barrier at install time. ++ self._a2a_barrier: QKVAllToAllBarrier | None = None ++ self._a2a_event: Any | None = None ++ # ``(slot, gpu_shards, gpu_weights, evt)`` of a gather armed but not yet ++ # issued, so ``post_forward`` can flush it if the attention never ran. ++ self._deferred_allgather: tuple[int, dict, dict, Any] | None = None ++ if order_allgather_after_attn_a2a: ++ self._a2a_event = current_omni_platform.Event() ++ self._a2a_barrier = QKVAllToAllBarrier(self._allgather_after_attn_a2a) ++ + # Shared shard (AllGather input) buffers — assigned by backend. + self.gpu_shard_buffers: list[dict[torch.dtype, torch.Tensor] | None] = [None, None] + +@@ -653,49 +669,23 @@ class DistributedLayerwiseOffloadHook(ModelHook): + gpu_shard = shard_bufs[dtype][: cpu_shard.numel()] + gpu_shard.copy_(cpu_shard, non_blocking=non_blocking) + gpu_shards[dtype] = gpu_shard +- self.comm_stream.wait_stream(self.copy_stream) +- with current_omni_platform.stream(self.comm_stream): +- for dtype, local_shard in gpu_shards.items(): +- gw = gpu_weights[dtype][: self._ag_output_sizes[dtype]] +- torch.distributed.all_gather_into_tensor( +- gw, +- local_shard, +- group=self.dp_group, +- ) +- evt.record(self.comm_stream) ++ ++ if self._a2a_barrier is not None and non_blocking and getattr(self, "_nz_cast_fn", None) is None: ++ # Hold the collective until the block's attention reports its ++ # QKV AllToAll, so the background gather cannot take the links ++ # ahead of the critical-path all-to-all. The H2D above has ++ # already been enqueued; only the gather moves, so it still ++ # overlaps the rest of the block. Deferral needs a repoint that ++ # emits no stream work, hence the NZ-cast guard. ++ self._deferred_allgather = (slot, gpu_shards, gpu_weights, evt) ++ self._a2a_barrier.arm() ++ else: ++ self._issue_allgather(slot, gpu_shards, gpu_weights, evt) ++ elif self._a2a_barrier is not None and non_blocking: ++ self._deferred_chunked_allgather = (slot, gpu_weights, evt) ++ self._a2a_barrier.arm() + else: +- for dtype_manifest in self.manifest.dtypes: +- cpu_shard = self.cpu_shards[dtype_manifest.dtype] +- buf = shard_bufs[dtype_manifest.dtype] +- for chunk in dtype_manifest.chunks: +- if buf.numel() >= 2 * chunk.local_numel: +- width = buf.numel() // 2 +- input_slot = chunk.chunk_id % 2 +- else: +- width = buf.numel() +- input_slot = 0 +- reuse = self._chunk_reuse_events[input_slot] +- if reuse is not None: +- self.copy_stream.wait_event(reuse) +- local_input = buf[input_slot * width : input_slot * width + chunk.local_numel] +- source = cpu_shard[chunk.cpu_offset : chunk.cpu_offset + chunk.local_numel] +- with current_omni_platform.stream(self.copy_stream): +- local_input.copy_(source, non_blocking=non_blocking) +- self._h2d_done_events[input_slot].record(self.copy_stream) +- self.comm_stream.wait_event(self._h2d_done_events[input_slot]) +- with current_omni_platform.stream(self.comm_stream): +- full_out = gpu_weights[dtype_manifest.dtype][ +- chunk.full_offset : chunk.full_offset + chunk.padded_numel +- ] +- torch.distributed.all_gather_into_tensor( +- full_out, +- local_input, +- group=self.dp_group, +- ) +- self._ag_done_events[input_slot].record(self.comm_stream) +- self._chunk_reuse_events[input_slot] = self._ag_done_events[input_slot] +- with current_omni_platform.stream(self.comm_stream): +- evt.record(self.comm_stream) ++ self._issue_chunked_allgather(slot, gpu_weights, evt, non_blocking=non_blocking) + + self.ready_events[slot] = evt + self._prefetch_done = evt +@@ -719,6 +709,131 @@ class DistributedLayerwiseOffloadHook(ModelHook): + ), + ) + ++ def _issue_chunked_allgather(self, slot, gpu_weights, evt, *, gated=False, non_blocking=True) -> None: ++ """Submit chunk copies and gathers together after the foreground gate. ++ ++ Chunk staging slots are bounded and reused. Deferring only gathers ++ would overwrite an input slot before its collective consumes it. ++ """ ++ shard_bufs = self.gpu_shard_buffers[slot] ++ assert shard_bufs is not None and self.manifest is not None ++ if gated: ++ self.comm_stream.wait_event(self._a2a_event) ++ for dtype_manifest in self.manifest.dtypes: ++ cpu_shard = self.cpu_shards[dtype_manifest.dtype] ++ buf = shard_bufs[dtype_manifest.dtype] ++ for chunk in dtype_manifest.chunks: ++ if buf.numel() >= 2 * chunk.local_numel: ++ width = buf.numel() // 2 ++ input_slot = chunk.chunk_id % 2 ++ else: ++ width = buf.numel() ++ input_slot = 0 ++ reuse = self._chunk_reuse_events[input_slot] ++ if reuse is not None: ++ self.copy_stream.wait_event(reuse) ++ local_input = buf[input_slot * width : input_slot * width + chunk.local_numel] ++ source = cpu_shard[chunk.cpu_offset : chunk.cpu_offset + chunk.local_numel] ++ with current_omni_platform.stream(self.copy_stream): ++ local_input.copy_(source, non_blocking=non_blocking) ++ self._h2d_done_events[input_slot].record(self.copy_stream) ++ self.comm_stream.wait_event(self._h2d_done_events[input_slot]) ++ with current_omni_platform.stream(self.comm_stream): ++ full_out = gpu_weights[dtype_manifest.dtype][ ++ chunk.full_offset : chunk.full_offset + chunk.padded_numel ++ ] ++ torch.distributed.all_gather_into_tensor( ++ full_out, ++ local_input, ++ group=self.dp_group, ++ ) ++ self._ag_done_events[input_slot].record(self.comm_stream) ++ self._chunk_reuse_events[input_slot] = self._ag_done_events[input_slot] ++ with current_omni_platform.stream(self.comm_stream): ++ evt.record(self.comm_stream) ++ ++ def _issue_allgather( ++ self, ++ slot: int, ++ gpu_shards: dict[torch.dtype, torch.Tensor], ++ gpu_weights: dict[torch.dtype, torch.Tensor], ++ evt: Any, ++ *, ++ gated: bool = False, ++ ) -> None: ++ """Submit this block's weight AllGather on ``comm_stream``. ++ ++ ``gated`` additionally waits on the QKV AllToAll barrier event recorded ++ by the block's attention, so the foreground collective gets the links ++ first and this gather queues behind it. ++ """ ++ self.comm_stream.wait_stream(self.copy_stream) ++ if gated: ++ self.comm_stream.wait_event(self._a2a_event) ++ with current_omni_platform.stream(self.comm_stream): ++ for dtype, local_shard in gpu_shards.items(): ++ # Slice the shared (max-sized) output buffer down to this ++ # block's actual AllGather output size. The buffers are ++ # sized to the *largest* block across all groups, so for any ++ # smaller block the full buffer would violate the ++ # all_gather_into_tensor contract ++ # (output.numel() == world_size * input.numel()). ++ # Repoint offsets are relative to the block's flattened ++ # buffer, so a prefix slice is safe. ++ gw = gpu_weights[dtype][: self._ag_output_sizes[dtype]] ++ torch.distributed.all_gather_into_tensor( ++ gw, ++ local_shard, ++ group=self.dp_group, ++ ) ++ evt.record(self.comm_stream) ++ ++ def _allgather_after_attn_a2a(self) -> None: ++ """Barrier callback: the block's QKV AllToAll is now enqueued. ++ ++ Recording on the compute stream here (rather than at block entry) is ++ what makes the gate effective: an event wait snapshots the records ++ enqueued before it, so the wait must be enqueued *after* the all-to-all. ++ """ ++ chunked = self._deferred_chunked_allgather ++ if chunked is not None: ++ self._deferred_chunked_allgather = None ++ assert self._a2a_event is not None ++ self._a2a_event.record(current_omni_platform.current_stream()) ++ self._issue_chunked_allgather(*chunked, gated=True) ++ return ++ deferred = self._deferred_allgather ++ if deferred is None: ++ return ++ self._deferred_allgather = None ++ assert self._a2a_event is not None ++ self._a2a_event.record(current_omni_platform.current_stream()) ++ slot, gpu_shards, gpu_weights, evt = deferred ++ self._issue_allgather(slot, gpu_shards, gpu_weights, evt, gated=True) ++ ++ def _flush_deferred_allgather(self) -> None: ++ """Issue a deferred gather whose attention never signalled. ++ ++ Keeps the double buffer live: a block whose attention is skipped ++ (cache-dit, an all-resident stage) still needs its weights, so the ++ gather is submitted ungated at the end of that block's forward. ++ """ ++ chunked = self._deferred_chunked_allgather ++ if chunked is not None: ++ self._deferred_chunked_allgather = None ++ if self._a2a_barrier is not None: ++ self._a2a_barrier.disarm() ++ self._issue_chunked_allgather(*chunked) ++ return ++ deferred = self._deferred_allgather ++ if deferred is None: ++ return ++ self._deferred_allgather = None ++ if self._a2a_barrier is not None: ++ self._a2a_barrier.disarm() ++ slot, gpu_shards, gpu_weights, evt = deferred ++ self._issue_allgather(slot, gpu_shards, gpu_weights, evt) ++ + def _wait_for_weights(self, slot: int) -> None: + """Wait until the slot holding this block is ready for compute. + +@@ -840,6 +955,10 @@ class DistributedLayerwiseOffloadHook(ModelHook): + return args, kwargs + + def post_forward(self, module: nn.Module, output: Any) -> Any: ++ # Pair with pre_forward: if the block's attention never signalled the ++ # gate, the deferred gather must still go out before this block's ++ # storage is released (offload_layer waits on _prefetch_done). ++ self._flush_deferred_allgather() + self.offload_layer() + return output + +@@ -864,6 +983,7 @@ def apply_distributed_block_hook( + tensor_transforms: dict[int, Any] | None = None, + materialization_probe_tensor: torch.Tensor | None = None, + chunk_size_bytes: int = 64 * 1024 * 1024, ++ order_allgather_after_attn_a2a: bool = False, + ) -> DistributedLayerwiseOffloadHook: + """Register a DistributedLayerwiseOffloadHook on *module*.""" + registry = HookRegistry.get_or_create(module) +@@ -881,11 +1001,32 @@ def apply_distributed_block_hook( + tensor_transforms=tensor_transforms, + materialization_probe_tensor=materialization_probe_tensor, + chunk_size_bytes=chunk_size_bytes, ++ order_allgather_after_attn_a2a=order_allgather_after_attn_a2a, + ) + registry.register_hook(DistributedLayerwiseOffloadHook._HOOK_NAME, hook) + return hook + + ++def attach_collective_gate(block: nn.Module, hook: DistributedLayerwiseOffloadHook) -> int: ++ """Hand ``hook``'s barrier to every Attention layer inside *block*. ++ ++ The attention signals the barrier right after its QKV AllToAll. The import ++ is local to keep this module independent of the attention package at import ++ time. ++ ++ Returns the number of attention layers wired (0 means the hook will rely on ++ the ``post_forward`` flush instead). ++ """ ++ from vllm_omni.diffusion.attention.layer import Attention ++ ++ attached = 0 ++ for module in block.modules(): ++ if isinstance(module, Attention): ++ module._collective_gate = hook._a2a_barrier ++ attached += 1 ++ return attached ++ ++ + def remove_distributed_block_hook(module: nn.Module) -> None: + """Remove the distributed layerwise offload hook from *module*.""" + registry: HookRegistry | None = getattr(module, "_hook_registry", None) +@@ -1566,29 +1707,45 @@ class DistributedLayerwiseOffloadBackend(OffloadBackend): + self._all_hook_groups.append(hooks) + self._blocks.append(block_list) + probes = {id(block): module_materialization_probe(block) for block in block_list} ++ # Only the DiT blocks issue a QKV AllToAll the gather can queue behind. ++ order_gate = bool(getattr(self.config, "dlo_order_allgather_after_attn_a2a", False)) and ( ++ component == DIT_COMPONENT ++ ) ++ if order_gate and not getattr(self, "_collective_gate_logged", False): ++ self._collective_gate_logged = True ++ logger.info( ++ "Distributed layerwise offload will hold each DiT block's weight AllGather " ++ "until that block's QKV AllToAll is on the compute stream" ++ ) + for block, next_block in zip( + chain((block_list[-1],), block_list[:-1]), + block_list, + strict=True, + ): +- hooks.append( +- apply_distributed_block_hook( +- block, +- next_block, +- self.device, +- group, +- group_size, +- group_rank, +- self.copy_stream, +- self.comm_stream, +- self.config.pin_cpu_memory, +- shared_buffers=[None, None], +- rank_local_mmap=self._using_rank_local_mmap if use_dit_mmap else False, +- tensor_transforms=self._mmap_transforms_by_tensor_id if use_dit_mmap else None, +- materialization_probe_tensor=probes[id(block)], +- chunk_size_bytes=self.config.chunk_size_bytes, +- ) ++ hook = apply_distributed_block_hook( ++ block, ++ next_block, ++ self.device, ++ group, ++ group_size, ++ group_rank, ++ self.copy_stream, ++ self.comm_stream, ++ self.config.pin_cpu_memory, ++ shared_buffers=[None, None], ++ rank_local_mmap=self._using_rank_local_mmap if use_dit_mmap else False, ++ tensor_transforms=self._mmap_transforms_by_tensor_id if use_dit_mmap else None, ++ materialization_probe_tensor=probes[id(block)], ++ chunk_size_bytes=self.config.chunk_size_bytes, ++ order_allgather_after_attn_a2a=order_gate, + ) ++ if hook._a2a_barrier is not None and attach_collective_gate(block, hook) == 0: ++ logger.warning( ++ "DLO AllGather ordering is enabled but block %s exposes no Attention layer; " ++ "its gather will fall back to the ungated path", ++ block.__class__.__name__, ++ ) ++ hooks.append(hook) + + # hooks = [last -> first, block0 -> block1, ...]. Alternating slots + # keep a prefetch from overwriting the current block for any ring size. +diff --git a/vllm_omni/engine/arg_utils.py b/vllm_omni/engine/arg_utils.py +index 0a876b9..bf8dcdc 100644 +--- a/vllm_omni/engine/arg_utils.py ++++ b/vllm_omni/engine/arg_utils.py +@@ -807,6 +807,7 @@ class OrchestratorArgs: + dlo_use_allgather: bool = True + dlo_chunk_size_mb: int = 64 + dlo_resident_layers: int = 0 ++ dlo_order_allgather_after_attn_a2a: bool = False + host_weight_runtime_mode: str = "disabled" + host_weight_runtime_root: str | None = None + dlo_host_registration_limit_gib: float = 0.0 +diff --git a/vllm_omni/entrypoints/cli/serve.py b/vllm_omni/entrypoints/cli/serve.py +index f2b55e7..aff71b5 100644 +--- a/vllm_omni/entrypoints/cli/serve.py ++++ b/vllm_omni/entrypoints/cli/serve.py +@@ -1041,6 +1041,17 @@ class OmniServeCommand(CLISubcommand): + help="Full-chunk target size in MiB for distributed layerwise offload " + "(each chunk is one host-to-device copy + AllGather unit).", + ) ++ omni_config_group.add_argument( ++ "--dlo-order-allgather-after-attn-a2a", ++ action="store_true", ++ default=False, ++ help=( ++ "Hold each DiT block's weight AllGather until that block's QKV AllToAll " ++ "has been enqueued, so the background gather does not take the collective " ++ "links ahead of the critical-path all-to-all. Requires distributed " ++ "layerwise offload with AllGather." ++ ), ++ ) + omni_config_group.add_argument( + "--host-weight-runtime-mode", + choices=("disabled", "preferred", "required"), diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c21-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c21-source-identity.json new file mode 100644 index 000000000..160c55a95 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c21-source-identity.json @@ -0,0 +1,28 @@ +{ + "head": "d48fa51d03dd748568cbcf57ed40f533c691c9bd", + "tree": "637647dbe51ccbd092f076c25e0ced0fe0404207", + "base": "d027b4d0c53bf40876215d4e3856bc17450a4c39", + "branch": "rsi/h3-297-video-pre-gate", + "files": { + "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "8933b536317652d0dbc7a974ea7e0bca928e7b3bdf438988999ff5a498a6ec9d", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "019bf71106ba3263b55ad7b682064e3a176eecb81725cb48bbf178387356d5d9", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "04a9212994b5c8593f8d196820300ded077fed9af4274b64d87d9e6bf71eae07", + "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" + }, + "published": false, + "patch_base": "be319692b96413918973b5bd399cdda3f320c574" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c22-candidate.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c22-candidate.patch new file mode 100644 index 000000000..bcf1323e4 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c22-candidate.patch @@ -0,0 +1,2187 @@ +diff --git a/tests/diffusion/distributed/test_collective_gate.py b/tests/diffusion/distributed/test_collective_gate.py +new file mode 100644 +index 0000000..e66fdae +--- /dev/null ++++ b/tests/diffusion/distributed/test_collective_gate.py +@@ -0,0 +1,205 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++"""Unit tests for the DLO/QKV-AllToAll ordering gate. ++ ++Covers the barrier rendezvous and the hook's deferred-AllGather state machine ++that keeps the background weight gather from taking the links ahead of the ++block's foreground AllToAll. ++""" ++ ++from unittest.mock import Mock ++ ++import pytest ++import torch ++from torch import nn ++ ++import vllm_omni.diffusion.offloader.distributed_layerwise_backend as dist_backend_module ++from tests.diffusion.offloader.helpers import DummyStream, patch_offload_runtime ++from vllm_omni.diffusion.distributed.collective_gate import QKVAllToAllBarrier ++from vllm_omni.diffusion.offloader.distributed_layerwise_backend import ( ++ DistributedLayerwiseOffloadHook, ++ attach_collective_gate, ++) ++ ++pytestmark = [pytest.mark.diffusion, pytest.mark.cpu, pytest.mark.core_model] ++ ++ ++@pytest.fixture ++def patched_offload_runtime(monkeypatch): ++ patch_offload_runtime(monkeypatch, dist_backend_module.current_omni_platform, synchronize=True) ++ ++ ++class TestQKVAllToAllBarrier: ++ def test_signal_without_arm_is_a_noop(self): ++ on_signal = Mock() ++ barrier = QKVAllToAllBarrier(on_signal) ++ ++ barrier.signal() ++ ++ on_signal.assert_not_called() ++ assert not barrier.armed ++ ++ def test_arm_then_signal_fires_once(self): ++ on_signal = Mock() ++ barrier = QKVAllToAllBarrier(on_signal) ++ ++ barrier.arm() ++ assert barrier.armed ++ barrier.signal() ++ ++ on_signal.assert_called_once_with() ++ assert not barrier.armed ++ ++ # A second signal in the same forward must not re-issue the gather. ++ barrier.signal() ++ on_signal.assert_called_once_with() ++ ++ def test_disarm_suppresses_a_pending_signal(self): ++ on_signal = Mock() ++ barrier = QKVAllToAllBarrier(on_signal) ++ ++ barrier.arm() ++ barrier.disarm() ++ barrier.signal() ++ ++ on_signal.assert_not_called() ++ ++ ++def _gated_hook() -> DistributedLayerwiseOffloadHook: ++ """A hook wired for the gated path, with hand-set shard buffers.""" ++ hook = DistributedLayerwiseOffloadHook( ++ next_block=nn.Linear(2, 2, bias=False), ++ device=torch.device("cpu"), ++ dp_group=object(), ++ dp_size=2, ++ rank=0, ++ copy_stream=DummyStream(), ++ comm_stream=DummyStream(), ++ pin_memory=False, ++ order_allgather_after_attn_a2a=True, ++ ) ++ device_buf = {torch.float32: torch.zeros(4)} ++ shard_buf = {torch.float32: torch.zeros(2)} ++ hook.gpu_buffers = [dict(device_buf), dict(device_buf)] ++ hook.gpu_shard_buffers = [dict(shard_buf), dict(shard_buf)] ++ hook.cpu_shards = {torch.float32: torch.zeros(2)} ++ hook._ag_output_sizes = {torch.float32: 4} ++ hook.metadata = {} ++ hook._cached_repoint = () ++ hook.rank_local_mmap = False ++ return hook ++ ++ ++class TestDeferredAllGather: ++ def test_prefetch_defers_the_gather_until_the_barrier_signals(self, patched_offload_runtime, monkeypatch): ++ hook = _gated_hook() ++ gather = Mock() ++ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) ++ ++ hook.prefetch_layer(0, non_blocking=True) ++ ++ # H2D ran; the collective is held back for the block's AllToAll. ++ gather.assert_not_called() ++ assert hook._a2a_barrier.armed ++ assert hook._deferred_allgather is not None ++ # The ready event must still be published for the consumer. ++ assert hook.ready_events[0] is hook._prefetch_done ++ ++ hook._a2a_barrier.signal() ++ ++ gather.assert_called_once() ++ assert hook._deferred_allgather is None ++ assert not hook._a2a_barrier.armed ++ ++ def test_post_forward_flush_issues_an_unsignalled_gather(self, patched_offload_runtime, monkeypatch): ++ """A block whose attention never ran must still get its weights.""" ++ hook = _gated_hook() ++ gather = Mock() ++ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) ++ ++ hook.prefetch_layer(0, non_blocking=True) ++ gather.assert_not_called() ++ ++ # pre_forward ran, attention did not: the pairing flush must fire. ++ hook.post_forward(hook.next_block, None) ++ ++ gather.assert_called_once() ++ assert hook._deferred_allgather is None ++ assert not hook._a2a_barrier.armed ++ ++ def test_sync_prefetch_issues_immediately_without_arming(self, patched_offload_runtime, monkeypatch): ++ """The cache-dit sync fallback has no attention signal to wait for.""" ++ hook = _gated_hook() ++ gather = Mock() ++ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) ++ ++ hook.prefetch_layer(0, non_blocking=False) ++ ++ gather.assert_called_once() ++ assert not hook._a2a_barrier.armed ++ assert hook._deferred_allgather is None ++ ++ def test_disabled_by_default(self, patched_offload_runtime, monkeypatch): ++ hook = DistributedLayerwiseOffloadHook( ++ next_block=nn.Linear(2, 2, bias=False), ++ device=torch.device("cpu"), ++ dp_group=object(), ++ dp_size=2, ++ rank=0, ++ copy_stream=DummyStream(), ++ comm_stream=DummyStream(), ++ pin_memory=False, ++ ) ++ hook.gpu_buffers = [{torch.float32: torch.zeros(4)}] * 2 ++ hook.gpu_shard_buffers = [{torch.float32: torch.zeros(2)}] * 2 ++ hook.cpu_shards = {torch.float32: torch.zeros(2)} ++ hook._ag_output_sizes = {torch.float32: 4} ++ hook.metadata = {} ++ hook._cached_repoint = () ++ gather = Mock() ++ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) ++ ++ hook.prefetch_layer(0, non_blocking=True) ++ ++ assert hook._a2a_barrier is None ++ gather.assert_called_once() ++ ++ ++class TestAttachCollectiveGate: ++ def test_block_without_attention_reports_zero(self): ++ hook = _gated_hook() ++ ++ assert attach_collective_gate(nn.Linear(2, 2), hook) == 0 ++ ++ ++@pytest.mark.parametrize("flush", [False, True]) ++def test_chunked_gate_defers_copies_and_reuses_bounded_slots(patched_offload_runtime, monkeypatch, flush): ++ from types import SimpleNamespace ++ ++ hook = _gated_hook() ++ chunks = tuple( ++ SimpleNamespace(chunk_id=i, local_numel=1, cpu_offset=i, full_offset=2 * i, padded_numel=2) for i in range(3) ++ ) ++ hook.manifest = SimpleNamespace(dtypes=(SimpleNamespace(dtype=torch.float32, chunks=chunks),)) ++ hook.cpu_shards = {torch.float32: torch.tensor([11.0, 22.0, 33.0])} ++ hook.gpu_buffers = [{torch.float32: torch.zeros(6)} for _ in range(2)] ++ calls = [] ++ ++ def gather(output, local, **kwargs): ++ calls.append(local.clone()) ++ output.copy_(local.repeat(2)) ++ ++ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) ++ hook.prefetch_layer(0, non_blocking=True) ++ assert not calls and hook._deferred_chunked_allgather is not None ++ assert torch.equal(hook.gpu_shard_buffers[0][torch.float32], torch.zeros(2)) ++ if flush: ++ hook._flush_deferred_allgather() ++ else: ++ hook._a2a_barrier.signal() ++ assert len(calls) == 3 and hook._deferred_chunked_allgather is None ++ assert not hook._a2a_barrier.armed ++ assert torch.equal(hook.gpu_buffers[0][torch.float32], torch.tensor([11.0, 11.0, 22.0, 22.0, 33.0, 33.0])) ++ hook._a2a_barrier.signal() ++ assert len(calls) == 3 +diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py +new file mode 100644 +index 0000000..19b8246 +--- /dev/null ++++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py +@@ -0,0 +1,199 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++from datetime import timedelta ++ ++import pytest ++import torch ++import torch.distributed as dist ++import torch.nn as nn ++ ++from tests.diffusion.models.minimax_h3.test_minimax_h3_video_row_cache import tiny_block ++from tests.helpers.mark import hardware_marks ++from vllm_omni.diffusion.layers.indexed_modulation import indexed_gate ++from vllm_omni.diffusion.models.minimax_h3.attention.flash_h3 import _prefix_bnsd, _subset_prefix_bnsd ++from vllm_omni.diffusion.models.minimax_h3.video_row_cache import MiniMaxH3VideoRowCache ++from vllm_omni.platforms import current_omni_platform ++ ++pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=4)] ++ ++ ++def make_state(start=0, length=4): ++ state = MiniMaxH3VideoRowCache(torch.tensor([1, 2]), seq_len=4, used_len=3, num_layers=1, pre_gate=True) ++ state.prepare_local_span(start, length) ++ return state ++ ++ ++def test_fixed_pre_gate_schedule_storage_and_request_cleanup(): ++ state = make_state() ++ x = torch.ones(4, 2, dtype=torch.bfloat16) ++ full = [] ++ for step in range(50): ++ state.begin_step(step) ++ if not state.partial: ++ full.append(step) ++ for stage in ("attn", "mlp"): ++ out = state.pre_gate_output(0, x, stage) ++ assert out.dtype == torch.bfloat16 ++ if state.partial: ++ state.record_fresh_gate(stage) ++ state.end_step() ++ assert full == [0, 1, 4, 9, 19, 29, 39, 49] ++ assert state.full_blocks == 8 and state.partial_blocks == 42 ++ assert state.fresh_attention_gates == state.fresh_mlp_gates == 42 ++ assert all(t.dtype == torch.bfloat16 for t in [*state._attention_outputs.values(), *state._mlp_outputs.values()]) ++ assert state.stats()["live_cache_bytes"] == 16 ++ state.close() ++ assert state.stats()["live_cache_bytes"] == 0 and state.closed ++ ++ ++@pytest.mark.parametrize("failure", ["missing_mlp", "mlp_first", "gated_capture", "gated_restore"]) ++def test_pre_gate_stage_order_cannot_silently_use_block_residual(failure): ++ state = make_state() ++ x = torch.ones(4, 2, dtype=torch.bfloat16) ++ state.begin_step(0) ++ with pytest.raises(RuntimeError): ++ if failure == "missing_mlp": ++ state.pre_gate_output(0, x, "attn") ++ state.end_step() ++ elif failure == "mlp_first": ++ state.pre_gate_output(0, x, "mlp") ++ elif failure == "gated_capture": ++ state.capture(0, x, x) ++ else: ++ state.restore(0, x) ++ state.close() ++ ++ ++class ConstantMLP(nn.Module): ++ def __init__(self): ++ super().__init__() ++ self.rows = [] ++ ++ def forward(self, x): ++ self.rows.append(x.shape[0]) ++ return torch.full_like(x, 3) ++ ++ ++@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) ++def test_real_block_reapplies_changed_gates_and_preserves_empty_dense_shards(start, length): ++ state = make_state(start, length) ++ block = tiny_block() ++ gates = [1.0, 1.0] ++ block.adaln_proj = lambda t: ( ++ torch.zeros(3, 2), ++ torch.zeros(3, 2), ++ torch.full((3, 2), gates[0]), ++ torch.zeros(3, 2), ++ torch.zeros(3, 2), ++ torch.full((3, 2), gates[1]), ++ ) ++ block.attn = lambda x, **kwargs: torch.full_like(x, 2) ++ block.mlp = ConstantMLP() ++ x = torch.ones(length, 2, dtype=torch.bfloat16) ++ kwargs = dict( ++ t_emb=torch.zeros(1), ++ combined_indices=torch.zeros(length, dtype=torch.long), ++ rope_table=None, ++ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), ++ max_seqlen=3, ++ packed_total=4, ++ ) ++ for step in (0, 1): ++ state.begin_step(step) ++ expected = block(x, **kwargs) ++ actual = block(x, video_row_cache=state, **kwargs) ++ assert torch.equal(actual.view(torch.uint8), expected.view(torch.uint8)) ++ state.end_step() ++ gates[:] = [0.5, -0.25] ++ current = x * 2 ++ saved = current.clone() ++ expected = block(current, **kwargs) ++ block.mlp.rows.clear() ++ state.begin_step(2) ++ actual = block(current, video_row_cache=state, **kwargs) ++ state.end_step() ++ rows = state.local_dense_positions ++ assert block.mlp.rows == ([rows.numel()] if rows.numel() else []) ++ assert torch.equal( ++ actual[: min(length, 3 - start)].view(torch.uint8), expected[: min(length, 3 - start)].view(torch.uint8) ++ ) ++ assert torch.equal( ++ actual.index_select(0, state.local_video_positions), torch.full((state.local_video_positions.numel(), 2), 2.25) ++ ) ++ assert not torch.equal( ++ actual.index_select(0, state.local_video_positions), current.index_select(0, state.local_video_positions) + 5 ++ ) ++ assert torch.equal(current, saved) ++ assert state.fresh_attention_gates == state.fresh_mlp_gates == 1 ++ state.close() ++ ++ ++def _native_pre_gate_worker(rank, secant=False): ++ import torch_npu # noqa: F401 ++ ++ torch.npu.set_device(rank) ++ dist.init_process_group( ++ "hccl", init_method="tcp://127.0.0.1:29597", rank=rank, world_size=4, timeout=timedelta(seconds=120) ++ ) ++ try: ++ torch.manual_seed(219) ++ full = [torch.randn(32, 4, 128, dtype=torch.bfloat16).npu() for _ in range(3)] ++ exchanged = [] ++ for tensor in full: ++ local = tensor[rank * 8 : (rank + 1) * 8] ++ send = local.reshape(8, 4, 1, 128).permute(1, 0, 2, 3).contiguous() ++ receive = torch.empty_like(send) ++ dist.all_to_all_single(receive, send) ++ exchanged.append(receive.reshape(1, 32, 1, 128)) ++ state = MiniMaxH3VideoRowCache( ++ torch.arange(8, 24, device="npu"), seq_len=32, used_len=30, num_layers=1, pre_gate=True, secant=secant ++ ) ++ state.prepare_local_span(rank * 8, 8) ++ x = torch.full((8, 512), 2, dtype=torch.bfloat16, device="npu") ++ for step in (0, 1): ++ state.begin_step(step, sigma=1 - step / 10) ++ state.pre_gate_output(0, torch.full_like(x, 0.5 + (step * 0.25 if secant else 0)), "attn") ++ state.pre_gate_output(0, torch.full_like(x, 0.25 + (step * 0.125 if secant else 0)), "mlp") ++ state.end_step() ++ state.begin_step(2, sigma=0.8) ++ out = _subset_prefix_bnsd(exchanged[0], exchanged[1][:, :30], exchanged[2][:, :30], 128**-0.5, state) ++ send, receive = out.reshape(4, 8, 1, 128).contiguous(), torch.empty_like(out.reshape(4, 8, 1, 128)) ++ dist.all_to_all_single(receive, send) ++ local_out = receive.permute(1, 0, 2, 3).reshape(8, 512) ++ expected = _prefix_bnsd(full[0].unsqueeze(0), full[1][:30].unsqueeze(0), full[2][:30].unsqueeze(0), 128**-0.5)[ ++ 0 ++ ].reshape(32, 512) ++ dense = state.local_dense_positions ++ torch.testing.assert_close( ++ local_out.index_select(0, dense).float(), ++ expected[rank * 8 : (rank + 1) * 8].index_select(0, dense).float(), ++ atol=0.03125, ++ rtol=0.02, ++ ) ++ attn = state.pre_gate_output(0, local_out, "attn") ++ indices = torch.zeros(8, dtype=torch.long, device="npu") ++ gate = torch.full((1, 512), 0.5, dtype=torch.float32, device="npu") ++ after_attn = indexed_gate(x, gate, attn, indices) ++ state.record_fresh_gate("attn") ++ mlp = state.pre_gate_output(0, torch.zeros_like(x), "mlp") ++ result = indexed_gate(after_attn, gate, mlp, indices) ++ state.record_fresh_gate("mlp") ++ video = state.local_video_positions ++ assert torch.equal( ++ result.index_select(0, video), ++ torch.full((video.numel(), 512), 2.75 if secant else 2.375, dtype=torch.bfloat16, device="npu"), ++ ) ++ assert dense.numel() == [8, 0, 0, 6][rank] ++ state.end_step() ++ state.close() ++ assert state.stats()["live_cache_bytes"] == 0 ++ torch.npu.synchronize() ++ dist.barrier() ++ finally: ++ dist.destroy_process_group() ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires four native Ascend devices") ++def test_native_four_rank_pre_gate_subset_attention_and_current_gates(): ++ torch.multiprocessing.spawn(_native_pre_gate_worker, nprocs=4, join=True) +diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_secant.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_secant.py +new file mode 100644 +index 0000000..307723f +--- /dev/null ++++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_secant.py +@@ -0,0 +1,83 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++import pytest ++import torch ++ ++from tests.diffusion.models.minimax_h3.test_minimax_h3_pre_gate_cache import _native_pre_gate_worker ++from tests.helpers.mark import hardware_marks ++from vllm_omni.diffusion.models.minimax_h3.video_row_cache import MiniMaxH3VideoRowCache ++from vllm_omni.platforms import current_omni_platform ++ ++pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=4)] ++ ++ ++def state(): ++ cache = MiniMaxH3VideoRowCache( ++ torch.tensor([1, 2]), seq_len=4, used_len=3, num_layers=1, pre_gate=True, secant=True ++ ) ++ cache.prepare_local_span(0, 4) ++ return cache ++ ++ ++@pytest.mark.parametrize("sigma", [None, float("nan"), float("inf"), -0.1, 1.1]) ++def test_secant_requires_explicit_finite_solver_sigma(sigma): ++ cache = state() ++ with pytest.raises(ValueError, match="finite solver sigma"): ++ cache.begin_step(0, sigma=sigma) ++ cache.close() ++ ++ ++def test_secant_cannot_select_gated_block_residual(): ++ with pytest.raises(ValueError, match="pre-gate"): ++ MiniMaxH3VideoRowCache(torch.tensor([1]), seq_len=4, used_len=3, num_layers=1, secant=True) ++ ++ ++def test_two_anchor_forecast_follows_sigma_and_preserves_dense_rows(): ++ cache = state() ++ for step, sigma, values in [(0, 1.0, [4, -8]), (1, 0.5, [2, -4])]: ++ cache.begin_step(step, sigma=sigma) ++ x = torch.tensor([values] * 4, dtype=torch.bfloat16) ++ for stage in ("attn", "mlp"): ++ cache.pre_gate_output(0, x, stage) ++ cache.end_step() ++ assert all(t.dtype == torch.int8 for t in cache._slopes.values()) ++ assert all(t.dtype == torch.float32 for t in cache._slope_scales.values()) ++ assert cache.stats()["live_cache_bytes"] == 40 ++ cache.begin_step(2, sigma=0.25) ++ current = torch.full((4, 2), 100, dtype=torch.bfloat16) ++ saved = current.clone() ++ for stage in ("attn", "mlp"): ++ out = cache.pre_gate_output(0, current, stage) ++ assert torch.equal(out[1:3], torch.tensor([[1, -2], [1, -2]], dtype=torch.bfloat16)) ++ assert torch.equal(out[[0, 3]], current[[0, 3]]) ++ out.zero_() ++ cache.end_step() ++ assert torch.equal(current, saved) ++ assert cache.secant_forecasts == 2 ++ with pytest.raises(ValueError, match="strictly decreasing"): ++ cache.begin_step(3, sigma=0.25) ++ cache.close() ++ assert cache.stats()["live_cache_bytes"] == 0 ++ assert not cache._slopes and not cache._anchor_sigmas ++ ++ ++def test_linear_trajectory_and_zero_slope_cover_fixed_fifty_steps(): ++ cache = state() ++ for step in range(50): ++ cache.begin_step(step, sigma=1 - step / 50) ++ # The first channel drifts; the second channel's exact zero difference ++ # exercises finite scale handling without adding an artificial epsilon. ++ actual = torch.tensor([[1 + step / 8, 2]] * 4, dtype=torch.bfloat16) ++ for stage in ("attn", "mlp"): ++ out = cache.pre_gate_output(0, actual, stage) ++ torch.testing.assert_close(out[1:3].float(), actual[1:3].float(), atol=0.03125, rtol=0) ++ cache.end_step() ++ assert cache.secant_forecasts == 84 ++ assert cache.full_steps == 8 and cache.partial_steps == 42 ++ cache.close() ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires four native Ascend devices") ++def test_native_four_rank_sigma_secant_subset_attention_and_current_gates(): ++ torch.multiprocessing.spawn(_native_pre_gate_worker, args=(True,), nprocs=4, join=True) +diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py +new file mode 100644 +index 0000000..94ff958 +--- /dev/null ++++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py +@@ -0,0 +1,406 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++from datetime import timedelta ++from types import SimpleNamespace ++ ++import pytest ++import torch ++import torch.distributed as dist ++import torch.nn as nn ++ ++from tests.helpers.mark import hardware_marks ++from vllm_omni.diffusion.models.minimax_h3 import minimax_h3_blocks as blocks ++from vllm_omni.diffusion.models.minimax_h3.attention.flash_h3 import _prefix_bnsd, _subset_prefix_bnsd ++from vllm_omni.diffusion.models.minimax_h3.video_row_cache import MiniMaxH3VideoRowCache ++from vllm_omni.platforms import current_omni_platform ++ ++pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=4)] ++ ++ ++def state_for_rows(device="cpu", start=0, length=4): ++ state = MiniMaxH3VideoRowCache(torch.tensor([1, 2], device=device), seq_len=4, used_len=3, num_layers=1) ++ state.prepare_local_span(start, length) ++ return state ++ ++ ++def anchor(state, x): ++ for step in range(5): ++ state.begin_step(step) ++ state.capture(0, x, x + 0.125) ++ state.end_step() ++ state.begin_step(5) ++ ++ ++def test_fixed_schedule_and_row_ownership_excludes_padding(): ++ assert [i for i in range(50) if MiniMaxH3VideoRowCache.is_full_step(i)] == [ ++ 0, ++ 1, ++ 2, ++ 3, ++ 4, ++ 9, ++ 14, ++ 19, ++ 24, ++ 29, ++ 34, ++ 39, ++ 44, ++ 46, ++ 47, ++ 48, ++ 49, ++ ] ++ state = state_for_rows() ++ assert state.global_dense_positions.tolist() == [0] ++ assert state.local_dense_positions.tolist() == [0] ++ assert state.local_video_positions.tolist() == [1, 2] ++ with pytest.raises(RuntimeError, match="ownership"): ++ state.prepare_local_span(0, 3) ++ ++ ++@pytest.mark.parametrize("positions", [[1, 1], [2, 1], [-1], [3]]) ++def test_invalid_target_positions_are_rejected(positions): ++ with pytest.raises(ValueError): ++ MiniMaxH3VideoRowCache(torch.tensor(positions), seq_len=4, used_len=3, num_layers=1) ++ ++ ++def test_delta_is_fp32_and_restore_owns_output(): ++ state = state_for_rows() ++ x = torch.tensor([[1, 2], [3, 4], [5, 6], [7, 8]], dtype=torch.bfloat16) ++ anchor(state, x) ++ assert state._deltas[0].dtype == torch.float32 ++ before = x.clone() ++ output = state.restore(0, x) ++ assert torch.equal(output[1:3], x[1:3] + 0.125) ++ assert torch.equal(output[[0, 3]], x[[0, 3]]) ++ output.zero_() ++ assert torch.equal(x, before) ++ state.end_step() ++ state.close() ++ assert state.closed and not state._deltas and state.global_dense_positions.numel() == 0 ++ ++ ++def large_delta_roundtrip(device="cpu"): ++ state = state_for_rows(device=device) ++ x = torch.full((4, 2), 100000, dtype=torch.bfloat16, device=device) ++ expected = -x ++ delta = expected.float() - x.float() ++ assert torch.isfinite(delta).all() ++ assert not torch.isfinite(delta.to(torch.float16)).all() ++ for step in range(5): ++ state.begin_step(step) ++ state.capture(0, x, expected) ++ state.end_step() ++ state.begin_step(5) ++ actual = state.restore(0, x) ++ assert torch.isfinite(actual).all() ++ assert torch.equal(actual[1:3].cpu().view(torch.uint8), expected[1:3].cpu().view(torch.uint8)) ++ assert torch.equal(actual[[0, 3]], x[[0, 3]]) ++ state.end_step() ++ state.close() ++ assert state.stats()["live_cache_bytes"] == 0 ++ ++ ++def test_large_finite_bf16_delta_roundtrip_exceeds_fp16_storage_range(): ++ large_delta_roundtrip() ++ ++ ++class CountMLP(nn.Module): ++ def __init__(self): ++ super().__init__() ++ self.rows = [] ++ ++ def forward(self, x): ++ self.rows.append(x.shape[0]) ++ return x + 0.25 ++ ++ ++def tiny_block(): ++ block = object.__new__(blocks.MiniMaxH3DiTBlock) ++ nn.Module.__init__(block) ++ block.layer_index = 0 ++ block.norm1 = SimpleNamespace(weight=torch.ones(2), variance_epsilon=1e-6) ++ block.norm2 = SimpleNamespace(weight=torch.ones(2), variance_epsilon=1e-6) ++ block.adaln_proj = lambda t: ( ++ torch.zeros(3, 2), ++ torch.zeros(3, 2), ++ torch.ones(3, 2), ++ torch.zeros(3, 2), ++ torch.zeros(3, 2), ++ torch.ones(3, 2), ++ ) ++ block.attn = lambda x, **kwargs: x * 0.5 ++ block.mlp = CountMLP() ++ return block ++ ++ ++@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) ++def test_real_block_dense_path_and_partial_mlp_empty_rows(start, length): ++ state = state_for_rows(start=start, length=length) ++ block = tiny_block() ++ x = torch.arange(length * 2).reshape(length, 2).to(torch.bfloat16) ++ before = x.clone() ++ kwargs = dict( ++ t_emb=torch.zeros(1), ++ combined_indices=torch.zeros(length, dtype=torch.long), ++ rope_table=None, ++ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), ++ max_seqlen=3, ++ packed_total=4, ++ ) ++ expected = block(x, **kwargs) ++ for step in range(5): ++ state.begin_step(step) ++ actual = block(x, video_row_cache=state, **kwargs) ++ assert torch.equal(actual.view(torch.uint8), expected.view(torch.uint8)) ++ state.end_step() ++ block.mlp.rows.clear() ++ state.begin_step(5) ++ actual = block(x, video_row_cache=state, **kwargs) ++ state.end_step() ++ rows = state.local_dense_positions ++ assert block.mlp.rows == ([rows.numel()] if rows.numel() else []) ++ assert torch.equal(actual.index_select(0, rows), expected.index_select(0, rows)) ++ assert torch.equal(x, before) ++ ++ ++@pytest.mark.parametrize("failure", ["success", "model", "cancel"]) ++def test_request_state_closed_and_removed_on_every_exit(monkeypatch, failure): ++ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop ++ ++ monkeypatch.setenv("H3_VIDEO_ROW_CACHE", "1") ++ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) ++ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) ++ state = MiniMaxH3VideoRowCache(torch.tensor([1]), seq_len=4, used_len=3, num_layers=1) ++ ++ class Model: ++ def create_video_row_cache(self, *args, **kwargs): ++ return state ++ ++ def __call__(self, **kwargs): ++ assert kwargs["video_row_cache"] is state ++ if failure == "model": ++ raise RuntimeError("model failed") ++ state.prepare_local_span(0, 4) ++ x = torch.ones(4, 2, dtype=torch.bfloat16) ++ if state.partial: ++ state.restore(0, x) ++ else: ++ state.capture(0, x, x + 0.125) ++ return torch.zeros(2, 96), torch.zeros(1, 32) ++ ++ class Branch: ++ task = "ref2va" ++ seq_len = 4 ++ used_len = 3 ++ img_pos = torch.tensor([0, 1]) ++ update_mask = torch.tensor([False, True]) ++ update_mask_dev = update_mask ++ audio_update_mask_dev = torch.tensor([True]) ++ locked_audio_rows = None ++ static_kwargs = {"video_token_layout": SimpleNamespace(video_spans=[SimpleNamespace(role="target", start=1)])} ++ ++ def forward_kwargs(self, **kwargs): ++ return dict(self.static_kwargs) ++ ++ monkeypatch.setattr( ++ loop, ++ "minimax_h3_prepare_denoise_rows", ++ lambda **kwargs: (kwargs["initial_video_rows"], kwargs["initial_audio_rows"], torch.zeros(1, 96), None), ++ ) ++ ++ def cancellation(**kwargs): ++ if failure == "cancel": ++ raise RuntimeError("request cancelled") ++ ++ monkeypatch.setattr(loop, "check_request_cancellation", cancellation) ++ branch, model = Branch(), Model() ++ kwargs = dict( ++ model=model, ++ positive=branch, ++ initial_video_rows=torch.zeros(2, 96), ++ initial_audio_rows=torch.zeros(1, 32), ++ keyframe_cond_rows=torch.zeros(1, 96), ++ sigmas_video=[1 - i / 50 for i in range(51)], ++ sigmas_audio=[1 - i / 50 for i in range(51)], ++ device=torch.device("cpu"), ++ ) ++ if failure == "success": ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert state.full_steps == 17 and state.partial_steps == 33 ++ else: ++ with pytest.raises(RuntimeError): ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert state.closed and not state._deltas and "video_row_cache" not in branch.static_kwargs ++ assert model._video_row_cache_stats["closed"] ++ ++ ++def test_default_factory_does_not_create_state(monkeypatch): ++ from vllm_omni.diffusion.models.minimax_h3.minimax_h3_transformer import MiniMaxH3DiTModel ++ ++ monkeypatch.delenv("H3_VIDEO_ROW_CACHE", raising=False) ++ assert ( ++ MiniMaxH3DiTModel.create_video_row_cache(None, torch.tensor([1]), seq_len=4, used_len=3, num_steps=50) is None ++ ) ++ ++ ++@pytest.mark.parametrize("ref2va", [False, True]) ++def test_real_packing_task_selects_ref2va_and_rejects_fl2va(monkeypatch, ref2va): ++ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop ++ from vllm_omni.diffusion.models.minimax_h3.packed_sequence import ( ++ minimax_h3_packed_sequence, ++ minimax_h3_packed_sequence_ref2va_blocks, ++ ) ++ ++ common = dict(text_len=2, latent_t=2, latent_h=4, latent_w=4, audio_t=2) ++ packed = ( ++ minimax_h3_packed_sequence_ref2va_blocks(**common, ref_blocks=[{"kind": "image", "latent_h": 4, "latent_w": 4}]) ++ if ref2va ++ else minimax_h3_packed_sequence(**common, include_keyframe_cond=True, keyframe_frame_indices=[0], frame_count=5) ++ ) ++ branch = loop.MiniMaxH3DenoiseBranch( ++ packed=packed, ++ text_embeddings=torch.zeros(2, 5120), ++ token_tags=packed["token_tags"], ++ device=torch.device("cpu"), ++ task="ref2va" if ref2va else "fl2va", ++ ) ++ layout = branch.static_kwargs["video_token_layout"] ++ assert not any(span.role == "reference" for span in layout.video_spans) ++ monkeypatch.setenv("H3_VIDEO_ROW_CACHE", "1") ++ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) ++ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) ++ monkeypatch.setattr(loop, "check_request_cancellation", lambda **kwargs: None) ++ states = [] ++ ++ class Model: ++ def create_video_row_cache(self, positions, **kwargs): ++ state = MiniMaxH3VideoRowCache( ++ positions, seq_len=kwargs["seq_len"], used_len=kwargs["used_len"], num_layers=1 ++ ) ++ state.prepare_local_span(0, branch.seq_len) ++ states.append(state) ++ return state ++ ++ def __call__(self, **kwargs): ++ state = kwargs["video_row_cache"] ++ x = torch.ones(branch.seq_len, 2, dtype=torch.bfloat16) ++ if state.partial: ++ state.restore(0, x) ++ else: ++ state.capture(0, x, x + 0.125) ++ return torch.zeros(branch.img_pos.numel(), 96), torch.zeros(branch.audio_pos.numel(), 32) ++ ++ kwargs = dict( ++ model=Model(), ++ positive=branch, ++ initial_video_rows=torch.zeros(branch.img_pos.numel(), 96), ++ initial_audio_rows=torch.zeros(branch.audio_pos.numel(), 32), ++ keyframe_cond_rows=torch.zeros(int((~branch.update_mask).sum()), 96), ++ sigmas_video=[1 - i / 50 for i in range(51)], ++ sigmas_audio=[1 - i / 50 for i in range(51)], ++ device=torch.device("cpu"), ++ ) ++ if ref2va: ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert len(states) == 1 and states[0].closed ++ assert states[0].full_steps == 17 and states[0].partial_steps == 33 ++ else: ++ with pytest.raises(ValueError, match="requires Ref2VA"): ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert not states ++ ++ ++@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) ++def test_attention_keeps_collective_path_and_projects_only_dense_local_rows(start, length): ++ state = state_for_rows(start=start, length=length) ++ x = torch.ones(length, 2, dtype=torch.bfloat16) ++ anchor(state, x) ++ attention = object.__new__(blocks.MiniMaxH3Attention) ++ nn.Module.__init__(attention) ++ attention.num_heads = 1 ++ attention.head_dim = 2 ++ attention.vdn = None ++ attention.to_gate_compress = None ++ calls, projected_rows = [], [] ++ attention._project_qkv = lambda *args: (x.reshape(length, 1, 2),) * 5 ++ ++ def packed(q, k, v, **kwargs): ++ calls.append(q.shape[0]) ++ assert kwargs["video_row_cache"] is state ++ return q ++ ++ def project(rows): ++ projected_rows.append(rows.shape[0]) ++ return rows * 2, None ++ ++ attention._run_packed_attention = packed ++ attention.out_proj = project ++ output = attention( ++ x, ++ rope_table=None, ++ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), ++ max_seqlen=3, ++ packed_total=4, ++ video_row_cache=state, ++ ) ++ rows = state.local_dense_positions ++ assert calls == [length] ++ assert projected_rows == ([rows.numel()] if rows.numel() else []) ++ assert torch.equal(output.index_select(0, rows), (x * 2).index_select(0, rows)) ++ assert torch.count_nonzero(output.index_select(0, state.local_video_positions)) == 0 ++ ++ ++def _native_worker(rank): ++ import torch_npu # noqa: F401 ++ ++ torch.npu.set_device(rank) ++ dist.init_process_group( ++ "hccl", init_method="tcp://127.0.0.1:29597", rank=rank, world_size=4, timeout=timedelta(seconds=120) ++ ) ++ try: ++ large_delta_roundtrip(device=f"npu:{rank}") ++ torch.manual_seed(19) ++ full = [torch.randn(32, 4, 128, dtype=torch.bfloat16).npu() for _ in range(3)] ++ exchanged = [] ++ for tensor in full: ++ local = tensor[rank * 8 : (rank + 1) * 8] ++ send = local.reshape(8, 4, 1, 128).permute(1, 0, 2, 3).contiguous() ++ receive = torch.empty_like(send) ++ dist.all_to_all_single(receive, send) ++ exchanged.append(receive.reshape(1, 32, 1, 128)) ++ state = MiniMaxH3VideoRowCache(torch.arange(8, 24, device="npu"), seq_len=32, used_len=30, num_layers=1) ++ state.prepare_local_span(rank * 8, 8) ++ x = torch.ones(8, 2, dtype=torch.bfloat16, device="npu") ++ anchor(state, x) ++ output = _subset_prefix_bnsd(exchanged[0], exchanged[1][:, :30], exchanged[2][:, :30], 128**-0.5, state) ++ send = output.reshape(4, 8, 1, 128).contiguous() ++ receive = torch.empty_like(send) ++ dist.all_to_all_single(receive, send) ++ local_output = receive.permute(1, 0, 2, 3).reshape(8, 4, 128) ++ expected = _prefix_bnsd(full[0].unsqueeze(0), full[1][:30].unsqueeze(0), full[2][:30].unsqueeze(0), 128**-0.5)[ ++ 0 ++ ] ++ rows = state.local_dense_positions ++ torch.testing.assert_close( ++ local_output.index_select(0, rows).float(), ++ expected[rank * 8 : (rank + 1) * 8].index_select(0, rows).float(), ++ atol=0.03125, ++ rtol=0.02, ++ ) ++ assert state.subset_attention_calls == 1 and state.subset_query_rows == 14 ++ assert rows.numel() == [8, 0, 0, 6][rank] ++ state.restore(0, x) ++ state.end_step() ++ state.close() ++ torch.npu.synchronize() ++ dist.barrier() ++ finally: ++ dist.destroy_process_group() ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires four native Ascend devices") ++def test_native_four_rank_subset_attention_with_empty_local_dense_rows(): ++ torch.multiprocessing.spawn(_native_worker, nprocs=4, join=True) +diff --git a/vllm_omni/config/omni_config.py b/vllm_omni/config/omni_config.py +index 0a803cf..44fc48c 100644 +--- a/vllm_omni/config/omni_config.py ++++ b/vllm_omni/config/omni_config.py +@@ -863,6 +863,7 @@ class _DiffusionConfigProjection(_TrackExplicitConfigFields): + dlo_use_allgather: bool = True + dlo_chunk_size_mb: int = Field(default=64, gt=0, strict=True) + dlo_resident_layers: int = Field(default=0, ge=0) ++ dlo_order_allgather_after_attn_a2a: bool = False + host_weight_runtime_mode: Literal["disabled", "preferred", "required"] = "disabled" + host_weight_runtime_root: str | None = None + dlo_host_registration_limit_gib: float = Field(default=0.0, ge=0) +diff --git a/vllm_omni/config/stage_config.py b/vllm_omni/config/stage_config.py +index 76a58a9..3c69986 100644 +--- a/vllm_omni/config/stage_config.py ++++ b/vllm_omni/config/stage_config.py +@@ -672,6 +672,7 @@ class StageDeployConfig: + dlo_use_allgather: bool | None = None + dlo_chunk_size_mb: int | None = None + dlo_resident_layers: int | None = None ++ dlo_order_allgather_after_attn_a2a: bool | None = None + host_weight_runtime_mode: str | None = None + host_weight_runtime_root: str | None = None + dlo_host_registration_limit_gib: float | None = None +diff --git a/vllm_omni/diffusion/attention/layer.py b/vllm_omni/diffusion/attention/layer.py +index 5520390..574eafa 100644 +--- a/vllm_omni/diffusion/attention/layer.py ++++ b/vllm_omni/diffusion/attention/layer.py +@@ -9,7 +9,7 @@ + + from collections.abc import Mapping + from dataclasses import replace +-from typing import cast ++from typing import Any, cast + + import torch + import torch.nn as nn +@@ -333,6 +333,11 @@ class Attention(nn.Module): + # Local strategy when SP is intentionally inactive outside sharded regions. + self._no_parallel_strategy = NoParallelAttention() + ++ # Optional cross-module rendezvous (see distributed/collective_gate.py). ++ # Installed by the DLO backend when it must keep its background weight ++ # AllGather from taking the links ahead of this layer's QKV AllToAll. ++ self._collective_gate: Any | None = None ++ + self.layer_idx: int | None = _try_extract_layer_index(prefix) + + self._kv_cache_dtype: str | None = None +@@ -506,10 +511,12 @@ class Attention(nn.Module): + reason = self._cross_backend_limitation(attn_metadata) + if reason is not None: + blocked = self._cross_fallback_blocker(attn_metadata) +- result = self.sdpa_fallback.resolve_execution_path(resolved_context, query, key, value, attn_metadata) ++ result = cast(SDPAImpl, self.sdpa_fallback).resolve_execution_path( ++ resolved_context, query, key, value, attn_metadata ++ ) + return replace( + result, +- backend=self.attn_backend.get_name() if blocked else "SDPA", ++ backend=cast(type[AttentionBackend], self.attn_backend).get_name() if blocked else "SDPA", + path="cross_unsupported" if blocked else "cross_sdpa_fallback", + support=CapabilityResult.unsupported(blocked) if blocked else CapabilityResult.unmigrated(reason), + ) +@@ -591,6 +598,12 @@ class Attention(nn.Module): + # For Ring: Concat joint_q + query, key, value, attn_metadata, ctx = strategy.pre_attention(query, key, value, attn_metadata) + ++ # The QKV AllToAll is now enqueued on this stream. Release any gather ++ # that was deferred to give this collective the links first. ++ gate = self._collective_gate ++ if gate is not None: ++ gate.signal() ++ + # MiniMax-H3 Tier2 stores reference K/V in the balanced layout produced + # by Ulysses. Keeping this hook at the generic post-parallel boundary + # avoids a model import in the parallel strategy and leaves other models +@@ -760,7 +773,7 @@ class Attention(nn.Module): + def _cross_fallback_blocker(self, attn_metadata: AttentionMetadata | None) -> str | None: + if self.backend_explicit: + return ( +- f"Explicit backend {self.attn_backend.get_name()} does not support " ++ f"Explicit backend {cast(type[AttentionBackend], self.attn_backend).get_name()} does not support " + f"{self._cross_backend_limitation(attn_metadata)}" + ) + if attn_metadata is not None and attn_metadata.extra.get("kv_cache_dtype") not in (None, "float"): +@@ -793,7 +806,7 @@ class Attention(nn.Module): + blocked = self._cross_fallback_blocker(attn_metadata) + if blocked is not None: + raise NotImplementedError(blocked) +- return self.sdpa_fallback.forward(query, key, value, attn_metadata) ++ return cast(SDPAImpl, self.sdpa_fallback).forward(query, key, value, attn_metadata) + dense_mask = attn_metadata.attn_mask if attn_metadata is not None else None + # Piecewise backends honor full_attn_spans ahead of attn_mask and never read the + # dense mask (e.g. HunyuanImage3, which sends both), so there is nothing to divert. +diff --git a/vllm_omni/diffusion/data.py b/vllm_omni/diffusion/data.py +index 8528574..fcbeec0 100644 +--- a/vllm_omni/diffusion/data.py ++++ b/vllm_omni/diffusion/data.py +@@ -962,6 +962,10 @@ class OmniDiffusionConfig: + dlo_chunk_size_mb: int = 64 + # Leading main-DiT blocks kept resident by distributed layerwise offload. + dlo_resident_layers: int = 0 ++ # Hold each DiT block's weight AllGather until that block's QKV AllToAll has ++ # been enqueued, so the background collective does not take the links ahead ++ # of the critical-path all-to-all. ++ dlo_order_allgather_after_attn_a2a: bool = False + # Final-layout Host Weight Runtime policy. The loader only activates this + # for eligible no-AllGather DLO; all other configurations preserve their + # existing loader/storage path. +diff --git a/vllm_omni/diffusion/distributed/collective_gate.py b/vllm_omni/diffusion/distributed/collective_gate.py +new file mode 100644 +index 0000000..1386d89 +--- /dev/null ++++ b/vllm_omni/diffusion/distributed/collective_gate.py +@@ -0,0 +1,72 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++"""Cross-module rendezvous that orders background collectives after the ++foreground all-to-all of a transformer block. ++ ++Distributed layerwise offload (DLO) issues the *next* block's weight AllGather ++on a dedicated ``comm_stream`` while the *current* block computes. Both that ++AllGather and the block's QKV sequence-parallel AllToAll are collectives over ++the same ranks, so whichever reaches the links first wins: the AllGather is ++submitted at block entry, well before the block reaches its attention, and the ++critical-path AllToAll ends up queued behind it. ++ ++A plain event wait cannot fix this by itself. ``wait_event`` snapshots the ++records enqueued *before* the call, so "enqueue the wait at block entry, record ++the event later in the block" is a no-op -- the wait resolves against the ++previous iteration's record. The gather therefore has to be *issued* after the ++AllToAll: the hook arms this barrier at block entry and defers the gather, the ++attention signals it right after ``strategy.pre_attention``, and the signal ++records the event and lets the hook submit the gather gated on it. ++ ++The gather still overlaps the rest of the block (attention output, MLP, the ++next block's projections); it simply cannot start before the AllToAll finished. ++""" ++ ++from __future__ import annotations ++ ++from collections.abc import Callable ++ ++__all__ = ["QKVAllToAllBarrier"] ++ ++ ++class QKVAllToAllBarrier: ++ """One-block rendezvous between a DLO hook and the block's attention. ++ ++ Owned by a single :class:`DistributedLayerwiseOffloadHook` (one per ++ streaming block) and shared with the attention layer inside that block. ++ Both live on the same block, so the arming and the signalling always come ++ from the same module forward. ++ ++ Args: ++ on_signal: Invoked at most once per :meth:`arm` when the block's ++ attention reports its QKV AllToAll. It must submit the deferred ++ weight AllGather. ++ """ ++ ++ def __init__(self, on_signal: Callable[[], None]) -> None: ++ self._on_signal = on_signal ++ self._armed = False ++ ++ @property ++ def armed(self) -> bool: ++ """True between :meth:`arm` and the matching :meth:`signal`.""" ++ return self._armed ++ ++ def arm(self) -> None: ++ """Mark the next attention QKV AllToAll as the gather's gate.""" ++ self._armed = True ++ ++ def disarm(self) -> None: ++ """Drop a pending arm without firing it (the caller flushes instead).""" ++ self._armed = False ++ ++ def signal(self) -> None: ++ """Report that the QKV AllToAll is now enqueued on the compute stream. ++ ++ A no-op unless armed, so it is cheap to call unconditionally from the ++ attention hot path. ++ """ ++ if not self._armed: ++ return ++ self._armed = False ++ self._on_signal() +diff --git a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py +index 6ebfb10..044c1f0 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py ++++ b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py +@@ -36,6 +36,16 @@ def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, sc + return output.reshape(batch, heads, rows, dim).transpose(1, 2).contiguous() + + ++def _subset_prefix_bnsd(query, key, value, scale, state): ++ """Compute only fresh conditioning queries; preserve the shared SP shape.""" ++ rows = state.global_dense_positions ++ dense = _prefix_bnsd(query.index_select(1, rows), key, value, scale) ++ output = torch.zeros_like(query) ++ output.index_copy_(1, rows, dense) ++ state.record_subset_attention(rows.numel()) ++ return output ++ ++ + class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + """Use independent head batches for explicit long BF16 prefix attention. + +@@ -51,7 +61,10 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + value: torch.Tensor, + attn_metadata: AttentionMetadata | None = None, + ) -> torch.Tensor: ++ state = attn_metadata.extra.get("minimax_h3_video_row_cache") if attn_metadata is not None else None + if os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" or attn_metadata is None: ++ if state is not None: ++ raise RuntimeError("H3 video-row cache requires the explicit fused attention path") + return super().forward_fa_npu(query, key, value, attn_metadata) + extra = attn_metadata.extra + if ( +@@ -68,9 +81,17 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + or query.shape[2:] != value.shape[2:] + or key.shape[1] != value.shape[1] + ): ++ if state is not None: ++ raise RuntimeError("H3 video-row cache received unsupported fused attention inputs") + return super().forward_fa_npu(query, key, value, attn_metadata) + resolved = self._resolve_packed_seq_npu(query, key, extra) + if resolved is None: ++ if state is not None: ++ raise RuntimeError("H3 video-row cache requires a resolved packed prefix") + return super().forward_fa_npu(query, key, value, attn_metadata) + used_k = resolved[1][0] ++ if state is not None: ++ if not state.partial or used_k != state.used_len or query.shape[1] != state.seq_len: ++ raise RuntimeError("H3 video-row cache attention layout changed") ++ return _subset_prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale, state) + return _prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale) +diff --git a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py +index 8cca219..eef175b 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py ++++ b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py +@@ -9,6 +9,7 @@ rows stay pinned to their noised step-0 anchors. + + from __future__ import annotations + ++import os + from collections.abc import Callable + from contextlib import AbstractContextManager, nullcontext + from typing import Any +@@ -69,7 +70,9 @@ class MiniMaxH3DenoiseBranch: + text_embeddings: torch.Tensor, + token_tags: torch.Tensor, + device: torch.device, ++ task: str | None = None, + ) -> None: ++ self.task = task + seq_len = int(packed["seq_len"]) + self.seq_len = seq_len + self.img_pos = packed["img_pos"].view(-1).to(torch.long) +@@ -380,6 +383,24 @@ def minimax_h3_denoise_loop( + positive.static_kwargs["reference_kv_state"] = reference_kv_state + + try: ++ video_row_cache = None ++ create_video_cache = getattr(model, "create_video_row_cache", None) ++ if os.environ.get("H3_VIDEO_ROW_CACHE") == "1": ++ if ( ++ not callable(create_video_cache) ++ or video_edit is not None ++ or audio_edit is not None ++ or keyframe_cond_rows is None ++ or positive.task != "ref2va" ++ ): ++ raise ValueError("H3 video-row cache requires Ref2VA image references without editing") ++ video_row_cache = create_video_cache( ++ positive.img_pos[positive.update_mask].to(device), ++ seq_len=positive.seq_len, ++ used_len=positive.used_len, ++ num_steps=num_steps, ++ ) ++ positive.static_kwargs["video_row_cache"] = video_row_cache + for step in range(num_steps): + check_request_cancellation() + step_cm = step_profiler(step) if step_profiler is not None else nullcontext() +@@ -388,6 +409,8 @@ def minimax_h3_denoise_loop( + s_a = sigmas_audio[step] + if reference_kv_state is not None: + reference_kv_state.begin_step(step, sigma=float(s_v)) ++ if video_row_cache is not None: ++ video_row_cache.begin_step(step, sigma=float(s_v)) + # Publish where we are so step-gated attention features (the dense + # warmup of RAINFUSION_ATTN, the timestep gate of TRTLLM_ATTN) can + # see it. Gates use the scheduler-style descending timestep, which +@@ -430,6 +453,8 @@ def minimax_h3_denoise_loop( + v_video, v_audio = model(**fk) + if reference_kv_state is not None: + reference_kv_state.end_step() ++ if video_row_cache is not None: ++ video_row_cache.end_step() + mv_video_t = v_video.float()[update] + mv_audio_t = v_audio.float()[audio_update] + +@@ -476,6 +501,10 @@ def minimax_h3_denoise_loop( + + finally: + minimax_h3_publish_denoise_progress(None, None, None) ++ if video_row_cache is not None: ++ positive.static_kwargs.pop("video_row_cache", None) ++ video_row_cache.close() ++ model._video_row_cache_stats = video_row_cache.stats() + if reference_kv_state is not None: + positive.static_kwargs.pop("reference_kv_state", None) + reference_kv_state.close() +diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py +index 09f2e7d..601e2ae 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py ++++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py +@@ -45,6 +45,7 @@ if TYPE_CHECKING: + from vllm.model_executor.layers.quantization.base_config import QuantizationConfig + + from .reference_kv_cache import MiniMaxH3ReferenceKVTier2State ++ from .video_row_cache import MiniMaxH3VideoRowCache + + _BF16_DTYPE = torch.bfloat16 + _FP32_DTYPE = torch.float32 +@@ -291,6 +292,7 @@ class MiniMaxH3Attention(nn.Module): + reference_kv_layer_index: int | None = None, + reference_kv_compact: bool = False, + vdn_window: VDNLayout | None = None, ++ video_row_cache: MiniMaxH3VideoRowCache | None = None, + ) -> torch.Tensor: + """Run packed attention as a small eager island. + +@@ -368,6 +370,8 @@ class MiniMaxH3Attention(nn.Module): + "vsa_h3_ref_policy": self.vsa_ref_policy, + "vsa_h3_ref_keep_rate": self.vsa_ref_keep_rate, + } ++ if video_row_cache is not None and video_row_cache.partial: ++ extra["minimax_h3_video_row_cache"] = video_row_cache + if gate_compress is not None: + extra["gate_compress"] = gate_compress.unsqueeze(0) + if video_layout is not None and video_layout.video_spans: +@@ -455,6 +459,7 @@ class MiniMaxH3Attention(nn.Module): + reference_kv_layer_index: int | None = None, + reference_kv_compact: bool = False, + vdn_window: VDNLayout | None = None, ++ video_row_cache: MiniMaxH3VideoRowCache | None = None, + ) -> torch.Tensor: + """x: [T, hidden] packed thd rows -> [T, hidden]. + +@@ -508,10 +513,19 @@ class MiniMaxH3Attention(nn.Module): + reference_kv_layer_index=reference_kv_layer_index, + reference_kv_compact=reference_kv_compact, + vdn_window=vdn_window, ++ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), + ) + if self.vdn is not None: + out = self.vdn.gate_softmax(out, x) + out = out.reshape(total, self.num_heads * self.head_dim) ++ if video_row_cache is not None and video_row_cache.partial: ++ rows = video_row_cache.local_dense_positions ++ assert rows is not None ++ projected = x.new_zeros((total, x.shape[-1])) ++ if rows.numel(): ++ dense, _ = self.out_proj(out.index_select(0, rows)) ++ projected.index_copy_(0, rows, dense) ++ return projected + out, _ = self.out_proj(out) + if self.vdn is not None: + out = self.vdn.add_linear(out, x, q_raw, k_raw, v, vdn_window) +@@ -742,6 +756,7 @@ class MiniMaxH3DiTBlock(nn.Module): + reference_kv_state: MiniMaxH3ReferenceKVTier2State | None = None, + reference_kv_compact: bool = False, + vdn_window: VDNLayout | None = None, ++ video_row_cache: MiniMaxH3VideoRowCache | None = None, + ) -> torch.Tensor: + """x: [T, H]; t_emb: [M, t_dim]; combined_indices: [T] + (= inverse_indices * modality_num + token_tags.clamp(min=0)). +@@ -759,6 +774,7 @@ class MiniMaxH3DiTBlock(nn.Module): + gate_mlp, + ) = self.adaln_proj(t_emb) + ++ block_input = x + residual = x + h = rms_norm_indexed_scale_shift( + x, +@@ -782,7 +798,10 @@ class MiniMaxH3DiTBlock(nn.Module): + reference_kv_layer_index=self.layer_index, + reference_kv_compact=reference_kv_compact, + vdn_window=vdn_window, ++ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), + ) ++ if video_row_cache is not None and video_row_cache.pre_gate: ++ h = video_row_cache.pre_gate_output(self.layer_index, h, "attn") + x, h = indexed_gate_rms_norm_scale_shift( + residual, + gate_msa, +@@ -793,6 +812,36 @@ class MiniMaxH3DiTBlock(nn.Module): + combined_indices, + self.norm2.variance_epsilon, + ) ++ if video_row_cache is not None and video_row_cache.pre_gate and video_row_cache.partial: ++ video_row_cache.record_fresh_gate("attn") + residual = x ++ if video_row_cache is not None and video_row_cache.partial: ++ rows = video_row_cache.local_dense_positions ++ assert rows is not None ++ if video_row_cache.pre_gate: ++ mlp_output = torch.zeros_like(h) ++ if rows.numel(): ++ mlp_output.index_copy_(0, rows, self.mlp(h.index_select(0, rows))) ++ mlp_output = video_row_cache.pre_gate_output(self.layer_index, mlp_output, "mlp") ++ output = indexed_gate(residual, gate_mlp, mlp_output, combined_indices) ++ video_row_cache.record_fresh_gate("mlp") ++ video_row_cache.partial_mlp_rows += rows.numel() ++ return output ++ output = video_row_cache.restore(self.layer_index, block_input) ++ if rows.numel(): ++ dense = indexed_gate( ++ residual.index_select(0, rows), ++ gate_mlp, ++ self.mlp(h.index_select(0, rows)), ++ combined_indices.index_select(0, rows), ++ ) ++ output.index_copy_(0, rows, dense) ++ video_row_cache.partial_mlp_rows += rows.numel() ++ return output + h = self.mlp(h) +- return indexed_gate(residual, gate_mlp, h, combined_indices) ++ if video_row_cache is not None and video_row_cache.pre_gate: ++ h = video_row_cache.pre_gate_output(self.layer_index, h, "mlp") ++ output = indexed_gate(residual, gate_mlp, h, combined_indices) ++ if video_row_cache is not None and not video_row_cache.pre_gate: ++ video_row_cache.capture(self.layer_index, block_input, output) ++ return output +diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py +index 9518828..3953f86 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py ++++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py +@@ -10,6 +10,7 @@ layout. + from __future__ import annotations + + import math ++import os + from bisect import bisect_left + from collections.abc import Iterable, Mapping + from dataclasses import replace +@@ -66,6 +67,8 @@ if TYPE_CHECKING: + from vllm_omni.diffusion.data import OmniDiffusionConfig + from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache import MiniMaxH3ReferenceKVTier2State + ++ from .video_row_cache import MiniMaxH3VideoRowCache ++ + logger = init_logger(__name__) + + +@@ -245,6 +248,7 @@ _FORWARD_SUPPORTED_KWARGS = frozenset( + "control_rows", + "control_context_scale", + "reference_kv_state", ++ "video_row_cache", + } + ) + +@@ -1232,6 +1236,42 @@ class MiniMaxH3DiTModel(nn.Module): + t_emb = self.time_embedder(unique_timesteps) + return embeddings, t_emb + ++ def create_video_row_cache( ++ self, target_positions: torch.Tensor, *, seq_len: int, used_len: int, num_steps: int ++ ) -> MiniMaxH3VideoRowCache | None: ++ if os.environ.get("H3_VIDEO_ROW_CACHE") != "1": ++ return None ++ p = self.parallel_config ++ if ( ++ not current_omni_platform.is_npu() ++ or num_steps != 50 ++ or p.tensor_parallel_size != 1 ++ or p.data_parallel_size not in (None, 1) ++ or p.cfg_parallel_size != 1 ++ or p.pipeline_parallel_size != 1 ++ or p.ulysses_degree != 4 ++ or p.ring_degree != 1 ++ or p.allgather_degree != 1 ++ or p.ulysses_mode != "strict" ++ or self.od_config.cache_backend not in (None, "none") ++ or self.od_config.kv_offload_config is not None ++ or self.vdn_config is not None ++ or os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" ++ or any(block.attn.attention.attn_backend.get_name() != "FLASH_ATTN" for block in self.blocks) ++ ): ++ raise ValueError("H3 video-row cache requires dense NPU BF16 Ref2VA with strict Ulysses4, TP/DP/CFG/PP1") ++ from .video_row_cache import MiniMaxH3VideoRowCache ++ ++ self._video_row_cache_stats = None ++ return MiniMaxH3VideoRowCache( ++ target_positions, ++ seq_len=seq_len, ++ used_len=used_len, ++ num_layers=len(self.blocks), ++ pre_gate=os.environ.get("H3_VIDEO_PRE_GATE_CACHE") == "1", ++ secant=os.environ.get("H3_VIDEO_PRE_GATE_SECANT") == "1", ++ ) ++ + def create_reference_kv_state( + self, + *, +@@ -1308,6 +1348,14 @@ class MiniMaxH3DiTModel(nn.Module): + refiner_max = int(self._psp_field(refiner_psp, "refiner_packed_seq_params", "max_seqlen_q")) + video_layout = kwargs.get("video_token_layout") + reference_kv_state = kwargs.get("reference_kv_state") ++ video_row_cache = kwargs.get("video_row_cache") ++ if video_row_cache is not None and ( ++ num_requests != 1 ++ or reference_kv_state is not None ++ or self.vdn_config is not None ++ or kwargs.get("control_rows") is not None ++ ): ++ raise ValueError("H3 video-row cache cannot combine with packed requests, reference KV, VDN or control") + vdn_window = None + if self.vdn_config is not None: + # Text rows lead the packed document. +@@ -1442,6 +1490,11 @@ class MiniMaxH3DiTModel(nn.Module): + if reference_kv_state is not None: + reference_kv_state.set_local_sequence_rows(hidden.shape[0]) + ++ if video_row_cache is not None: ++ if hidden.shape[0] != local_len: ++ raise RuntimeError("H3 video-row cache requires the strict local sequence shard") ++ video_row_cache.prepare_local_span(local_start, local_len) ++ + block_video_layout = video_layout + if compact_reference: + block_video_layout = None +@@ -1505,6 +1558,7 @@ class MiniMaxH3DiTModel(nn.Module): + reference_kv_state=reference_kv_state, + reference_kv_compact=compact_reference, + vdn_window=vdn_window, ++ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), + ) + if block_index in hints: + hidden = hidden + hints.pop(block_index) * control_scale +diff --git a/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py b/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py +index 476c3cc..ebce8fc 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py ++++ b/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py +@@ -2115,6 +2115,7 @@ class MiniMaxH3Pipeline( + text_embeddings=text_embeddings, + token_tags=tags, + device=self.device, ++ task=task, + ) + + if control_rows is not None: +diff --git a/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py b/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py +new file mode 100644 +index 0000000..19a550d +--- /dev/null ++++ b/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py +@@ -0,0 +1,291 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++"""Request-owned, experimental target-video block residuals for H3 Ref2VA.""" ++ ++from __future__ import annotations ++ ++import math ++ ++import torch ++ ++ ++class MiniMaxH3VideoRowCache: ++ """Keep non-video computation dense while reusing video block updates. ++ ++ The first experimental profile is deliberately fixed. No state is shared ++ across requests, and every rank retains all attention collectives. ++ """ ++ ++ def __init__( ++ self, ++ target_positions: torch.Tensor, ++ *, ++ seq_len: int, ++ used_len: int, ++ num_layers: int, ++ pre_gate: bool = False, ++ secant: bool = False, ++ ) -> None: ++ if secant and not pre_gate: ++ raise ValueError("sigma secants require pre-gate outputs") ++ positions = target_positions.detach().cpu().tolist() ++ if not 0 < used_len <= seq_len or not positions or num_layers < 1: ++ raise ValueError("video-row cache requires a nonempty valid packed document") ++ if positions != sorted(set(positions)) or positions[0] < 0 or positions[-1] >= used_len: ++ raise ValueError("target-video positions must be sorted unique valid rows") ++ target_set = set(positions) ++ dense = [i for i in range(used_len) if i not in target_set] ++ if not dense: ++ raise ValueError("video-row cache requires non-video conditioning rows") ++ self.seq_len = seq_len ++ self.used_len = used_len ++ self.num_layers = num_layers ++ self._target_positions = positions ++ self.global_dense_positions = torch.tensor(dense, dtype=torch.long, device=target_positions.device) ++ self.local_dense_positions: torch.Tensor | None = None ++ self.local_video_positions: torch.Tensor | None = None ++ self._local_span: tuple[int, int] | None = None ++ self._deltas: dict[int, torch.Tensor] = {} ++ self.pre_gate = pre_gate ++ self.secant = secant ++ self._sigma: float | None = None ++ self._anchor_sigmas: dict[tuple[str, int], float] = {} ++ self._slope_intervals: dict[tuple[str, int], float] = {} ++ self._slopes: dict[tuple[str, int], torch.Tensor] = {} ++ self._slope_scales: dict[tuple[str, int], torch.Tensor] = {} ++ self.secant_forecasts = 0 ++ self._attention_outputs: dict[int, torch.Tensor] = {} ++ self._mlp_outputs: dict[int, torch.Tensor] = {} ++ self._awaiting_mlp_layer: int | None = None ++ self.fresh_attention_gates = 0 ++ self.fresh_mlp_gates = 0 ++ self._step = -1 ++ self._blocks_this_step = 0 ++ self.partial = False ++ self.closed = False ++ self.full_steps = 0 ++ self.partial_steps = 0 ++ self.full_blocks = 0 ++ self.partial_blocks = 0 ++ self.subset_attention_calls = 0 ++ self.subset_query_rows = 0 ++ self.omitted_query_rows = 0 ++ self.partial_mlp_rows = 0 ++ self.peak_cache_bytes = 0 ++ ++ @staticmethod ++ def is_full_step(step: int) -> bool: ++ if not 0 <= step < 50: ++ raise ValueError("the video-row cache profile requires 50 steps") ++ return step < 4 or step >= 46 or (step - 4) % 5 == 0 ++ ++ def begin_step(self, step: int, *, sigma: float | None = None) -> None: ++ if self.closed or step != self._step + 1: ++ raise RuntimeError("video-row cache step order changed") ++ if self._step >= 0 and self._blocks_this_step != self.num_layers: ++ raise RuntimeError("previous video-row cache step is incomplete") ++ full = self.is_full_step(step) ++ if self.pre_gate: ++ full = step in (0, 1, 4, 9, 19, 29, 39, 49) ++ self.partial = not full ++ if self._awaiting_mlp_layer is not None: ++ raise RuntimeError("previous video pre-gate sublayer is incomplete") ++ complete = ( ++ len(self._attention_outputs) == len(self._mlp_outputs) == self.num_layers ++ if self.pre_gate ++ else len(self._deltas) == self.num_layers ++ ) ++ if self.partial and not complete: ++ raise RuntimeError("partial video step has no complete dense anchor") ++ if self.secant: ++ if sigma is None or not math.isfinite(sigma) or not 0 <= sigma <= 1: ++ raise ValueError("video secant requires a finite solver sigma") ++ if self._sigma is not None and sigma >= self._sigma: ++ raise ValueError("video secant requires strictly decreasing solver sigma") ++ if self.partial and len(self._slopes) != 2 * self.num_layers: ++ raise RuntimeError("video secant requires two complete dense anchors") ++ self._sigma = float(sigma) ++ self._step = step ++ self._blocks_this_step = 0 ++ if self.partial: ++ self.partial_steps += 1 ++ else: ++ self.full_steps += 1 ++ ++ def prepare_local_span(self, start: int, length: int) -> None: ++ if self.closed or start < 0 or length < 1 or start + length > self.seq_len: ++ raise ValueError("invalid local packed span") ++ if self._local_span is not None: ++ if self._local_span != (start, length): ++ raise RuntimeError("video-row cache local ownership changed") ++ return ++ device = self.global_dense_positions.device ++ target = set(self._target_positions) ++ self.local_video_positions = torch.tensor( ++ [i - start for i in self._target_positions if start <= i < start + length], dtype=torch.long, device=device ++ ) ++ self.local_dense_positions = torch.tensor( ++ [i - start for i in range(start, min(start + length, self.used_len)) if i not in target], ++ dtype=torch.long, ++ device=device, ++ ) ++ self._local_span = (start, length) ++ ++ def _visit(self, layer: int, x: torch.Tensor) -> None: ++ if self.closed or self._step < 0 or layer != self._blocks_this_step: ++ raise RuntimeError("video-row cache block order changed") ++ if self._local_span is None or x.shape[0] != self._local_span[1] or x.dtype != torch.bfloat16: ++ raise ValueError("video-row cache requires the prepared BF16 row shard") ++ self._blocks_this_step += 1 ++ ++ def capture(self, layer: int, x: torch.Tensor, output: torch.Tensor) -> None: ++ if self.pre_gate: ++ raise RuntimeError("pre-gate state cannot capture a gated block residual") ++ if self.partial: ++ raise RuntimeError("cannot replace the dense anchor during a partial step") ++ self._visit(layer, x) ++ rows = self.local_video_positions ++ assert rows is not None ++ # BF16 endpoints may have a finite difference outside FP16's range. ++ # Keep that difference in FP32; only the reconstructed block output is ++ # rounded to the original BF16 activation dtype. ++ self._deltas[layer] = output.index_select(0, rows).float() - x.index_select(0, rows).float() ++ self.peak_cache_bytes = max( ++ self.peak_cache_bytes, sum(t.numel() * t.element_size() for t in self._deltas.values()) ++ ) ++ self.full_blocks += 1 ++ ++ def restore(self, layer: int, x: torch.Tensor) -> torch.Tensor: ++ if self.pre_gate: ++ raise RuntimeError("pre-gate state cannot restore a gated block residual") ++ if not self.partial: ++ raise RuntimeError("cannot reuse a video residual on a full step") ++ self._visit(layer, x) ++ rows = self.local_video_positions ++ assert rows is not None ++ # This copy owns the returned full shard, preserving caller input and ++ # unused padding while fresh non-video rows are filled by the block. ++ output = x.clone() ++ if rows.numel(): ++ output.index_copy_(0, rows, (x.index_select(0, rows).float() + self._deltas[layer].float()).to(x.dtype)) ++ self.partial_blocks += 1 ++ return output ++ ++ def _cache_bytes(self) -> int: ++ return sum( ++ tensor.numel() * tensor.element_size() ++ for cache in (self._deltas, self._attention_outputs, self._mlp_outputs, self._slopes, self._slope_scales) ++ for tensor in cache.values() ++ ) ++ ++ def pre_gate_output(self, layer: int, output: torch.Tensor, stage: str) -> torch.Tensor: ++ """Capture or fill only target rows before applying the current gate.""" ++ if not self.pre_gate or stage not in ("attn", "mlp"): ++ raise ValueError("pre-gate output requires the selected attention or MLP stage") ++ if stage == "attn": ++ if self._awaiting_mlp_layer is not None: ++ raise RuntimeError("previous video pre-gate sublayer is incomplete") ++ self._visit(layer, output) ++ self._awaiting_mlp_layer = layer ++ if self.partial: ++ self.partial_blocks += 1 ++ else: ++ self.full_blocks += 1 ++ else: ++ if self.closed or self._awaiting_mlp_layer != layer: ++ raise RuntimeError("video pre-gate MLP order changed") ++ if output.dtype != torch.bfloat16 or output.shape[0] != self._local_span[1]: ++ raise ValueError("video pre-gate output must retain the BF16 local shard") ++ self._awaiting_mlp_layer = None ++ rows = self.local_video_positions ++ assert rows is not None ++ cache = self._attention_outputs if stage == "attn" else self._mlp_outputs ++ key = (stage, layer) ++ if self.partial: ++ restored = output.clone() ++ if self.secant: ++ ratio = (self._sigma - self._anchor_sigmas[key]) / self._slope_intervals[key] ++ predicted = cache[layer].float() + ratio * self._slopes[key].float() * self._slope_scales[key] ++ restored.index_copy_(0, rows, predicted.to(output.dtype)) ++ self.secant_forecasts += 1 ++ else: ++ restored.index_copy_(0, rows, cache[layer]) ++ return restored ++ anchor = output.index_select(0, rows) ++ if self.secant: ++ if layer in cache: ++ delta = anchor.float() - cache[layer].float() ++ if delta.shape[0]: ++ maximum = delta.abs().amax(dim=0, keepdim=True) ++ scale = torch.where(maximum > 0, maximum / 127, torch.ones_like(maximum)) ++ else: ++ scale = torch.ones((1, anchor.shape[1]), dtype=torch.float32, device=anchor.device) ++ self._slopes[key] = (delta / scale).round().clamp(-127, 127).to(torch.int8) ++ self._slope_scales[key] = scale ++ self._slope_intervals[key] = self._sigma - self._anchor_sigmas[key] ++ self._anchor_sigmas[key] = self._sigma ++ cache[layer] = anchor ++ self.peak_cache_bytes = max(self.peak_cache_bytes, self._cache_bytes()) ++ return output ++ ++ def record_fresh_gate(self, stage: str) -> None: ++ if not self.pre_gate or not self.partial or stage not in ("attn", "mlp"): ++ raise RuntimeError("unexpected fresh gate record") ++ if stage == "attn": ++ self.fresh_attention_gates += 1 ++ else: ++ self.fresh_mlp_gates += 1 ++ ++ def record_subset_attention(self, rows: int) -> None: ++ if not self.partial or rows != self.global_dense_positions.numel(): ++ raise RuntimeError("unexpected subset attention query count") ++ self.subset_attention_calls += 1 ++ self.subset_query_rows += rows ++ self.omitted_query_rows += len(self._target_positions) ++ ++ def end_step(self) -> None: ++ if self._blocks_this_step != self.num_layers or self._awaiting_mlp_layer is not None: ++ raise RuntimeError("video-row cache step did not visit every main block") ++ ++ def stats(self) -> dict[str, int | bool | str]: ++ counters: dict[str, int | bool | str] = { ++ name: getattr(self, name) ++ for name in [ ++ "full_steps", ++ "partial_steps", ++ "full_blocks", ++ "partial_blocks", ++ "subset_attention_calls", ++ "subset_query_rows", ++ "omitted_query_rows", ++ "partial_mlp_rows", ++ "peak_cache_bytes", ++ "closed", ++ ] ++ } ++ counters["storage_dtype"] = "torch.bfloat16" if self.pre_gate else "torch.float32" ++ counters["cache_stage"] = ( ++ "pre_gate_secant" if self.secant else "pre_gate" if self.pre_gate else "block_residual" ++ ) ++ counters["secant_forecasts"] = self.secant_forecasts ++ counters["slope_codec"] = "int8_per_channel" if self.secant else "none" ++ counters["fresh_attention_gates"] = self.fresh_attention_gates ++ counters["fresh_mlp_gates"] = self.fresh_mlp_gates ++ counters["live_cache_bytes"] = self._cache_bytes() ++ return counters ++ ++ def close(self) -> None: ++ self._deltas.clear() ++ self._attention_outputs.clear() ++ self._mlp_outputs.clear() ++ self._slopes.clear() ++ self._slope_scales.clear() ++ self._anchor_sigmas.clear() ++ self._slope_intervals.clear() ++ self._sigma = None ++ self._awaiting_mlp_layer = None ++ self._target_positions.clear() ++ self.global_dense_positions = torch.empty(0, dtype=torch.long) ++ self.local_dense_positions = None ++ self.local_video_positions = None ++ self.closed = True +diff --git a/vllm_omni/diffusion/offloader/base.py b/vllm_omni/diffusion/offloader/base.py +index 2505d48..50b73e5 100644 +--- a/vllm_omni/diffusion/offloader/base.py ++++ b/vllm_omni/diffusion/offloader/base.py +@@ -71,6 +71,10 @@ class OffloadConfig: + dlo_use_allgather: bool = True + dlo_resident_layers: int = 0 # leading DiT layers kept on device + chunk_size_bytes: int = 64 * 1024 * 1024 ++ # Hold each DiT block's weight AllGather until that block's QKV AllToAll has ++ # been enqueued, so the background collective does not take the links ahead ++ # of the critical-path all-to-all. See ``collective_gate.py``. ++ dlo_order_allgather_after_attn_a2a: bool = False + # Optional per-worker ceiling for registering an HWR mmap. Zero means no + # additional ceiling; pin_cpu_memory controls whether registration is tried. + dlo_host_registration_limit_gib: float = 0.0 +@@ -216,6 +220,7 @@ class OffloadConfig: + dlo_use_allgather=dit_uses_allgather, + dlo_resident_layers=dlo_resident_layers, + chunk_size_bytes=chunk_size_bytes, ++ dlo_order_allgather_after_attn_a2a=bool(getattr(od_config, "dlo_order_allgather_after_attn_a2a", False)), + dlo_host_registration_limit_gib=dlo_host_registration_limit_gib, + components=components, + dlo_transfers=dlo_transfers, +diff --git a/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py b/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py +index 7f0f081..8c4f87f 100644 +--- a/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py ++++ b/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py +@@ -31,6 +31,7 @@ import torch.distributed + from torch import nn + from vllm.logger import init_logger + ++from vllm_omni.diffusion.distributed.collective_gate import QKVAllToAllBarrier + from vllm_omni.diffusion.hooks import HookRegistry, ModelHook + from vllm_omni.diffusion.model_loader.host_weight_plan import ( + HostWeightPlan, +@@ -142,6 +143,7 @@ class DistributedLayerwiseOffloadHook(ModelHook): + tensor_transforms: dict[int, Any] | None = None, + materialization_probe_tensor: torch.Tensor | None = None, + chunk_size_bytes: int = 64 * 1024 * 1024, ++ order_allgather_after_attn_a2a: bool = False, + ): + assert isinstance(next_block, nn.Module), "transformer block must be type `torch.nn.Module`" + if type(dp_size) is not int or dp_size < 1: +@@ -167,6 +169,7 @@ class DistributedLayerwiseOffloadHook(ModelHook): + + self.chunk_size_bytes = chunk_size_bytes + self.manifest: PartManifest | None = None ++ self._deferred_chunked_allgather = None + self._h2d_done_events = [current_omni_platform.Event(), current_omni_platform.Event()] + self._ag_done_events = [current_omni_platform.Event(), current_omni_platform.Event()] + # Hook-local events protect chunk-input reuse within this hook. Across +@@ -239,6 +242,19 @@ class DistributedLayerwiseOffloadHook(ModelHook): + # Per-block synchronization primitive: set after H2D copy completes. + self._prefetch_done: Any | None = None + ++ # Optional ordering: hold the background weight AllGather until the ++ # block's QKV AllToAll is on the compute stream. See ++ # distributed/collective_gate.py. Off by default; the attention layer ++ # inside this block is handed the same barrier at install time. ++ self._a2a_barrier: QKVAllToAllBarrier | None = None ++ self._a2a_event: Any | None = None ++ # ``(slot, gpu_shards, gpu_weights, evt)`` of a gather armed but not yet ++ # issued, so ``post_forward`` can flush it if the attention never ran. ++ self._deferred_allgather: tuple[int, dict, dict, Any] | None = None ++ if order_allgather_after_attn_a2a: ++ self._a2a_event = current_omni_platform.Event() ++ self._a2a_barrier = QKVAllToAllBarrier(self._allgather_after_attn_a2a) ++ + # Shared shard (AllGather input) buffers — assigned by backend. + self.gpu_shard_buffers: list[dict[torch.dtype, torch.Tensor] | None] = [None, None] + +@@ -653,49 +669,23 @@ class DistributedLayerwiseOffloadHook(ModelHook): + gpu_shard = shard_bufs[dtype][: cpu_shard.numel()] + gpu_shard.copy_(cpu_shard, non_blocking=non_blocking) + gpu_shards[dtype] = gpu_shard +- self.comm_stream.wait_stream(self.copy_stream) +- with current_omni_platform.stream(self.comm_stream): +- for dtype, local_shard in gpu_shards.items(): +- gw = gpu_weights[dtype][: self._ag_output_sizes[dtype]] +- torch.distributed.all_gather_into_tensor( +- gw, +- local_shard, +- group=self.dp_group, +- ) +- evt.record(self.comm_stream) ++ ++ if self._a2a_barrier is not None and non_blocking and getattr(self, "_nz_cast_fn", None) is None: ++ # Hold the collective until the block's attention reports its ++ # QKV AllToAll, so the background gather cannot take the links ++ # ahead of the critical-path all-to-all. The H2D above has ++ # already been enqueued; only the gather moves, so it still ++ # overlaps the rest of the block. Deferral needs a repoint that ++ # emits no stream work, hence the NZ-cast guard. ++ self._deferred_allgather = (slot, gpu_shards, gpu_weights, evt) ++ self._a2a_barrier.arm() ++ else: ++ self._issue_allgather(slot, gpu_shards, gpu_weights, evt) ++ elif self._a2a_barrier is not None and non_blocking: ++ self._deferred_chunked_allgather = (slot, gpu_weights, evt) ++ self._a2a_barrier.arm() + else: +- for dtype_manifest in self.manifest.dtypes: +- cpu_shard = self.cpu_shards[dtype_manifest.dtype] +- buf = shard_bufs[dtype_manifest.dtype] +- for chunk in dtype_manifest.chunks: +- if buf.numel() >= 2 * chunk.local_numel: +- width = buf.numel() // 2 +- input_slot = chunk.chunk_id % 2 +- else: +- width = buf.numel() +- input_slot = 0 +- reuse = self._chunk_reuse_events[input_slot] +- if reuse is not None: +- self.copy_stream.wait_event(reuse) +- local_input = buf[input_slot * width : input_slot * width + chunk.local_numel] +- source = cpu_shard[chunk.cpu_offset : chunk.cpu_offset + chunk.local_numel] +- with current_omni_platform.stream(self.copy_stream): +- local_input.copy_(source, non_blocking=non_blocking) +- self._h2d_done_events[input_slot].record(self.copy_stream) +- self.comm_stream.wait_event(self._h2d_done_events[input_slot]) +- with current_omni_platform.stream(self.comm_stream): +- full_out = gpu_weights[dtype_manifest.dtype][ +- chunk.full_offset : chunk.full_offset + chunk.padded_numel +- ] +- torch.distributed.all_gather_into_tensor( +- full_out, +- local_input, +- group=self.dp_group, +- ) +- self._ag_done_events[input_slot].record(self.comm_stream) +- self._chunk_reuse_events[input_slot] = self._ag_done_events[input_slot] +- with current_omni_platform.stream(self.comm_stream): +- evt.record(self.comm_stream) ++ self._issue_chunked_allgather(slot, gpu_weights, evt, non_blocking=non_blocking) + + self.ready_events[slot] = evt + self._prefetch_done = evt +@@ -719,6 +709,131 @@ class DistributedLayerwiseOffloadHook(ModelHook): + ), + ) + ++ def _issue_chunked_allgather(self, slot, gpu_weights, evt, *, gated=False, non_blocking=True) -> None: ++ """Submit chunk copies and gathers together after the foreground gate. ++ ++ Chunk staging slots are bounded and reused. Deferring only gathers ++ would overwrite an input slot before its collective consumes it. ++ """ ++ shard_bufs = self.gpu_shard_buffers[slot] ++ assert shard_bufs is not None and self.manifest is not None ++ if gated: ++ self.comm_stream.wait_event(self._a2a_event) ++ for dtype_manifest in self.manifest.dtypes: ++ cpu_shard = self.cpu_shards[dtype_manifest.dtype] ++ buf = shard_bufs[dtype_manifest.dtype] ++ for chunk in dtype_manifest.chunks: ++ if buf.numel() >= 2 * chunk.local_numel: ++ width = buf.numel() // 2 ++ input_slot = chunk.chunk_id % 2 ++ else: ++ width = buf.numel() ++ input_slot = 0 ++ reuse = self._chunk_reuse_events[input_slot] ++ if reuse is not None: ++ self.copy_stream.wait_event(reuse) ++ local_input = buf[input_slot * width : input_slot * width + chunk.local_numel] ++ source = cpu_shard[chunk.cpu_offset : chunk.cpu_offset + chunk.local_numel] ++ with current_omni_platform.stream(self.copy_stream): ++ local_input.copy_(source, non_blocking=non_blocking) ++ self._h2d_done_events[input_slot].record(self.copy_stream) ++ self.comm_stream.wait_event(self._h2d_done_events[input_slot]) ++ with current_omni_platform.stream(self.comm_stream): ++ full_out = gpu_weights[dtype_manifest.dtype][ ++ chunk.full_offset : chunk.full_offset + chunk.padded_numel ++ ] ++ torch.distributed.all_gather_into_tensor( ++ full_out, ++ local_input, ++ group=self.dp_group, ++ ) ++ self._ag_done_events[input_slot].record(self.comm_stream) ++ self._chunk_reuse_events[input_slot] = self._ag_done_events[input_slot] ++ with current_omni_platform.stream(self.comm_stream): ++ evt.record(self.comm_stream) ++ ++ def _issue_allgather( ++ self, ++ slot: int, ++ gpu_shards: dict[torch.dtype, torch.Tensor], ++ gpu_weights: dict[torch.dtype, torch.Tensor], ++ evt: Any, ++ *, ++ gated: bool = False, ++ ) -> None: ++ """Submit this block's weight AllGather on ``comm_stream``. ++ ++ ``gated`` additionally waits on the QKV AllToAll barrier event recorded ++ by the block's attention, so the foreground collective gets the links ++ first and this gather queues behind it. ++ """ ++ self.comm_stream.wait_stream(self.copy_stream) ++ if gated: ++ self.comm_stream.wait_event(self._a2a_event) ++ with current_omni_platform.stream(self.comm_stream): ++ for dtype, local_shard in gpu_shards.items(): ++ # Slice the shared (max-sized) output buffer down to this ++ # block's actual AllGather output size. The buffers are ++ # sized to the *largest* block across all groups, so for any ++ # smaller block the full buffer would violate the ++ # all_gather_into_tensor contract ++ # (output.numel() == world_size * input.numel()). ++ # Repoint offsets are relative to the block's flattened ++ # buffer, so a prefix slice is safe. ++ gw = gpu_weights[dtype][: self._ag_output_sizes[dtype]] ++ torch.distributed.all_gather_into_tensor( ++ gw, ++ local_shard, ++ group=self.dp_group, ++ ) ++ evt.record(self.comm_stream) ++ ++ def _allgather_after_attn_a2a(self) -> None: ++ """Barrier callback: the block's QKV AllToAll is now enqueued. ++ ++ Recording on the compute stream here (rather than at block entry) is ++ what makes the gate effective: an event wait snapshots the records ++ enqueued before it, so the wait must be enqueued *after* the all-to-all. ++ """ ++ chunked = self._deferred_chunked_allgather ++ if chunked is not None: ++ self._deferred_chunked_allgather = None ++ assert self._a2a_event is not None ++ self._a2a_event.record(current_omni_platform.current_stream()) ++ self._issue_chunked_allgather(*chunked, gated=True) ++ return ++ deferred = self._deferred_allgather ++ if deferred is None: ++ return ++ self._deferred_allgather = None ++ assert self._a2a_event is not None ++ self._a2a_event.record(current_omni_platform.current_stream()) ++ slot, gpu_shards, gpu_weights, evt = deferred ++ self._issue_allgather(slot, gpu_shards, gpu_weights, evt, gated=True) ++ ++ def _flush_deferred_allgather(self) -> None: ++ """Issue a deferred gather whose attention never signalled. ++ ++ Keeps the double buffer live: a block whose attention is skipped ++ (cache-dit, an all-resident stage) still needs its weights, so the ++ gather is submitted ungated at the end of that block's forward. ++ """ ++ chunked = self._deferred_chunked_allgather ++ if chunked is not None: ++ self._deferred_chunked_allgather = None ++ if self._a2a_barrier is not None: ++ self._a2a_barrier.disarm() ++ self._issue_chunked_allgather(*chunked) ++ return ++ deferred = self._deferred_allgather ++ if deferred is None: ++ return ++ self._deferred_allgather = None ++ if self._a2a_barrier is not None: ++ self._a2a_barrier.disarm() ++ slot, gpu_shards, gpu_weights, evt = deferred ++ self._issue_allgather(slot, gpu_shards, gpu_weights, evt) ++ + def _wait_for_weights(self, slot: int) -> None: + """Wait until the slot holding this block is ready for compute. + +@@ -840,6 +955,10 @@ class DistributedLayerwiseOffloadHook(ModelHook): + return args, kwargs + + def post_forward(self, module: nn.Module, output: Any) -> Any: ++ # Pair with pre_forward: if the block's attention never signalled the ++ # gate, the deferred gather must still go out before this block's ++ # storage is released (offload_layer waits on _prefetch_done). ++ self._flush_deferred_allgather() + self.offload_layer() + return output + +@@ -864,6 +983,7 @@ def apply_distributed_block_hook( + tensor_transforms: dict[int, Any] | None = None, + materialization_probe_tensor: torch.Tensor | None = None, + chunk_size_bytes: int = 64 * 1024 * 1024, ++ order_allgather_after_attn_a2a: bool = False, + ) -> DistributedLayerwiseOffloadHook: + """Register a DistributedLayerwiseOffloadHook on *module*.""" + registry = HookRegistry.get_or_create(module) +@@ -881,11 +1001,32 @@ def apply_distributed_block_hook( + tensor_transforms=tensor_transforms, + materialization_probe_tensor=materialization_probe_tensor, + chunk_size_bytes=chunk_size_bytes, ++ order_allgather_after_attn_a2a=order_allgather_after_attn_a2a, + ) + registry.register_hook(DistributedLayerwiseOffloadHook._HOOK_NAME, hook) + return hook + + ++def attach_collective_gate(block: nn.Module, hook: DistributedLayerwiseOffloadHook) -> int: ++ """Hand ``hook``'s barrier to every Attention layer inside *block*. ++ ++ The attention signals the barrier right after its QKV AllToAll. The import ++ is local to keep this module independent of the attention package at import ++ time. ++ ++ Returns the number of attention layers wired (0 means the hook will rely on ++ the ``post_forward`` flush instead). ++ """ ++ from vllm_omni.diffusion.attention.layer import Attention ++ ++ attached = 0 ++ for module in block.modules(): ++ if isinstance(module, Attention): ++ module._collective_gate = hook._a2a_barrier ++ attached += 1 ++ return attached ++ ++ + def remove_distributed_block_hook(module: nn.Module) -> None: + """Remove the distributed layerwise offload hook from *module*.""" + registry: HookRegistry | None = getattr(module, "_hook_registry", None) +@@ -1566,29 +1707,45 @@ class DistributedLayerwiseOffloadBackend(OffloadBackend): + self._all_hook_groups.append(hooks) + self._blocks.append(block_list) + probes = {id(block): module_materialization_probe(block) for block in block_list} ++ # Only the DiT blocks issue a QKV AllToAll the gather can queue behind. ++ order_gate = bool(getattr(self.config, "dlo_order_allgather_after_attn_a2a", False)) and ( ++ component == DIT_COMPONENT ++ ) ++ if order_gate and not getattr(self, "_collective_gate_logged", False): ++ self._collective_gate_logged = True ++ logger.info( ++ "Distributed layerwise offload will hold each DiT block's weight AllGather " ++ "until that block's QKV AllToAll is on the compute stream" ++ ) + for block, next_block in zip( + chain((block_list[-1],), block_list[:-1]), + block_list, + strict=True, + ): +- hooks.append( +- apply_distributed_block_hook( +- block, +- next_block, +- self.device, +- group, +- group_size, +- group_rank, +- self.copy_stream, +- self.comm_stream, +- self.config.pin_cpu_memory, +- shared_buffers=[None, None], +- rank_local_mmap=self._using_rank_local_mmap if use_dit_mmap else False, +- tensor_transforms=self._mmap_transforms_by_tensor_id if use_dit_mmap else None, +- materialization_probe_tensor=probes[id(block)], +- chunk_size_bytes=self.config.chunk_size_bytes, +- ) ++ hook = apply_distributed_block_hook( ++ block, ++ next_block, ++ self.device, ++ group, ++ group_size, ++ group_rank, ++ self.copy_stream, ++ self.comm_stream, ++ self.config.pin_cpu_memory, ++ shared_buffers=[None, None], ++ rank_local_mmap=self._using_rank_local_mmap if use_dit_mmap else False, ++ tensor_transforms=self._mmap_transforms_by_tensor_id if use_dit_mmap else None, ++ materialization_probe_tensor=probes[id(block)], ++ chunk_size_bytes=self.config.chunk_size_bytes, ++ order_allgather_after_attn_a2a=order_gate, + ) ++ if hook._a2a_barrier is not None and attach_collective_gate(block, hook) == 0: ++ logger.warning( ++ "DLO AllGather ordering is enabled but block %s exposes no Attention layer; " ++ "its gather will fall back to the ungated path", ++ block.__class__.__name__, ++ ) ++ hooks.append(hook) + + # hooks = [last -> first, block0 -> block1, ...]. Alternating slots + # keep a prefetch from overwriting the current block for any ring size. +diff --git a/vllm_omni/engine/arg_utils.py b/vllm_omni/engine/arg_utils.py +index 0a876b9..bf8dcdc 100644 +--- a/vllm_omni/engine/arg_utils.py ++++ b/vllm_omni/engine/arg_utils.py +@@ -807,6 +807,7 @@ class OrchestratorArgs: + dlo_use_allgather: bool = True + dlo_chunk_size_mb: int = 64 + dlo_resident_layers: int = 0 ++ dlo_order_allgather_after_attn_a2a: bool = False + host_weight_runtime_mode: str = "disabled" + host_weight_runtime_root: str | None = None + dlo_host_registration_limit_gib: float = 0.0 +diff --git a/vllm_omni/entrypoints/cli/serve.py b/vllm_omni/entrypoints/cli/serve.py +index f2b55e7..aff71b5 100644 +--- a/vllm_omni/entrypoints/cli/serve.py ++++ b/vllm_omni/entrypoints/cli/serve.py +@@ -1041,6 +1041,17 @@ class OmniServeCommand(CLISubcommand): + help="Full-chunk target size in MiB for distributed layerwise offload " + "(each chunk is one host-to-device copy + AllGather unit).", + ) ++ omni_config_group.add_argument( ++ "--dlo-order-allgather-after-attn-a2a", ++ action="store_true", ++ default=False, ++ help=( ++ "Hold each DiT block's weight AllGather until that block's QKV AllToAll " ++ "has been enqueued, so the background gather does not take the collective " ++ "links ahead of the critical-path all-to-all. Requires distributed " ++ "layerwise offload with AllGather." ++ ), ++ ) + omni_config_group.add_argument( + "--host-weight-runtime-mode", + choices=("disabled", "preferred", "required"), diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c22-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c22-source-identity.json new file mode 100644 index 000000000..bce9b1703 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c22-source-identity.json @@ -0,0 +1,29 @@ +{ + "head": "7b0d12bfefdf05868e93e74bc91f04c70ee34128", + "tree": "1390788d5fcee97e48ba874be718db25ae89b064", + "base": "d48fa51d03dd748568cbcf57ed40f533c691c9bd", + "branch": "rsi/h3-297-video-pre-gate-secant", + "files": { + "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "b8394d22b054d60c1ab8e069e9600e1947cc73b5826e1f206e66c548123f632d", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_secant.py": "70a215fbb0dc63b90652d96f8f6d106a5785bd7d64dae75a2c4f6ec5c627ac1c", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "b991ebb5d43242357d5ab8cfb476808a76fc0658d6048db6f3b7131122c8a993", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "8933b536317652d0dbc7a974ea7e0bca928e7b3bdf438988999ff5a498a6ec9d", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "3904bc8ac6679bc51614b95f5510789331a41bf20a43a517f3a99d33505201a6", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "e21e0afe8f8948d805d374428a34f3b2c5c0b5041d63ea791f047af8f455cf16", + "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" + }, + "published": false, + "patch_base": "be319692b96413918973b5bd399cdda3f320c574" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c23-candidate.patch b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c23-candidate.patch new file mode 100644 index 000000000..f863e0140 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c23-candidate.patch @@ -0,0 +1,2430 @@ +diff --git a/tests/diffusion/distributed/test_collective_gate.py b/tests/diffusion/distributed/test_collective_gate.py +new file mode 100644 +index 0000000..e66fdae +--- /dev/null ++++ b/tests/diffusion/distributed/test_collective_gate.py +@@ -0,0 +1,205 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++"""Unit tests for the DLO/QKV-AllToAll ordering gate. ++ ++Covers the barrier rendezvous and the hook's deferred-AllGather state machine ++that keeps the background weight gather from taking the links ahead of the ++block's foreground AllToAll. ++""" ++ ++from unittest.mock import Mock ++ ++import pytest ++import torch ++from torch import nn ++ ++import vllm_omni.diffusion.offloader.distributed_layerwise_backend as dist_backend_module ++from tests.diffusion.offloader.helpers import DummyStream, patch_offload_runtime ++from vllm_omni.diffusion.distributed.collective_gate import QKVAllToAllBarrier ++from vllm_omni.diffusion.offloader.distributed_layerwise_backend import ( ++ DistributedLayerwiseOffloadHook, ++ attach_collective_gate, ++) ++ ++pytestmark = [pytest.mark.diffusion, pytest.mark.cpu, pytest.mark.core_model] ++ ++ ++@pytest.fixture ++def patched_offload_runtime(monkeypatch): ++ patch_offload_runtime(monkeypatch, dist_backend_module.current_omni_platform, synchronize=True) ++ ++ ++class TestQKVAllToAllBarrier: ++ def test_signal_without_arm_is_a_noop(self): ++ on_signal = Mock() ++ barrier = QKVAllToAllBarrier(on_signal) ++ ++ barrier.signal() ++ ++ on_signal.assert_not_called() ++ assert not barrier.armed ++ ++ def test_arm_then_signal_fires_once(self): ++ on_signal = Mock() ++ barrier = QKVAllToAllBarrier(on_signal) ++ ++ barrier.arm() ++ assert barrier.armed ++ barrier.signal() ++ ++ on_signal.assert_called_once_with() ++ assert not barrier.armed ++ ++ # A second signal in the same forward must not re-issue the gather. ++ barrier.signal() ++ on_signal.assert_called_once_with() ++ ++ def test_disarm_suppresses_a_pending_signal(self): ++ on_signal = Mock() ++ barrier = QKVAllToAllBarrier(on_signal) ++ ++ barrier.arm() ++ barrier.disarm() ++ barrier.signal() ++ ++ on_signal.assert_not_called() ++ ++ ++def _gated_hook() -> DistributedLayerwiseOffloadHook: ++ """A hook wired for the gated path, with hand-set shard buffers.""" ++ hook = DistributedLayerwiseOffloadHook( ++ next_block=nn.Linear(2, 2, bias=False), ++ device=torch.device("cpu"), ++ dp_group=object(), ++ dp_size=2, ++ rank=0, ++ copy_stream=DummyStream(), ++ comm_stream=DummyStream(), ++ pin_memory=False, ++ order_allgather_after_attn_a2a=True, ++ ) ++ device_buf = {torch.float32: torch.zeros(4)} ++ shard_buf = {torch.float32: torch.zeros(2)} ++ hook.gpu_buffers = [dict(device_buf), dict(device_buf)] ++ hook.gpu_shard_buffers = [dict(shard_buf), dict(shard_buf)] ++ hook.cpu_shards = {torch.float32: torch.zeros(2)} ++ hook._ag_output_sizes = {torch.float32: 4} ++ hook.metadata = {} ++ hook._cached_repoint = () ++ hook.rank_local_mmap = False ++ return hook ++ ++ ++class TestDeferredAllGather: ++ def test_prefetch_defers_the_gather_until_the_barrier_signals(self, patched_offload_runtime, monkeypatch): ++ hook = _gated_hook() ++ gather = Mock() ++ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) ++ ++ hook.prefetch_layer(0, non_blocking=True) ++ ++ # H2D ran; the collective is held back for the block's AllToAll. ++ gather.assert_not_called() ++ assert hook._a2a_barrier.armed ++ assert hook._deferred_allgather is not None ++ # The ready event must still be published for the consumer. ++ assert hook.ready_events[0] is hook._prefetch_done ++ ++ hook._a2a_barrier.signal() ++ ++ gather.assert_called_once() ++ assert hook._deferred_allgather is None ++ assert not hook._a2a_barrier.armed ++ ++ def test_post_forward_flush_issues_an_unsignalled_gather(self, patched_offload_runtime, monkeypatch): ++ """A block whose attention never ran must still get its weights.""" ++ hook = _gated_hook() ++ gather = Mock() ++ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) ++ ++ hook.prefetch_layer(0, non_blocking=True) ++ gather.assert_not_called() ++ ++ # pre_forward ran, attention did not: the pairing flush must fire. ++ hook.post_forward(hook.next_block, None) ++ ++ gather.assert_called_once() ++ assert hook._deferred_allgather is None ++ assert not hook._a2a_barrier.armed ++ ++ def test_sync_prefetch_issues_immediately_without_arming(self, patched_offload_runtime, monkeypatch): ++ """The cache-dit sync fallback has no attention signal to wait for.""" ++ hook = _gated_hook() ++ gather = Mock() ++ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) ++ ++ hook.prefetch_layer(0, non_blocking=False) ++ ++ gather.assert_called_once() ++ assert not hook._a2a_barrier.armed ++ assert hook._deferred_allgather is None ++ ++ def test_disabled_by_default(self, patched_offload_runtime, monkeypatch): ++ hook = DistributedLayerwiseOffloadHook( ++ next_block=nn.Linear(2, 2, bias=False), ++ device=torch.device("cpu"), ++ dp_group=object(), ++ dp_size=2, ++ rank=0, ++ copy_stream=DummyStream(), ++ comm_stream=DummyStream(), ++ pin_memory=False, ++ ) ++ hook.gpu_buffers = [{torch.float32: torch.zeros(4)}] * 2 ++ hook.gpu_shard_buffers = [{torch.float32: torch.zeros(2)}] * 2 ++ hook.cpu_shards = {torch.float32: torch.zeros(2)} ++ hook._ag_output_sizes = {torch.float32: 4} ++ hook.metadata = {} ++ hook._cached_repoint = () ++ gather = Mock() ++ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) ++ ++ hook.prefetch_layer(0, non_blocking=True) ++ ++ assert hook._a2a_barrier is None ++ gather.assert_called_once() ++ ++ ++class TestAttachCollectiveGate: ++ def test_block_without_attention_reports_zero(self): ++ hook = _gated_hook() ++ ++ assert attach_collective_gate(nn.Linear(2, 2), hook) == 0 ++ ++ ++@pytest.mark.parametrize("flush", [False, True]) ++def test_chunked_gate_defers_copies_and_reuses_bounded_slots(patched_offload_runtime, monkeypatch, flush): ++ from types import SimpleNamespace ++ ++ hook = _gated_hook() ++ chunks = tuple( ++ SimpleNamespace(chunk_id=i, local_numel=1, cpu_offset=i, full_offset=2 * i, padded_numel=2) for i in range(3) ++ ) ++ hook.manifest = SimpleNamespace(dtypes=(SimpleNamespace(dtype=torch.float32, chunks=chunks),)) ++ hook.cpu_shards = {torch.float32: torch.tensor([11.0, 22.0, 33.0])} ++ hook.gpu_buffers = [{torch.float32: torch.zeros(6)} for _ in range(2)] ++ calls = [] ++ ++ def gather(output, local, **kwargs): ++ calls.append(local.clone()) ++ output.copy_(local.repeat(2)) ++ ++ monkeypatch.setattr(torch.distributed, "all_gather_into_tensor", gather) ++ hook.prefetch_layer(0, non_blocking=True) ++ assert not calls and hook._deferred_chunked_allgather is not None ++ assert torch.equal(hook.gpu_shard_buffers[0][torch.float32], torch.zeros(2)) ++ if flush: ++ hook._flush_deferred_allgather() ++ else: ++ hook._a2a_barrier.signal() ++ assert len(calls) == 3 and hook._deferred_chunked_allgather is None ++ assert not hook._a2a_barrier.armed ++ assert torch.equal(hook.gpu_buffers[0][torch.float32], torch.tensor([11.0, 11.0, 22.0, 22.0, 33.0, 33.0])) ++ hook._a2a_barrier.signal() ++ assert len(calls) == 3 +diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py +new file mode 100644 +index 0000000..4e9979e +--- /dev/null ++++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py +@@ -0,0 +1,198 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++from datetime import timedelta ++ ++import pytest ++import torch ++import torch.distributed as dist ++import torch.nn as nn ++ ++from tests.diffusion.models.minimax_h3.test_minimax_h3_video_row_cache import tiny_block ++from tests.helpers.mark import hardware_marks ++from vllm_omni.diffusion.layers.indexed_modulation import indexed_gate ++from vllm_omni.diffusion.models.minimax_h3.attention.flash_h3 import _prefix_bnsd, _subset_prefix_bnsd ++from vllm_omni.diffusion.models.minimax_h3.video_row_cache import MiniMaxH3VideoRowCache ++from vllm_omni.platforms import current_omni_platform ++ ++pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=4)] ++ ++ ++def make_state(start=0, length=4): ++ state = MiniMaxH3VideoRowCache(torch.tensor([1, 2]), seq_len=4, used_len=3, num_layers=1, pre_gate=True) ++ state.prepare_local_span(start, length) ++ return state ++ ++ ++def test_fixed_pre_gate_schedule_storage_and_request_cleanup(): ++ state = make_state() ++ x = torch.ones(4, 2, dtype=torch.bfloat16) ++ full = [] ++ for step in range(50): ++ state.begin_step(step) ++ if not state.partial: ++ full.append(step) ++ for stage in ("attn", "mlp"): ++ out = state.pre_gate_output(0, x, stage) ++ assert out.dtype == torch.bfloat16 ++ if state.partial: ++ state.record_fresh_gate(stage) ++ state.end_step() ++ assert full == [0, 1, 4, 9, 19, 29, 39, 49] ++ assert state.full_blocks == 8 and state.partial_blocks == 42 ++ assert state.fresh_attention_gates == state.fresh_mlp_gates == 42 ++ assert all(t.dtype == torch.bfloat16 for t in [*state._attention_outputs.values(), *state._mlp_outputs.values()]) ++ assert state.stats()["live_cache_bytes"] == 16 ++ state.close() ++ assert state.stats()["live_cache_bytes"] == 0 and state.closed ++ ++ ++@pytest.mark.parametrize("failure", ["missing_mlp", "mlp_first", "gated_capture", "gated_restore"]) ++def test_pre_gate_stage_order_cannot_silently_use_block_residual(failure): ++ state = make_state() ++ x = torch.ones(4, 2, dtype=torch.bfloat16) ++ state.begin_step(0) ++ with pytest.raises(RuntimeError): ++ if failure == "missing_mlp": ++ state.pre_gate_output(0, x, "attn") ++ state.end_step() ++ elif failure == "mlp_first": ++ state.pre_gate_output(0, x, "mlp") ++ elif failure == "gated_capture": ++ state.capture(0, x, x) ++ else: ++ state.restore(0, x) ++ state.close() ++ ++ ++class ConstantMLP(nn.Module): ++ def __init__(self): ++ super().__init__() ++ self.rows = [] ++ ++ def forward(self, x): ++ self.rows.append(x.shape[0]) ++ return torch.full_like(x, 3) ++ ++ ++@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) ++def test_real_block_reapplies_changed_gates_and_preserves_empty_dense_shards(start, length): ++ state = make_state(start, length) ++ block = tiny_block() ++ gates = [1.0, 1.0] ++ block.adaln_proj = lambda t: ( ++ torch.zeros(3, 2), ++ torch.zeros(3, 2), ++ torch.full((3, 2), gates[0]), ++ torch.zeros(3, 2), ++ torch.zeros(3, 2), ++ torch.full((3, 2), gates[1]), ++ ) ++ block.attn = lambda x, **kwargs: torch.full_like(x, 2) ++ block.mlp = ConstantMLP() ++ x = torch.ones(length, 2, dtype=torch.bfloat16) ++ kwargs = dict( ++ t_emb=torch.zeros(1), ++ combined_indices=torch.zeros(length, dtype=torch.long), ++ rope_table=None, ++ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), ++ max_seqlen=3, ++ packed_total=4, ++ ) ++ for step in (0, 1): ++ state.begin_step(step) ++ expected = block(x, **kwargs) ++ actual = block(x, video_row_cache=state, **kwargs) ++ assert torch.equal(actual.view(torch.uint8), expected.view(torch.uint8)) ++ state.end_step() ++ gates[:] = [0.5, -0.25] ++ current = x * 2 ++ saved = current.clone() ++ expected = block(current, **kwargs) ++ block.mlp.rows.clear() ++ state.begin_step(2) ++ actual = block(current, video_row_cache=state, **kwargs) ++ state.end_step() ++ rows = state.local_dense_positions ++ assert block.mlp.rows == ([rows.numel()] if rows.numel() else []) ++ assert torch.equal( ++ actual[: min(length, 3 - start)].view(torch.uint8), expected[: min(length, 3 - start)].view(torch.uint8) ++ ) ++ assert torch.equal( ++ actual.index_select(0, state.local_video_positions), torch.full((state.local_video_positions.numel(), 2), 2.25) ++ ) ++ assert not torch.equal( ++ actual.index_select(0, state.local_video_positions), current.index_select(0, state.local_video_positions) + 5 ++ ) ++ assert torch.equal(current, saved) ++ assert state.fresh_attention_gates == state.fresh_mlp_gates == 1 ++ state.close() ++ ++ ++def _native_pre_gate_worker(rank): ++ import torch_npu # noqa: F401 ++ ++ torch.npu.set_device(rank) ++ dist.init_process_group( ++ "hccl", init_method="tcp://127.0.0.1:29597", rank=rank, world_size=4, timeout=timedelta(seconds=120) ++ ) ++ try: ++ torch.manual_seed(219) ++ full = [torch.randn(32, 4, 128, dtype=torch.bfloat16).npu() for _ in range(3)] ++ exchanged = [] ++ for tensor in full: ++ local = tensor[rank * 8 : (rank + 1) * 8] ++ send = local.reshape(8, 4, 1, 128).permute(1, 0, 2, 3).contiguous() ++ receive = torch.empty_like(send) ++ dist.all_to_all_single(receive, send) ++ exchanged.append(receive.reshape(1, 32, 1, 128)) ++ state = MiniMaxH3VideoRowCache( ++ torch.arange(8, 24, device="npu"), seq_len=32, used_len=30, num_layers=1, pre_gate=True ++ ) ++ state.prepare_local_span(rank * 8, 8) ++ x = torch.full((8, 512), 2, dtype=torch.bfloat16, device="npu") ++ for step in (0, 1): ++ state.begin_step(step) ++ state.pre_gate_output(0, torch.full_like(x, 0.5), "attn") ++ state.pre_gate_output(0, torch.full_like(x, 0.25), "mlp") ++ state.end_step() ++ state.begin_step(2) ++ out = _subset_prefix_bnsd(exchanged[0], exchanged[1][:, :30], exchanged[2][:, :30], 128**-0.5, state) ++ send, receive = out.reshape(4, 8, 1, 128).contiguous(), torch.empty_like(out.reshape(4, 8, 1, 128)) ++ dist.all_to_all_single(receive, send) ++ local_out = receive.permute(1, 0, 2, 3).reshape(8, 512) ++ expected = _prefix_bnsd(full[0].unsqueeze(0), full[1][:30].unsqueeze(0), full[2][:30].unsqueeze(0), 128**-0.5)[ ++ 0 ++ ].reshape(32, 512) ++ dense = state.local_dense_positions ++ torch.testing.assert_close( ++ local_out.index_select(0, dense).float(), ++ expected[rank * 8 : (rank + 1) * 8].index_select(0, dense).float(), ++ atol=0.03125, ++ rtol=0.02, ++ ) ++ attn = state.pre_gate_output(0, local_out, "attn") ++ indices = torch.zeros(8, dtype=torch.long, device="npu") ++ gate = torch.full((1, 512), 0.5, dtype=torch.float32, device="npu") ++ after_attn = indexed_gate(x, gate, attn, indices) ++ state.record_fresh_gate("attn") ++ mlp = state.pre_gate_output(0, torch.zeros_like(x), "mlp") ++ result = indexed_gate(after_attn, gate, mlp, indices) ++ state.record_fresh_gate("mlp") ++ video = state.local_video_positions ++ assert torch.equal( ++ result.index_select(0, video), torch.full((video.numel(), 512), 2.375, dtype=torch.bfloat16, device="npu") ++ ) ++ assert dense.numel() == [8, 0, 0, 6][rank] ++ state.end_step() ++ state.close() ++ assert state.stats()["live_cache_bytes"] == 0 ++ torch.npu.synchronize() ++ dist.barrier() ++ finally: ++ dist.destroy_process_group() ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires four native Ascend devices") ++def test_native_four_rank_pre_gate_subset_attention_and_current_gates(): ++ torch.multiprocessing.spawn(_native_pre_gate_worker, nprocs=4, join=True) +diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_video_context.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_context.py +new file mode 100644 +index 0000000..71c6dd5 +--- /dev/null ++++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_context.py +@@ -0,0 +1,225 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++from datetime import timedelta ++ ++import pytest ++import torch ++import torch.distributed as dist ++ ++from tests.diffusion.models.minimax_h3.test_minimax_h3_video_row_cache import tiny_block ++from tests.helpers.mark import hardware_marks ++from vllm_omni.diffusion.models.minimax_h3.attention import video_context as context ++from vllm_omni.diffusion.models.minimax_h3.attention.flash_h3 import _prefix_bnsd ++from vllm_omni.diffusion.models.minimax_h3.video_row_cache import MiniMaxH3VideoRowCache ++from vllm_omni.platforms import current_omni_platform ++ ++pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=4)] ++ ++ ++def oracle(query, key, value, scale): ++ q, k, v = [x.transpose(1, 2).float() for x in (query, key, value)] ++ logits = q @ k.transpose(-2, -1) * scale ++ output = (torch.softmax(logits, -1) @ v).transpose(1, 2).to(query.dtype) ++ return output.contiguous(), torch.logsumexp(logits, -1).transpose(1, 2).unsqueeze(-1).contiguous() ++ ++ ++def reference_mix(a, b, lse_a, lse_b): ++ maximum = torch.maximum(lse_a, lse_b) ++ wa, wb = (lse_a - maximum).exp(), (lse_b - maximum).exp() ++ return ((wa * a.float() + wb * b.float()) / (wa + wb)).to(a.dtype) ++ ++ ++def make_state(seq=12, used=10, prefix=3, device="cpu", start=0, length=None): ++ state = MiniMaxH3VideoRowCache( ++ torch.arange(prefix, used, device=device), seq_len=seq, used_len=used, num_layers=1, context_only=True ++ ) ++ state.prepare_local_span(start, seq if length is None else length) ++ return state ++ ++ ++@pytest.mark.parametrize("positions,pregate", [([1], False), ([1, 2], True)]) ++def test_context_mode_rejects_nonfinal_video_or_pre_gate_ffn(positions, pregate): ++ with pytest.raises(ValueError, match="final target span"): ++ MiniMaxH3VideoRowCache( ++ torch.tensor(positions), seq_len=4, used_len=3, num_layers=1, context_only=True, pre_gate=pregate ++ ) ++ ++ ++def test_partition_matches_dense_and_refreshes_prefix_without_mutating_inputs(monkeypatch): ++ monkeypatch.setattr(context, "_prefix_bnsd", lambda *args: oracle(*args)[0]) ++ monkeypatch.setattr(context, "prefix_attention_lse", oracle) ++ monkeypatch.setattr(context, "mix_context", reference_mix) ++ torch.manual_seed(23) ++ q = torch.randn(1, 12, 2, 8, dtype=torch.bfloat16) ++ k, v = [torch.randn(1, 10, 2, 8, dtype=torch.bfloat16) for _ in range(2)] ++ state = make_state() ++ dummy = torch.ones(12, 2, dtype=torch.bfloat16) ++ for step in (0, 1): ++ state.begin_step(step) ++ output = context.video_context_attention(q, k, v, 0.25, state) ++ expected = oracle(q[:, :10], k, v, 0.25)[0] ++ torch.testing.assert_close(output[:, :10].float(), expected.float(), atol=0.015625, rtol=0.02) ++ assert torch.count_nonzero(output[:, 10:]) == 0 ++ state.finish_context_block(0, dummy) ++ state.end_step() ++ previous = output.clone() ++ anchors = [tensor.clone() for tensor in state._video_contexts[0]] ++ v = v.clone() ++ v[:, :3] += 1 ++ saved = [t.clone() for t in (q, k, v)] ++ state.begin_step(2) ++ output = context.video_context_attention(q, k, v, 0.25, state) ++ expected = oracle(q[:, :10], k, v, 0.25)[0] ++ torch.testing.assert_close(output[:, :10].float(), expected.float(), atol=0.015625, rtol=0.02) ++ assert not torch.equal(output[:, 3:10], previous[:, 3:10]) ++ assert all(torch.equal(a, b) for a, b in zip(saved, (q, k, v))) ++ assert all(torch.equal(a, b) for a, b in zip(anchors, state._video_contexts[0])) ++ state.finish_context_block(0, dummy) ++ state.end_step() ++ assert state.video_context_refreshes == 2 and state.video_context_reuses == 1 ++ assert state.fresh_video_prefix_calls == 3 ++ state.close() ++ assert state.stats()["live_cache_bytes"] == 0 ++ ++ ++@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) ++def test_real_block_keeps_all_ffn_rows_fresh_on_context_reuse(start, length): ++ state = make_state(seq=4, used=3, prefix=1, start=start, length=length) ++ block = tiny_block() ++ ++ def attention(x, **kwargs): ++ cache = kwargs.get("video_row_cache") ++ if cache is not None: ++ if cache.partial: ++ cache.read_video_context() ++ else: ++ cache.store_video_context(torch.zeros(1, 2, 1, 2, dtype=torch.bfloat16), torch.zeros(1, 2, 1, 1)) ++ cache.record_context_attention() ++ return x * 0.5 ++ ++ block.attn = attention ++ x = torch.ones(length, 2, dtype=torch.bfloat16) ++ kwargs = dict( ++ t_emb=torch.zeros(1), ++ combined_indices=torch.zeros(length, dtype=torch.long), ++ rope_table=None, ++ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), ++ max_seqlen=3, ++ packed_total=4, ++ ) ++ expected = block(x, **kwargs) ++ block.mlp.rows.clear() ++ for step in (0, 1, 2): ++ state.begin_step(step) ++ actual = block(x, video_row_cache=state, **kwargs) ++ assert torch.equal(actual.view(torch.uint8), expected.view(torch.uint8)) ++ state.end_step() ++ assert block.mlp.rows == [length] * 3 ++ assert state.full_row_mlp_blocks == 3 ++ state.close() ++ ++ ++def test_context_lifecycle_requires_actual_attention(): ++ state = make_state() ++ state.begin_step(0) ++ dummy = torch.ones(12, 2, dtype=torch.bfloat16) ++ with pytest.raises(RuntimeError, match="not executed"): ++ state.finish_context_block(0, dummy) ++ state.close() ++ ++ ++def test_long_forward_selects_context_on_full_and_partial_steps(monkeypatch): ++ from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata ++ from vllm_omni.diffusion.models.minimax_h3.attention.flash_h3 import MiniMaxH3FlashAttentionImpl ++ ++ monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score") ++ state = make_state(seq=4192, used=4160, prefix=64) ++ impl = object.__new__(MiniMaxH3FlashAttentionImpl) ++ impl.causal, impl.softmax_scale = False, 0.25 ++ impl._resolve_packed_seq_npu = lambda *args: ([4160], [4160]) ++ q = torch.ones(1, 4192, 1, 128, dtype=torch.bfloat16) ++ metadata = AttentionMetadata(extra={"npu_attn_varlen": True, "minimax_h3_video_row_cache": state}) ++ calls = [] ++ ++ def selected(query, key, value, scale, cache): ++ assert query is q and key.shape[1] == value.shape[1] == 4160 ++ calls.append(cache.partial) ++ if cache.partial: ++ cache.read_video_context() ++ else: ++ cache.store_video_context(torch.zeros(1, 4096, 1, 128, dtype=torch.bfloat16), torch.zeros(1, 4096, 1, 1)) ++ cache.record_context_attention() ++ return q ++ ++ monkeypatch.setattr(context, "video_context_attention", selected) ++ dummy = torch.ones(4192, 2, dtype=torch.bfloat16) ++ for step in (0, 1, 2): ++ state.begin_step(step) ++ assert impl.forward_fa_npu(q, q, q, metadata) is q ++ state.finish_context_block(0, dummy) ++ state.end_step() ++ assert calls == [False, False, True] ++ state.close() ++ ++ ++def _native_worker(rank): ++ import torch_npu # noqa: F401 ++ ++ torch.npu.set_device(rank) ++ dist.init_process_group( ++ "hccl", init_method="tcp://127.0.0.1:29597", rank=rank, world_size=4, timeout=timedelta(seconds=120) ++ ) ++ try: ++ torch.manual_seed(223) ++ full = [torch.randn(32, 4, 128, dtype=torch.bfloat16).npu() for _ in range(3)] ++ state = make_state(seq=32, used=30, prefix=8, device="npu", start=rank * 8, length=8) ++ dummy = torch.ones(8, 2, dtype=torch.bfloat16, device="npu") ++ for step in (0, 1, 2): ++ state.begin_step(step) ++ if step == 2: ++ full[2][:8] += 0.125 ++ exchanged = [] ++ for tensor in full: ++ send = tensor[rank * 8 : (rank + 1) * 8].reshape(8, 4, 1, 128).permute(1, 0, 2, 3).contiguous() ++ receive = torch.empty_like(send) ++ dist.all_to_all_single(receive, send) ++ exchanged.append(receive.reshape(1, 32, 1, 128)) ++ output = context.video_context_attention( ++ exchanged[0], exchanged[1][:, :30], exchanged[2][:, :30], 128**-0.5, state ++ ) ++ send = output.reshape(4, 8, 1, 128).contiguous() ++ receive = torch.empty_like(send) ++ dist.all_to_all_single(receive, send) ++ local = receive.permute(1, 0, 2, 3).reshape(8, 4, 128) ++ expected = _prefix_bnsd( ++ full[0].unsqueeze(0), full[1][:30].unsqueeze(0), full[2][:30].unsqueeze(0), 128**-0.5 ++ )[0] ++ valid = min(8, max(30 - rank * 8, 0)) ++ torch.testing.assert_close( ++ local[:valid].float(), expected[rank * 8 : rank * 8 + valid].float(), atol=0.03125, rtol=0.02 ++ ) ++ state.finish_context_block(0, dummy) ++ state.end_step() ++ assert state.video_context_refreshes == 2 and state.video_context_reuses == 1 ++ assert state.local_dense_positions.numel() == [8, 0, 0, 0][rank] ++ state.close() ++ assert state.stats()["live_cache_bytes"] == 0 ++ if rank == 0: ++ # Regression shape exceeded the first manual merge grid. Native ++ # tensor operations must cover the complete output and its tail. ++ shape = (1, 65536 * 8 + 1, 1, 128) ++ a = torch.zeros(shape, dtype=torch.bfloat16, device="npu") ++ b = torch.ones_like(a) ++ lse = torch.zeros(shape[:-1] + (1,), dtype=torch.float32, device="npu") ++ mixed = context.mix_context(a, b, lse, lse) ++ assert torch.equal(mixed, torch.full_like(a, 0.5)) ++ torch.npu.synchronize() ++ dist.barrier() ++ finally: ++ dist.destroy_process_group() ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires four native Ascend devices") ++def test_native_four_rank_video_context_lse_merge_and_prefix_updates(): ++ torch.multiprocessing.spawn(_native_worker, nprocs=4, join=True) +diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py +new file mode 100644 +index 0000000..94ff958 +--- /dev/null ++++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py +@@ -0,0 +1,406 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++ ++from datetime import timedelta ++from types import SimpleNamespace ++ ++import pytest ++import torch ++import torch.distributed as dist ++import torch.nn as nn ++ ++from tests.helpers.mark import hardware_marks ++from vllm_omni.diffusion.models.minimax_h3 import minimax_h3_blocks as blocks ++from vllm_omni.diffusion.models.minimax_h3.attention.flash_h3 import _prefix_bnsd, _subset_prefix_bnsd ++from vllm_omni.diffusion.models.minimax_h3.video_row_cache import MiniMaxH3VideoRowCache ++from vllm_omni.platforms import current_omni_platform ++ ++pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=4)] ++ ++ ++def state_for_rows(device="cpu", start=0, length=4): ++ state = MiniMaxH3VideoRowCache(torch.tensor([1, 2], device=device), seq_len=4, used_len=3, num_layers=1) ++ state.prepare_local_span(start, length) ++ return state ++ ++ ++def anchor(state, x): ++ for step in range(5): ++ state.begin_step(step) ++ state.capture(0, x, x + 0.125) ++ state.end_step() ++ state.begin_step(5) ++ ++ ++def test_fixed_schedule_and_row_ownership_excludes_padding(): ++ assert [i for i in range(50) if MiniMaxH3VideoRowCache.is_full_step(i)] == [ ++ 0, ++ 1, ++ 2, ++ 3, ++ 4, ++ 9, ++ 14, ++ 19, ++ 24, ++ 29, ++ 34, ++ 39, ++ 44, ++ 46, ++ 47, ++ 48, ++ 49, ++ ] ++ state = state_for_rows() ++ assert state.global_dense_positions.tolist() == [0] ++ assert state.local_dense_positions.tolist() == [0] ++ assert state.local_video_positions.tolist() == [1, 2] ++ with pytest.raises(RuntimeError, match="ownership"): ++ state.prepare_local_span(0, 3) ++ ++ ++@pytest.mark.parametrize("positions", [[1, 1], [2, 1], [-1], [3]]) ++def test_invalid_target_positions_are_rejected(positions): ++ with pytest.raises(ValueError): ++ MiniMaxH3VideoRowCache(torch.tensor(positions), seq_len=4, used_len=3, num_layers=1) ++ ++ ++def test_delta_is_fp32_and_restore_owns_output(): ++ state = state_for_rows() ++ x = torch.tensor([[1, 2], [3, 4], [5, 6], [7, 8]], dtype=torch.bfloat16) ++ anchor(state, x) ++ assert state._deltas[0].dtype == torch.float32 ++ before = x.clone() ++ output = state.restore(0, x) ++ assert torch.equal(output[1:3], x[1:3] + 0.125) ++ assert torch.equal(output[[0, 3]], x[[0, 3]]) ++ output.zero_() ++ assert torch.equal(x, before) ++ state.end_step() ++ state.close() ++ assert state.closed and not state._deltas and state.global_dense_positions.numel() == 0 ++ ++ ++def large_delta_roundtrip(device="cpu"): ++ state = state_for_rows(device=device) ++ x = torch.full((4, 2), 100000, dtype=torch.bfloat16, device=device) ++ expected = -x ++ delta = expected.float() - x.float() ++ assert torch.isfinite(delta).all() ++ assert not torch.isfinite(delta.to(torch.float16)).all() ++ for step in range(5): ++ state.begin_step(step) ++ state.capture(0, x, expected) ++ state.end_step() ++ state.begin_step(5) ++ actual = state.restore(0, x) ++ assert torch.isfinite(actual).all() ++ assert torch.equal(actual[1:3].cpu().view(torch.uint8), expected[1:3].cpu().view(torch.uint8)) ++ assert torch.equal(actual[[0, 3]], x[[0, 3]]) ++ state.end_step() ++ state.close() ++ assert state.stats()["live_cache_bytes"] == 0 ++ ++ ++def test_large_finite_bf16_delta_roundtrip_exceeds_fp16_storage_range(): ++ large_delta_roundtrip() ++ ++ ++class CountMLP(nn.Module): ++ def __init__(self): ++ super().__init__() ++ self.rows = [] ++ ++ def forward(self, x): ++ self.rows.append(x.shape[0]) ++ return x + 0.25 ++ ++ ++def tiny_block(): ++ block = object.__new__(blocks.MiniMaxH3DiTBlock) ++ nn.Module.__init__(block) ++ block.layer_index = 0 ++ block.norm1 = SimpleNamespace(weight=torch.ones(2), variance_epsilon=1e-6) ++ block.norm2 = SimpleNamespace(weight=torch.ones(2), variance_epsilon=1e-6) ++ block.adaln_proj = lambda t: ( ++ torch.zeros(3, 2), ++ torch.zeros(3, 2), ++ torch.ones(3, 2), ++ torch.zeros(3, 2), ++ torch.zeros(3, 2), ++ torch.ones(3, 2), ++ ) ++ block.attn = lambda x, **kwargs: x * 0.5 ++ block.mlp = CountMLP() ++ return block ++ ++ ++@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) ++def test_real_block_dense_path_and_partial_mlp_empty_rows(start, length): ++ state = state_for_rows(start=start, length=length) ++ block = tiny_block() ++ x = torch.arange(length * 2).reshape(length, 2).to(torch.bfloat16) ++ before = x.clone() ++ kwargs = dict( ++ t_emb=torch.zeros(1), ++ combined_indices=torch.zeros(length, dtype=torch.long), ++ rope_table=None, ++ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), ++ max_seqlen=3, ++ packed_total=4, ++ ) ++ expected = block(x, **kwargs) ++ for step in range(5): ++ state.begin_step(step) ++ actual = block(x, video_row_cache=state, **kwargs) ++ assert torch.equal(actual.view(torch.uint8), expected.view(torch.uint8)) ++ state.end_step() ++ block.mlp.rows.clear() ++ state.begin_step(5) ++ actual = block(x, video_row_cache=state, **kwargs) ++ state.end_step() ++ rows = state.local_dense_positions ++ assert block.mlp.rows == ([rows.numel()] if rows.numel() else []) ++ assert torch.equal(actual.index_select(0, rows), expected.index_select(0, rows)) ++ assert torch.equal(x, before) ++ ++ ++@pytest.mark.parametrize("failure", ["success", "model", "cancel"]) ++def test_request_state_closed_and_removed_on_every_exit(monkeypatch, failure): ++ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop ++ ++ monkeypatch.setenv("H3_VIDEO_ROW_CACHE", "1") ++ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) ++ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) ++ state = MiniMaxH3VideoRowCache(torch.tensor([1]), seq_len=4, used_len=3, num_layers=1) ++ ++ class Model: ++ def create_video_row_cache(self, *args, **kwargs): ++ return state ++ ++ def __call__(self, **kwargs): ++ assert kwargs["video_row_cache"] is state ++ if failure == "model": ++ raise RuntimeError("model failed") ++ state.prepare_local_span(0, 4) ++ x = torch.ones(4, 2, dtype=torch.bfloat16) ++ if state.partial: ++ state.restore(0, x) ++ else: ++ state.capture(0, x, x + 0.125) ++ return torch.zeros(2, 96), torch.zeros(1, 32) ++ ++ class Branch: ++ task = "ref2va" ++ seq_len = 4 ++ used_len = 3 ++ img_pos = torch.tensor([0, 1]) ++ update_mask = torch.tensor([False, True]) ++ update_mask_dev = update_mask ++ audio_update_mask_dev = torch.tensor([True]) ++ locked_audio_rows = None ++ static_kwargs = {"video_token_layout": SimpleNamespace(video_spans=[SimpleNamespace(role="target", start=1)])} ++ ++ def forward_kwargs(self, **kwargs): ++ return dict(self.static_kwargs) ++ ++ monkeypatch.setattr( ++ loop, ++ "minimax_h3_prepare_denoise_rows", ++ lambda **kwargs: (kwargs["initial_video_rows"], kwargs["initial_audio_rows"], torch.zeros(1, 96), None), ++ ) ++ ++ def cancellation(**kwargs): ++ if failure == "cancel": ++ raise RuntimeError("request cancelled") ++ ++ monkeypatch.setattr(loop, "check_request_cancellation", cancellation) ++ branch, model = Branch(), Model() ++ kwargs = dict( ++ model=model, ++ positive=branch, ++ initial_video_rows=torch.zeros(2, 96), ++ initial_audio_rows=torch.zeros(1, 32), ++ keyframe_cond_rows=torch.zeros(1, 96), ++ sigmas_video=[1 - i / 50 for i in range(51)], ++ sigmas_audio=[1 - i / 50 for i in range(51)], ++ device=torch.device("cpu"), ++ ) ++ if failure == "success": ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert state.full_steps == 17 and state.partial_steps == 33 ++ else: ++ with pytest.raises(RuntimeError): ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert state.closed and not state._deltas and "video_row_cache" not in branch.static_kwargs ++ assert model._video_row_cache_stats["closed"] ++ ++ ++def test_default_factory_does_not_create_state(monkeypatch): ++ from vllm_omni.diffusion.models.minimax_h3.minimax_h3_transformer import MiniMaxH3DiTModel ++ ++ monkeypatch.delenv("H3_VIDEO_ROW_CACHE", raising=False) ++ assert ( ++ MiniMaxH3DiTModel.create_video_row_cache(None, torch.tensor([1]), seq_len=4, used_len=3, num_steps=50) is None ++ ) ++ ++ ++@pytest.mark.parametrize("ref2va", [False, True]) ++def test_real_packing_task_selects_ref2va_and_rejects_fl2va(monkeypatch, ref2va): ++ from vllm_omni.diffusion.models.minimax_h3 import denoise_loop as loop ++ from vllm_omni.diffusion.models.minimax_h3.packed_sequence import ( ++ minimax_h3_packed_sequence, ++ minimax_h3_packed_sequence_ref2va_blocks, ++ ) ++ ++ common = dict(text_len=2, latent_t=2, latent_h=4, latent_w=4, audio_t=2) ++ packed = ( ++ minimax_h3_packed_sequence_ref2va_blocks(**common, ref_blocks=[{"kind": "image", "latent_h": 4, "latent_w": 4}]) ++ if ref2va ++ else minimax_h3_packed_sequence(**common, include_keyframe_cond=True, keyframe_frame_indices=[0], frame_count=5) ++ ) ++ branch = loop.MiniMaxH3DenoiseBranch( ++ packed=packed, ++ text_embeddings=torch.zeros(2, 5120), ++ token_tags=packed["token_tags"], ++ device=torch.device("cpu"), ++ task="ref2va" if ref2va else "fl2va", ++ ) ++ layout = branch.static_kwargs["video_token_layout"] ++ assert not any(span.role == "reference" for span in layout.video_spans) ++ monkeypatch.setenv("H3_VIDEO_ROW_CACHE", "1") ++ monkeypatch.setattr(loop.current_omni_platform, "is_npu", lambda: False) ++ monkeypatch.setattr(loop, "minimax_h3_publish_denoise_progress", lambda *args: None) ++ monkeypatch.setattr(loop, "check_request_cancellation", lambda **kwargs: None) ++ states = [] ++ ++ class Model: ++ def create_video_row_cache(self, positions, **kwargs): ++ state = MiniMaxH3VideoRowCache( ++ positions, seq_len=kwargs["seq_len"], used_len=kwargs["used_len"], num_layers=1 ++ ) ++ state.prepare_local_span(0, branch.seq_len) ++ states.append(state) ++ return state ++ ++ def __call__(self, **kwargs): ++ state = kwargs["video_row_cache"] ++ x = torch.ones(branch.seq_len, 2, dtype=torch.bfloat16) ++ if state.partial: ++ state.restore(0, x) ++ else: ++ state.capture(0, x, x + 0.125) ++ return torch.zeros(branch.img_pos.numel(), 96), torch.zeros(branch.audio_pos.numel(), 32) ++ ++ kwargs = dict( ++ model=Model(), ++ positive=branch, ++ initial_video_rows=torch.zeros(branch.img_pos.numel(), 96), ++ initial_audio_rows=torch.zeros(branch.audio_pos.numel(), 32), ++ keyframe_cond_rows=torch.zeros(int((~branch.update_mask).sum()), 96), ++ sigmas_video=[1 - i / 50 for i in range(51)], ++ sigmas_audio=[1 - i / 50 for i in range(51)], ++ device=torch.device("cpu"), ++ ) ++ if ref2va: ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert len(states) == 1 and states[0].closed ++ assert states[0].full_steps == 17 and states[0].partial_steps == 33 ++ else: ++ with pytest.raises(ValueError, match="requires Ref2VA"): ++ loop.minimax_h3_denoise_loop(**kwargs) ++ assert not states ++ ++ ++@pytest.mark.parametrize("start,length", [(0, 4), (1, 2)]) ++def test_attention_keeps_collective_path_and_projects_only_dense_local_rows(start, length): ++ state = state_for_rows(start=start, length=length) ++ x = torch.ones(length, 2, dtype=torch.bfloat16) ++ anchor(state, x) ++ attention = object.__new__(blocks.MiniMaxH3Attention) ++ nn.Module.__init__(attention) ++ attention.num_heads = 1 ++ attention.head_dim = 2 ++ attention.vdn = None ++ attention.to_gate_compress = None ++ calls, projected_rows = [], [] ++ attention._project_qkv = lambda *args: (x.reshape(length, 1, 2),) * 5 ++ ++ def packed(q, k, v, **kwargs): ++ calls.append(q.shape[0]) ++ assert kwargs["video_row_cache"] is state ++ return q ++ ++ def project(rows): ++ projected_rows.append(rows.shape[0]) ++ return rows * 2, None ++ ++ attention._run_packed_attention = packed ++ attention.out_proj = project ++ output = attention( ++ x, ++ rope_table=None, ++ cu_seqlens=torch.tensor([0, 3, 4], dtype=torch.int32), ++ max_seqlen=3, ++ packed_total=4, ++ video_row_cache=state, ++ ) ++ rows = state.local_dense_positions ++ assert calls == [length] ++ assert projected_rows == ([rows.numel()] if rows.numel() else []) ++ assert torch.equal(output.index_select(0, rows), (x * 2).index_select(0, rows)) ++ assert torch.count_nonzero(output.index_select(0, state.local_video_positions)) == 0 ++ ++ ++def _native_worker(rank): ++ import torch_npu # noqa: F401 ++ ++ torch.npu.set_device(rank) ++ dist.init_process_group( ++ "hccl", init_method="tcp://127.0.0.1:29597", rank=rank, world_size=4, timeout=timedelta(seconds=120) ++ ) ++ try: ++ large_delta_roundtrip(device=f"npu:{rank}") ++ torch.manual_seed(19) ++ full = [torch.randn(32, 4, 128, dtype=torch.bfloat16).npu() for _ in range(3)] ++ exchanged = [] ++ for tensor in full: ++ local = tensor[rank * 8 : (rank + 1) * 8] ++ send = local.reshape(8, 4, 1, 128).permute(1, 0, 2, 3).contiguous() ++ receive = torch.empty_like(send) ++ dist.all_to_all_single(receive, send) ++ exchanged.append(receive.reshape(1, 32, 1, 128)) ++ state = MiniMaxH3VideoRowCache(torch.arange(8, 24, device="npu"), seq_len=32, used_len=30, num_layers=1) ++ state.prepare_local_span(rank * 8, 8) ++ x = torch.ones(8, 2, dtype=torch.bfloat16, device="npu") ++ anchor(state, x) ++ output = _subset_prefix_bnsd(exchanged[0], exchanged[1][:, :30], exchanged[2][:, :30], 128**-0.5, state) ++ send = output.reshape(4, 8, 1, 128).contiguous() ++ receive = torch.empty_like(send) ++ dist.all_to_all_single(receive, send) ++ local_output = receive.permute(1, 0, 2, 3).reshape(8, 4, 128) ++ expected = _prefix_bnsd(full[0].unsqueeze(0), full[1][:30].unsqueeze(0), full[2][:30].unsqueeze(0), 128**-0.5)[ ++ 0 ++ ] ++ rows = state.local_dense_positions ++ torch.testing.assert_close( ++ local_output.index_select(0, rows).float(), ++ expected[rank * 8 : (rank + 1) * 8].index_select(0, rows).float(), ++ atol=0.03125, ++ rtol=0.02, ++ ) ++ assert state.subset_attention_calls == 1 and state.subset_query_rows == 14 ++ assert rows.numel() == [8, 0, 0, 6][rank] ++ state.restore(0, x) ++ state.end_step() ++ state.close() ++ torch.npu.synchronize() ++ dist.barrier() ++ finally: ++ dist.destroy_process_group() ++ ++ ++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires four native Ascend devices") ++def test_native_four_rank_subset_attention_with_empty_local_dense_rows(): ++ torch.multiprocessing.spawn(_native_worker, nprocs=4, join=True) +diff --git a/vllm_omni/config/omni_config.py b/vllm_omni/config/omni_config.py +index 0a803cf..44fc48c 100644 +--- a/vllm_omni/config/omni_config.py ++++ b/vllm_omni/config/omni_config.py +@@ -863,6 +863,7 @@ class _DiffusionConfigProjection(_TrackExplicitConfigFields): + dlo_use_allgather: bool = True + dlo_chunk_size_mb: int = Field(default=64, gt=0, strict=True) + dlo_resident_layers: int = Field(default=0, ge=0) ++ dlo_order_allgather_after_attn_a2a: bool = False + host_weight_runtime_mode: Literal["disabled", "preferred", "required"] = "disabled" + host_weight_runtime_root: str | None = None + dlo_host_registration_limit_gib: float = Field(default=0.0, ge=0) +diff --git a/vllm_omni/config/stage_config.py b/vllm_omni/config/stage_config.py +index 76a58a9..3c69986 100644 +--- a/vllm_omni/config/stage_config.py ++++ b/vllm_omni/config/stage_config.py +@@ -672,6 +672,7 @@ class StageDeployConfig: + dlo_use_allgather: bool | None = None + dlo_chunk_size_mb: int | None = None + dlo_resident_layers: int | None = None ++ dlo_order_allgather_after_attn_a2a: bool | None = None + host_weight_runtime_mode: str | None = None + host_weight_runtime_root: str | None = None + dlo_host_registration_limit_gib: float | None = None +diff --git a/vllm_omni/diffusion/attention/layer.py b/vllm_omni/diffusion/attention/layer.py +index 5520390..574eafa 100644 +--- a/vllm_omni/diffusion/attention/layer.py ++++ b/vllm_omni/diffusion/attention/layer.py +@@ -9,7 +9,7 @@ + + from collections.abc import Mapping + from dataclasses import replace +-from typing import cast ++from typing import Any, cast + + import torch + import torch.nn as nn +@@ -333,6 +333,11 @@ class Attention(nn.Module): + # Local strategy when SP is intentionally inactive outside sharded regions. + self._no_parallel_strategy = NoParallelAttention() + ++ # Optional cross-module rendezvous (see distributed/collective_gate.py). ++ # Installed by the DLO backend when it must keep its background weight ++ # AllGather from taking the links ahead of this layer's QKV AllToAll. ++ self._collective_gate: Any | None = None ++ + self.layer_idx: int | None = _try_extract_layer_index(prefix) + + self._kv_cache_dtype: str | None = None +@@ -506,10 +511,12 @@ class Attention(nn.Module): + reason = self._cross_backend_limitation(attn_metadata) + if reason is not None: + blocked = self._cross_fallback_blocker(attn_metadata) +- result = self.sdpa_fallback.resolve_execution_path(resolved_context, query, key, value, attn_metadata) ++ result = cast(SDPAImpl, self.sdpa_fallback).resolve_execution_path( ++ resolved_context, query, key, value, attn_metadata ++ ) + return replace( + result, +- backend=self.attn_backend.get_name() if blocked else "SDPA", ++ backend=cast(type[AttentionBackend], self.attn_backend).get_name() if blocked else "SDPA", + path="cross_unsupported" if blocked else "cross_sdpa_fallback", + support=CapabilityResult.unsupported(blocked) if blocked else CapabilityResult.unmigrated(reason), + ) +@@ -591,6 +598,12 @@ class Attention(nn.Module): + # For Ring: Concat joint_q + query, key, value, attn_metadata, ctx = strategy.pre_attention(query, key, value, attn_metadata) + ++ # The QKV AllToAll is now enqueued on this stream. Release any gather ++ # that was deferred to give this collective the links first. ++ gate = self._collective_gate ++ if gate is not None: ++ gate.signal() ++ + # MiniMax-H3 Tier2 stores reference K/V in the balanced layout produced + # by Ulysses. Keeping this hook at the generic post-parallel boundary + # avoids a model import in the parallel strategy and leaves other models +@@ -760,7 +773,7 @@ class Attention(nn.Module): + def _cross_fallback_blocker(self, attn_metadata: AttentionMetadata | None) -> str | None: + if self.backend_explicit: + return ( +- f"Explicit backend {self.attn_backend.get_name()} does not support " ++ f"Explicit backend {cast(type[AttentionBackend], self.attn_backend).get_name()} does not support " + f"{self._cross_backend_limitation(attn_metadata)}" + ) + if attn_metadata is not None and attn_metadata.extra.get("kv_cache_dtype") not in (None, "float"): +@@ -793,7 +806,7 @@ class Attention(nn.Module): + blocked = self._cross_fallback_blocker(attn_metadata) + if blocked is not None: + raise NotImplementedError(blocked) +- return self.sdpa_fallback.forward(query, key, value, attn_metadata) ++ return cast(SDPAImpl, self.sdpa_fallback).forward(query, key, value, attn_metadata) + dense_mask = attn_metadata.attn_mask if attn_metadata is not None else None + # Piecewise backends honor full_attn_spans ahead of attn_mask and never read the + # dense mask (e.g. HunyuanImage3, which sends both), so there is nothing to divert. +diff --git a/vllm_omni/diffusion/data.py b/vllm_omni/diffusion/data.py +index 8528574..fcbeec0 100644 +--- a/vllm_omni/diffusion/data.py ++++ b/vllm_omni/diffusion/data.py +@@ -962,6 +962,10 @@ class OmniDiffusionConfig: + dlo_chunk_size_mb: int = 64 + # Leading main-DiT blocks kept resident by distributed layerwise offload. + dlo_resident_layers: int = 0 ++ # Hold each DiT block's weight AllGather until that block's QKV AllToAll has ++ # been enqueued, so the background collective does not take the links ahead ++ # of the critical-path all-to-all. ++ dlo_order_allgather_after_attn_a2a: bool = False + # Final-layout Host Weight Runtime policy. The loader only activates this + # for eligible no-AllGather DLO; all other configurations preserve their + # existing loader/storage path. +diff --git a/vllm_omni/diffusion/distributed/collective_gate.py b/vllm_omni/diffusion/distributed/collective_gate.py +new file mode 100644 +index 0000000..1386d89 +--- /dev/null ++++ b/vllm_omni/diffusion/distributed/collective_gate.py +@@ -0,0 +1,72 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++"""Cross-module rendezvous that orders background collectives after the ++foreground all-to-all of a transformer block. ++ ++Distributed layerwise offload (DLO) issues the *next* block's weight AllGather ++on a dedicated ``comm_stream`` while the *current* block computes. Both that ++AllGather and the block's QKV sequence-parallel AllToAll are collectives over ++the same ranks, so whichever reaches the links first wins: the AllGather is ++submitted at block entry, well before the block reaches its attention, and the ++critical-path AllToAll ends up queued behind it. ++ ++A plain event wait cannot fix this by itself. ``wait_event`` snapshots the ++records enqueued *before* the call, so "enqueue the wait at block entry, record ++the event later in the block" is a no-op -- the wait resolves against the ++previous iteration's record. The gather therefore has to be *issued* after the ++AllToAll: the hook arms this barrier at block entry and defers the gather, the ++attention signals it right after ``strategy.pre_attention``, and the signal ++records the event and lets the hook submit the gather gated on it. ++ ++The gather still overlaps the rest of the block (attention output, MLP, the ++next block's projections); it simply cannot start before the AllToAll finished. ++""" ++ ++from __future__ import annotations ++ ++from collections.abc import Callable ++ ++__all__ = ["QKVAllToAllBarrier"] ++ ++ ++class QKVAllToAllBarrier: ++ """One-block rendezvous between a DLO hook and the block's attention. ++ ++ Owned by a single :class:`DistributedLayerwiseOffloadHook` (one per ++ streaming block) and shared with the attention layer inside that block. ++ Both live on the same block, so the arming and the signalling always come ++ from the same module forward. ++ ++ Args: ++ on_signal: Invoked at most once per :meth:`arm` when the block's ++ attention reports its QKV AllToAll. It must submit the deferred ++ weight AllGather. ++ """ ++ ++ def __init__(self, on_signal: Callable[[], None]) -> None: ++ self._on_signal = on_signal ++ self._armed = False ++ ++ @property ++ def armed(self) -> bool: ++ """True between :meth:`arm` and the matching :meth:`signal`.""" ++ return self._armed ++ ++ def arm(self) -> None: ++ """Mark the next attention QKV AllToAll as the gather's gate.""" ++ self._armed = True ++ ++ def disarm(self) -> None: ++ """Drop a pending arm without firing it (the caller flushes instead).""" ++ self._armed = False ++ ++ def signal(self) -> None: ++ """Report that the QKV AllToAll is now enqueued on the compute stream. ++ ++ A no-op unless armed, so it is cheap to call unconditionally from the ++ attention hot path. ++ """ ++ if not self._armed: ++ return ++ self._armed = False ++ self._on_signal() +diff --git a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py +index 6ebfb10..da008bf 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py ++++ b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py +@@ -36,6 +36,16 @@ def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, sc + return output.reshape(batch, heads, rows, dim).transpose(1, 2).contiguous() + + ++def _subset_prefix_bnsd(query, key, value, scale, state): ++ """Compute only fresh conditioning queries; preserve the shared SP shape.""" ++ rows = state.global_dense_positions ++ dense = _prefix_bnsd(query.index_select(1, rows), key, value, scale) ++ output = torch.zeros_like(query) ++ output.index_copy_(1, rows, dense) ++ state.record_subset_attention(rows.numel()) ++ return output ++ ++ + class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + """Use independent head batches for explicit long BF16 prefix attention. + +@@ -51,7 +61,10 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + value: torch.Tensor, + attn_metadata: AttentionMetadata | None = None, + ) -> torch.Tensor: ++ state = attn_metadata.extra.get("minimax_h3_video_row_cache") if attn_metadata is not None else None + if os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" or attn_metadata is None: ++ if state is not None: ++ raise RuntimeError("H3 video-row cache requires the explicit fused attention path") + return super().forward_fa_npu(query, key, value, attn_metadata) + extra = attn_metadata.extra + if ( +@@ -68,9 +81,25 @@ class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl): + or query.shape[2:] != value.shape[2:] + or key.shape[1] != value.shape[1] + ): ++ if state is not None: ++ raise RuntimeError("H3 video-row cache received unsupported fused attention inputs") + return super().forward_fa_npu(query, key, value, attn_metadata) + resolved = self._resolve_packed_seq_npu(query, key, extra) + if resolved is None: ++ if state is not None: ++ raise RuntimeError("H3 video-row cache requires a resolved packed prefix") + return super().forward_fa_npu(query, key, value, attn_metadata) + used_k = resolved[1][0] ++ if state is not None: ++ if ( ++ (not state.partial and not state.context_only) ++ or used_k != state.used_len ++ or query.shape[1] != state.seq_len ++ ): ++ raise RuntimeError("H3 video-row cache attention layout changed") ++ if state.context_only: ++ from .video_context import video_context_attention ++ ++ return video_context_attention(query, key[:, :used_k], value[:, :used_k], self.softmax_scale, state) ++ return _subset_prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale, state) + return _prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale) +diff --git a/vllm_omni/diffusion/models/minimax_h3/attention/video_context.py b/vllm_omni/diffusion/models/minimax_h3/attention/video_context.py +new file mode 100644 +index 0000000..4603233 +--- /dev/null ++++ b/vllm_omni/diffusion/models/minimax_h3/attention/video_context.py +@@ -0,0 +1,77 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++"""Request-owned video context with fresh global conditioning attention.""" ++ ++from __future__ import annotations ++ ++import torch ++ ++from .flash_h3 import _prefix_bnsd ++ ++ ++def prefix_attention_lse(query, key, value, scale): ++ """Use native FAS output and its FP32 softmax statistics in BSND order.""" ++ import torch_npu ++ ++ batch, rows, heads, dim = query.shape ++ ++ def fold(tensor): ++ return tensor.transpose(1, 2).contiguous().reshape(batch * heads, 1, tensor.shape[1], dim) ++ ++ output, maximum, total, *_ = torch_npu.npu_fusion_attention( ++ fold(query), ++ fold(key), ++ fold(value), ++ head_num=1, ++ input_layout="BNSD", ++ scale=scale, ++ pre_tockens=2147483647, ++ next_tockens=2147483647, ++ sync=False, ++ ) ++ output = output.reshape(batch, heads, rows, dim).transpose(1, 2).contiguous() ++ lse = (maximum[..., 0].float() + total[..., 0].float().log()).reshape(batch, heads, rows) ++ return output, lse.transpose(1, 2).unsqueeze(-1).contiguous() ++ ++ ++def mix_context(prefix, video, prefix_lse, video_lse): ++ if ( ++ prefix.shape != video.shape ++ or prefix.dtype != torch.bfloat16 ++ or video.dtype != prefix.dtype ++ or prefix_lse.shape != prefix.shape[:-1] + (1,) ++ or video_lse.shape != prefix_lse.shape ++ or prefix_lse.dtype != torch.float32 ++ or video_lse.dtype != torch.float32 ++ or not all(t.is_contiguous() for t in (prefix, video, prefix_lse, video_lse)) ++ ): ++ raise ValueError("H3 context merge requires contiguous BF16 outputs and FP32 row/head LSE") ++ maximum = torch.maximum(prefix_lse, video_lse) ++ weight_a, weight_b = (prefix_lse - maximum).exp(), (video_lse - maximum).exp() ++ return ((weight_a * prefix.float() + weight_b * video.float()) / (weight_a + weight_b)).to(prefix.dtype) ++ ++ ++def video_context_attention(query, key, value, scale, state): ++ prefix = state.target_start ++ if ( ++ not state.context_only ++ or query.shape[0] != 1 ++ or query.shape[1] != state.seq_len ++ or key.shape[1] != state.used_len ++ or value.shape != key.shape ++ or query.shape[2:] != key.shape[2:] ++ or not 0 < prefix < state.used_len ++ ): ++ raise ValueError("H3 video context requires the final target span and original packed query order") ++ output = torch.zeros_like(query) ++ output[:, :prefix] = _prefix_bnsd(query[:, :prefix], key, value, scale) ++ video_query = query[:, prefix : state.used_len] ++ fresh, fresh_lse = prefix_attention_lse(video_query, key[:, :prefix], value[:, :prefix], scale) ++ if state.partial: ++ video, video_lse = state.read_video_context() ++ else: ++ video, video_lse = prefix_attention_lse(video_query, key[:, prefix:], value[:, prefix:], scale) ++ state.store_video_context(video, video_lse) ++ output[:, prefix : state.used_len] = mix_context(fresh, video, fresh_lse, video_lse) ++ state.record_context_attention() ++ return output +diff --git a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py +index 8cca219..07fb372 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py ++++ b/vllm_omni/diffusion/models/minimax_h3/denoise_loop.py +@@ -9,6 +9,7 @@ rows stay pinned to their noised step-0 anchors. + + from __future__ import annotations + ++import os + from collections.abc import Callable + from contextlib import AbstractContextManager, nullcontext + from typing import Any +@@ -69,7 +70,9 @@ class MiniMaxH3DenoiseBranch: + text_embeddings: torch.Tensor, + token_tags: torch.Tensor, + device: torch.device, ++ task: str | None = None, + ) -> None: ++ self.task = task + seq_len = int(packed["seq_len"]) + self.seq_len = seq_len + self.img_pos = packed["img_pos"].view(-1).to(torch.long) +@@ -380,6 +383,24 @@ def minimax_h3_denoise_loop( + positive.static_kwargs["reference_kv_state"] = reference_kv_state + + try: ++ video_row_cache = None ++ create_video_cache = getattr(model, "create_video_row_cache", None) ++ if os.environ.get("H3_VIDEO_ROW_CACHE") == "1": ++ if ( ++ not callable(create_video_cache) ++ or video_edit is not None ++ or audio_edit is not None ++ or keyframe_cond_rows is None ++ or positive.task != "ref2va" ++ ): ++ raise ValueError("H3 video-row cache requires Ref2VA image references without editing") ++ video_row_cache = create_video_cache( ++ positive.img_pos[positive.update_mask].to(device), ++ seq_len=positive.seq_len, ++ used_len=positive.used_len, ++ num_steps=num_steps, ++ ) ++ positive.static_kwargs["video_row_cache"] = video_row_cache + for step in range(num_steps): + check_request_cancellation() + step_cm = step_profiler(step) if step_profiler is not None else nullcontext() +@@ -388,6 +409,8 @@ def minimax_h3_denoise_loop( + s_a = sigmas_audio[step] + if reference_kv_state is not None: + reference_kv_state.begin_step(step, sigma=float(s_v)) ++ if video_row_cache is not None: ++ video_row_cache.begin_step(step) + # Publish where we are so step-gated attention features (the dense + # warmup of RAINFUSION_ATTN, the timestep gate of TRTLLM_ATTN) can + # see it. Gates use the scheduler-style descending timestep, which +@@ -430,6 +453,8 @@ def minimax_h3_denoise_loop( + v_video, v_audio = model(**fk) + if reference_kv_state is not None: + reference_kv_state.end_step() ++ if video_row_cache is not None: ++ video_row_cache.end_step() + mv_video_t = v_video.float()[update] + mv_audio_t = v_audio.float()[audio_update] + +@@ -476,6 +501,10 @@ def minimax_h3_denoise_loop( + + finally: + minimax_h3_publish_denoise_progress(None, None, None) ++ if video_row_cache is not None: ++ positive.static_kwargs.pop("video_row_cache", None) ++ video_row_cache.close() ++ model._video_row_cache_stats = video_row_cache.stats() + if reference_kv_state is not None: + positive.static_kwargs.pop("reference_kv_state", None) + reference_kv_state.close() +diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py +index 09f2e7d..ae870ed 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py ++++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py +@@ -45,6 +45,7 @@ if TYPE_CHECKING: + from vllm.model_executor.layers.quantization.base_config import QuantizationConfig + + from .reference_kv_cache import MiniMaxH3ReferenceKVTier2State ++ from .video_row_cache import MiniMaxH3VideoRowCache + + _BF16_DTYPE = torch.bfloat16 + _FP32_DTYPE = torch.float32 +@@ -291,6 +292,7 @@ class MiniMaxH3Attention(nn.Module): + reference_kv_layer_index: int | None = None, + reference_kv_compact: bool = False, + vdn_window: VDNLayout | None = None, ++ video_row_cache: MiniMaxH3VideoRowCache | None = None, + ) -> torch.Tensor: + """Run packed attention as a small eager island. + +@@ -368,6 +370,8 @@ class MiniMaxH3Attention(nn.Module): + "vsa_h3_ref_policy": self.vsa_ref_policy, + "vsa_h3_ref_keep_rate": self.vsa_ref_keep_rate, + } ++ if video_row_cache is not None and (video_row_cache.partial or video_row_cache.context_only): ++ extra["minimax_h3_video_row_cache"] = video_row_cache + if gate_compress is not None: + extra["gate_compress"] = gate_compress.unsqueeze(0) + if video_layout is not None and video_layout.video_spans: +@@ -455,6 +459,7 @@ class MiniMaxH3Attention(nn.Module): + reference_kv_layer_index: int | None = None, + reference_kv_compact: bool = False, + vdn_window: VDNLayout | None = None, ++ video_row_cache: MiniMaxH3VideoRowCache | None = None, + ) -> torch.Tensor: + """x: [T, hidden] packed thd rows -> [T, hidden]. + +@@ -508,10 +513,19 @@ class MiniMaxH3Attention(nn.Module): + reference_kv_layer_index=reference_kv_layer_index, + reference_kv_compact=reference_kv_compact, + vdn_window=vdn_window, ++ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), + ) + if self.vdn is not None: + out = self.vdn.gate_softmax(out, x) + out = out.reshape(total, self.num_heads * self.head_dim) ++ if video_row_cache is not None and video_row_cache.partial and not video_row_cache.context_only: ++ rows = video_row_cache.local_dense_positions ++ assert rows is not None ++ projected = x.new_zeros((total, x.shape[-1])) ++ if rows.numel(): ++ dense, _ = self.out_proj(out.index_select(0, rows)) ++ projected.index_copy_(0, rows, dense) ++ return projected + out, _ = self.out_proj(out) + if self.vdn is not None: + out = self.vdn.add_linear(out, x, q_raw, k_raw, v, vdn_window) +@@ -742,6 +756,7 @@ class MiniMaxH3DiTBlock(nn.Module): + reference_kv_state: MiniMaxH3ReferenceKVTier2State | None = None, + reference_kv_compact: bool = False, + vdn_window: VDNLayout | None = None, ++ video_row_cache: MiniMaxH3VideoRowCache | None = None, + ) -> torch.Tensor: + """x: [T, H]; t_emb: [M, t_dim]; combined_indices: [T] + (= inverse_indices * modality_num + token_tags.clamp(min=0)). +@@ -759,6 +774,7 @@ class MiniMaxH3DiTBlock(nn.Module): + gate_mlp, + ) = self.adaln_proj(t_emb) + ++ block_input = x + residual = x + h = rms_norm_indexed_scale_shift( + x, +@@ -782,7 +798,10 @@ class MiniMaxH3DiTBlock(nn.Module): + reference_kv_layer_index=self.layer_index, + reference_kv_compact=reference_kv_compact, + vdn_window=vdn_window, ++ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), + ) ++ if video_row_cache is not None and video_row_cache.pre_gate: ++ h = video_row_cache.pre_gate_output(self.layer_index, h, "attn") + x, h = indexed_gate_rms_norm_scale_shift( + residual, + gate_msa, +@@ -793,6 +812,39 @@ class MiniMaxH3DiTBlock(nn.Module): + combined_indices, + self.norm2.variance_epsilon, + ) ++ if video_row_cache is not None and video_row_cache.pre_gate and video_row_cache.partial: ++ video_row_cache.record_fresh_gate("attn") + residual = x ++ if video_row_cache is not None and video_row_cache.partial and not video_row_cache.context_only: ++ rows = video_row_cache.local_dense_positions ++ assert rows is not None ++ if video_row_cache.pre_gate: ++ mlp_output = torch.zeros_like(h) ++ if rows.numel(): ++ mlp_output.index_copy_(0, rows, self.mlp(h.index_select(0, rows))) ++ mlp_output = video_row_cache.pre_gate_output(self.layer_index, mlp_output, "mlp") ++ output = indexed_gate(residual, gate_mlp, mlp_output, combined_indices) ++ video_row_cache.record_fresh_gate("mlp") ++ video_row_cache.partial_mlp_rows += rows.numel() ++ return output ++ output = video_row_cache.restore(self.layer_index, block_input) ++ if rows.numel(): ++ dense = indexed_gate( ++ residual.index_select(0, rows), ++ gate_mlp, ++ self.mlp(h.index_select(0, rows)), ++ combined_indices.index_select(0, rows), ++ ) ++ output.index_copy_(0, rows, dense) ++ video_row_cache.partial_mlp_rows += rows.numel() ++ return output + h = self.mlp(h) +- return indexed_gate(residual, gate_mlp, h, combined_indices) ++ if video_row_cache is not None and video_row_cache.pre_gate: ++ h = video_row_cache.pre_gate_output(self.layer_index, h, "mlp") ++ output = indexed_gate(residual, gate_mlp, h, combined_indices) ++ if video_row_cache is not None and not video_row_cache.pre_gate: ++ if video_row_cache.context_only: ++ video_row_cache.finish_context_block(self.layer_index, block_input) ++ else: ++ video_row_cache.capture(self.layer_index, block_input, output) ++ return output +diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py +index 9518828..189aa31 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py ++++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py +@@ -10,6 +10,7 @@ layout. + from __future__ import annotations + + import math ++import os + from bisect import bisect_left + from collections.abc import Iterable, Mapping + from dataclasses import replace +@@ -66,6 +67,8 @@ if TYPE_CHECKING: + from vllm_omni.diffusion.data import OmniDiffusionConfig + from vllm_omni.diffusion.models.minimax_h3.reference_kv_cache import MiniMaxH3ReferenceKVTier2State + ++ from .video_row_cache import MiniMaxH3VideoRowCache ++ + logger = init_logger(__name__) + + +@@ -245,6 +248,7 @@ _FORWARD_SUPPORTED_KWARGS = frozenset( + "control_rows", + "control_context_scale", + "reference_kv_state", ++ "video_row_cache", + } + ) + +@@ -1232,6 +1236,42 @@ class MiniMaxH3DiTModel(nn.Module): + t_emb = self.time_embedder(unique_timesteps) + return embeddings, t_emb + ++ def create_video_row_cache( ++ self, target_positions: torch.Tensor, *, seq_len: int, used_len: int, num_steps: int ++ ) -> MiniMaxH3VideoRowCache | None: ++ if os.environ.get("H3_VIDEO_ROW_CACHE") != "1": ++ return None ++ p = self.parallel_config ++ if ( ++ not current_omni_platform.is_npu() ++ or num_steps != 50 ++ or p.tensor_parallel_size != 1 ++ or p.data_parallel_size not in (None, 1) ++ or p.cfg_parallel_size != 1 ++ or p.pipeline_parallel_size != 1 ++ or p.ulysses_degree != 4 ++ or p.ring_degree != 1 ++ or p.allgather_degree != 1 ++ or p.ulysses_mode != "strict" ++ or self.od_config.cache_backend not in (None, "none") ++ or self.od_config.kv_offload_config is not None ++ or self.vdn_config is not None ++ or os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" ++ or any(block.attn.attention.attn_backend.get_name() != "FLASH_ATTN" for block in self.blocks) ++ ): ++ raise ValueError("H3 video-row cache requires dense NPU BF16 Ref2VA with strict Ulysses4, TP/DP/CFG/PP1") ++ from .video_row_cache import MiniMaxH3VideoRowCache ++ ++ self._video_row_cache_stats = None ++ return MiniMaxH3VideoRowCache( ++ target_positions, ++ seq_len=seq_len, ++ used_len=used_len, ++ num_layers=len(self.blocks), ++ pre_gate=os.environ.get("H3_VIDEO_PRE_GATE_CACHE") == "1", ++ context_only=os.environ.get("H3_VIDEO_CONTEXT_CACHE") == "1", ++ ) ++ + def create_reference_kv_state( + self, + *, +@@ -1308,6 +1348,14 @@ class MiniMaxH3DiTModel(nn.Module): + refiner_max = int(self._psp_field(refiner_psp, "refiner_packed_seq_params", "max_seqlen_q")) + video_layout = kwargs.get("video_token_layout") + reference_kv_state = kwargs.get("reference_kv_state") ++ video_row_cache = kwargs.get("video_row_cache") ++ if video_row_cache is not None and ( ++ num_requests != 1 ++ or reference_kv_state is not None ++ or self.vdn_config is not None ++ or kwargs.get("control_rows") is not None ++ ): ++ raise ValueError("H3 video-row cache cannot combine with packed requests, reference KV, VDN or control") + vdn_window = None + if self.vdn_config is not None: + # Text rows lead the packed document. +@@ -1442,6 +1490,11 @@ class MiniMaxH3DiTModel(nn.Module): + if reference_kv_state is not None: + reference_kv_state.set_local_sequence_rows(hidden.shape[0]) + ++ if video_row_cache is not None: ++ if hidden.shape[0] != local_len: ++ raise RuntimeError("H3 video-row cache requires the strict local sequence shard") ++ video_row_cache.prepare_local_span(local_start, local_len) ++ + block_video_layout = video_layout + if compact_reference: + block_video_layout = None +@@ -1505,6 +1558,7 @@ class MiniMaxH3DiTModel(nn.Module): + reference_kv_state=reference_kv_state, + reference_kv_compact=compact_reference, + vdn_window=vdn_window, ++ **({"video_row_cache": video_row_cache} if video_row_cache is not None else {}), + ) + if block_index in hints: + hidden = hidden + hints.pop(block_index) * control_scale +diff --git a/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py b/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py +index 476c3cc..ebce8fc 100644 +--- a/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py ++++ b/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py +@@ -2115,6 +2115,7 @@ class MiniMaxH3Pipeline( + text_embeddings=text_embeddings, + token_tags=tags, + device=self.device, ++ task=task, + ) + + if control_rows is not None: +diff --git a/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py b/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py +new file mode 100644 +index 0000000..0670cf1 +--- /dev/null ++++ b/vllm_omni/diffusion/models/minimax_h3/video_row_cache.py +@@ -0,0 +1,299 @@ ++# SPDX-License-Identifier: Apache-2.0 ++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project ++"""Request-owned, experimental target-video block residuals for H3 Ref2VA.""" ++ ++from __future__ import annotations ++ ++import torch ++ ++ ++class MiniMaxH3VideoRowCache: ++ """Keep non-video computation dense while reusing video block updates. ++ ++ The first experimental profile is deliberately fixed. No state is shared ++ across requests, and every rank retains all attention collectives. ++ """ ++ ++ def __init__( ++ self, ++ target_positions: torch.Tensor, ++ *, ++ seq_len: int, ++ used_len: int, ++ num_layers: int, ++ pre_gate: bool = False, ++ context_only: bool = False, ++ ) -> None: ++ positions = target_positions.detach().cpu().tolist() ++ if not 0 < used_len <= seq_len or not positions or num_layers < 1: ++ raise ValueError("video-row cache requires a nonempty valid packed document") ++ if positions != sorted(set(positions)) or positions[0] < 0 or positions[-1] >= used_len: ++ raise ValueError("target-video positions must be sorted unique valid rows") ++ target_set = set(positions) ++ dense = [i for i in range(used_len) if i not in target_set] ++ if not dense: ++ raise ValueError("video-row cache requires non-video conditioning rows") ++ if context_only and (pre_gate or positions != list(range(positions[0], used_len))): ++ raise ValueError("video context cache requires one final target span and fresh FFN") ++ self.seq_len = seq_len ++ self.used_len = used_len ++ self.num_layers = num_layers ++ self._target_positions = positions ++ self.global_dense_positions = torch.tensor(dense, dtype=torch.long, device=target_positions.device) ++ self.local_dense_positions: torch.Tensor | None = None ++ self.local_video_positions: torch.Tensor | None = None ++ self._local_span: tuple[int, int] | None = None ++ self._deltas: dict[int, torch.Tensor] = {} ++ self.pre_gate = pre_gate ++ self.context_only = context_only ++ self.target_start = positions[0] ++ self._video_contexts: dict[int, tuple[torch.Tensor, torch.Tensor]] = {} ++ self._context_calls_this_step = 0 ++ self.video_context_refreshes = 0 ++ self.video_context_reuses = 0 ++ self.fresh_video_prefix_calls = 0 ++ self.full_row_mlp_blocks = 0 ++ self._attention_outputs: dict[int, torch.Tensor] = {} ++ self._mlp_outputs: dict[int, torch.Tensor] = {} ++ self._awaiting_mlp_layer: int | None = None ++ self.fresh_attention_gates = 0 ++ self.fresh_mlp_gates = 0 ++ self._step = -1 ++ self._blocks_this_step = 0 ++ self.partial = False ++ self.closed = False ++ self.full_steps = 0 ++ self.partial_steps = 0 ++ self.full_blocks = 0 ++ self.partial_blocks = 0 ++ self.subset_attention_calls = 0 ++ self.subset_query_rows = 0 ++ self.omitted_query_rows = 0 ++ self.partial_mlp_rows = 0 ++ self.peak_cache_bytes = 0 ++ ++ @staticmethod ++ def is_full_step(step: int) -> bool: ++ if not 0 <= step < 50: ++ raise ValueError("the video-row cache profile requires 50 steps") ++ return step < 4 or step >= 46 or (step - 4) % 5 == 0 ++ ++ def begin_step(self, step: int) -> None: ++ if self.closed or step != self._step + 1: ++ raise RuntimeError("video-row cache step order changed") ++ if self._step >= 0 and self._blocks_this_step != self.num_layers: ++ raise RuntimeError("previous video-row cache step is incomplete") ++ full = self.is_full_step(step) ++ if self.pre_gate or self.context_only: ++ full = step in (0, 1, 4, 9, 19, 29, 39, 49) ++ self.partial = not full ++ if self._awaiting_mlp_layer is not None: ++ raise RuntimeError("previous video pre-gate sublayer is incomplete") ++ complete = ( ++ len(self._attention_outputs) == len(self._mlp_outputs) == self.num_layers ++ if self.pre_gate ++ else len(self._video_contexts) == self.num_layers ++ if self.context_only ++ else len(self._deltas) == self.num_layers ++ ) ++ if self.partial and not complete: ++ raise RuntimeError("partial video step has no complete dense anchor") ++ self._step = step ++ self._blocks_this_step = 0 ++ self._context_calls_this_step = 0 ++ if self.partial: ++ self.partial_steps += 1 ++ else: ++ self.full_steps += 1 ++ ++ def prepare_local_span(self, start: int, length: int) -> None: ++ if self.closed or start < 0 or length < 1 or start + length > self.seq_len: ++ raise ValueError("invalid local packed span") ++ if self._local_span is not None: ++ if self._local_span != (start, length): ++ raise RuntimeError("video-row cache local ownership changed") ++ return ++ device = self.global_dense_positions.device ++ target = set(self._target_positions) ++ self.local_video_positions = torch.tensor( ++ [i - start for i in self._target_positions if start <= i < start + length], dtype=torch.long, device=device ++ ) ++ self.local_dense_positions = torch.tensor( ++ [i - start for i in range(start, min(start + length, self.used_len)) if i not in target], ++ dtype=torch.long, ++ device=device, ++ ) ++ self._local_span = (start, length) ++ ++ def _visit(self, layer: int, x: torch.Tensor) -> None: ++ if self.closed or self._step < 0 or layer != self._blocks_this_step: ++ raise RuntimeError("video-row cache block order changed") ++ if self._local_span is None or x.shape[0] != self._local_span[1] or x.dtype != torch.bfloat16: ++ raise ValueError("video-row cache requires the prepared BF16 row shard") ++ self._blocks_this_step += 1 ++ ++ def capture(self, layer: int, x: torch.Tensor, output: torch.Tensor) -> None: ++ if self.context_only: ++ raise RuntimeError("video context uses its attention/FFN completion point, not a block residual") ++ if self.pre_gate: ++ raise RuntimeError("pre-gate state cannot capture a gated block residual") ++ if self.partial: ++ raise RuntimeError("cannot replace the dense anchor during a partial step") ++ self._visit(layer, x) ++ rows = self.local_video_positions ++ assert rows is not None ++ # BF16 endpoints may have a finite difference outside FP16's range. ++ # Keep that difference in FP32; only the reconstructed block output is ++ # rounded to the original BF16 activation dtype. ++ self._deltas[layer] = output.index_select(0, rows).float() - x.index_select(0, rows).float() ++ self.peak_cache_bytes = max( ++ self.peak_cache_bytes, sum(t.numel() * t.element_size() for t in self._deltas.values()) ++ ) ++ self.full_blocks += 1 ++ ++ def restore(self, layer: int, x: torch.Tensor) -> torch.Tensor: ++ if self.pre_gate: ++ raise RuntimeError("pre-gate state cannot restore a gated block residual") ++ if not self.partial: ++ raise RuntimeError("cannot reuse a video residual on a full step") ++ self._visit(layer, x) ++ rows = self.local_video_positions ++ assert rows is not None ++ # This copy owns the returned full shard, preserving caller input and ++ # unused padding while fresh non-video rows are filled by the block. ++ output = x.clone() ++ if rows.numel(): ++ output.index_copy_(0, rows, (x.index_select(0, rows).float() + self._deltas[layer].float()).to(x.dtype)) ++ self.partial_blocks += 1 ++ return output ++ ++ def _cache_bytes(self) -> int: ++ return sum( ++ tensor.numel() * tensor.element_size() ++ for cache in (self._deltas, self._attention_outputs, self._mlp_outputs) ++ for tensor in cache.values() ++ ) + sum(t.numel() * t.element_size() for pair in self._video_contexts.values() for t in pair) ++ ++ def store_video_context(self, output: torch.Tensor, lse: torch.Tensor) -> None: ++ if not self.context_only or self.partial or self.closed or self._step < 0: ++ raise RuntimeError("cannot refresh video context on this step") ++ layer = self._blocks_this_step ++ if layer >= self.num_layers or output.dtype != torch.bfloat16 or lse.dtype != torch.float32: ++ raise ValueError("video context requires native BF16 output and FP32 LSE") ++ self._video_contexts[layer] = (output.detach(), lse.detach()) ++ self.video_context_refreshes += 1 ++ self.peak_cache_bytes = max(self.peak_cache_bytes, self._cache_bytes()) ++ ++ def finish_context_block(self, layer: int, x: torch.Tensor) -> None: ++ if not self.context_only: ++ raise RuntimeError("video context completion requires the selected mode") ++ self._visit(layer, x) ++ if self._context_calls_this_step != self._blocks_this_step or layer not in self._video_contexts: ++ raise RuntimeError("video context attention was not executed before the fresh FFN") ++ self.full_row_mlp_blocks += 1 ++ if self.partial: ++ self.partial_blocks += 1 ++ else: ++ self.full_blocks += 1 ++ ++ def read_video_context(self) -> tuple[torch.Tensor, torch.Tensor]: ++ if not self.context_only or not self.partial or self.closed: ++ raise RuntimeError("cannot reuse video context on this step") ++ self.video_context_reuses += 1 ++ return self._video_contexts[self._blocks_this_step] ++ ++ def record_context_attention(self) -> None: ++ if not self.context_only or self.closed or self._context_calls_this_step != self._blocks_this_step: ++ raise RuntimeError("video context attention order changed") ++ self._context_calls_this_step += 1 ++ self.fresh_video_prefix_calls += 1 ++ ++ def pre_gate_output(self, layer: int, output: torch.Tensor, stage: str) -> torch.Tensor: ++ """Capture or fill only target rows before applying the current gate.""" ++ if not self.pre_gate or stage not in ("attn", "mlp"): ++ raise ValueError("pre-gate output requires the selected attention or MLP stage") ++ if stage == "attn": ++ if self._awaiting_mlp_layer is not None: ++ raise RuntimeError("previous video pre-gate sublayer is incomplete") ++ self._visit(layer, output) ++ self._awaiting_mlp_layer = layer ++ if self.partial: ++ self.partial_blocks += 1 ++ else: ++ self.full_blocks += 1 ++ else: ++ if self.closed or self._awaiting_mlp_layer != layer: ++ raise RuntimeError("video pre-gate MLP order changed") ++ if output.dtype != torch.bfloat16 or output.shape[0] != self._local_span[1]: ++ raise ValueError("video pre-gate output must retain the BF16 local shard") ++ self._awaiting_mlp_layer = None ++ rows = self.local_video_positions ++ assert rows is not None ++ cache = self._attention_outputs if stage == "attn" else self._mlp_outputs ++ if self.partial: ++ restored = output.clone() ++ restored.index_copy_(0, rows, cache[layer]) ++ return restored ++ cache[layer] = output.index_select(0, rows) ++ self.peak_cache_bytes = max(self.peak_cache_bytes, self._cache_bytes()) ++ return output ++ ++ def record_fresh_gate(self, stage: str) -> None: ++ if not self.pre_gate or not self.partial or stage not in ("attn", "mlp"): ++ raise RuntimeError("unexpected fresh gate record") ++ if stage == "attn": ++ self.fresh_attention_gates += 1 ++ else: ++ self.fresh_mlp_gates += 1 ++ ++ def record_subset_attention(self, rows: int) -> None: ++ if not self.partial or rows != self.global_dense_positions.numel(): ++ raise RuntimeError("unexpected subset attention query count") ++ self.subset_attention_calls += 1 ++ self.subset_query_rows += rows ++ self.omitted_query_rows += len(self._target_positions) ++ ++ def end_step(self) -> None: ++ if self._blocks_this_step != self.num_layers or self._awaiting_mlp_layer is not None: ++ raise RuntimeError("video-row cache step did not visit every main block") ++ ++ def stats(self) -> dict[str, int | bool | str]: ++ counters: dict[str, int | bool | str] = { ++ name: getattr(self, name) ++ for name in [ ++ "full_steps", ++ "partial_steps", ++ "full_blocks", ++ "partial_blocks", ++ "subset_attention_calls", ++ "subset_query_rows", ++ "omitted_query_rows", ++ "partial_mlp_rows", ++ "peak_cache_bytes", ++ "closed", ++ ] ++ } ++ counters["storage_dtype"] = "torch.bfloat16" if self.pre_gate or self.context_only else "torch.float32" ++ counters["cache_stage"] = ( ++ "video_context_lse" if self.context_only else "pre_gate" if self.pre_gate else "block_residual" ++ ) ++ counters["video_context_refreshes"] = self.video_context_refreshes ++ counters["video_context_reuses"] = self.video_context_reuses ++ counters["fresh_video_prefix_calls"] = self.fresh_video_prefix_calls ++ counters["full_row_mlp_blocks"] = self.full_row_mlp_blocks ++ counters["fresh_attention_gates"] = self.fresh_attention_gates ++ counters["fresh_mlp_gates"] = self.fresh_mlp_gates ++ counters["live_cache_bytes"] = self._cache_bytes() ++ return counters ++ ++ def close(self) -> None: ++ self._deltas.clear() ++ self._attention_outputs.clear() ++ self._mlp_outputs.clear() ++ self._video_contexts.clear() ++ self._awaiting_mlp_layer = None ++ self._target_positions.clear() ++ self.global_dense_positions = torch.empty(0, dtype=torch.long) ++ self.local_dense_positions = None ++ self.local_video_positions = None ++ self.closed = True +diff --git a/vllm_omni/diffusion/offloader/base.py b/vllm_omni/diffusion/offloader/base.py +index 2505d48..50b73e5 100644 +--- a/vllm_omni/diffusion/offloader/base.py ++++ b/vllm_omni/diffusion/offloader/base.py +@@ -71,6 +71,10 @@ class OffloadConfig: + dlo_use_allgather: bool = True + dlo_resident_layers: int = 0 # leading DiT layers kept on device + chunk_size_bytes: int = 64 * 1024 * 1024 ++ # Hold each DiT block's weight AllGather until that block's QKV AllToAll has ++ # been enqueued, so the background collective does not take the links ahead ++ # of the critical-path all-to-all. See ``collective_gate.py``. ++ dlo_order_allgather_after_attn_a2a: bool = False + # Optional per-worker ceiling for registering an HWR mmap. Zero means no + # additional ceiling; pin_cpu_memory controls whether registration is tried. + dlo_host_registration_limit_gib: float = 0.0 +@@ -216,6 +220,7 @@ class OffloadConfig: + dlo_use_allgather=dit_uses_allgather, + dlo_resident_layers=dlo_resident_layers, + chunk_size_bytes=chunk_size_bytes, ++ dlo_order_allgather_after_attn_a2a=bool(getattr(od_config, "dlo_order_allgather_after_attn_a2a", False)), + dlo_host_registration_limit_gib=dlo_host_registration_limit_gib, + components=components, + dlo_transfers=dlo_transfers, +diff --git a/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py b/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py +index 7f0f081..8c4f87f 100644 +--- a/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py ++++ b/vllm_omni/diffusion/offloader/distributed_layerwise_backend.py +@@ -31,6 +31,7 @@ import torch.distributed + from torch import nn + from vllm.logger import init_logger + ++from vllm_omni.diffusion.distributed.collective_gate import QKVAllToAllBarrier + from vllm_omni.diffusion.hooks import HookRegistry, ModelHook + from vllm_omni.diffusion.model_loader.host_weight_plan import ( + HostWeightPlan, +@@ -142,6 +143,7 @@ class DistributedLayerwiseOffloadHook(ModelHook): + tensor_transforms: dict[int, Any] | None = None, + materialization_probe_tensor: torch.Tensor | None = None, + chunk_size_bytes: int = 64 * 1024 * 1024, ++ order_allgather_after_attn_a2a: bool = False, + ): + assert isinstance(next_block, nn.Module), "transformer block must be type `torch.nn.Module`" + if type(dp_size) is not int or dp_size < 1: +@@ -167,6 +169,7 @@ class DistributedLayerwiseOffloadHook(ModelHook): + + self.chunk_size_bytes = chunk_size_bytes + self.manifest: PartManifest | None = None ++ self._deferred_chunked_allgather = None + self._h2d_done_events = [current_omni_platform.Event(), current_omni_platform.Event()] + self._ag_done_events = [current_omni_platform.Event(), current_omni_platform.Event()] + # Hook-local events protect chunk-input reuse within this hook. Across +@@ -239,6 +242,19 @@ class DistributedLayerwiseOffloadHook(ModelHook): + # Per-block synchronization primitive: set after H2D copy completes. + self._prefetch_done: Any | None = None + ++ # Optional ordering: hold the background weight AllGather until the ++ # block's QKV AllToAll is on the compute stream. See ++ # distributed/collective_gate.py. Off by default; the attention layer ++ # inside this block is handed the same barrier at install time. ++ self._a2a_barrier: QKVAllToAllBarrier | None = None ++ self._a2a_event: Any | None = None ++ # ``(slot, gpu_shards, gpu_weights, evt)`` of a gather armed but not yet ++ # issued, so ``post_forward`` can flush it if the attention never ran. ++ self._deferred_allgather: tuple[int, dict, dict, Any] | None = None ++ if order_allgather_after_attn_a2a: ++ self._a2a_event = current_omni_platform.Event() ++ self._a2a_barrier = QKVAllToAllBarrier(self._allgather_after_attn_a2a) ++ + # Shared shard (AllGather input) buffers — assigned by backend. + self.gpu_shard_buffers: list[dict[torch.dtype, torch.Tensor] | None] = [None, None] + +@@ -653,49 +669,23 @@ class DistributedLayerwiseOffloadHook(ModelHook): + gpu_shard = shard_bufs[dtype][: cpu_shard.numel()] + gpu_shard.copy_(cpu_shard, non_blocking=non_blocking) + gpu_shards[dtype] = gpu_shard +- self.comm_stream.wait_stream(self.copy_stream) +- with current_omni_platform.stream(self.comm_stream): +- for dtype, local_shard in gpu_shards.items(): +- gw = gpu_weights[dtype][: self._ag_output_sizes[dtype]] +- torch.distributed.all_gather_into_tensor( +- gw, +- local_shard, +- group=self.dp_group, +- ) +- evt.record(self.comm_stream) ++ ++ if self._a2a_barrier is not None and non_blocking and getattr(self, "_nz_cast_fn", None) is None: ++ # Hold the collective until the block's attention reports its ++ # QKV AllToAll, so the background gather cannot take the links ++ # ahead of the critical-path all-to-all. The H2D above has ++ # already been enqueued; only the gather moves, so it still ++ # overlaps the rest of the block. Deferral needs a repoint that ++ # emits no stream work, hence the NZ-cast guard. ++ self._deferred_allgather = (slot, gpu_shards, gpu_weights, evt) ++ self._a2a_barrier.arm() ++ else: ++ self._issue_allgather(slot, gpu_shards, gpu_weights, evt) ++ elif self._a2a_barrier is not None and non_blocking: ++ self._deferred_chunked_allgather = (slot, gpu_weights, evt) ++ self._a2a_barrier.arm() + else: +- for dtype_manifest in self.manifest.dtypes: +- cpu_shard = self.cpu_shards[dtype_manifest.dtype] +- buf = shard_bufs[dtype_manifest.dtype] +- for chunk in dtype_manifest.chunks: +- if buf.numel() >= 2 * chunk.local_numel: +- width = buf.numel() // 2 +- input_slot = chunk.chunk_id % 2 +- else: +- width = buf.numel() +- input_slot = 0 +- reuse = self._chunk_reuse_events[input_slot] +- if reuse is not None: +- self.copy_stream.wait_event(reuse) +- local_input = buf[input_slot * width : input_slot * width + chunk.local_numel] +- source = cpu_shard[chunk.cpu_offset : chunk.cpu_offset + chunk.local_numel] +- with current_omni_platform.stream(self.copy_stream): +- local_input.copy_(source, non_blocking=non_blocking) +- self._h2d_done_events[input_slot].record(self.copy_stream) +- self.comm_stream.wait_event(self._h2d_done_events[input_slot]) +- with current_omni_platform.stream(self.comm_stream): +- full_out = gpu_weights[dtype_manifest.dtype][ +- chunk.full_offset : chunk.full_offset + chunk.padded_numel +- ] +- torch.distributed.all_gather_into_tensor( +- full_out, +- local_input, +- group=self.dp_group, +- ) +- self._ag_done_events[input_slot].record(self.comm_stream) +- self._chunk_reuse_events[input_slot] = self._ag_done_events[input_slot] +- with current_omni_platform.stream(self.comm_stream): +- evt.record(self.comm_stream) ++ self._issue_chunked_allgather(slot, gpu_weights, evt, non_blocking=non_blocking) + + self.ready_events[slot] = evt + self._prefetch_done = evt +@@ -719,6 +709,131 @@ class DistributedLayerwiseOffloadHook(ModelHook): + ), + ) + ++ def _issue_chunked_allgather(self, slot, gpu_weights, evt, *, gated=False, non_blocking=True) -> None: ++ """Submit chunk copies and gathers together after the foreground gate. ++ ++ Chunk staging slots are bounded and reused. Deferring only gathers ++ would overwrite an input slot before its collective consumes it. ++ """ ++ shard_bufs = self.gpu_shard_buffers[slot] ++ assert shard_bufs is not None and self.manifest is not None ++ if gated: ++ self.comm_stream.wait_event(self._a2a_event) ++ for dtype_manifest in self.manifest.dtypes: ++ cpu_shard = self.cpu_shards[dtype_manifest.dtype] ++ buf = shard_bufs[dtype_manifest.dtype] ++ for chunk in dtype_manifest.chunks: ++ if buf.numel() >= 2 * chunk.local_numel: ++ width = buf.numel() // 2 ++ input_slot = chunk.chunk_id % 2 ++ else: ++ width = buf.numel() ++ input_slot = 0 ++ reuse = self._chunk_reuse_events[input_slot] ++ if reuse is not None: ++ self.copy_stream.wait_event(reuse) ++ local_input = buf[input_slot * width : input_slot * width + chunk.local_numel] ++ source = cpu_shard[chunk.cpu_offset : chunk.cpu_offset + chunk.local_numel] ++ with current_omni_platform.stream(self.copy_stream): ++ local_input.copy_(source, non_blocking=non_blocking) ++ self._h2d_done_events[input_slot].record(self.copy_stream) ++ self.comm_stream.wait_event(self._h2d_done_events[input_slot]) ++ with current_omni_platform.stream(self.comm_stream): ++ full_out = gpu_weights[dtype_manifest.dtype][ ++ chunk.full_offset : chunk.full_offset + chunk.padded_numel ++ ] ++ torch.distributed.all_gather_into_tensor( ++ full_out, ++ local_input, ++ group=self.dp_group, ++ ) ++ self._ag_done_events[input_slot].record(self.comm_stream) ++ self._chunk_reuse_events[input_slot] = self._ag_done_events[input_slot] ++ with current_omni_platform.stream(self.comm_stream): ++ evt.record(self.comm_stream) ++ ++ def _issue_allgather( ++ self, ++ slot: int, ++ gpu_shards: dict[torch.dtype, torch.Tensor], ++ gpu_weights: dict[torch.dtype, torch.Tensor], ++ evt: Any, ++ *, ++ gated: bool = False, ++ ) -> None: ++ """Submit this block's weight AllGather on ``comm_stream``. ++ ++ ``gated`` additionally waits on the QKV AllToAll barrier event recorded ++ by the block's attention, so the foreground collective gets the links ++ first and this gather queues behind it. ++ """ ++ self.comm_stream.wait_stream(self.copy_stream) ++ if gated: ++ self.comm_stream.wait_event(self._a2a_event) ++ with current_omni_platform.stream(self.comm_stream): ++ for dtype, local_shard in gpu_shards.items(): ++ # Slice the shared (max-sized) output buffer down to this ++ # block's actual AllGather output size. The buffers are ++ # sized to the *largest* block across all groups, so for any ++ # smaller block the full buffer would violate the ++ # all_gather_into_tensor contract ++ # (output.numel() == world_size * input.numel()). ++ # Repoint offsets are relative to the block's flattened ++ # buffer, so a prefix slice is safe. ++ gw = gpu_weights[dtype][: self._ag_output_sizes[dtype]] ++ torch.distributed.all_gather_into_tensor( ++ gw, ++ local_shard, ++ group=self.dp_group, ++ ) ++ evt.record(self.comm_stream) ++ ++ def _allgather_after_attn_a2a(self) -> None: ++ """Barrier callback: the block's QKV AllToAll is now enqueued. ++ ++ Recording on the compute stream here (rather than at block entry) is ++ what makes the gate effective: an event wait snapshots the records ++ enqueued before it, so the wait must be enqueued *after* the all-to-all. ++ """ ++ chunked = self._deferred_chunked_allgather ++ if chunked is not None: ++ self._deferred_chunked_allgather = None ++ assert self._a2a_event is not None ++ self._a2a_event.record(current_omni_platform.current_stream()) ++ self._issue_chunked_allgather(*chunked, gated=True) ++ return ++ deferred = self._deferred_allgather ++ if deferred is None: ++ return ++ self._deferred_allgather = None ++ assert self._a2a_event is not None ++ self._a2a_event.record(current_omni_platform.current_stream()) ++ slot, gpu_shards, gpu_weights, evt = deferred ++ self._issue_allgather(slot, gpu_shards, gpu_weights, evt, gated=True) ++ ++ def _flush_deferred_allgather(self) -> None: ++ """Issue a deferred gather whose attention never signalled. ++ ++ Keeps the double buffer live: a block whose attention is skipped ++ (cache-dit, an all-resident stage) still needs its weights, so the ++ gather is submitted ungated at the end of that block's forward. ++ """ ++ chunked = self._deferred_chunked_allgather ++ if chunked is not None: ++ self._deferred_chunked_allgather = None ++ if self._a2a_barrier is not None: ++ self._a2a_barrier.disarm() ++ self._issue_chunked_allgather(*chunked) ++ return ++ deferred = self._deferred_allgather ++ if deferred is None: ++ return ++ self._deferred_allgather = None ++ if self._a2a_barrier is not None: ++ self._a2a_barrier.disarm() ++ slot, gpu_shards, gpu_weights, evt = deferred ++ self._issue_allgather(slot, gpu_shards, gpu_weights, evt) ++ + def _wait_for_weights(self, slot: int) -> None: + """Wait until the slot holding this block is ready for compute. + +@@ -840,6 +955,10 @@ class DistributedLayerwiseOffloadHook(ModelHook): + return args, kwargs + + def post_forward(self, module: nn.Module, output: Any) -> Any: ++ # Pair with pre_forward: if the block's attention never signalled the ++ # gate, the deferred gather must still go out before this block's ++ # storage is released (offload_layer waits on _prefetch_done). ++ self._flush_deferred_allgather() + self.offload_layer() + return output + +@@ -864,6 +983,7 @@ def apply_distributed_block_hook( + tensor_transforms: dict[int, Any] | None = None, + materialization_probe_tensor: torch.Tensor | None = None, + chunk_size_bytes: int = 64 * 1024 * 1024, ++ order_allgather_after_attn_a2a: bool = False, + ) -> DistributedLayerwiseOffloadHook: + """Register a DistributedLayerwiseOffloadHook on *module*.""" + registry = HookRegistry.get_or_create(module) +@@ -881,11 +1001,32 @@ def apply_distributed_block_hook( + tensor_transforms=tensor_transforms, + materialization_probe_tensor=materialization_probe_tensor, + chunk_size_bytes=chunk_size_bytes, ++ order_allgather_after_attn_a2a=order_allgather_after_attn_a2a, + ) + registry.register_hook(DistributedLayerwiseOffloadHook._HOOK_NAME, hook) + return hook + + ++def attach_collective_gate(block: nn.Module, hook: DistributedLayerwiseOffloadHook) -> int: ++ """Hand ``hook``'s barrier to every Attention layer inside *block*. ++ ++ The attention signals the barrier right after its QKV AllToAll. The import ++ is local to keep this module independent of the attention package at import ++ time. ++ ++ Returns the number of attention layers wired (0 means the hook will rely on ++ the ``post_forward`` flush instead). ++ """ ++ from vllm_omni.diffusion.attention.layer import Attention ++ ++ attached = 0 ++ for module in block.modules(): ++ if isinstance(module, Attention): ++ module._collective_gate = hook._a2a_barrier ++ attached += 1 ++ return attached ++ ++ + def remove_distributed_block_hook(module: nn.Module) -> None: + """Remove the distributed layerwise offload hook from *module*.""" + registry: HookRegistry | None = getattr(module, "_hook_registry", None) +@@ -1566,29 +1707,45 @@ class DistributedLayerwiseOffloadBackend(OffloadBackend): + self._all_hook_groups.append(hooks) + self._blocks.append(block_list) + probes = {id(block): module_materialization_probe(block) for block in block_list} ++ # Only the DiT blocks issue a QKV AllToAll the gather can queue behind. ++ order_gate = bool(getattr(self.config, "dlo_order_allgather_after_attn_a2a", False)) and ( ++ component == DIT_COMPONENT ++ ) ++ if order_gate and not getattr(self, "_collective_gate_logged", False): ++ self._collective_gate_logged = True ++ logger.info( ++ "Distributed layerwise offload will hold each DiT block's weight AllGather " ++ "until that block's QKV AllToAll is on the compute stream" ++ ) + for block, next_block in zip( + chain((block_list[-1],), block_list[:-1]), + block_list, + strict=True, + ): +- hooks.append( +- apply_distributed_block_hook( +- block, +- next_block, +- self.device, +- group, +- group_size, +- group_rank, +- self.copy_stream, +- self.comm_stream, +- self.config.pin_cpu_memory, +- shared_buffers=[None, None], +- rank_local_mmap=self._using_rank_local_mmap if use_dit_mmap else False, +- tensor_transforms=self._mmap_transforms_by_tensor_id if use_dit_mmap else None, +- materialization_probe_tensor=probes[id(block)], +- chunk_size_bytes=self.config.chunk_size_bytes, +- ) ++ hook = apply_distributed_block_hook( ++ block, ++ next_block, ++ self.device, ++ group, ++ group_size, ++ group_rank, ++ self.copy_stream, ++ self.comm_stream, ++ self.config.pin_cpu_memory, ++ shared_buffers=[None, None], ++ rank_local_mmap=self._using_rank_local_mmap if use_dit_mmap else False, ++ tensor_transforms=self._mmap_transforms_by_tensor_id if use_dit_mmap else None, ++ materialization_probe_tensor=probes[id(block)], ++ chunk_size_bytes=self.config.chunk_size_bytes, ++ order_allgather_after_attn_a2a=order_gate, + ) ++ if hook._a2a_barrier is not None and attach_collective_gate(block, hook) == 0: ++ logger.warning( ++ "DLO AllGather ordering is enabled but block %s exposes no Attention layer; " ++ "its gather will fall back to the ungated path", ++ block.__class__.__name__, ++ ) ++ hooks.append(hook) + + # hooks = [last -> first, block0 -> block1, ...]. Alternating slots + # keep a prefetch from overwriting the current block for any ring size. +diff --git a/vllm_omni/engine/arg_utils.py b/vllm_omni/engine/arg_utils.py +index 0a876b9..bf8dcdc 100644 +--- a/vllm_omni/engine/arg_utils.py ++++ b/vllm_omni/engine/arg_utils.py +@@ -807,6 +807,7 @@ class OrchestratorArgs: + dlo_use_allgather: bool = True + dlo_chunk_size_mb: int = 64 + dlo_resident_layers: int = 0 ++ dlo_order_allgather_after_attn_a2a: bool = False + host_weight_runtime_mode: str = "disabled" + host_weight_runtime_root: str | None = None + dlo_host_registration_limit_gib: float = 0.0 +diff --git a/vllm_omni/entrypoints/cli/serve.py b/vllm_omni/entrypoints/cli/serve.py +index f2b55e7..aff71b5 100644 +--- a/vllm_omni/entrypoints/cli/serve.py ++++ b/vllm_omni/entrypoints/cli/serve.py +@@ -1041,6 +1041,17 @@ class OmniServeCommand(CLISubcommand): + help="Full-chunk target size in MiB for distributed layerwise offload " + "(each chunk is one host-to-device copy + AllGather unit).", + ) ++ omni_config_group.add_argument( ++ "--dlo-order-allgather-after-attn-a2a", ++ action="store_true", ++ default=False, ++ help=( ++ "Hold each DiT block's weight AllGather until that block's QKV AllToAll " ++ "has been enqueued, so the background gather does not take the collective " ++ "links ahead of the critical-path all-to-all. Requires distributed " ++ "layerwise offload with AllGather." ++ ), ++ ) + omni_config_group.add_argument( + "--host-weight-runtime-mode", + choices=("disabled", "preferred", "required"), diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c23-source-identity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c23-source-identity.json new file mode 100644 index 000000000..1eb4e74f5 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/patch/c23-source-identity.json @@ -0,0 +1,30 @@ +{ + "head": "6eb7bb094effb40853431208d1631effaf452a3f", + "tree": "685c971a476b9df16cb5f79b3db41e12930306ab", + "base": "d48fa51d03dd748568cbcf57ed40f533c691c9bd", + "branch": "rsi/h3-297-video-context-lse", + "files": { + "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_context.py": "1e76074b71f0ea4e070c435db623add702d1414a351427fd86ebb40e8dfd67fd", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "48533c941a50719995206d18fa9198330824b940ecba9399b46ba7615b969ccb", + "vllm_omni/diffusion/models/minimax_h3/attention/video_context.py": "f4cfd2225514b6f99d36567879f06fb4a42f9d235ac0114fc643001b345cb22e", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "73c96090e6fb0e13ce734bc96b23472fd5adb83bceb71f3eba2815d6343a33e4", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "577e290c2d92fc02822cd516cc499c3c4d1f465145ad2a837edfdfeaa09ed9fb", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cff17568fbd856c28394d104d393a806931613b7b057f288006ad8c6fb9aa05a", + "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" + }, + "published": false, + "patch_base": "be319692b96413918973b5bd399cdda3f320c574" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/plan/README.md b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/README.md new file mode 100644 index 000000000..d9cd612b2 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/README.md @@ -0,0 +1,45 @@ +# Frozen campaign contracts and execution checkpoint + +The governing issue is [#297](https://github.com/InferMatrixLab/InferRSI/issues/297). +This tracking snapshot records actual execution; it does not authorize a new +budget, replace original B0 or establish campaign acceptance. + +- [Execution manifest](execution-checkpoint.json): source/model identities, hardware, + private-stack versions, workload, original deadline and outstanding scope. +- [Lossless contract](frozen-lossless-contract-v1.json): strict latent/schedule and + decoded RGB24/PCM checks, qualified original source, and frozen timing protocol. +- [Lossy contract](frozen-lossy-contract-v1.json): original pre-search scorer bytes, + revisions/weights, ten fixed samples, aggregation and automatic quality gates. +- [Requester decisions](requester-decisions.json): user-selected tracks, no human + output checks, target and shared ceiling; later execution does not reset them. + +Original B0 repeatability passed at 5 s and 15 s. Migration numerical comparison +passed at 5 s; default dense migration 15 s remains unqualified. C1 subsequently +passed full strict 5 s and15 s comparison,439 checks each. These are numerical conclusions, not latency +acceptance. The model's 81 files / 144051182613 bytes were fully SHA/LFS verified +in private preparation; the raw private manifest is not bundled. + +Every actual diagnostic/native/operator run used the pinned InferRSI external +executor. The first native CLI clean 5 s AB pair completed: strict quality PASS35 +plus the separate PASS439 audit, source integrity and cleanup passed. Performance +FAIL: 1.5729% reduction versus the required50%. Physical NPU discovery IDs1,2 map to the +four execution logical devices2,3,4,5; native inventory remains8physical/16logical. +The case-local driver owns fresh servers, readiness, warmups, measured requests, +media identities and native cleanup; it returns complete HTTP latency in ms. + +Clean screening requires 2 warmups and 3 measured requests per fresh server. +Final confirmation requires 3 fresh pairs in AB/BA/AB order, the same inner +request counts, and each pair plus the aggregate reaching the 50% reduction bar. +Keep profiler, audits and CPU scoring out of clean timing. The completed +[5 s screening](../results/c1-clean-5s-screening.json) is not final confirmation; +15 s clean timing and complete four-track acceptance remain outstanding. + +The current requester-authorized ceiling is 24 directions / 32 wall hours / +128 logical-device-hours, including preparation, failures, retries, scoring and +cleanup. It started at 2026-11-01 17:19 +08 and ends at 2026-11-03 01:19 +08. +The original 12-direction / 16-hour / 64-device-hour allowance is retained as +decision history in requester-decisions.json. The original start never changed. +All 24 families are charged. No extra final-confirmation amount is authorized. + +[Replay and availability](reproduction.md) separates local byte checks from +device replay requiring the external artifacts and original private environment. diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/plan/c11-modality-sigma-residual.json b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/c11-modality-sigma-residual.json new file mode 100644 index 000000000..70334a17c --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/c11-modality-sigma-residual.json @@ -0,0 +1,45 @@ +{ + "id": "C11-H3-modality-sigma-residual", + "configuration_key": "modality-sigma-quadratic", + "source_base": "922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb", + "branch": "rsi/h3-297-modality-sigma-residual", + "track": "lossy", + "files": [ + "vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py", + "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py", + "tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py" + ], + "production_changes": [ + "vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py", + "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py", + "tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py" + ], + "cache_config": { + "Fn_compute_blocks": 1, + "Bn_compute_blocks": 0, + "max_warmup_steps": 4, + "scm_steps_mask_policy": null, + "residual_diff_threshold": 0.08, + "max_continuous_cached_steps": 3, + "enable_taylorseer": false, + "minimax_h3_sigma_cache": true + }, + "hypothesis": "Replace integer-step Taylor residual forecasting with a quadratic through the last three actual middle-block residuals, using video sigma for video/text rows and audio sigma for audio rows. Nonuniform schedule and different modality clocks are verified source facts; improvement of frozen audio/visual quality is unverified. Keep parent reuse decisions plus insufficient/ill-conditioned history veto, all50solver steps, weights, attention and reference conditioning unchanged. No C10 audio-veto or Laser composition.", + "ownership": "Denoise request owns three detached cloned BF16/FP32 residual anchors and clocks, reset before each request and cleared in finally on success/cancel/error. Persistent wrapper keeps transient metadata with try/finally and weak owner references; model retains scalar-only stats after cleanup. Default disabled; explicit unsupported config fails.", + "selection": "Existing DiffusionCacheConfig extra field minimax_h3_sigma_cache, strictly boolean, only explicit cache_dit Ref2VA DP1/CFG1 without RefKV/control/edit; reject incompatible combinations before execution. No dependency or generic cache API edit.", + "stability": "Three distinct monotonically descending sigma anchors per modality; quadratic Lagrange coefficient absolute-sum<=64 on both clocks or full compute. Host-only shared clocks keep rank decision agreement. Accumulate FP32, cast forecast to raw residual dtype, add to Fn hidden states using parent residual semantics. Text/padding use video clock, padding remains masked by existing attention.", + "validation": [ + "Ruff0.14.10 check/format, exactfive-path diff and private base identity.", + "Native pytest polynomial recovery on nonuniform two-clock fixtures, inputimmutability, parentdenial/historyveto, disabled/unsupportedselection, transientrelease and denoise error/cancellation cleanup. RealNPUtest must execute.", + "InferRSI native trial is first C11family charge. Commit testedbytes privately before fullmodel.", + "Full50step 5s-00 then5s-01 automaticquality using distinctkey, with actual computed/predicted counts. Reject any observedfrozenviolation. MissingcohortremainsINC.", + "Only retainedcandidate proceeds to independent15s, completefive-samplecohorts, cleanpairedtiming and final3freshpairs within remainingbudget." + ], + "quality_contract_sha256": "b8fd9e634e6db5ebfbce3842f3360f1b2241a08119953ee71bfc26553dbbafed", + "budget": "Original start17:19+08 deadline09:19+08 and64logical-device-hours unchanged; lastunspentfamily C11. No extra finalbudget. Nativecap8min, first5smodelcap15min, no parallel selected-device job.", + "status": "Prospective plan before edits; no native/model result and no quality/speed acceptance." +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/plan/c11-sigma-fp32-residual.json b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/c11-sigma-fp32-residual.json new file mode 100644 index 000000000..791f83829 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/c11-sigma-fp32-residual.json @@ -0,0 +1,52 @@ +{ + "id": "C11-H3-modality-sigma-residual", + "configuration_key": "modality-sigma-fp32-anchor", + "source_base": "61b8efa06e88372bc5ab87b9e2668a133eaac282", + "branch": "rsi/h3-297-sigma-fp32-residual", + "track": "lossy", + "files": [ + "vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py", + "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py", + "tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py" + ], + "production_changes": [ + "vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py", + "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py", + "tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py" + ], + "cache_config": { + "Fn_compute_blocks": 1, + "Bn_compute_blocks": 0, + "max_warmup_steps": 4, + "scm_steps_mask_policy": null, + "residual_diff_threshold": 0.08, + "max_continuous_cached_steps": 3, + "enable_taylorseer": false, + "minimax_h3_sigma_cache": true + }, + "hypothesis": "Original Fn1Bn0sigma passed38/39guards but failedcaption; Bn1passedcaption but failedaudiolevel. Preserve bothfailures. Source shows BF16packedstates, BF16middle-residual subtraction and a BF16forecast cast before BF16prefixaddition. Test only residualprecision: subtractactualcomputedmiddle-end.float()-prefix.float(), own threeFP32anchors, same dualsigmaquadratic, addforecast+prefixinFP32 and castonce to originalBF16hidden. This preserves more information from actualBF16endpoints; whether it improves modelquality is unverified. No threshold/tail/step/model change and no reference-output-dependent normalization.", + "ownership": "Same request-ownedthreeanchors/finallycleanup; anchorstorage FP32. ParentFn decision/bookkeeping delegated. The newlycomputedrawMnresidual is FP32; wrappercasts the reconstructedhidden to incominghidden dtype before anytail/finalprojection. Untargeted/defaultoffpath unchanged.", + "selection": "Existing DiffusionCacheConfig extra field minimax_h3_sigma_cache, strictly boolean, only explicit cache_dit Ref2VA DP1/CFG1 without RefKV/control/edit; reject incompatible combinations before execution. No dependency or generic cache API edit.", + "stability": "Three distinct monotonically descending sigma anchors per modality; quadratic Lagrange coefficient absolute-sum<=64 on both clocks or full compute. Host-only shared clocks keep rank decision agreement. Accumulate FP32, cast forecast to raw residual dtype, add to Fn hidden states using parent residual semantics. Text/padding use video clock, padding remains masked by existing attention.", + "validation": [ + "Ruff0.14.10 check/format, exactfive-path diff and private base identity.", + "Native pytest polynomial recovery on nonuniform two-clock fixtures, inputimmutability, parentdenial/historyveto, disabled/unsupportedselection, transientrelease and denoise error/cancellation cleanup. RealNPUtest must execute.", + "InferRSI native trial is first C11family charge. Commit testedbytes privately before fullmodel.", + "Full50step 5s-00 then5s-01 automaticquality using distinctkey, with actual computed/predicted counts. Reject any observedfrozenviolation. MissingcohortremainsINC.", + "Only retainedcandidate proceeds to independent15s, completefive-samplecohorts, cleanpairedtiming and final3freshpairs within remainingbudget." + ], + "quality_contract_sha256": "b8fd9e634e6db5ebfbce3842f3360f1b2241a08119953ee71bfc26553dbbafed", + "budget": "Original start17:19+08 deadline09:19+08 and64logical-device-hours unchanged; lastunspentfamily C11. No extra finalbudget. Nativecap8min, first5smodelcap15min, no parallel selected-device job.", + "status": "Prospective revision2 plan before source edits or native execution.", + "new_delta_files": [ + "vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py", + "tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py" + ], + "native_validation": "Retain20focusedtests; add nativeBF16middle-endpoint residual test proving exactFP32differenceandBF16reconstruction, mixedFP32history/BF16current admission, and outputdtype/immutability. RunactualnativepaththroughInferRSI; sourceproofbeforeprivatecommit andfullmodel. Keepold61b8efacommit/nativeproof separate.", + "scope": "Exactlytwochangedpaths over61b8efa; overallfive-fileH3scope over922funchanged. No genericCacheDit/backendconfig/dependency change. ExistingC11family, no13thfamily.", + "stop": "Firstfull5s-00; reject anyobserved frozenfailure. Ifretainedindependent5s-01 then15s-00/fullcohorts/cleanpairs withinremainingbudget. Ifprecision controlalsofails, stop predictor variants; no threshold/tail grid." +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json new file mode 100644 index 000000000..064a27086 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/execution-checkpoint.json @@ -0,0 +1,130 @@ +{ + "schema_version": 1, + "governing_issue": "https://github.com/InferMatrixLab/InferRSI/issues/297", + "recorded_at_utc": "2026-11-02T07:51:55.651531+00:00", + "campaign_verdict": "INCONCLUSIVE", + "baseline_source": "df1c1b05936381dbaa3891c1ae4c4b21ab375a5d", + "migration_source": "922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb", + "inferrsi_source": "50c19da43e0b358e6f990c00719891ad308a8ac1", + "model_revision": "6818f6c32d12b210915e44ad56a4228c2608f160", + "hardware": { + "backend": "ascend", + "model": "910C", + "logical_device_count": 4, + "physical_dual_die_device_count": 2, + "memory_gib_per_logical_device": 64 + }, + "stack": { + "os": "private Ubuntu22.04 namespace", + "glibc": "2.35", + "cann": "9.1", + "torch": "2.10.0+cpu", + "torch_npu": "2.10.0.post4", + "mindie_sd": "3.1.0", + "mindie_sd_source_revision": "UNKNOWN", + "vllm": "0.32.0+empty", + "vllm_ascend": "450e55698" + }, + "workload": { + "steps": 50, + "width": 1344, + "height": 768, + "fps": 24, + "durations_frames": { + "5": 124, + "15": 362 + }, + "seed": 3100, + "flow_shift": 12, + "audio_flow_shift": 3, + "parameters": "BF16 dense original model; original FP32 intermediate/latent state preserved", + "parallel": "USP4, ring1, TE TP4, tiled VAE patch parallel4; DLO on for reference" + }, + "frozen_quality_contract_sha256": "b8fd9e634e6db5ebfbce3842f3360f1b2241a08119953ee71bfc26553dbbafed", + "frozen_strict_validator_sha256": "b83f2bff553ff6ca2970a16e25525cac5d9523612e9c05ea563852c814019028", + "latency_protocol": { + "numerical_qualification": { + "fresh_server": true, + "warmup": 0, + "measured_requests": 2, + "audit": true, + "not_clean_timing": true + }, + "screening_per_arm": { + "fresh_server": true, + "same_shape_warmups": 2, + "measured_requests": 3, + "profiler": false, + "audit": false + }, + "final_confirmation": { + "fresh_server_pairs": 3, + "order": [ + "AB", + "BA", + "AB" + ], + "warmups_per_server": 2, + "measurements_per_server": 3, + "statistical_unit": "server median", + "requirement": "Each paired candidate server median and overall server-median comparison meet 50% reduction; all quality/lifecycle gates complete.", + "statistical_significance": "not claimed", + "budget": "Check remaining total ceiling or separately authorize an additional phase; never exceed the active deadline." + }, + "revision_note": "Final 3-pair proposal replaces the unfrozen 5-pair draft before any candidate outcome is observed, to bound execution cost. The quality and 50% bars are unchanged." + }, + "acceptance": { + "complete_http_reduction_pct": 50, + "per_duration_and_quality_track": true, + "clean_paired_runs_completed": 1, + "final_confirmation_pairs_completed": 0, + "human_output_checks_in_rsi": false + }, + "budget": { + "start": "2026-11-01T17:19:00+08:00", + "deadline": "2026-11-03T01:19:00+08:00", + "wall_hours_ceiling": 32.0, + "logical_device_hours_ceiling": 128, + "directions_ceiling": 24, + "charged_direction_families": [ + "C0", + "C1", + "C2", + "C3", + "C4", + "C5", + "C6", + "C7", + "C8", + "C9", + "C10", + "C11", + "C12", + "C13", + "C14", + "C15", + "C16", + "C17", + "C18", + "C19", + "C20", + "C21", + "C22", + "C23" + ], + "extra_final_confirmation_budget_authorized": false + }, + "limitations": [ + "Diagnostic HTTP values are not clean timing or accepted gains.", + "C1 numerical qualification covers fixed5s/15s requests; 15s clean timing and final confirmation remain incomplete.", + "Quality corpus incomplete; heldout requests have not been generated.", + "Raw private captures,model weights and quality dependency weights are not included.", + "Draft tracking snapshot; independent review, full quality coverage and four-track goal acceptance remain incomplete." + ], + "publication": { + "scope": "Sanitized projections and source patch artifacts", + "device_experiments_rerun_for_publication": false, + "runtime_harness_revision_unchanged": true, + "inventory_registration": "Layout and available-byte verification only; not campaign acceptance" + } +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json new file mode 100644 index 000000000..88b269794 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossless-contract-v1.json @@ -0,0 +1,108 @@ +{ + "schema_version": 1, + "frozen_at": "2026-11-01T10:09:45.855117+00:00", + "issue": "https://github.com/InferMatrixLab/InferRSI/issues/297", + "track_scope": [ + "5s-lossless", + "15s-lossless" + ], + "status": "Prospective numerical contract; performance confirmation remains separate.", + "baseline_source": "df1c1b05936381dbaa3891c1ae4c4b21ab375a5d", + "first_migration_candidate": "922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb", + "model_revision": "6818f6c32d12b210915e44ad56a4228c2608f160", + "model_manifest_sha256": "38e36913cf7a7dea81b2c7fc8ae6dd2ad1d0f9d27a8cd4deee3b1927f593b512", + "device_ids": [ + 2, + 3, + 4, + 5 + ], + "quality": { + "mode": "strict_bit_exact", + "points": [ + "initial_noise", + "post_step_000", + "post_step_009", + "post_step_024", + "post_step_049", + "vae_inputs" + ], + "streams": [ + "video", + "audio" + ], + "ranks": [ + 0, + 1, + 2, + 3 + ], + "dtype": "torch.float32", + "compare": [ + "shape", + "dtype", + "logical_bytes_sha256" + ], + "raw_rank0_payload_sha_and_finite_required": true, + "global_replicated_state_equality_required": true, + "sampler_and_all_51_sigma_boundaries_equal": true, + "decoded_pixels": "every RGB24 frame, exact bytes", + "decoded_audio": "every interleaved float32 PCM sample, exact bytes", + "container_bytes_identity": "reported, not a substitute for pixels/PCM", + "human_checks": false + }, + "workload": { + "steps": 50, + "width": 1344, + "height": 768, + "fps": 24, + "durations_frames": { + "5": 124, + "15": 362 + }, + "seed": 3100, + "flow_shift": 12, + "audio_flow_shift": 3, + "parameters": "BF16 dense original model; original FP32 intermediate/latent state preserved", + "parallel": "USP4, ring1, TE TP4, tiled VAE patch parallel4; DLO on for reference" + }, + "latency_target": { + "scope": "full HTTP request through complete MP4 response body", + "min_reduction_pct": 50, + "min_speedup": 2, + "diagnostic_latencies_are_not_qualifying": true + }, + "protocol": { + "numerical_qualification": { + "fresh_server": true, + "warmup": 0, + "measured_requests": 2, + "audit": true, + "not_clean_timing": true + }, + "screening_per_arm": { + "fresh_server": true, + "same_shape_warmups": 2, + "measured_requests": 3, + "profiler": false, + "audit": false + }, + "final_confirmation": { + "fresh_server_pairs": 3, + "order": [ + "AB", + "BA", + "AB" + ], + "warmups_per_server": 2, + "measurements_per_server": 3, + "statistical_unit": "server median", + "requirement": "Each paired candidate server median and overall server-median comparison meet 50% reduction; all quality/lifecycle gates complete.", + "statistical_significance": "not claimed", + "budget": "Check remaining total ceiling or separately authorize an additional phase; never exceed the active deadline." + }, + "revision_note": "Final 3-pair proposal replaces the unfrozen 5-pair draft before any candidate outcome is observed, to bound execution cost. The quality and 50% bars are unchanged." + }, + "validator_sha256": "b83f2bff553ff6ca2970a16e25525cac5d9523612e9c05ea563852c814019028", + "lossy_contract": "not frozen; no lossy candidate qualification" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json new file mode 100644 index 000000000..10d7753d8 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/frozen-lossy-contract-v1.json @@ -0,0 +1,341 @@ +{ + "schema_version": 1, + "frozen_at": "2026-11-01T11:08:31.611111+00:00", + "issue": "https://github.com/InferMatrixLab/InferRSI/issues/297", + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "decision": "Prospective automatic RSI quality contract; user retains final acceptance. No real lossy candidate has been generated or scored.", + "baseline_source": "df1c1b05936381dbaa3891c1ae4c4b21ab375a5d", + "model_revision": "6818f6c32d12b210915e44ad56a4228c2608f160", + "reference_image_sha256": "3c3196accd6fb0de0642945bcaed1ddd4bfff04885b070455004c707fbfcfe3b", + "scorer_sha256": "99f5a76436a3c8d37462fab83f63d24eae74b4458c30e5a00b60c19fde139f3e", + "comparison_sha256": "99e8e23463f8ce83b52a22de9cdc697ef4bbf14d910a0c67e76bfd01e94c4b3d", + "source_revisions": { + "vbench": "fd18b3d055cb0fc6f066ca90fe2c3c8cbb698490", + "clip": "d05afc436d78f1c48dc0dbf8e5980a9d471f35f6", + "dino": "7c446df5b9f45747937fb0d72314eb9f7b66930a", + "clap": "8fa0f1c6d0433df6e97c127f64b2a1d6c0dcda8a" + }, + "samples": [ + { + "id": "5s-00", + "split": "search", + "request": { + "model": "h3-ref2va-baseline", + "prompt": "A slow cinematic camera move across the sunset landscape shown in the reference image. Preserve the open fields, distant hills, and warm orange sky. Gentle cloud movement and natural wind ambience.", + "width": "1344", + "height": "768", + "aspect_ratio": "16:9", + "fps": "24", + "seconds": "5", + "num_inference_steps": "50", + "seed": "3100", + "flow_shift": "12", + "extra_params": "{\"task\":\"ref2va\",\"duration\":5.0,\"audio_flow_shift\":3.0}", + "image_reference": "{\"image_url\":\"http://127.0.0.1:19193/reference.png\"}", + "quality": "lossless" + }, + "audio_caption": "Natural wind over open fields with a quiet outdoor ambience." + }, + { + "id": "5s-01", + "split": "search", + "request": { + "model": "h3-ref2va-baseline", + "prompt": "A locked camera observes the sunset landscape in the reference image. Preserve the fields, distant hills, and orange sky. Clouds drift gently while a light breeze moves the grass. Natural wind ambience.", + "width": "1344", + "height": "768", + "aspect_ratio": "16:9", + "fps": "24", + "seconds": "5", + "num_inference_steps": "50", + "seed": "3101", + "flow_shift": "12", + "extra_params": "{\"task\":\"ref2va\",\"duration\":5.0,\"audio_flow_shift\":3.0}", + "image_reference": "{\"image_url\":\"http://127.0.0.1:19193/reference.png\"}", + "quality": "lossless" + }, + "audio_caption": "A light outdoor breeze and soft grass rustling." + }, + { + "id": "5s-02", + "split": "search", + "request": { + "model": "h3-ref2va-baseline", + "prompt": "Slowly pan right across the sunset fields shown in the reference image. Keep the distant hills and warm sky consistent. Grass sways in a steady breeze with natural rustling sounds.", + "width": "1344", + "height": "768", + "aspect_ratio": "16:9", + "fps": "24", + "seconds": "5", + "num_inference_steps": "50", + "seed": "3102", + "flow_shift": "12", + "extra_params": "{\"task\":\"ref2va\",\"duration\":5.0,\"audio_flow_shift\":3.0}", + "image_reference": "{\"image_url\":\"http://127.0.0.1:19193/reference.png\"}", + "quality": "lossless" + }, + "audio_caption": "Steady wind and grass rustling in open fields." + }, + { + "id": "5s-03", + "split": "search", + "request": { + "model": "h3-ref2va-baseline", + "prompt": "Move the camera slowly forward over the field in the reference image. Preserve the sunset colors and distant horizon. Clouds and grass move naturally, accompanied by quiet wind.", + "width": "1344", + "height": "768", + "aspect_ratio": "16:9", + "fps": "24", + "seconds": "5", + "num_inference_steps": "50", + "seed": "3103", + "flow_shift": "12", + "extra_params": "{\"task\":\"ref2va\",\"duration\":5.0,\"audio_flow_shift\":3.0}", + "image_reference": "{\"image_url\":\"http://127.0.0.1:19193/reference.png\"}", + "quality": "lossless" + }, + "audio_caption": "Quiet wind over an open field." + }, + { + "id": "5s-04", + "split": "held_out", + "request": { + "model": "h3-ref2va-baseline", + "prompt": "A gentle leftward camera move reveals the same rural sunset landscape. Preserve the orange clouds and distant hills. A soft breeze moves the vegetation; faint birds can be heard in the distance.", + "width": "1344", + "height": "768", + "aspect_ratio": "16:9", + "fps": "24", + "seconds": "5", + "num_inference_steps": "50", + "seed": "3104", + "flow_shift": "12", + "extra_params": "{\"task\":\"ref2va\",\"duration\":5.0,\"audio_flow_shift\":3.0}", + "image_reference": "{\"image_url\":\"http://127.0.0.1:19193/reference.png\"}", + "quality": "lossless" + }, + "audio_caption": "Soft wind and faint distant bird calls." + }, + { + "id": "15s-00", + "split": "search", + "request": { + "model": "h3-ref2va-baseline", + "prompt": "A slow cinematic camera move across the sunset landscape shown in the reference image. Preserve the open fields, distant hills, and warm orange sky. Gentle cloud movement and natural wind ambience.", + "width": "1344", + "height": "768", + "aspect_ratio": "16:9", + "fps": "24", + "seconds": "15", + "num_inference_steps": "50", + "seed": "3100", + "flow_shift": "12", + "extra_params": "{\"task\":\"ref2va\",\"duration\":15.0,\"audio_flow_shift\":3.0}", + "image_reference": "{\"image_url\":\"http://127.0.0.1:19193/reference.png\"}", + "quality": "lossless" + }, + "audio_caption": "Natural wind over open fields with a quiet outdoor ambience." + }, + { + "id": "15s-01", + "split": "search", + "request": { + "model": "h3-ref2va-baseline", + "prompt": "A locked camera observes the sunset landscape in the reference image. Preserve the fields, distant hills, and orange sky. Clouds drift gently while a light breeze moves the grass. Natural wind ambience.", + "width": "1344", + "height": "768", + "aspect_ratio": "16:9", + "fps": "24", + "seconds": "15", + "num_inference_steps": "50", + "seed": "3106", + "flow_shift": "12", + "extra_params": "{\"task\":\"ref2va\",\"duration\":15.0,\"audio_flow_shift\":3.0}", + "image_reference": "{\"image_url\":\"http://127.0.0.1:19193/reference.png\"}", + "quality": "lossless" + }, + "audio_caption": "A light outdoor breeze and soft grass rustling." + }, + { + "id": "15s-02", + "split": "search", + "request": { + "model": "h3-ref2va-baseline", + "prompt": "Slowly pan right across the sunset fields shown in the reference image. Keep the distant hills and warm sky consistent. Grass sways in a steady breeze with natural rustling sounds.", + "width": "1344", + "height": "768", + "aspect_ratio": "16:9", + "fps": "24", + "seconds": "15", + "num_inference_steps": "50", + "seed": "3107", + "flow_shift": "12", + "extra_params": "{\"task\":\"ref2va\",\"duration\":15.0,\"audio_flow_shift\":3.0}", + "image_reference": "{\"image_url\":\"http://127.0.0.1:19193/reference.png\"}", + "quality": "lossless" + }, + "audio_caption": "Steady wind and grass rustling in open fields." + }, + { + "id": "15s-03", + "split": "search", + "request": { + "model": "h3-ref2va-baseline", + "prompt": "Move the camera slowly forward over the field in the reference image. Preserve the sunset colors and distant horizon. Clouds and grass move naturally, accompanied by quiet wind.", + "width": "1344", + "height": "768", + "aspect_ratio": "16:9", + "fps": "24", + "seconds": "15", + "num_inference_steps": "50", + "seed": "3108", + "flow_shift": "12", + "extra_params": "{\"task\":\"ref2va\",\"duration\":15.0,\"audio_flow_shift\":3.0}", + "image_reference": "{\"image_url\":\"http://127.0.0.1:19193/reference.png\"}", + "quality": "lossless" + }, + "audio_caption": "Quiet wind over an open field." + }, + { + "id": "15s-04", + "split": "held_out", + "request": { + "model": "h3-ref2va-baseline", + "prompt": "A gentle leftward camera move reveals the same rural sunset landscape. Preserve the orange clouds and distant hills. A soft breeze moves the vegetation; faint birds can be heard in the distance.", + "width": "1344", + "height": "768", + "aspect_ratio": "16:9", + "fps": "24", + "seconds": "15", + "num_inference_steps": "50", + "seed": "3109", + "flow_shift": "12", + "extra_params": "{\"task\":\"ref2va\",\"duration\":15.0,\"audio_flow_shift\":3.0}", + "image_reference": "{\"image_url\":\"http://127.0.0.1:19193/reference.png\"}", + "quality": "lossless" + }, + "audio_caption": "Soft wind and faint distant bird calls." + } + ], + "gates": { + "reduced_vbench_relative_error_max": 0.01, + "dimension_relative_error_max": 0.01, + "per_clip_reduced_vbench_relative_error_max": 0.03, + "dino_paired_mean_cosine_min": 0.98, + "dino_paired_p05_cosine_min": 0.95, + "clip_paired_mean_cosine_min": 0.98, + "clap_paired_mean_cosine_min": 0.99, + "clap_paired_segment_cosine_min": 0.98, + "adherence_max_absolute_cosine_drop": 0.01, + "audio_rms_max_abs_db": 1.0, + "clipping_max_fraction_increase": 0.0001, + "audio_envelope_relative_l2_max": 0.25, + "motion_floor": 1e-06, + "motion_ratio_min": 0.8, + "motion_ratio_max": 1.25 + }, + "aggregation": "For each duration independently, arithmetic mean of the five clip scores for each of two VBench dimensions, then equal-weight mean across dimensions. Require absolute relative error <1% for total and each dimension. All observed per-clip guards must pass. Missing coverage is INCONCLUSIVE.", + "held_out": "For each duration separately, release its held-out sample once after selecting that track final candidate; include both held-out samples in the ten-sample decision. Search uses the other eight.", + "scoring": "CPU FP32, official unmodified dimension functions with read_frame=True; only decord import deferred in the CPU source adapter. Every RGB frame is processed. CLAP sees the complete waveform in one (5 s) or three (15 s) equal contiguous windows after rational 32-to-48 kHz resampling. Random preprocessing seed resets to zero per window.", + "workload": "Original 50 steps, original model weights; no distilled model or step reduction. Optimization controls are recorded separately; only request quality and declared cache hints may differ.", + "weight_files": [ + { + "path": "quality-cache/weights/ViT-B-32.pt", + "sha256": "40d365715913c9da98579312b702a82c18be219cc2a73407c4526f58eba950af" + }, + { + "path": "quality-cache/weights/dino_vitbase16_pretrain.pth", + "sha256": "bf34ad0f424b9029b593e8dc3ed553bf26e88bcba0d32bf3e62a6209cb64c85e" + }, + { + "path": "quality-cache/clap/8fa0f1c6d0433df6e97c127f64b2a1d6c0dcda8a/config.json", + "sha256": "9efb9557bc804f2ca6e394486af2e45dfed0b18554909735a99c6220b84e4288" + }, + { + "path": "quality-cache/clap/8fa0f1c6d0433df6e97c127f64b2a1d6c0dcda8a/preprocessor_config.json", + "sha256": "9739f58296aa6f9ac18008fd0150fb2649bc554985fbde86d0a4041c882ac753" + }, + { + "path": "quality-cache/clap/8fa0f1c6d0433df6e97c127f64b2a1d6c0dcda8a/pytorch_model.bin", + "sha256": "1cd3c601bc4afe0fa87be3de4c13dd2cfadd249fac1e29acf74a9b296c3219bb" + }, + { + "path": "quality-cache/clap/8fa0f1c6d0433df6e97c127f64b2a1d6c0dcda8a/merges.txt", + "sha256": "fe36cab26d4f4421ed725e10a2e9ddb7f799449c603a96e7f29b5a3c82a95862" + }, + { + "path": "quality-cache/clap/8fa0f1c6d0433df6e97c127f64b2a1d6c0dcda8a/vocab.json", + "sha256": "ed19656ea1707df69134c4af35c8ceda2cc9860bf2c3495026153a133670ab5e" + }, + { + "path": "quality-cache/clap/8fa0f1c6d0433df6e97c127f64b2a1d6c0dcda8a/tokenizer.json", + "sha256": "77ef92283d67f0d97e1454909a964afcbfa2019f0fb9f18f8e88d5c25c3ba729" + }, + { + "path": "quality-cache/clap/8fa0f1c6d0433df6e97c127f64b2a1d6c0dcda8a/tokenizer_config.json", + "sha256": "377f91458f7729a4574a84c77bdce67dbc3c58c1a345a29bbf8c4eb1307948a3" + }, + { + "path": "quality-cache/clap/8fa0f1c6d0433df6e97c127f64b2a1d6c0dcda8a/special_tokens_map.json", + "sha256": "06e405a36dfe4b9604f484f6a1e619af1a7f7d09e34a8555eb0b77b66318067f" + } + ], + "source_files": [ + { + "path": "quality-adapter/vbench/__init__.py", + "sha256": "ed75ffe21eda639114cae10a0c81565f6f4b290b844895f4d4574fded2fc8cda" + }, + { + "path": "quality-adapter/vbench/background_consistency.py", + "sha256": "7def92057819df0dad01c857eecf3b39b8b555c8cdfe881a672cc2b19dc226b4" + }, + { + "path": "quality-adapter/vbench/distributed.py", + "sha256": "13178b64ac62d71a01339ce6d8e29452163e7a653016a7e2d3af907e0d51f3fe" + }, + { + "path": "quality-adapter/vbench/subject_consistency.py", + "sha256": "55f5c9a3e5be543380a709b94712903fa67a76340c3c0cb2414ac090dbd1b78d" + }, + { + "path": "quality-adapter/vbench/utils.py", + "sha256": "c06c92d03cffaa05f1952e9378f06b2b3535b22dab65772df5a33f4fe9939174" + }, + { + "path": "source/clip/clip/simple_tokenizer.py", + "sha256": "d1b09f10ed3d1e2c343619cb98cbfacb92363dd8607526a43aa7fe2d60d15bad" + }, + { + "path": "source/clip/clip/__init__.py", + "sha256": "d7930eb290ebed432116795c91473200d098451a94c0aedd4ae0f934f97955e8" + }, + { + "path": "source/clip/clip/clip.py", + "sha256": "9540f200fbf8145479fa655382a56dab048d238cc698b9cbd8df3b6d86d3f1b6" + }, + { + "path": "source/clip/clip/model.py", + "sha256": "9902cbe5ee90a1da2aa3e6f043e8a23dc1f8831193b963785c9af03d5c7bef2c" + }, + { + "path": "source/dino/vision_transformer.py", + "sha256": "b1f998d5f49ab43666b9fc6d007c5f6540c3ead2e8645e144f74045f63ff44d7" + }, + { + "path": "source/dino/utils.py", + "sha256": "962a97e1acda1c986dfd921275325e4083f9016dddadcf018f1ccf03fa600eab" + }, + { + "path": "validator/check_media.py", + "sha256": "617c79d3c55dd74b5ccd2a10ba51f70241e36fe95be28a08e3f714fddcdbfa38" + }, + { + "path": "envs/b0/lib/python3.12/site-packages/transformers/models/clap/modeling_clap.py", + "sha256": "0c8335dc90fb7633aa7d838be0ec960d00826244a9a2194cdd2e7ae0acd8cc28" + }, + { + "path": "envs/b0/lib/python3.12/site-packages/transformers/models/clap/feature_extraction_clap.py", + "sha256": "6b7a51f63ae1b7d9eeb5351b678b8d0786cc59c44156377255f4d2f5f57b476f" + } + ] +} \ No newline at end of file diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/plan/native-cli-910c-device-map.json b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/native-cli-910c-device-map.json new file mode 100644 index 000000000..0fbd748f2 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/native-cli-910c-device-map.json @@ -0,0 +1,49 @@ +{ + "verified_at_utc": "2026-11-01T17:58:30.353138+00:00", + "inferrsi_source": "50c19da43e0b358e6f990c00719891ad308a8ac1", + "inspection_owner": "src/inferrsi/infra/platforms/__init__.py:_npu_smi_records reads physical NPU rows", + "native_inventory": { + "physical_dual_die_npus": 8, + "logical_devices": 16, + "unchanged_native_inventory": true + }, + "native_cli_device_ids": [ + "1", + "2" + ], + "native_cli_id_semantics": "physical NPU IDs in npu-smi info", + "execution_logical_device_ids": [ + 2, + 3, + 4, + 5 + ], + "mapping": [ + { + "physical_npu_id": 1, + "chip_id": 0, + "logical_id": 2 + }, + { + "physical_npu_id": 1, + "chip_id": 1, + "logical_id": 3 + }, + { + "physical_npu_id": 2, + "chip_id": 0, + "logical_id": 4 + }, + { + "physical_npu_id": 2, + "chip_id": 1, + "logical_id": 5 + } + ], + "runtime_enforcement": "Existing unchanged runtime.sh exports ASCEND_RT_VISIBLE_DEVICES=2,3,4,5; launch_server num_gpus4/USP4. InferRSI runner external commands do not derive runtime visibility from discovery IDs. Each service independently checks native foreign processes and records environment.", + "scope": "Only metadata in prospective unexecuted C1 CLI argv corrected. Original v1 retained. No allocation, execution devices, native inventory, numerical/quality/timing contract or source revision changed. CLI plan/run still unexecuted.", + "native_readback_sha256": "280c0d3e579acf7e11ad1092ba7d894ee7507ea49de881691313bed6bb6bdc4c", + "native_readback": "Private rawreceipt retained outside this draft; selected mapping projected here.", + "corrected_plan_argv": "plan/c1-screening-plan-argv-v2.json", + "raw_receipt_included": false +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/plan/reproduction.md b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/reproduction.md new file mode 100644 index 000000000..d9ff0710c --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/reproduction.md @@ -0,0 +1,67 @@ +# Replay and artifact availability + +This snapshot records device experiments performed with InferRSI +`50c19da43e0b358e6f990c00719891ad308a8ac1`. The PR's repository base may be newer; +publishing evidence does not rerun or migrate those experiments. + +## Checks available from this checkout + +With the repository's development dependencies installed: + +```sh +python tools/validate_evidence.py +python -m pytest tests/test_evidence_inventory.py tests/test_smoke.py tests/test_claim.py +``` + +The inventory checks the publication digest map and eight byte-preserved frozen +snapshots. Its external/unavailable list is expected: it does not download model +weights or raw media, replay NPU execution, certify privacy, or change a verdict. + +Read [the experiment report](../results/report.md), then +[the execution snapshot](execution-checkpoint.json), +[the frozen lossless contract](frozen-lossless-contract-v1.json) and +[the frozen lossy contract](frozen-lossy-contract-v1.json). The lossy contract +includes fixed requests and loopback image URLs. Supply the original reference +image with the recorded SHA256; a replacement image changes the workload. +Held-out requests have not been generated or scored and remain reserved for +selection under that contract. + +## Engine source reconstruction + +The engine changes are patch artifacts under [patch/](../patch/README.md), not +changes to the installed InferRSI runtime. Each identity states the tested +private engine revision and patch base. The private commit object itself may +be unavailable publicly; the diff is supplied for source inspection. + +For C1, use an isolated vLLM-Omni checkout at +`922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb`, then apply +`patch/c1-h3-bf16-prefix.patch`. C12/C13/C15–C23 patches are cumulative over +C1 `be319692b96413918973b5bd399cdda3f320c574`; apply C1 first, then exactly one +selected cumulative candidate patch. They are alternative experiments, not a +sequence to stack. C10/C11 patch bases are stated separately in their identities. +Use `git apply --check` before applying a patch. Do not overwrite a shared checkout. + +The identity's raw file hashes describe the tested Windows/Linux working bytes; +Git can normalize CRLF/LF while preserving the Git tree. A reconstructed checkout +does not recreate a private signed commit or prove numerical equivalence. + +## Additional requirements for device replay + +An authorized replay needs four idle logical 910C devices with 64 GiB each, the +exact original model and reference image, the recorded dependency stack, +isolated reference/candidate/evaluator environments and an independently frozen +resource/timing plan. The private Ubuntu namespace, CANN installation, request +driver, diagnostics and bootstrap scripts are not bundled. Full device replay +is therefore not self-contained from this PR. + +The [strict validator](../validator/strict_compare.py) compares original audit +layouts and raw tensors/decoded media; the archived scoring scripts require the +dependency weights and directory layout recorded in the frozen contract. Those +scripts are preserved evidence, not a newly generalized public runner. Retained +source/weight hashes identify missing inputs but do not make them available. + +Never substitute diagnostic request latency for a clean pair. Original B0 remains +the comparator; the migration branch and candidate use separate source identities. +Final confirmation still requires all four tracks and the prospective three-pair +protocol. Known quality failures remain FAIL and should not be rerun to select +favorable samples. No device replay is launched by this tracking publication. diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/plan/requester-decisions.json b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/requester-decisions.json new file mode 100644 index 000000000..3a3a63823 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/plan/requester-decisions.json @@ -0,0 +1,36 @@ +{ + "lossless": { + "mode": "bit-exact", + "comparison_points_and_dtypes": "frozen in frozen-lossless-contract-v1.json", + "baseline_repeatability": "Original5s and15s PASS219; see private raw receipts and numerical scope" + }, + "lossy": { + "framework": "reduced vBench relative BF16 score error <1%", + "human_checks_in_rsi": false, + "automated_video_audio_reference_temporal_gates": "frozen-lossy-contract-v1.json; incomplete corpus,observed violations retained" + }, + "search_ceiling": { + "candidate_directions": 24, + "wall_hours": 32, + "logical_device_hours": 128, + "includes_failures_and_retries": true, + "allocation_and_accounting": "Original start17:19+08 unchanged; requester extended ceilingto24directions/32wallhours/128logical-device-hours,deadline11-03 01:19+08; see execution-manifest.json", + "final_confirmation_budget": "No additional amount authorized; must fit original ceiling or remain pending", + "extension_basis": "Requester decisions recorded by the accountable owner in the campaign session; no restart or extra final budget." + }, + "latency_objective": { + "metric": "complete HTTP latency versus matched B0", + "min_reduction_pct": 50, + "min_speedup": 2, + "scope": "each of the four tracks independently" + }, + "campaign_verdict": "INCONCLUSIVE", + "candidate_search_started": true, + "decision_source": "https://github.com/InferMatrixLab/InferRSI/issues/297#issuecomment-6555458151", + "status": "Execution checkpoint; no accepted complete-HTTP gain", + "initial_search_ceiling": { + "candidate_directions": 12, + "wall_hours": 16, + "logical_device_hours": 64 + } +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/README.md b/evidence/h3-ref2va-910c-4logical-20261101-297/results/README.md new file mode 100644 index 000000000..77b62dc27 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/README.md @@ -0,0 +1,104 @@ +# Results status + +Campaign verdict: **INCONCLUSIVE**. No accepted complete-HTTP gain exists. + +- [C1 strict 5 s comparison](c1-5s-strict-comparison.json): 439 checks pass, + zero failures/missing, two complete 50-step requests compared with the original + B0; selected video/audio latent bits, schedule and every decoded RGB24/PCM sample. + This qualifies this fixed 5 s request, not performance. +- [C1 strict 15 s comparison](c1-15s-strict-comparison.json): 439 checks pass, + zero failures/missing, two complete 50-step requests compared directly with + original B0. [Execution projection](c1-15s-execution.json) records actual + prefix-attention calls: 2,500 per rank per request. Its HTTP values are audited + diagnostics, not clean performance qualification. +- [Automatic quality checkpoint](automatic-quality-checkpoint.json): observed + cache, Laser and RainFusion failures are retained with missing sample coverage. + In particular, the Taylor2 .08/max3 profile failed both 15s-00 and 5s-01 despite + its first 5s-00 observed gates passing. No human output checks were used. +- [Query-folding operator study](c1-queryfold-operator.json): 2/4/8 query tiles + remain bit-exact on one real 5 s fixture but are slower by approximately + 2.0%/2.4%/5.2%, including repeated K/V and output-layout restoration. Do not integrate. + +- Original C10 Fn1/Bn1 failed 5s-00 CLAP and audio adherence. Its Bn0 + variant passed the first clip's observed checks but failed independent 5s-01 + CLAP. Both variants are rejected for 5 s; 15 s remains independently untested. +- Taylor3 Fn1/Bn1 failed 5s-01 CLAP and a CLAP segment. Diagnostic HTTP + 176.088 s does not qualify this rejected configuration. +- [C11 native execution](c11-native-execution.json): 20 tests passed, including + two executed Ascend tests covering two-clock forecasting and the actual + Cache-DiT wrapper's compute/forecast path. Source and cleanup checks passed. + The private five-file patch is bound to 61b8efa. Both Fn1/Bn0 and Fn1/Bn1 + failed different frozen 5s-00 audio guards; neither is promoted. +- [FP32 residual native revision](c11-fp32-native-execution.json): 23 tests + passed, including three executed NPU tests. Two test-reference failures are + retained. Production bytes stayed unchanged across those native attempts. + The new private patch is bound to d4d25b4. It failed5s-00 adherence + and five independent15s-00 visual/audio guards; neither duration is promoted. + [5s audit](c11-fp32-5s-00-audit.json) and + [15s audit](c11-fp32-15s-00-audit.json) retain actual counts and violations. + +Original private executor receipts, native inventory and raw captures remain +outside the repository. These files are selected field projections, not complete +portable trial archives. + +- [First clean 5 s AB screening](c1-clean-5s-screening.json): original B0 + median291.247763 s, migration+C1 median286.666760 s;1.572888689% reduction. + Native performance FAIL at the unchanged50% bar; strict pair quality PASS35 + with separate audit PASS439; source integrity and cleanup passed. One fresh + pair, two warmups plus three measured requests per arm; no significance or + final acceptance claim. The delta includes migration+C1, without an isolated + migration-only timing arm. + +15 s clean timing, final three-pair confirmation, full lossy quality coverage +and all four final track decisions remain outstanding. + +## Earlier C12–C17 execution checkpoint + +[Earlier projection](c12-through-c17-execution-checkpoint.json) retains failures +and actual counters. C13 direct-B0 strict5s comparison PASS439; clean performance +and15s remain unqualified. C14 block-FP8/INT8 probes failed before FIA2; public +MindIE dev already has quant_attention, while the installed3.1.0 wheel lacks it. +C15 FP16 residual failed finite velocity. [C16 actual model proof](c16-5s-00-execution-proof.json) +shows finite full50steps and134input checks but three audio quality failures. +C17 passed15native tests and a real5s operator gate:69.603→25.266ms,63.7004% +reduction, intentionally nonbit output. Its full model failed11frozen quality checks. +No newly accepted complete-HTTP gain exists. All four final tracks remain open. + +## Earlier checkpoint through C20 + +[Earlier C12–C20 projection](c12-through-c20-execution-checkpoint.json) and +[C17 model proof](c17-5s-00-execution-proof.json) retain the completed failures. +C18/C19/C20 each passed17native tests, then failed the prospective operator +screen:7.25%/52.85%/48.42%operation reduction,relL2 .218013/.258805/.263361. +The unchanged primitive bounds require>=50%and relL2<.258052; no full models +were run for these three. All source branches and bundles are preserved. +At that checkpoint selected devices/ports were idle and 21/24 directions were +charged. Runtime diagnostic remains v17. No accepted complete-HTTP gain exists. + +## Current checkpoint through C23 + +[Current C12–C23 projection](c12-through-c23-execution-checkpoint.json) retains +the source-bound native, operator and full-model attempts. C21 passed 36 native +tests but failed seven frozen quality checks on its complete 5s-00 request. +C22 passed 45 native tests but failed nine frozen quality checks. Neither fixed +profile proceeds to 15 s or clean timing. The audited controller-alias repair +preserves the raw budget snapshot, failed attempts and all time charges. + +C23 V1 passed 44 native tests but reduced its static real-input 50-call attention +cycle by only 38.26%, below the unchanged 50% operator gate. Component diagnosis +found a bit-equal Torch merge faster than the manual Triton implementation. +C23 V2 passed 44 native tests, three executed four-rank suites, exact real-input +merge replay and the unchanged operator gate: 3477.473 to 1166.387 ms per cycle, +66.46% reduction, finite outputs and maximum relative L2 0.00201662 < 0.005. +This excludes FFN, changing-input quality, full-model timing and HTTP. +[Full 5s-00 evidence](c23-5s-00-execution-proof.json) records completed 50 steps, +134 input checks, expected cache/FFN/DLO counters and cleanup. Diagnostic HTTP +was 199.065 s; 39 frozen quality checks produced eight failures in visual/audio +similarity, reference/audio adherence, audio level and envelope. The fixed +profile is rejected; no 15 s or clean timing extension follows. +[Retirement proof](c23-retired-proof.json) records idle devices, free ports, +unchanged tested source, runtime v17 and frozen validators at 15:48:59 +08. + +All 24 direction families are charged. The original 32 h / 128 logical-device- +hours and 2026-11-03 01:19 +08 deadline remain unchanged. All four final tracks +remain open; heldout requests remain sealed and no public publication occurred. diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/automatic-quality-checkpoint.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/automatic-quality-checkpoint.json new file mode 100644 index 000000000..1e5707182 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/automatic-quality-checkpoint.json @@ -0,0 +1,1847 @@ +{ + "campaign_verdict": "INCONCLUSIVE", + "comparisons": [ + { + "configuration_key": "audio-cache-guard", + "sample_id": "5s-00", + "comparison_sha256": "7e1ae17e18f57e1fa59c9e4eab199ea73ccdfcbc450be8d765f29b954dd98f6a", + "verdict": "FAIL", + "checks": 76, + "failures": [ + "5s-00:clap: similarity", + "5s-00:audio_caption_clap_cosine: adherence" + ], + "missing": [ + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.9661745814288535, + "background_consistency": 0.9675477634115917 + }, + "cosines": { + "dino": { + "mean": 0.9991652965545654, + "min": 0.9982653260231018, + "p05": 0.9986943453550339 + }, + "clip": { + "mean": 0.9960272908210754, + "min": 0.9868044853210449, + "p05": 0.9926694452762603 + }, + "clap": { + "mean": 0.985465407371521, + "min": 0.985465407371521, + "p05": 0.985465407371521 + } + }, + "reduced_relative_error": 0.0007849554886155881, + "audio_rms_db": -0.30719158571507965, + "audio_envelope_relative_l2": 0.12341521943906826, + "motion_ratio": 0.9977740956441081 + }, + "5s-01": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9213417344461612, + "background_consistency": 0.9521126235888256 + }, + "candidate": { + "subject_consistency": 0.9199168895318256, + "background_consistency": 0.962323812692146 + }, + "cosines": { + "dino": { + "mean": 0.9927577376365662, + "min": 0.978660523891449, + "p05": 0.9818145453929901 + }, + "clip": { + "mean": 0.9899253249168396, + "min": 0.9589802026748657, + "p05": 0.9826266527175903 + }, + "clap": { + "mean": 0.9935711026191711, + "min": 0.9935711026191711, + "p05": 0.9935711026191711 + } + }, + "reduced_relative_error": 0.004689916330921762, + "audio_rms_db": 0.08751117168316971, + "audio_envelope_relative_l2": 0.01751661659815182, + "motion_ratio": 1.001235231687485 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "audio-cache-guard", + "sample_id": "5s-01", + "comparison_sha256": "e9c8225f5437078f47ecbcbdd62bd6c5dc1c831dafb25cffa0b92fdda4e7bcfa", + "verdict": "INCONCLUSIVE", + "checks": 39, + "failures": [], + "missing": [ + "5s-00", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-01": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9213417344461612, + "background_consistency": 0.9521126235888256 + }, + "candidate": { + "subject_consistency": 0.9199168895318256, + "background_consistency": 0.962323812692146 + }, + "cosines": { + "dino": { + "mean": 0.9927577376365662, + "min": 0.978660523891449, + "p05": 0.9818145453929901 + }, + "clip": { + "mean": 0.9899253249168396, + "min": 0.9589802026748657, + "p05": 0.9826266527175903 + }, + "clap": { + "mean": 0.9935711026191711, + "min": 0.9935711026191711, + "p05": 0.9935711026191711 + } + }, + "reduced_relative_error": 0.004689916330921762, + "audio_rms_db": 0.08751117168316971, + "audio_envelope_relative_l2": 0.01751661659815182, + "motion_ratio": 1.001235231687485 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "audio-guard-bn0", + "sample_id": "5s-00", + "comparison_sha256": "1c06eb16613344654fd0d2d3e5f53faaebc8ae521e5a2fc6fde710db02f1a10e", + "verdict": "INCONCLUSIVE", + "checks": 39, + "failures": [], + "missing": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.9660068995099726, + "background_consistency": 0.9688905959691458 + }, + "cosines": { + "dino": { + "mean": 0.9992567300796509, + "min": 0.9986443519592285, + "p05": 0.9990008890628814 + }, + "clip": { + "mean": 0.9961938261985779, + "min": 0.9860672950744629, + "p05": 0.9928072810173034 + }, + "clap": { + "mean": 0.9955079555511475, + "min": 0.9955079555511475, + "p05": 0.9955079555511475 + } + }, + "reduced_relative_error": 0.0013931467745854867, + "audio_rms_db": -0.0815295231710133, + "audio_envelope_relative_l2": 0.09144958691574608, + "motion_ratio": 0.9973141825132419 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "audio-guard-bn0", + "sample_id": "5s-01", + "comparison_sha256": "95c023ee20e9e7a32ac2d763b6dda759fa42bf0f21b819fab5a4d855a75ae694", + "verdict": "FAIL", + "checks": 76, + "failures": [ + "5s-01:clap: similarity" + ], + "missing": [ + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.9660068995099726, + "background_consistency": 0.9688905959691458 + }, + "cosines": { + "dino": { + "mean": 0.9992567300796509, + "min": 0.9986443519592285, + "p05": 0.9990008890628814 + }, + "clip": { + "mean": 0.9961938261985779, + "min": 0.9860672950744629, + "p05": 0.9928072810173034 + }, + "clap": { + "mean": 0.9955079555511475, + "min": 0.9955079555511475, + "p05": 0.9955079555511475 + } + }, + "reduced_relative_error": 0.0013931467745854867, + "audio_rms_db": -0.0815295231710133, + "audio_envelope_relative_l2": 0.09144958691574608, + "motion_ratio": 0.9973141825132419 + }, + "5s-01": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9213417344461612, + "background_consistency": 0.9521126235888256 + }, + "candidate": { + "subject_consistency": 0.9195573150627012, + "background_consistency": 0.9624702191449762 + }, + "cosines": { + "dino": { + "mean": 0.9923730492591858, + "min": 0.9756415486335754, + "p05": 0.9809045672416687 + }, + "clip": { + "mean": 0.9897363781929016, + "min": 0.9703266620635986, + "p05": 0.979900699853897 + }, + "clap": { + "mean": 0.983326256275177, + "min": 0.983326256275177, + "p05": 0.983326256275177 + } + }, + "reduced_relative_error": 0.004576132925748447, + "audio_rms_db": 0.12675221378818413, + "audio_envelope_relative_l2": 0.028032476073448085, + "motion_ratio": 0.9837732598446114 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "cache-balanced-008-max2", + "sample_id": "5s-00", + "comparison_sha256": "e760ac0f421d8188407b5e057720a938d984366e299dc324bbc3bb2c71259b14", + "verdict": "FAIL", + "checks": 39, + "failures": [ + "5s-00:clap: similarity", + "5s-00: clap segment", + "5s-00: audio temporal envelope" + ], + "missing": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.9651354716076115, + "background_consistency": 0.966441892996067 + }, + "cosines": { + "dino": { + "mean": 0.9980329871177673, + "min": 0.9964174628257751, + "p05": 0.997240173816681 + }, + "clip": { + "mean": 0.9923331141471863, + "min": 0.9756749868392944, + "p05": 0.9842084020376205 + }, + "clap": { + "mean": 0.9760740995407104, + "min": 0.9760740995407104, + "p05": 0.9760740995407104 + } + }, + "reduced_relative_error": 0.0003251645637818601, + "audio_rms_db": -0.3659990076999208, + "audio_envelope_relative_l2": 0.26497643494616335, + "motion_ratio": 0.9831136213720726 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "cache-taylor2-004-max1-bn1", + "sample_id": "5s-01", + "comparison_sha256": "423b5e479f3519755109661cd48298f2e6d71ce803fbfcc4b1a5360696a7fc2a", + "verdict": "FAIL", + "checks": 39, + "failures": [ + "5s-01:clap: similarity", + "5s-01:audio_caption_clap_cosine: adherence" + ], + "missing": [ + "5s-00", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-01": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9213417344461612, + "background_consistency": 0.9521126235888256 + }, + "candidate": { + "subject_consistency": 0.9194389666483654, + "background_consistency": 0.9604502952680355 + }, + "cosines": { + "dino": { + "mean": 0.9951377511024475, + "min": 0.9648282527923584, + "p05": 0.9873370617628098 + }, + "clip": { + "mean": 0.9926162362098694, + "min": 0.9726531505584717, + "p05": 0.9865877836942673 + }, + "clap": { + "mean": 0.9898052215576172, + "min": 0.9898052215576172, + "p05": 0.9898052215576172 + } + }, + "reduced_relative_error": 0.0034347801716203, + "audio_rms_db": -0.04559134981289249, + "audio_envelope_relative_l2": 0.01794616595761191, + "motion_ratio": 0.992853542121268 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "cache-taylor2-008-max3", + "sample_id": "15s-00", + "comparison_sha256": "101ba21fcd0245a72b9003dadcae4a6f59525c044f1f117f51d73e5b826073f0", + "verdict": "FAIL", + "checks": 76, + "failures": [ + "15s-00:dino: similarity", + "15s-00:clip: similarity", + "15s-00:clap: similarity", + "15s-00: dino lower tail", + "15s-00: clap segment" + ], + "missing": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.9654010900152408, + "background_consistency": 0.9671699547670721 + }, + "cosines": { + "dino": { + "mean": 0.9983525276184082, + "min": 0.9966764450073242, + "p05": 0.9975502401590347 + }, + "clip": { + "mean": 0.9948052167892456, + "min": 0.9867727160453796, + "p05": 0.9891456693410874 + }, + "clap": { + "mean": 0.9920889735221863, + "min": 0.9920889735221863, + "p05": 0.9920889735221863 + } + }, + "reduced_relative_error": 0.00018910790971873886, + "audio_rms_db": -0.2371983786171961, + "audio_envelope_relative_l2": 0.18912491681824536, + "motion_ratio": 0.9983713680460938 + }, + "15s-00": { + "seconds": "15", + "baseline": { + "subject_consistency": 0.8167167416157155, + "background_consistency": 0.8849122500320551 + }, + "candidate": { + "subject_consistency": 0.814778186318947, + "background_consistency": 0.9055140411424505 + }, + "cosines": { + "dino": { + "mean": 0.9708892107009888, + "min": 0.8723620176315308, + "p05": 0.9347653150558471 + }, + "clip": { + "mean": 0.9714338779449463, + "min": 0.8665174245834351, + "p05": 0.9213008940219879 + }, + "clap": { + "mean": 0.979804515838623, + "min": 0.9607230424880981, + "p05": 0.963468861579895 + } + }, + "reduced_relative_error": 0.010967864267259823, + "audio_rms_db": 0.0547138050738972, + "audio_envelope_relative_l2": 0.07447108624639824, + "motion_ratio": 0.983092625811758 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "cache-taylor2-008-max3", + "sample_id": "5s-00", + "comparison_sha256": "8a436513633dcc897d1f28607054928d04011cf0263abbbd41948ff29af9b232", + "verdict": "INCONCLUSIVE", + "checks": 39, + "failures": [], + "missing": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.9654010900152408, + "background_consistency": 0.9671699547670721 + }, + "cosines": { + "dino": { + "mean": 0.9983525276184082, + "min": 0.9966764450073242, + "p05": 0.9975502401590347 + }, + "clip": { + "mean": 0.9948052167892456, + "min": 0.9867727160453796, + "p05": 0.9891456693410874 + }, + "clap": { + "mean": 0.9920889735221863, + "min": 0.9920889735221863, + "p05": 0.9920889735221863 + } + }, + "reduced_relative_error": 0.00018910790971873886, + "audio_rms_db": -0.2371983786171961, + "audio_envelope_relative_l2": 0.18912491681824536, + "motion_ratio": 0.9983713680460938 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "cache-taylor2-008-max3", + "sample_id": "5s-01", + "comparison_sha256": "4fcb44d3efcf482968a4ba83da7de10dd96a722da392373e11f6d03415043069", + "verdict": "FAIL", + "checks": 113, + "failures": [ + "5s-01:clap: similarity", + "5s-01: clap segment", + "5s-01:audio_caption_clap_cosine: adherence", + "15s-00:dino: similarity", + "15s-00:clip: similarity", + "15s-00:clap: similarity", + "15s-00: dino lower tail", + "15s-00: clap segment" + ], + "missing": [ + "5s-02", + "5s-03", + "5s-04", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.9654010900152408, + "background_consistency": 0.9671699547670721 + }, + "cosines": { + "dino": { + "mean": 0.9983525276184082, + "min": 0.9966764450073242, + "p05": 0.9975502401590347 + }, + "clip": { + "mean": 0.9948052167892456, + "min": 0.9867727160453796, + "p05": 0.9891456693410874 + }, + "clap": { + "mean": 0.9920889735221863, + "min": 0.9920889735221863, + "p05": 0.9920889735221863 + } + }, + "reduced_relative_error": 0.00018910790971873886, + "audio_rms_db": -0.2371983786171961, + "audio_envelope_relative_l2": 0.18912491681824536, + "motion_ratio": 0.9983713680460938 + }, + "5s-01": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9213417344461612, + "background_consistency": 0.9521126235888256 + }, + "candidate": { + "subject_consistency": 0.9207581187166819, + "background_consistency": 0.9634070793787638 + }, + "cosines": { + "dino": { + "mean": 0.9845563769340515, + "min": 0.9531147480010986, + "p05": 0.9630601137876511 + }, + "clip": { + "mean": 0.9853087663650513, + "min": 0.9452791810035706, + "p05": 0.9642424941062927 + }, + "clap": { + "mean": 0.9720799326896667, + "min": 0.9720799326896667, + "p05": 0.9720799326896667 + } + }, + "reduced_relative_error": 0.005717160930300574, + "audio_rms_db": 0.4509269168547554, + "audio_envelope_relative_l2": 0.09309591765214825, + "motion_ratio": 1.018571144526001 + }, + "15s-00": { + "seconds": "15", + "baseline": { + "subject_consistency": 0.8167167416157155, + "background_consistency": 0.8849122500320551 + }, + "candidate": { + "subject_consistency": 0.814778186318947, + "background_consistency": 0.9055140411424505 + }, + "cosines": { + "dino": { + "mean": 0.9708892107009888, + "min": 0.8723620176315308, + "p05": 0.9347653150558471 + }, + "clip": { + "mean": 0.9714338779449463, + "min": 0.8665174245834351, + "p05": 0.9213008940219879 + }, + "clap": { + "mean": 0.979804515838623, + "min": 0.9607230424880981, + "p05": 0.963468861579895 + } + }, + "reduced_relative_error": 0.010967864267259823, + "audio_rms_db": 0.0547138050738972, + "audio_envelope_relative_l2": 0.07447108624639824, + "motion_ratio": 0.983092625811758 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "cache-taylor2-008-max3-bn1", + "sample_id": "5s-01", + "comparison_sha256": "0ced9712635d0ffbd609b436a77f1652aae2e6d981c098b8d0477dc0dc9f3f24", + "verdict": "FAIL", + "checks": 39, + "failures": [ + "5s-01:clap: similarity", + "5s-01: clap segment" + ], + "missing": [ + "5s-00", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-01": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9213417344461612, + "background_consistency": 0.9521126235888256 + }, + "candidate": { + "subject_consistency": 0.921592558787121, + "background_consistency": 0.9582489375176468 + }, + "cosines": { + "dino": { + "mean": 0.9852516651153564, + "min": 0.9538165330886841, + "p05": 0.9698864668607712 + }, + "clip": { + "mean": 0.9861429929733276, + "min": 0.9640627503395081, + "p05": 0.9699478477239609 + }, + "clap": { + "mean": 0.973999559879303, + "min": 0.973999559879303, + "p05": 0.973999559879303 + } + }, + "reduced_relative_error": 0.003409284161307452, + "audio_rms_db": 0.20808666707388404, + "audio_envelope_relative_l2": 0.03845678935899298, + "motion_ratio": 1.0336753638251845 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "cache-taylor2-008-max3-bn4", + "sample_id": "5s-01", + "comparison_sha256": "d2bf98312b742f1dada1e0acdb453bdc1759567902a878737b9464ad4dfc08f6", + "verdict": "FAIL", + "checks": 39, + "failures": [ + "5s-01:dino: similarity", + "5s-01:clip: similarity", + "5s-01:clap: similarity", + "5s-01: dino lower tail", + "5s-01: clap segment", + "5s-01:reference_dino_cosine: adherence", + "5s-01:audio_caption_clap_cosine: adherence" + ], + "missing": [ + "5s-00", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-01": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9213417344461612, + "background_consistency": 0.9521126235888256 + }, + "candidate": { + "subject_consistency": 0.9193919410065907, + "background_consistency": 0.9695822967261802 + }, + "cosines": { + "dino": { + "mean": 0.9559677839279175, + "min": 0.917838454246521, + "p05": 0.9317350178956986 + }, + "clip": { + "mean": 0.9704718589782715, + "min": 0.937095046043396, + "p05": 0.9475325375795365 + }, + "clap": { + "mean": 0.9627834558486938, + "min": 0.9627834558486938, + "p05": 0.9627834558486938 + } + }, + "reduced_relative_error": 0.00828409810531098, + "audio_rms_db": 0.04268223342783043, + "audio_envelope_relative_l2": 0.06482673389999478, + "motion_ratio": 1.2091018487554375 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "cache-taylor3-008-max3-bn1", + "sample_id": "5s-01", + "comparison_sha256": "a20e4bd5b4c919bb5a65cd11d2bb0cc7dc6aa045ec2f3b1bd5a9c6e28489411b", + "verdict": "FAIL", + "checks": 39, + "failures": [ + "5s-01:clap: similarity", + "5s-01: clap segment" + ], + "missing": [ + "5s-00", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-01": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9213417344461612, + "background_consistency": 0.9521126235888256 + }, + "candidate": { + "subject_consistency": 0.9205945810166801, + "background_consistency": 0.961419354124767 + }, + "cosines": { + "dino": { + "mean": 0.9861611127853394, + "min": 0.9520018696784973, + "p05": 0.9690358877182007 + }, + "clip": { + "mean": 0.9868983626365662, + "min": 0.94464111328125, + "p05": 0.9714847385883332 + }, + "clap": { + "mean": 0.973813533782959, + "min": 0.973813533782959, + "p05": 0.973813533782959 + } + }, + "reduced_relative_error": 0.004568874106673343, + "audio_rms_db": 0.32023464803440704, + "audio_envelope_relative_l2": 0.048263250142338246, + "motion_ratio": 1.037731901309175 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "hybrid-laser-video-bf16-context", + "sample_id": "5s-00", + "comparison_sha256": "60c4356e41b34ff791043bf34179a1fa693780b70c273405e0011e19269017d6", + "verdict": "FAIL", + "checks": 39, + "failures": [ + "5s-00:audio_caption_clap_cosine: adherence" + ], + "missing": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.9661054570015853, + "background_consistency": 0.9678300546436776 + }, + "cosines": { + "dino": { + "mean": 0.9992612600326538, + "min": 0.9987664222717285, + "p05": 0.9990039587020874 + }, + "clip": { + "mean": 0.9964061379432678, + "min": 0.986167848110199, + "p05": 0.9930137008428573 + }, + "clap": { + "mean": 0.9925766587257385, + "min": 0.9925766587257385, + "p05": 0.9925766587257385 + } + }, + "reduced_relative_error": 0.0008952785305148697, + "audio_rms_db": -0.04112309268137339, + "audio_envelope_relative_l2": 0.03829713389408406, + "motion_ratio": 0.9988179931035557 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "laser-scale256", + "sample_id": "5s-00", + "comparison_sha256": "91c2419327f5333f59b8d3f74cfc47d84634ad6e7d4f026e3abd61d422b9aad8", + "verdict": "FAIL", + "checks": 39, + "failures": [ + "5s-00:audio_caption_clap_cosine: adherence" + ], + "missing": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.9667792068264349, + "background_consistency": 0.9680149632740797 + }, + "cosines": { + "dino": { + "mean": 0.999154806137085, + "min": 0.9985394477844238, + "p05": 0.9986860930919648 + }, + "clip": { + "mean": 0.9963837265968323, + "min": 0.9901332855224609, + "p05": 0.9924647331237793 + }, + "clap": { + "mean": 0.9945911169052124, + "min": 0.9945911169052124, + "p05": 0.9945911169052124 + } + }, + "reduced_relative_error": 0.0013396714218789517, + "audio_rms_db": -0.11301796107390463, + "audio_envelope_relative_l2": 0.12640093834958974, + "motion_ratio": 1.0010309184980533 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "modality-sigma-fp32-anchor", + "sample_id": "15s-00", + "comparison_sha256": "1b75e2ea84298165d6c19e86b2a8e7ebd08835924c6e71832eed9b79d920f452", + "verdict": "FAIL", + "checks": 76, + "failures": [ + "5s-00:audio_caption_clap_cosine: adherence", + "15s-00:dino: similarity", + "15s-00:clip: similarity", + "15s-00:clap: similarity", + "15s-00: dino lower tail", + "15s-00: clap segment" + ], + "missing": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.9634841426116664, + "background_consistency": 0.9637262268764216 + }, + "cosines": { + "dino": { + "mean": 0.9979137182235718, + "min": 0.9962100982666016, + "p05": 0.9968585044145584 + }, + "clip": { + "mean": 0.993086576461792, + "min": 0.9763539433479309, + "p05": 0.9824884951114654 + }, + "clap": { + "mean": 0.9918106198310852, + "min": 0.9918106198310852, + "p05": 0.9918106198310852 + } + }, + "reduced_relative_error": 0.002585273427929995, + "audio_rms_db": -0.1567548220318038, + "audio_envelope_relative_l2": 0.08416217762953182, + "motion_ratio": 1.0089539814707082 + }, + "15s-00": { + "seconds": "15", + "baseline": { + "subject_consistency": 0.8167167416157155, + "background_consistency": 0.8849122500320551 + }, + "candidate": { + "subject_consistency": 0.8073931144875502, + "background_consistency": 0.8748147027809534 + }, + "cosines": { + "dino": { + "mean": 0.9541988372802734, + "min": 0.8350636959075928, + "p05": 0.9045321136713028 + }, + "clip": { + "mean": 0.9663479924201965, + "min": 0.8494424819946289, + "p05": 0.9202105909585953 + }, + "clap": { + "mean": 0.9531505703926086, + "min": 0.9011195302009583, + "p05": 0.9078725516796112 + } + }, + "reduced_relative_error": 0.011413283667940134, + "audio_rms_db": 0.1285537754516634, + "audio_envelope_relative_l2": 0.13562143033577656, + "motion_ratio": 1.0165335096317705 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "modality-sigma-fp32-anchor", + "sample_id": "5s-00", + "comparison_sha256": "5ea1cf7b5bdc342a7b298e0469dd4042cee7c981ba533df3510052538b65e8c0", + "verdict": "FAIL", + "checks": 39, + "failures": [ + "5s-00:audio_caption_clap_cosine: adherence" + ], + "missing": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.9634841426116664, + "background_consistency": 0.9637262268764216 + }, + "cosines": { + "dino": { + "mean": 0.9979137182235718, + "min": 0.9962100982666016, + "p05": 0.9968585044145584 + }, + "clip": { + "mean": 0.993086576461792, + "min": 0.9763539433479309, + "p05": 0.9824884951114654 + }, + "clap": { + "mean": 0.9918106198310852, + "min": 0.9918106198310852, + "p05": 0.9918106198310852 + } + }, + "reduced_relative_error": 0.002585273427929995, + "audio_rms_db": -0.1567548220318038, + "audio_envelope_relative_l2": 0.08416217762953182, + "motion_ratio": 1.0089539814707082 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "modality-sigma-quadratic", + "sample_id": "5s-00", + "comparison_sha256": "4b795a65ab2165390844e117813b8efd24aec3d5b7b75cf5652f73576b0a8099", + "verdict": "FAIL", + "checks": 39, + "failures": [ + "5s-00:audio_caption_clap_cosine: adherence" + ], + "missing": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.9642229327341405, + "background_consistency": 0.9690693287345452 + }, + "cosines": { + "dino": { + "mean": 0.9978399872779846, + "min": 0.9946956634521484, + "p05": 0.9965563297271729 + }, + "clip": { + "mean": 0.9933723211288452, + "min": 0.9708013534545898, + "p05": 0.9835974663496018 + }, + "clap": { + "mean": 0.9960051774978638, + "min": 0.9960051774978638, + "p05": 0.9960051774978638 + } + }, + "reduced_relative_error": 0.0005623687407243513, + "audio_rms_db": -0.141298705336359, + "audio_envelope_relative_l2": 0.08778629602289918, + "motion_ratio": 1.0088439516314678 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "modality-sigma-quadratic-bn1", + "sample_id": "5s-00", + "comparison_sha256": "6fde89564e6572398df194e82f7634af92020ff9a84d5f2d1383df18809354f7", + "verdict": "FAIL", + "checks": 39, + "failures": [ + "5s-00: audio level" + ], + "missing": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.9667953994215989, + "background_consistency": 0.9636606847852226 + }, + "cosines": { + "dino": { + "mean": 0.993774950504303, + "min": 0.9827978014945984, + "p05": 0.9883693248033524 + }, + "clip": { + "mean": 0.9904270172119141, + "min": 0.972300112247467, + "p05": 0.9838350206613541 + }, + "clap": { + "mean": 0.9939356446266174, + "min": 0.9939356446266174, + "p05": 0.9939356446266174 + } + }, + "reduced_relative_error": 0.0009054756694858051, + "audio_rms_db": -1.0007078553545243, + "audio_envelope_relative_l2": 0.19737799824495614, + "motion_ratio": 1.0067671166204435 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "rainfusion50-bf16-w8-tail8", + "sample_id": "5s-00", + "comparison_sha256": "7f64efe80fcca7b976e82a2cdd9f9876832c85b226de7f26992c7fe70d020864", + "verdict": "FAIL", + "checks": 39, + "failures": [ + "5s-00:clap: similarity", + "5s-00: clap segment", + "5s-00: audio temporal envelope" + ], + "missing": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.9667071133125119, + "background_consistency": 0.9689083639683762 + }, + "cosines": { + "dino": { + "mean": 0.9946473836898804, + "min": 0.9871225357055664, + "p05": 0.9900005757808685 + }, + "clip": { + "mean": 0.9907172918319702, + "min": 0.9721568822860718, + "p05": 0.9800667852163315 + }, + "clap": { + "mean": 0.9769790172576904, + "min": 0.9769790172576904, + "p05": 0.9769790172576904 + } + }, + "reduced_relative_error": 0.001764733412885843, + "audio_rms_db": -0.14111524718973395, + "audio_envelope_relative_l2": 0.2797538746282675, + "motion_ratio": 1.026263875056369 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "rainfusion80-bf16-w4-tail4", + "sample_id": "5s-00", + "comparison_sha256": "c739c5b0f2db605991f40a5d2c0adc7ced4d025717a581bbeed157f5f80a717e", + "verdict": "FAIL", + "checks": 39, + "failures": [ + "5s-00:dino: similarity", + "5s-00:clip: similarity", + "5s-00:clap: similarity", + "5s-00: clap segment", + "5s-00: audio level", + "5s-00: audio temporal envelope" + ], + "missing": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.9632140490097728, + "background_consistency": 0.9603373892908174 + }, + "cosines": { + "dino": { + "mean": 0.9794362187385559, + "min": 0.9407444000244141, + "p05": 0.9508843153715134 + }, + "clip": { + "mean": 0.9764617681503296, + "min": 0.9348478317260742, + "p05": 0.9474407732486725 + }, + "clap": { + "mean": 0.9637646079063416, + "min": 0.9637646079063416, + "p05": 0.9637646079063416 + } + }, + "reduced_relative_error": 0.00447892858239735, + "audio_rms_db": -2.173295241375743, + "audio_envelope_relative_l2": 0.36216840555897617, + "motion_ratio": 1.0477099329740793 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "refkv-device-refresh2", + "sample_id": "5s-00", + "comparison_sha256": "6682812c58757690b5114492834e68c54e17988e22834e45dd93856a3e4cfd27", + "verdict": "FAIL", + "checks": 39, + "failures": [ + "5s-00:clap: similarity" + ], + "missing": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.9673074055008772, + "background_consistency": 0.9747018482142348 + }, + "cosines": { + "dino": { + "mean": 0.9847110509872437, + "min": 0.937708854675293, + "p05": 0.9595070838928222 + }, + "clip": { + "mean": 0.9860638380050659, + "min": 0.9606876373291016, + "p05": 0.9736308753490448 + }, + "clap": { + "mean": 0.9891108274459839, + "min": 0.9891108274459839, + "p05": 0.9891108274459839 + } + }, + "reduced_relative_error": 0.005073789276672285, + "audio_rms_db": -0.11514590616150458, + "audio_envelope_relative_l2": 0.1555914434459509, + "motion_ratio": 0.8933582476698538 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "state-aligned-residual-v1", + "sample_id": "5s-00", + "comparison_sha256": "f4b14d6eba264c0c348cc01862c45ad62ed71409cadd5cd66465b85d63c89d60", + "verdict": "INCONCLUSIVE", + "checks": 39, + "failures": [], + "missing": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "cosines": { + "dino": { + "mean": 1.0, + "min": 0.9999998211860657, + "p05": 0.9999998807907104 + }, + "clip": { + "mean": 1.0, + "min": 0.9999997615814209, + "p05": 0.9999998807907104 + }, + "clap": { + "mean": 1.0, + "min": 1.0, + "p05": 1.0 + } + }, + "reduced_relative_error": 0.0, + "audio_rms_db": 0.0, + "audio_envelope_relative_l2": 0.0, + "motion_ratio": 1.0 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "tp4-all-resident", + "sample_id": "5s-00", + "comparison_sha256": "7b9e4ba13d0b6c098c7c529e6d3f23a885559681114850ae551f7a6cdeee8684", + "verdict": "FAIL", + "checks": 39, + "failures": [ + "5s-00:audio_caption_clap_cosine: adherence" + ], + "missing": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.966968424678818, + "background_consistency": 0.9685012627907885 + }, + "cosines": { + "dino": { + "mean": 0.9991623163223267, + "min": 0.9984581470489502, + "p05": 0.9986790925264358 + }, + "clip": { + "mean": 0.9960500597953796, + "min": 0.9835643768310547, + "p05": 0.991750293970108 + }, + "clap": { + "mean": 0.9936883449554443, + "min": 0.9936883449554443, + "p05": 0.9936883449554443 + } + }, + "reduced_relative_error": 0.001689280879494273, + "audio_rms_db": -0.5914884173619972, + "audio_envelope_relative_l2": 0.14660693414345288, + "motion_ratio": 0.9986264182059804 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "video-context-lse", + "sample_id": "5s-00", + "comparison_sha256": "2b3d31ae933c50c78571a71deee42a3a23ec6f4e6986b3fe4b1e76945544aa13", + "verdict": "FAIL", + "checks": 39, + "failures": [ + "5s-00:dino: similarity", + "5s-00:clip: similarity", + "5s-00:clap: similarity", + "5s-00: clap segment", + "5s-00:reference_dino_cosine: adherence", + "5s-00:audio_caption_clap_cosine: adherence", + "5s-00: audio level", + "5s-00: audio temporal envelope" + ], + "missing": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.9650656626961096, + "background_consistency": 0.9708255100540999 + }, + "cosines": { + "dino": { + "mean": 0.97896808385849, + "min": 0.9462954998016357, + "p05": 0.9537913143634796 + }, + "clip": { + "mean": 0.9749853014945984, + "min": 0.9158563017845154, + "p05": 0.9535898625850677 + }, + "clap": { + "mean": 0.9371471405029297, + "min": 0.9371471405029297, + "p05": 0.9371471405029297 + } + }, + "reduced_relative_error": 0.001907417743306655, + "audio_rms_db": -6.375729760723826, + "audio_envelope_relative_l2": 0.5422772289154131, + "motion_ratio": 0.9732356030412213 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "video-pre-gate", + "sample_id": "5s-00", + "comparison_sha256": "f821a089e7bacbbd51d2debe40169d23db13cc5ff744b2237acc3474765bf68c", + "verdict": "FAIL", + "checks": 39, + "failures": [ + "5s-00:dino: similarity", + "5s-00:clip: similarity", + "5s-00:clap: similarity", + "5s-00: dino lower tail", + "5s-00: clap segment", + "5s-00:reference_dino_cosine: adherence", + "5s-00: audio temporal envelope" + ], + "missing": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.9604354704298624, + "background_consistency": 0.9678790501462735 + }, + "cosines": { + "dino": { + "mean": 0.9699685573577881, + "min": 0.9310290217399597, + "p05": 0.9455805897712708 + }, + "clip": { + "mean": 0.9705709218978882, + "min": 0.9431105852127075, + "p05": 0.9498869240283966 + }, + "clap": { + "mean": 0.9706100225448608, + "min": 0.9706100225448608, + "p05": 0.9706100225448608 + } + }, + "reduced_relative_error": 0.0020138274804530603, + "audio_rms_db": -0.20747782268393988, + "audio_envelope_relative_l2": 0.2799833627923924, + "motion_ratio": 1.121115241596866 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "video-pre-gate-secant", + "sample_id": "5s-00", + "comparison_sha256": "6a809a10611ec2231cc9da15514924d8c1a61ecb3a6abdcf8baad1425a0d357c", + "verdict": "FAIL", + "checks": 39, + "failures": [ + "5s-00:dino: similarity", + "5s-00:clip: similarity", + "5s-00:clap: similarity", + "5s-00: dino lower tail", + "5s-00: clap segment", + "5s-00:reference_clip_cosine: adherence", + "5s-00:reference_dino_cosine: adherence", + "5s-00:audio_caption_clap_cosine: adherence", + "5s-00: motion energy" + ], + "missing": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.9611704327711245, + "background_consistency": 0.9549353379544204 + }, + "cosines": { + "dino": { + "mean": 0.9575454592704773, + "min": 0.9346060752868652, + "p05": 0.9436570793390274 + }, + "clip": { + "mean": 0.9614511132240295, + "min": 0.919788658618927, + "p05": 0.9337257981300354 + }, + "clap": { + "mean": 0.9732030630111694, + "min": 0.9732030630111694, + "p05": 0.9732030630111694 + } + }, + "reduced_relative_error": 0.008332383610497398, + "audio_rms_db": 0.5065753171403786, + "audio_envelope_relative_l2": 0.24859357669358687, + "motion_ratio": 1.2605876841404495 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "video-row-fp32", + "sample_id": "5s-00", + "comparison_sha256": "9226ab5741347e5928911c4e01a136c64f85bfc7406f812ff4a936b18a41c272", + "verdict": "FAIL", + "checks": 39, + "failures": [ + "5s-00:clap: similarity", + "5s-00: clap segment", + "5s-00: audio temporal envelope" + ], + "missing": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.9644393346658567, + "background_consistency": 0.9664301145367507 + }, + "cosines": { + "dino": { + "mean": 0.997343897819519, + "min": 0.9954850673675537, + "p05": 0.9960428774356842 + }, + "clip": { + "mean": 0.9929363131523132, + "min": 0.9782494902610779, + "p05": 0.9856640666723251 + }, + "clap": { + "mean": 0.9739290475845337, + "min": 0.9739290475845337, + "p05": 0.9739290475845337 + } + }, + "reduced_relative_error": 0.0006915414043044125, + "audio_rms_db": -0.10083169173992401, + "audio_envelope_relative_l2": 0.2743734098838323, + "motion_ratio": 1.0172588417564994 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + }, + { + "configuration_key": "video-temporal-window", + "sample_id": "5s-00", + "comparison_sha256": "f7f23452cea62d3ff53d6289a1b988e5279de9be52856c5ea7a4af849025ed40", + "verdict": "FAIL", + "checks": 39, + "failures": [ + "5s-00:dino: similarity", + "5s-00:clip: similarity", + "5s-00:clap: similarity", + "5s-00: dino lower tail", + "5s-00: clap segment", + "5s-00:reference_clip_cosine: adherence", + "5s-00:reference_dino_cosine: adherence", + "5s-00:prompt_clip_cosine: adherence", + "5s-00: audio level", + "5s-00: audio temporal envelope", + "5s-00: motion energy" + ], + "missing": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "observed": { + "5s-00": { + "seconds": "5", + "baseline": { + "subject_consistency": 0.9653875539457895, + "background_consistency": 0.9668180954650165 + }, + "candidate": { + "subject_consistency": 0.9351117109380117, + "background_consistency": 0.9415582151916938 + }, + "cosines": { + "dino": { + "mean": 0.8847789168357849, + "min": 0.7981574535369873, + "p05": 0.8222497254610062 + }, + "clip": { + "mean": 0.9208264350891113, + "min": 0.843837559223175, + "p05": 0.870898300409317 + }, + "clap": { + "mean": 0.06382762640714645, + "min": 0.06382762640714645, + "p05": 0.06382762640714645 + } + }, + "reduced_relative_error": 0.02874213896333196, + "audio_rms_db": -36.23827660321877, + "audio_envelope_relative_l2": 0.9866005095726371, + "motion_ratio": 5.481376339509837 + } + }, + "scope": "Fixed Ref2VA sunset-image corpus: relative BF16 VBench subject/background consistency plus reference, content, audio-feature, motion and media-timestamp guards. No human check. These proxies do not certify absolute audiovisual semantic synchrony or general model quality.", + "performance_acceptance": "separate" + } + ] +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-15s-execution.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-15s-execution.json new file mode 100644 index 000000000..76d33fdf2 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-15s-execution.json @@ -0,0 +1,47 @@ +{ + "source_commit": "be319692b96413918973b5bd399cdda3f320c574", + "scope": "http_complete_mp4", + "samples_ms": [ + 1707035.2496140404, + 1677409.5148360357 + ], + "latency_median_ms": 1692222.382225038, + "warmup": 0, + "repetitions": 2, + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true, + "actual_attention_calls": { + "0": { + "0": { + "_prefix_bnsd": 2500 + }, + "1": { + "_prefix_bnsd": 2500 + }, + "2": { + "_prefix_bnsd": 2500 + }, + "3": { + "_prefix_bnsd": 2500 + } + }, + "1": { + "0": { + "_prefix_bnsd": 2500 + }, + "1": { + "_prefix_bnsd": 2500 + }, + "2": { + "_prefix_bnsd": 2500 + }, + "3": { + "_prefix_bnsd": 2500 + } + } + }, + "numerical_result": "c1-15s-strict-comparison.json", + "performance_acceptance": "NOT_EVALUATED", + "campaign_verdict": "INCONCLUSIVE" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-15s-strict-comparison.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-15s-strict-comparison.json new file mode 100644 index 000000000..536b992d6 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-15s-strict-comparison.json @@ -0,0 +1,9 @@ +{ + "verdict": "PASS", + "scope": "specified latent points, schedule, decoded RGB24/PCM; no performance acceptance", + "comparison": "candidate versus baseline", + "checks": 439, + "failures": [], + "missing_evidence": [], + "quality_contract": "strict bit-exact; no tolerance or human check" +} \ No newline at end of file diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-5s-strict-comparison.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-5s-strict-comparison.json new file mode 100644 index 000000000..20c729eb6 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-5s-strict-comparison.json @@ -0,0 +1,9 @@ +{ + "verdict": "PASS", + "scope": "specified latent points, schedule, decoded RGB24/PCM; no performance acceptance", + "comparison": "candidate versus baseline", + "checks": 439, + "failures": [], + "missing_evidence": [], + "quality_contract": "strict bit-exact; no tolerance or human check" +} \ No newline at end of file diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-clean-5s-screening.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-clean-5s-screening.json new file mode 100644 index 000000000..cbb3c90af --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-clean-5s-screening.json @@ -0,0 +1,85 @@ +{ + "native_run_id": "20261101T181531Z-164cfec13c", + "inferrsi_revision": "50c19da43e0b358e6f990c00719891ad308a8ac1", + "candidate_patch_base": "922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb", + "gain_attribution_limit": "Original B0 versus migration+C1; no clean migration-only timing arm in this pair", + "duration_s": 5, + "frames": 124, + "solver_steps": 50, + "order": "AB", + "fresh_pairs": 1, + "kind": "screening", + "native_cli_discovery_physical_ids": [ + 1, + 2 + ], + "runtime_logical_devices": [ + 2, + 3, + 4, + 5 + ], + "arms": { + "baseline": { + "source_head": "df1c1b05936381dbaa3891c1ae4c4b21ab375a5d", + "samples_ms": [ + 291247.76331102476, + 291434.0972539503, + 291051.2047139928 + ], + "server_median_ms": 291247.76331102476, + "inner_range_pct_of_median": 0.13146625938158557, + "warmup": 2, + "repetitions": 3, + "clean_timing": true, + "diagnostic": "off", + "scope": "http_complete_mp4", + "cleanup_passed": true, + "helpers_unchanged": true + }, + "candidate": { + "source_head": "be319692b96413918973b5bd399cdda3f320c574", + "samples_ms": [ + 286812.4624299817, + 286666.7601839872, + 286197.25764304167 + ], + "server_median_ms": 286666.7601839872, + "inner_range_pct_of_median": 0.21460625101604527, + "warmup": 2, + "repetitions": 3, + "clean_timing": true, + "diagnostic": "off", + "scope": "http_complete_mp4", + "cleanup_passed": true, + "helpers_unchanged": true + } + }, + "http_reduction_pct": 1.5728886893271898, + "required_reduction_pct": 50.0, + "native_verdict": "FAIL", + "native_reasons": [ + "measured gain 1.57289% must be positive and meet 50% threshold" + ], + "run_state": "completed", + "source_integrity": { + "verified": true, + "reason": "Git source snapshots unchanged", + "command_checks": 3 + }, + "strict_quality": { + "verdict": "PASS", + "checks": 35, + "failures": [], + "scope": "matched strict qualification plus every measured clean RGB24/PCM response", + "final_protocol": false + }, + "strict_validation_execution": { + "status": "completed", + "returncode": 0 + }, + "statistical_significance": "not established", + "campaign_verdict": "INCONCLUSIVE: four tracks and final protocol are incomplete", + "raw_media_included": false, + "raw_records_included": false +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-queryfold-operator.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-queryfold-operator.json new file mode 100644 index 000000000..0354f9809 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-queryfold-operator.json @@ -0,0 +1,97 @@ +{ + "scope": "Single real5s first-long-attention fixture,logical device count1; not whole-model performance", + "cases": [ + { + "name": "bnsd_query_tiles2", + "bit_exact": true, + "unequal_elements": 0, + "compared_elements": 71269632, + "finite": true, + "reference_sha256": "7cabc1030b41d2e42c77aac352c20a2b676e194d1f9883b54d5c9b8f72b38c21", + "candidate_sha256": "7cabc1030b41d2e42c77aac352c20a2b676e194d1f9883b54d5c9b8f72b38c21", + "baseline_ms": [ + 71.68554002419114, + 71.42315799137577, + 71.29112700931728, + 71.63919898448512, + 71.81268103886396, + 71.30170601885766, + 71.39379699947312 + ], + "candidate_ms": [ + 73.04394000675529, + 72.85228901309893, + 72.96156999655068, + 73.32086301175877, + 72.36289500724524, + 72.75056897196919, + 72.87441898370162 + ], + "baseline_median_ms": 71.42315799137577, + "candidate_median_ms": 72.87441898370162, + "reduction_pct": -2.0319193846078276 + }, + { + "name": "bnsd_query_tiles4", + "bit_exact": true, + "unequal_elements": 0, + "compared_elements": 71269632, + "finite": true, + "reference_sha256": "7cabc1030b41d2e42c77aac352c20a2b676e194d1f9883b54d5c9b8f72b38c21", + "candidate_sha256": "7cabc1030b41d2e42c77aac352c20a2b676e194d1f9883b54d5c9b8f72b38c21", + "baseline_ms": [ + 71.867641992867, + 71.62905996665359, + 71.80321001214907, + 71.19845598936081, + 71.26071699894965, + 71.53511798242107, + 71.6239099856466 + ], + "candidate_ms": [ + 73.54129396844655, + 72.99410999985412, + 73.14522104570642, + 73.51523399120197, + 73.63296498078853, + 73.1829310534522, + 73.33417201880366 + ], + "baseline_median_ms": 71.6239099856466, + "candidate_median_ms": 73.33417201880366, + "reduction_pct": -2.387836734269033 + }, + { + "name": "bnsd_query_tiles8", + "bit_exact": true, + "unequal_elements": 0, + "compared_elements": 71269632, + "finite": true, + "reference_sha256": "7cabc1030b41d2e42c77aac352c20a2b676e194d1f9883b54d5c9b8f72b38c21", + "candidate_sha256": "7cabc1030b41d2e42c77aac352c20a2b676e194d1f9883b54d5c9b8f72b38c21", + "baseline_ms": [ + 71.52353797573596, + 71.5064779506065, + 71.6628700029105, + 71.23498595319688, + 71.53764902614057, + 71.93902198923752, + 72.01983203412965 + ], + "candidate_ms": [ + 74.3553499924019, + 75.21554798586294, + 76.08375401468948, + 75.20787802059203, + 75.23201801814139, + 75.40346903260797, + 75.78394201118499 + ], + "baseline_median_ms": 71.53764902614057, + "candidate_median_ms": 75.23201801814139, + "reduction_pct": -5.164230363023048 + } + ], + "campaign_verdict": "INCONCLUSIVE", + "decision": "All three variants are bit-exact but slower; do not integrate." +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-upstream-main-native.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-upstream-main-native.json new file mode 100644 index 000000000..1412027bb --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c1-upstream-main-native.json @@ -0,0 +1,74 @@ +{ + "upstream_pr": "https://github.com/vllm-project/vllm-omni/pull/9745", + "draft": true, + "base": "b84bade9b9be8fdad0a1a9b3e8dd3ab4273dc4d9", + "head": "7a20cf9b987b22fe3ce00221cb30cc3c6fb90e63", + "source_bound_files": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": { + "tested_working_sha256": "25989bba0e0f07b65c33d69e1c217655e65bf62722c982032f9c749bf560553c", + "published_blob_sha256": "25989bba0e0f07b65c33d69e1c217655e65bf62722c982032f9c749bf560553c", + "difference": "none" + }, + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": { + "tested_working_sha256": "e7e26ad0b2a9120b99281bf59a67eced7f7207f5679a8f02f2ede5859695be1a", + "published_blob_sha256": "e7e26ad0b2a9120b99281bf59a67eced7f7207f5679a8f02f2ede5859695be1a", + "difference": "none" + }, + "tests/diffusion/models/minimax_h3/test_minimax_h3_flash_attention.py": { + "tested_working_sha256": "94c17d91c4dfe1f611655002f7a4b10a6ea9e49cf21d37469c26daff58f1e766", + "published_blob_sha256": "94c17d91c4dfe1f611655002f7a4b10a6ea9e49cf21d37469c26daff58f1e766", + "difference": "none" + } + }, + "source_snapshot_sha256": "925d80d2c26f4e37e3cbf287b2d942d474744236200e12f130d2003cbcddc9cf", + "all_source_support_files_bound": 2285, + "native_run": "c1-upstream-pr-v2-native-a3c17706", + "hardware": { + "model": "Ascend910C", + "logical_device_count": 1, + "memory_gib": 64 + }, + "stack": { + "torch": "2.10.0+cpu", + "torch_npu": "2.10.0.post4", + "cann": "9.1", + "mindie_sd": "3.1.0", + "mindie_sd_source_revision": "UNKNOWN", + "vllm": "0.32.0+empty" + }, + "execution_status": "completed", + "tests_passed": 12, + "tests_failed": 0, + "tests_skipped": 0, + "required_native_executed": true, + "source_unchanged": true, + "cleanup_passed": true, + "executor_elapsed_seconds": 50.47770592797315, + "test_cases": [ + "test_h3_prefix_dispatch_preserves_unselected_paths[fused]", + "test_h3_prefix_dispatch_preserves_unselected_paths[default]", + "test_h3_prefix_dispatch_preserves_unselected_paths[laser]", + "test_h3_prefix_dispatch_preserves_unselected_paths[causal]", + "test_h3_prefix_dispatch_preserves_unselected_paths[masked]", + "test_h3_prefix_dispatch_preserves_unselected_paths[compact]", + "test_h3_prefix_dispatch_preserves_unselected_paths[gqa]", + "test_h3_prefix_dispatch_preserves_unselected_paths[unresolved]", + "test_h3_prefix_dispatch_preserves_unselected_paths[short]", + "test_h3_prefix_dispatch_preserves_unselected_paths[piecewise]", + "test_h3_prefix_dispatch_preserves_unselected_paths[ranges]", + "test_native_prefix_bnsd_excludes_padding_and_preserves_inputs" + ], + "prior_attempt": { + "run": "c1-upstream-pr-native-80d5d51f", + "status": "failed", + "phase": "collection", + "missing_existing_upstream_helper": "tests.model_executor.helpers", + "tests_executed": 0 + }, + "scope": "Main-based C1 dispatch/native operation checks; not full-model/HTTP requalification.", + "historical_c1_strict_results_apply_to": "be319692b96413918973b5bd399cdda3f320c574", + "full_model_5s_15s_on_new_base": "NOT_RUN", + "clean_http_on_new_base": "NOT_RUN", + "new_search_direction_charged": false, + "campaign_verdict": "INCONCLUSIVE" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c10-5s-00-audit.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c10-5s-00-audit.json new file mode 100644 index 000000000..328cadaf3 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c10-5s-00-audit.json @@ -0,0 +1,73 @@ +{ + "source_commit": "0af99184bc860cffa8d1e5343c2bff4664f667c5", + "sample_id": "5s-00", + "scope": "http_complete_mp4", + "diagnostic": "audit", + "clean_timing": false, + "samples_ms": [ + 243038.95941301016 + ], + "cleanup_passed": true, + "actual_execution": { + "0": { + "calls": { + "MiniMaxH3AudioGuardCachedBlocks.forward": 50 + }, + "guard": { + "checks": 42, + "vetoes": 27, + "audio_reuse_limit": 0.04 + } + }, + "1": { + "calls": { + "MiniMaxH3AudioGuardCachedBlocks.forward": 50 + }, + "guard": { + "checks": 42, + "vetoes": 27, + "audio_reuse_limit": 0.04 + } + }, + "2": { + "calls": { + "MiniMaxH3AudioGuardCachedBlocks.forward": 50 + }, + "guard": { + "checks": 42, + "vetoes": 27, + "audio_reuse_limit": 0.04 + } + }, + "3": { + "calls": { + "MiniMaxH3AudioGuardCachedBlocks.forward": 50 + }, + "guard": { + "checks": 42, + "vetoes": 27, + "audio_reuse_limit": 0.04 + } + } + }, + "quality_contract_sha256": "b8fd9e634e6db5ebfbce3842f3360f1b2241a08119953ee71bfc26553dbbafed", + "cumulative_observed_checks": 76, + "observed_failures": [ + "5s-00:clap: similarity", + "5s-00:audio_caption_clap_cosine: adherence" + ], + "quality_verdict": "FAIL", + "missing_samples": [ + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "decision": "Reject original C10 for5s; its first5s-01 observation did not generalize.", + "performance_acceptance": "NOT_EVALUATED", + "campaign_verdict": "INCONCLUSIVE" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c10-5s-01-audit.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c10-5s-01-audit.json new file mode 100644 index 000000000..4b1162530 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c10-5s-01-audit.json @@ -0,0 +1,31 @@ +{ + "source_commit": "0af99184bc860cffa8d1e5343c2bff4664f667c5", + "sample_id": "5s-01", + "scope": "http_complete_mp4", + "diagnostic": "audit", + "clean_timing": false, + "warmups": 0, + "repetitions": 1, + "samples_ms": [ + 236856.30676796427 + ], + "cleanup_passed": true, + "quality_contract_sha256": "b8fd9e634e6db5ebfbce3842f3360f1b2241a08119953ee71bfc26553dbbafed", + "observed_quality_checks": 39, + "observed_failures": [], + "quality_verdict": "INCONCLUSIVE", + "missing_samples": [ + "5s-00", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "execution_proof_limit": "The actual log selects the H3 wrapper and reports16cachehits. Post-denoise model-tree traversal missed its closure-held wrapper; direct guard check/veto counters remain unobserved for this run.", + "campaign_verdict": "INCONCLUSIVE", + "performance_acceptance": "NOT_EVALUATED" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c10-native-tests.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c10-native-tests.json new file mode 100644 index 000000000..6618ec113 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c10-native-tests.json @@ -0,0 +1,18 @@ +{ + "scope": "focused tests only; no full-model quality or performance acceptance", + "returncode": 0, + "status": "completed", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": true, + "tested_source_files": { + "vllm_omni/diffusion/models/minimax_h3/cache_audio_guard.py": "2844e749c9c876e52bffeef565f2c69904693e2d62c359da6abe66844930daed", + "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "d3381079ff012f32176a917bb0825d12288c058d3a058f5169ecc97d1e2c675c", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "1d5e806489de94cec2f9224f7d31ff96d9ace0a3fb80c23d8f8ba25f262d275d", + "tests/diffusion/models/minimax_h3/test_minimax_h3_audio_cache_guard.py": "c13e1ec677531457e92853d3a97e1e3fdcccb9547883ef0088f75852473d523d" + }, + "source_base_head": "922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb", + "source_commit": "0af99184bc860cffa8d1e5343c2bff4664f667c5", + "full_model_quality": "configuration-specific; original Fn1Bn1 failed5s; see quality records", + "performance_acceptance": "PENDING" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-15s-00-audit.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-15s-00-audit.json new file mode 100644 index 000000000..717956262 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-15s-00-audit.json @@ -0,0 +1,72 @@ +{ + "trial": "c11-sigma-fp32-quality-15s-00-aaa2b0d6", + "source_commit": "d4d25b420889965ab42459c96ab4a82389c9fa64", + "cache_configuration": { + "Fn_compute_blocks": 1, + "Bn_compute_blocks": 0, + "max_warmup_steps": 4, + "scm_steps_mask_policy": null, + "residual_diff_threshold": 0.08, + "max_continuous_cached_steps": 3, + "enable_taylorseer": false, + "minimax_h3_sigma_cache": true + }, + "latency_diagnostic_ms": 845953.8551190053, + "clean_timing": false, + "input_checks_passed": 134, + "cleanup_passed": true, + "actual_middle_counts_by_rank": [ + { + "computed": 23, + "predicted": 27, + "history_vetoes": 0 + }, + { + "computed": 23, + "predicted": 27, + "history_vetoes": 0 + }, + { + "computed": 23, + "predicted": 27, + "history_vetoes": 0 + }, + { + "computed": 23, + "predicted": 27, + "history_vetoes": 0 + } + ], + "quality_contract_sha256": "b8fd9e634e6db5ebfbce3842f3360f1b2241a08119953ee71bfc26553dbbafed", + "duration_verdict": "FAIL", + "duration_failures": [ + "15s-00:dino: similarity", + "15s-00:clip: similarity", + "15s-00:clap: similarity", + "15s-00: dino lower tail", + "15s-00: clap segment" + ], + "all_comparison_failures": [ + "5s-00:audio_caption_clap_cosine: adherence", + "15s-00:dino: similarity", + "15s-00:clip: similarity", + "15s-00:clap: similarity", + "15s-00: dino lower tail", + "15s-00: clap segment" + ], + "global_verdict": "FAIL", + "missing_samples": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "media_sha256": "afe42452f0852eabb49d123a8e6e328891b56876cf265fc71b7fdacff1f6bbbd", + "features_sha256": "561381462449683f8fb0ca3536fbff580789352aaf36b74efc9ece5c23b9fabf", + "decision": "Reject15s from its own five observed violations. Known5s failure is retained separately.", + "performance_acceptance": "PENDING; diagnostic HTTP cannot qualify a rejected candidate" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-15s-first-step.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-15s-first-step.json new file mode 100644 index 000000000..2968f3182 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-15s-first-step.json @@ -0,0 +1,102 @@ +{ + "checks": [ + { + "rank": 0, + "point": "initial_noise", + "stream": "video", + "equal": true + }, + { + "rank": 0, + "point": "initial_noise", + "stream": "audio", + "equal": true + }, + { + "rank": 0, + "point": "post_step_000", + "stream": "video", + "equal": true + }, + { + "rank": 0, + "point": "post_step_000", + "stream": "audio", + "equal": true + }, + { + "rank": 1, + "point": "initial_noise", + "stream": "video", + "equal": true + }, + { + "rank": 1, + "point": "initial_noise", + "stream": "audio", + "equal": true + }, + { + "rank": 1, + "point": "post_step_000", + "stream": "video", + "equal": true + }, + { + "rank": 1, + "point": "post_step_000", + "stream": "audio", + "equal": true + }, + { + "rank": 2, + "point": "initial_noise", + "stream": "video", + "equal": true + }, + { + "rank": 2, + "point": "initial_noise", + "stream": "audio", + "equal": true + }, + { + "rank": 2, + "point": "post_step_000", + "stream": "video", + "equal": true + }, + { + "rank": 2, + "point": "post_step_000", + "stream": "audio", + "equal": true + }, + { + "rank": 3, + "point": "initial_noise", + "stream": "video", + "equal": true + }, + { + "rank": 3, + "point": "initial_noise", + "stream": "audio", + "equal": true + }, + { + "rank": 3, + "point": "post_step_000", + "stream": "video", + "equal": true + }, + { + "rank": 3, + "point": "post_step_000", + "stream": "audio", + "equal": true + } + ], + "all_equal": true, + "scope": "15s initialnoise and full-compute poststep0 only; not later cachedstates, fullquality or timing acceptance" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-5s-00-audit.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-5s-00-audit.json new file mode 100644 index 000000000..abc73241f --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-5s-00-audit.json @@ -0,0 +1,62 @@ +{ + "trial": "c11-sigma-fp32-quality-5s-00-7c7b2032", + "source_commit": "d4d25b420889965ab42459c96ab4a82389c9fa64", + "cache_configuration": { + "Fn_compute_blocks": 1, + "Bn_compute_blocks": 0, + "max_warmup_steps": 4, + "scm_steps_mask_policy": null, + "residual_diff_threshold": 0.08, + "max_continuous_cached_steps": 3, + "enable_taylorseer": false, + "minimax_h3_sigma_cache": true + }, + "latency_diagnostic_ms": 169512.50694599003, + "clean_timing": false, + "input_checks_passed": 134, + "cleanup_passed": true, + "actual_middle_counts_by_rank": [ + { + "computed": 22, + "predicted": 28, + "history_vetoes": 0 + }, + { + "computed": 22, + "predicted": 28, + "history_vetoes": 0 + }, + { + "computed": 22, + "predicted": 28, + "history_vetoes": 0 + }, + { + "computed": 22, + "predicted": 28, + "history_vetoes": 0 + } + ], + "quality_contract_sha256": "b8fd9e634e6db5ebfbce3842f3360f1b2241a08119953ee71bfc26553dbbafed", + "quality_verdict": "FAIL", + "observed_checks": 39, + "observed_failures": [ + "5s-00:audio_caption_clap_cosine: adherence" + ], + "missing_samples": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "audio_caption_clap_cosine": 0.0498393215239048, + "media_sha256": "0d48934b0fec90326ae505deae54ce26a5f03733eaf0bca2e2fb5d13202c518b", + "features_sha256": "ff0b426c3e791ea782bf1d00cf2859f71678fd8c42289533f8f6f2842369133a", + "decision": "Reject5s-00 for frozenadherencefailure. No further predictorgrid. 15s requires independent qualification.", + "performance_acceptance": "PENDING; no clean pairedgain" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-5s-first-step.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-5s-first-step.json new file mode 100644 index 000000000..a1ab69b0f --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-5s-first-step.json @@ -0,0 +1,102 @@ +{ + "checks": [ + { + "rank": 0, + "point": "initial_noise", + "stream": "video", + "equal": true + }, + { + "rank": 0, + "point": "initial_noise", + "stream": "audio", + "equal": true + }, + { + "rank": 0, + "point": "post_step_000", + "stream": "video", + "equal": true + }, + { + "rank": 0, + "point": "post_step_000", + "stream": "audio", + "equal": true + }, + { + "rank": 1, + "point": "initial_noise", + "stream": "video", + "equal": true + }, + { + "rank": 1, + "point": "initial_noise", + "stream": "audio", + "equal": true + }, + { + "rank": 1, + "point": "post_step_000", + "stream": "video", + "equal": true + }, + { + "rank": 1, + "point": "post_step_000", + "stream": "audio", + "equal": true + }, + { + "rank": 2, + "point": "initial_noise", + "stream": "video", + "equal": true + }, + { + "rank": 2, + "point": "initial_noise", + "stream": "audio", + "equal": true + }, + { + "rank": 2, + "point": "post_step_000", + "stream": "video", + "equal": true + }, + { + "rank": 2, + "point": "post_step_000", + "stream": "audio", + "equal": true + }, + { + "rank": 3, + "point": "initial_noise", + "stream": "video", + "equal": true + }, + { + "rank": 3, + "point": "initial_noise", + "stream": "audio", + "equal": true + }, + { + "rank": 3, + "point": "post_step_000", + "stream": "video", + "equal": true + }, + { + "rank": 3, + "point": "post_step_000", + "stream": "audio", + "equal": true + } + ], + "all_equal": true, + "scope": "initialnoise and fullcompute poststep0 only; not later cached states, fullquality or timing acceptance" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-native-execution.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-native-execution.json new file mode 100644 index 000000000..7ec4c89a2 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-fp32-native-execution.json @@ -0,0 +1,76 @@ +{ + "attempts": [ + { + "trial": "c11-sigma-fp32-native-tests-b873db4a", + "status": "failed", + "cleanup_passed": true, + "source_unchanged": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py": "fb27fdebdf0cb45e678243c0180d56cf3e7a7470ff424bdf2d3bf144c8e2aee3", + "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "d3381079ff012f32176a917bb0825d12288c058d3a058f5169ecc97d1e2c675c", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a649a09e5b59256b5887c12fe508d8e14e3dedd9019bdb82c65db2d59f11de91", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "a9524734a4f3410a0c6f766fa64fa9e5bd5a4d41bd7481738e9024cac54ce7f9", + "tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py": "f48d3180de2b25b8ef03c7ca37eee5ff2adb0243268d18adae17142d5eedcd13" + }, + "native_test_names": [ + "test_native_two_clock_forecast_and_input_immutability", + "test_native_real_cache_manager_computes_then_forecasts" + ], + "elapsed_seconds": 51.38206260098377 + }, + { + "trial": "c11-sigma-fp32-native-tests-cc806d17", + "status": "failed", + "cleanup_passed": true, + "source_unchanged": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py": "fb27fdebdf0cb45e678243c0180d56cf3e7a7470ff424bdf2d3bf144c8e2aee3", + "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "d3381079ff012f32176a917bb0825d12288c058d3a058f5169ecc97d1e2c675c", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a649a09e5b59256b5887c12fe508d8e14e3dedd9019bdb82c65db2d59f11de91", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "a9524734a4f3410a0c6f766fa64fa9e5bd5a4d41bd7481738e9024cac54ce7f9", + "tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py": "f52b3050a8dab6141eb9d84abf6f62f9cb1b78c9233588117fe47817c62cc72c" + }, + "native_test_names": [ + "test_native_two_clock_forecast_and_input_immutability" + ], + "elapsed_seconds": 49.97501417802414 + }, + { + "trial": "c11-sigma-fp32-native-tests-9542a13d", + "status": "completed", + "cleanup_passed": true, + "source_unchanged": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py": "fb27fdebdf0cb45e678243c0180d56cf3e7a7470ff424bdf2d3bf144c8e2aee3", + "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "d3381079ff012f32176a917bb0825d12288c058d3a058f5169ecc97d1e2c675c", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a649a09e5b59256b5887c12fe508d8e14e3dedd9019bdb82c65db2d59f11de91", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "a9524734a4f3410a0c6f766fa64fa9e5bd5a4d41bd7481738e9024cac54ce7f9", + "tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py": "16d1e9329c0088eabf4bb17214fc364ec0c6176d92f871786456a2b92aaa418f" + }, + "native_test_names": [ + "test_native_two_clock_forecast_and_input_immutability", + "test_native_real_cache_manager_computes_then_forecasts", + "test_native_bf16_blocks_use_fp32_anchors_and_keep_output_dtype" + ], + "elapsed_seconds": 50.17931097600376 + } + ], + "tests_passed_after_fixture_correction": 23, + "corrections": [ + { + "failed_run": "c11-sigma-fp32-native-tests-b873db4a", + "old_test_sha256": "f48d3180de2b25b8ef03c7ca37eee5ff2adb0243268d18adae17142d5eedcd13", + "new_test_sha256": "f52b3050a8dab6141eb9d84abf6f62f9cb1b78c9233588117fe47817c62cc72c", + "production_unchanged": true, + "reason": "Old fixture ignored BF16 anchor quantization. Replace analytic unquantized reference with independent FP64 Vandermonde solve through actual anchors; exact full-compute checks and stricter forecast bound. Frozen quality evaluator unchanged." + }, + { + "failed_run": "c11-sigma-fp32-native-tests-cc806d17", + "old_test_sha256": "f52b3050a8dab6141eb9d84abf6f62f9cb1b78c9233588117fe47817c62cc72c", + "new_test_sha256": "16d1e9329c0088eabf4bb17214fc364ec0c6176d92f871786456a2b92aaa418f", + "production_unchanged": true, + "reason": "Move actual NPU anchor values to CPU before .double(): Ascend warns and downgrades double operations on device. CPU FP64 solve now receives matching Double A/B. No numerical/quality threshold change." + } + ], + "scope": "Two failed test-reference attempts retained, then23passed incl3executed NPUtests. Productionbytes unchanged across attempts. Frozen modelquality thresholds/scorers unchanged." +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-full-compute-first-step.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-full-compute-first-step.json new file mode 100644 index 000000000..e3be0bcb7 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-full-compute-first-step.json @@ -0,0 +1,102 @@ +{ + "checks": [ + { + "rank": 0, + "point": "initial_noise", + "stream": "video", + "equal": true + }, + { + "rank": 0, + "point": "initial_noise", + "stream": "audio", + "equal": true + }, + { + "rank": 0, + "point": "post_step_000", + "stream": "video", + "equal": true + }, + { + "rank": 0, + "point": "post_step_000", + "stream": "audio", + "equal": true + }, + { + "rank": 1, + "point": "initial_noise", + "stream": "video", + "equal": true + }, + { + "rank": 1, + "point": "initial_noise", + "stream": "audio", + "equal": true + }, + { + "rank": 1, + "point": "post_step_000", + "stream": "video", + "equal": true + }, + { + "rank": 1, + "point": "post_step_000", + "stream": "audio", + "equal": true + }, + { + "rank": 2, + "point": "initial_noise", + "stream": "video", + "equal": true + }, + { + "rank": 2, + "point": "initial_noise", + "stream": "audio", + "equal": true + }, + { + "rank": 2, + "point": "post_step_000", + "stream": "video", + "equal": true + }, + { + "rank": 2, + "point": "post_step_000", + "stream": "audio", + "equal": true + }, + { + "rank": 3, + "point": "initial_noise", + "stream": "video", + "equal": true + }, + { + "rank": 3, + "point": "initial_noise", + "stream": "audio", + "equal": true + }, + { + "rank": 3, + "point": "post_step_000", + "stream": "video", + "equal": true + }, + { + "rank": 3, + "point": "post_step_000", + "stream": "audio", + "equal": true + } + ], + "all_equal": true, + "scope": "initialnoise and full-compute poststep0 only; not bit-exact later cached steps or quality acceptance" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-native-execution.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-native-execution.json new file mode 100644 index 000000000..f2088a524 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-native-execution.json @@ -0,0 +1,21 @@ +{ + "trial": "c11-sigma-native-tests-e4a95751", + "status": "completed", + "source_base_head": "922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb", + "source_commit": "61b8efa06e88372bc5ab87b9e2668a133eaac282", + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/cache_sigma_prediction.py": "ac29ef338d569529e40f59210c89a57dd2e07c510b5e2952c0da451f183da52a", + "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "d3381079ff012f32176a917bb0825d12288c058d3a058f5169ecc97d1e2c675c", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a649a09e5b59256b5887c12fe508d8e14e3dedd9019bdb82c65db2d59f11de91", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "a9524734a4f3410a0c6f766fa64fa9e5bd5a4d41bd7481738e9024cac54ce7f9", + "tests/diffusion/models/minimax_h3/test_minimax_h3_sigma_cache.py": "9ba094c0bc0686e8d044ebe064beaa2606a6cad496449e63ec282411bb2e727a" + }, + "native_test_names": [ + "test_native_two_clock_forecast_and_input_immutability", + "test_native_real_cache_manager_computes_then_forecasts" + ], + "cleanup_passed": true, + "source_unchanged": true, + "elapsed_seconds": 49.4796573020285, + "scope": "20 focused tests including two executed native Ascend tests; not full-model quality or speed qualification" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-original-5s-00-audit.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-original-5s-00-audit.json new file mode 100644 index 000000000..44f81660d --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c11-original-5s-00-audit.json @@ -0,0 +1,58 @@ +{ + "trial": "c11-modality-sigma-quality-5s-00-eb726462", + "source_commit": "61b8efa06e88372bc5ab87b9e2668a133eaac282", + "cache_configuration": { + "Fn_compute_blocks": 1, + "Bn_compute_blocks": 0, + "max_warmup_steps": 4, + "scm_steps_mask_policy": null, + "residual_diff_threshold": 0.08, + "max_continuous_cached_steps": 3, + "enable_taylorseer": false, + "minimax_h3_sigma_cache": true + }, + "latency_diagnostic_ms": 171179.77748898556, + "clean_timing": false, + "input_checks_passed": 134, + "cleanup_passed": true, + "actual_middle_counts_by_rank": [ + { + "computed": 22, + "predicted": 28, + "history_vetoes": 0 + }, + { + "computed": 22, + "predicted": 28, + "history_vetoes": 0 + }, + { + "computed": 22, + "predicted": 28, + "history_vetoes": 0 + }, + { + "computed": 22, + "predicted": 28, + "history_vetoes": 0 + } + ], + "observed_quality_checks": 39, + "observed_failures": [ + "5s-00:audio_caption_clap_cosine: adherence" + ], + "quality_verdict": "FAIL", + "missing_samples": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "decision": "Reject original Fn1Bn0 for5s-00; frozen adherence failure is retained; Bn1 is a separate configuration.", + "performance_acceptance": "PENDING; diagnostic HTTP is not a clean paired gain" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json new file mode 100644 index 000000000..4b1b3752b --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c15-execution-checkpoint.json @@ -0,0 +1,509 @@ +{ + "campaign_verdict": "INCONCLUSIVE", + "performance_acceptance": "No newly accepted gain", + "runs": [ + { + "trial": "c12-alignment-quality-5s-00-9bf5c373", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 471.80311120999977 + }, + "model_diagnostic": { + "source_head": "a29648e5acd3077b7b22739bc9522017782fe1ac", + "samples_ms": [ + 318280.59680899605 + ], + "latency_median_ms": 318280.59680899605, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-1\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-2\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-3\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c12-state-alignment-native-tests-0dc5859b", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/cache_state_alignment.py": "3340ec21a77aef78531c601be241bcfb1e44107c795e8c8649907d1a18ed377f", + "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "f4cd0e9aa9e824c92ec1acd02c57a8f7d8684c35d4edf92ffdfae3345ff44e99", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "436ec52cd6329cbc64e91d94b450bfa6f773a9a953b8f974db73a5eb986a98bb", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "4dec8f02a04bfe7e8d2ada24fc6d5ae40a0058671d8515c8a266ecf88824b7b0", + "tests/diffusion/models/minimax_h3/test_minimax_h3_state_cache.py": "89cf57482b35d8d8ad8d4978aec87e9ee3ef45330e5b911161adef4379950a65" + }, + "scope": "focused tests only; no full-model quality or performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 82.06077561300481 + }, + { + "trial": "c13-baseline-cuda-staging-e22d8651", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "vllm_omni/diffusion/offloader/sequential_backend.py": "d542935b656a0d9dced78de8fb086b8e1c5aade26fac6cee5316b32a49f0d569", + "tests/diffusion/offloader/test_sequential_backend.py": "dcf1d64e65bed572d65b095c1350950f55d7e8d9ddf8c7dfd72d0ea6e721685a" + }, + "scope": "Reproduce stock CUDA-graph staging expectation on unchanged migration owning source in the same Ascend test environment", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 50.234599059971515 + }, + { + "trial": "c13-c1-dlo-5s-audit-cead10d1", + "model_controller": { + "status": "failed", + "returncode": 2, + "elapsed_seconds": 0.11969929095357656 + } + }, + { + "trial": "c13-c1-dlo-5s-audit-v2-b0b86c6d", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 749.505351218977 + }, + "model_diagnostic": { + "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "samples_ms": [ + 315593.86175096733, + 284456.94282796467 + ], + "latency_median_ms": 300025.402289466, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 2, + "kind": "diagnostic", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c13-c1-dlo-5s-calibration-c98129d8", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 768.8735517460154 + }, + "model_diagnostic": { + "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "samples_ms": [ + 317177.1984040388, + 285534.089674009 + ], + "latency_median_ms": 301355.6440390239, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 2, + "kind": "calibration", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c13-dlo-gate-tests-bc92a71d", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", + "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" + }, + "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 47.3752507569734 + }, + { + "trial": "c13-dlo-regression-tests-95e9ed2a", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", + "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" + }, + "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 62.34616496198578 + }, + { + "trial": "c13-dlo-regression-v2-12c217d5", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", + "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" + }, + "scope": "Expanded offloader tests in Ascend environment; some stock accelerator fixtures use NPU, not a full model test", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 65.14432259398745 + }, + { + "trial": "c14-int8-prerequisites-fdd6633b", + "cleanup_passed": true, + "source_unchanged": true, + "helpers_unchanged": true, + "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 20.795623040001374, + "native_prerequisite": { + "verdict": "FAIL", + "scope": "TorchNPU block-INT8 attention prerequisites only", + "phase": "dynamic_block_int8_quant_query", + "versions": { + "torch": "2.10.0+cpu", + "torch_npu": "2.10.0.post4", + "mindiesd": "3.1.0" + }, + "apis": { + "float8_e4m3fn": true, + "npu_dynamic_block_quant": true, + "npu_fused_infer_attention_score_v2": true + }, + "device_name": "Ascend910_9392", + "input": { + "shape": [ + 1, + 14, + 256, + 128 + ], + "dtype": "torch.bfloat16", + "visible_devices": "2" + }, + "error_type": "RuntimeError", + "probe_sha256": "fab18252f860231f9b7fcaa54587afbaa96fce15987fd551922246c8964f7217", + "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 561002\n[ERROR] 2026-11-02-02:00:57 (PID:2662830, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2662830] 2026-11-02-02:00:57.039.986 Invalid_Argument(EZ0024): Parameter row_block_size of DynamicBlockQuant has incorrect value 128. It should be 1.\n Solution: Check whether the parameter value is correct.\nTraceBack (most recent call last):\n The attr get failed.[FUNC:CheckParams][FILE:dynamic_block_quant_i8_tiling.cpp][LINE:78]\n The params check failed.[FUNC:Tiling4DynamicBlockQuant][FILE:dynamic_block_quant_tiling.cpp][LINE:564]\n Failed to execute operator DynamicBlockQuant_0. Reason: Failed to execute tiling function.\n Check NnopbaseExecutorDoTiling(executor) failed\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" + } + }, + { + "trial": "c14-quant-prerequisites-358b9b28", + "cleanup_passed": true, + "source_unchanged": true, + "helpers_unchanged": true, + "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 20.143024684977718, + "native_prerequisite": { + "verdict": "FAIL", + "scope": "TorchNPU FP8 prerequisites only", + "phase": "dynamic_block_quant_query", + "versions": { + "torch": "2.10.0+cpu", + "torch_npu": "2.10.0.post4", + "mindiesd": "3.1.0" + }, + "apis": { + "float8_e4m3fn": true, + "npu_dynamic_block_quant": true, + "npu_fused_infer_attention_score_v2": true + }, + "device_name": "Ascend910_9392", + "input": { + "shape": [ + 1, + 14, + 256, + 128 + ], + "dtype": "torch.bfloat16", + "visible_devices": "2" + }, + "error_type": "RuntimeError", + "probe_sha256": "df6465956e6f73bbca140fb9880057e30869fd4b7976b2dc091f71a575dd1096", + "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 161002\n[ERROR] 2026-11-02-01:45:24 (PID:2491012, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2491012] 2026-11-02-01:45:24.732.576 Execution_Error(EZ1009): Failed to execute operator DynamicBlockQuant_0. Reason: The dtype or format of the actual input or output parameter of the operator is inconsistent with that defined in the operator prototype OpDef, details of parameter are x(DT_BFLOAT16, ND), yOut(DT_FLOAT8_E4M3FN, ND), scaleOut(DT_FLOAT, ND).\nTraceBack (most recent call last):\n Supported dtype and format list is:\n SupportInfo[0]: x(DT_FLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n SupportInfo[1]: x(DT_BFLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" + } + }, + { + "trial": "c15-video-row-native-066a9a40", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "e35b551b8fc0ec3b0ba1384b4df96a8daa0e5179c5ed39d641125310b2ee957f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "6f890fcf84047bc5bc084f7b756a2b02e72573888a3bcc12ba7d0211e1bb62ef", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "d2a5bed10620a337ff9aaf99cf4a3740d94a8962ef903bf2da75d65e3b50e21c" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 82.27795292000519 + }, + { + "trial": "c15-video-row-native-3a9721a6", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "8ec1b3ae4cd252f5d95ab23e0232b983900939e8cb15d1e64f4740f7204abf39", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 81.91829407500336 + }, + { + "trial": "c15-video-row-native-a2a224de", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "875d0e70621fc76906be130519d7bb1242de5dd58159e31025e26da37e60d98c" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 82.86056435096543 + }, + { + "trial": "c15-video-row-native-bb5084cc", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "fa1ac8fd375d754503b3507fae006381138d8875afdad2418f1ea5ae338b9c26" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 81.50971999001922 + }, + { + "trial": "c15-video-row-quality-5s-00-710315e0", + "model_controller": { + "status": "failed", + "returncode": 1, + "elapsed_seconds": 153.44603093498154 + } + } + ], + "decisions": [ + "C12 fixed profile stopped: zero actual predictions.", + "C14 pinned block-FP8 and block-INT8 prerequisites failed; per-token INT8 is not excluded.", + "C13 numerical qualification and clean performance are separate.", + "C15 is a separate lossy fixed profile; native fixtures do not establish model quality." + ] +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json new file mode 100644 index 000000000..5ed1d87ab --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c17-execution-checkpoint.json @@ -0,0 +1,931 @@ +{ + "campaign_verdict": "INCONCLUSIVE", + "performance_acceptance": "No newly accepted gain", + "runs": [ + { + "trial": "c12-alignment-quality-5s-00-9bf5c373", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 471.80311120999977 + }, + "model_diagnostic": { + "source_head": "a29648e5acd3077b7b22739bc9522017782fe1ac", + "samples_ms": [ + 318280.59680899605 + ], + "latency_median_ms": 318280.59680899605, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-1\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-2\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-3\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c12-state-alignment-native-tests-0dc5859b", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/cache_state_alignment.py": "3340ec21a77aef78531c601be241bcfb1e44107c795e8c8649907d1a18ed377f", + "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "f4cd0e9aa9e824c92ec1acd02c57a8f7d8684c35d4edf92ffdfae3345ff44e99", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "436ec52cd6329cbc64e91d94b450bfa6f773a9a953b8f974db73a5eb986a98bb", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "4dec8f02a04bfe7e8d2ada24fc6d5ae40a0058671d8515c8a266ecf88824b7b0", + "tests/diffusion/models/minimax_h3/test_minimax_h3_state_cache.py": "89cf57482b35d8d8ad8d4978aec87e9ee3ef45330e5b911161adef4379950a65" + }, + "scope": "focused tests only; no full-model quality or performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 82.06077561300481 + }, + { + "trial": "c13-baseline-cuda-staging-e22d8651", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "vllm_omni/diffusion/offloader/sequential_backend.py": "d542935b656a0d9dced78de8fb086b8e1c5aade26fac6cee5316b32a49f0d569", + "tests/diffusion/offloader/test_sequential_backend.py": "dcf1d64e65bed572d65b095c1350950f55d7e8d9ddf8c7dfd72d0ea6e721685a" + }, + "scope": "Reproduce stock CUDA-graph staging expectation on unchanged migration owning source in the same Ascend test environment", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 50.234599059971515 + }, + { + "trial": "c13-c1-dlo-5s-audit-cead10d1", + "model_controller": { + "status": "failed", + "returncode": 2, + "elapsed_seconds": 0.11969929095357656 + } + }, + { + "trial": "c13-c1-dlo-5s-audit-v2-b0b86c6d", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 749.505351218977 + }, + "model_diagnostic": { + "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "samples_ms": [ + 315593.86175096733, + 284456.94282796467 + ], + "latency_median_ms": 300025.402289466, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 2, + "kind": "diagnostic", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c13-c1-dlo-5s-calibration-c98129d8", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 768.8735517460154 + }, + "model_diagnostic": { + "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "samples_ms": [ + 317177.1984040388, + 285534.089674009 + ], + "latency_median_ms": 301355.6440390239, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 2, + "kind": "calibration", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c13-dlo-gate-tests-bc92a71d", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", + "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" + }, + "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 47.3752507569734 + }, + { + "trial": "c13-dlo-regression-tests-95e9ed2a", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", + "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" + }, + "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 62.34616496198578 + }, + { + "trial": "c13-dlo-regression-v2-12c217d5", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", + "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" + }, + "scope": "Expanded offloader tests in Ascend environment; some stock accelerator fixtures use NPU, not a full model test", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 65.14432259398745 + }, + { + "trial": "c14-int8-prerequisites-fdd6633b", + "cleanup_passed": true, + "source_unchanged": true, + "helpers_unchanged": true, + "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 20.795623040001374, + "native_prerequisite": { + "verdict": "FAIL", + "scope": "TorchNPU block-INT8 attention prerequisites only", + "phase": "dynamic_block_int8_quant_query", + "versions": { + "torch": "2.10.0+cpu", + "torch_npu": "2.10.0.post4", + "mindiesd": "3.1.0" + }, + "apis": { + "float8_e4m3fn": true, + "npu_dynamic_block_quant": true, + "npu_fused_infer_attention_score_v2": true + }, + "device_name": "Ascend910_9392", + "input": { + "shape": [ + 1, + 14, + 256, + 128 + ], + "dtype": "torch.bfloat16", + "visible_devices": "2" + }, + "error_type": "RuntimeError", + "probe_sha256": "fab18252f860231f9b7fcaa54587afbaa96fce15987fd551922246c8964f7217", + "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 561002\n[ERROR] 2026-11-02-02:00:57 (PID:2662830, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2662830] 2026-11-02-02:00:57.039.986 Invalid_Argument(EZ0024): Parameter row_block_size of DynamicBlockQuant has incorrect value 128. It should be 1.\n Solution: Check whether the parameter value is correct.\nTraceBack (most recent call last):\n The attr get failed.[FUNC:CheckParams][FILE:dynamic_block_quant_i8_tiling.cpp][LINE:78]\n The params check failed.[FUNC:Tiling4DynamicBlockQuant][FILE:dynamic_block_quant_tiling.cpp][LINE:564]\n Failed to execute operator DynamicBlockQuant_0. Reason: Failed to execute tiling function.\n Check NnopbaseExecutorDoTiling(executor) failed\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" + } + }, + { + "trial": "c14-quant-prerequisites-358b9b28", + "cleanup_passed": true, + "source_unchanged": true, + "helpers_unchanged": true, + "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 20.143024684977718, + "native_prerequisite": { + "verdict": "FAIL", + "scope": "TorchNPU FP8 prerequisites only", + "phase": "dynamic_block_quant_query", + "versions": { + "torch": "2.10.0+cpu", + "torch_npu": "2.10.0.post4", + "mindiesd": "3.1.0" + }, + "apis": { + "float8_e4m3fn": true, + "npu_dynamic_block_quant": true, + "npu_fused_infer_attention_score_v2": true + }, + "device_name": "Ascend910_9392", + "input": { + "shape": [ + 1, + 14, + 256, + 128 + ], + "dtype": "torch.bfloat16", + "visible_devices": "2" + }, + "error_type": "RuntimeError", + "probe_sha256": "df6465956e6f73bbca140fb9880057e30869fd4b7976b2dc091f71a575dd1096", + "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 161002\n[ERROR] 2026-11-02-01:45:24 (PID:2491012, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2491012] 2026-11-02-01:45:24.732.576 Execution_Error(EZ1009): Failed to execute operator DynamicBlockQuant_0. Reason: The dtype or format of the actual input or output parameter of the operator is inconsistent with that defined in the operator prototype OpDef, details of parameter are x(DT_BFLOAT16, ND), yOut(DT_FLOAT8_E4M3FN, ND), scaleOut(DT_FLOAT, ND).\nTraceBack (most recent call last):\n Supported dtype and format list is:\n SupportInfo[0]: x(DT_FLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n SupportInfo[1]: x(DT_BFLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" + } + }, + { + "trial": "c15-video-row-native-066a9a40", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "e35b551b8fc0ec3b0ba1384b4df96a8daa0e5179c5ed39d641125310b2ee957f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "6f890fcf84047bc5bc084f7b756a2b02e72573888a3bcc12ba7d0211e1bb62ef", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "d2a5bed10620a337ff9aaf99cf4a3740d94a8962ef903bf2da75d65e3b50e21c" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 82.27795292000519 + }, + { + "trial": "c15-video-row-native-3a9721a6", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "8ec1b3ae4cd252f5d95ab23e0232b983900939e8cb15d1e64f4740f7204abf39", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 81.91829407500336 + }, + { + "trial": "c15-video-row-native-a2a224de", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "875d0e70621fc76906be130519d7bb1242de5dd58159e31025e26da37e60d98c" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 82.86056435096543 + }, + { + "trial": "c15-video-row-native-bb5084cc", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "fa1ac8fd375d754503b3507fae006381138d8875afdad2418f1ea5ae338b9c26" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 81.50971999001922 + }, + { + "trial": "c15-video-row-quality-5s-00-710315e0", + "model_controller": { + "status": "failed", + "returncode": 1, + "elapsed_seconds": 153.44603093498154 + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": {}, + "request-000\\rank-1\\attention_path_counts.json": {}, + "request-000\\rank-2\\attention_path_counts.json": {}, + "request-000\\rank-3\\attention_path_counts.json": {} + } + }, + { + "trial": "c15-video-row-v2-quality-5s-00-d6b921b9", + "model_controller": { + "status": "failed", + "returncode": 1, + "elapsed_seconds": 199.99670764501207 + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 2475, + "cache_bytes": 4019635200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 0, + "cache_bytes": 5330304000, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 251 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 251 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 251 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 251 + } + } + }, + { + "trial": "c15-video-row-v3-quality-5s-00-70703586", + "model_controller": { + "status": "failed", + "returncode": 1, + "elapsed_seconds": 191.09544755500974 + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 123750, + "cache_bytes": 4019635200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 0, + "cache_bytes": 5330304000, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 300 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 300 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 300 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 300 + } + } + }, + { + "trial": "c16-video-row-fp32-native-d3fe320c", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "c555d76f7bd6457a81238fca56100d11cb7ab043a4a649d55e23b6203d09bde2", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 78.8189791989862 + }, + { + "trial": "c16-video-row-fp32-quality-5s-00-8600757a", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 355.51310362201184 + }, + "model_diagnostic": { + "source_head": "e0a3462fcc44de698b5dedcf8ff2c99f9802f757", + "samples_ms": [ + 197716.36760496767 + ], + "latency_median_ms": 197716.36760496767, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 4083750, + "peak_cache_bytes": 8039270400, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10700390400, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10700390400, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10660608000, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c17-video-window-native-9b48cb56", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" + }, + "scope": "CPU window reference/routing and native batched BF16 FAS; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.878022204036824 + }, + { + "trial": "c17-video-window-operator-9bf7de3e", + "cleanup_passed": true, + "source_unchanged": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" + }, + "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.778123575029895, + "operator_benchmark": { + "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", + "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", + "fixture_tensor_sha256": { + "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", + "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", + "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" + }, + "query_shape": [ + 1, + 39808, + 14, + 128 + ], + "valid_kv": 39771, + "frames": 37, + "rows_per_frame": 1008, + "target_start": 2475, + "window": 3, + "finite": true, + "relative_l2_vs_dense": 0.5161040425300598, + "baseline_samples_ms": [ + 69.0619180095382, + 69.60304302629083, + 69.27597103640437, + 69.81903495034203, + 69.96654602698982 + ], + "candidate_samples_ms": [ + 25.191069988068193, + 25.301200977992266, + 25.325032009277493, + 25.26562102138996, + 25.219489994924515 + ], + "baseline_median_ms": 69.60304302629083, + "candidate_median_ms": 25.26562102138996, + "reduction_pct": 63.70040744936038, + "operator_gate_passed": true, + "warmups_per_operation": 2, + "measured_per_operation": 5, + "order": "AB/BA/AB/BA/AB", + "probe_sha256": "2465cf08173950a241ac3cc4713a4227d4c4b5a87f35358692a7ef6b60821824" + } + }, + { + "trial": "c17-video-window-quality-5s-00-a8c689e4", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 358.6752056070254 + }, + "model_diagnostic": { + "source_head": "c1589a2b5e4ada617e7cca38ef7c300fe9495c58", + "samples_ms": [ + 210397.12271699682 + ], + "latency_median_ms": 210397.12271699682, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + } + } + } + ], + "decisions": [ + "C12 fixed profile stopped: zero actual predictions.", + "C14 pinned block-FP8 and block-INT8 prerequisites failed; per-token INT8 is not excluded.", + "C13 numerical qualification and clean performance are separate.", + "C15 fixed FP16 video residual profile failed finite velocity at first reuse.", + "C16 FP32 residual completed but failed three frozen audio quality guards; fixed profile stopped.", + "C17 operator reduction is separate from full-model quality and complete HTTP acceptance." + ] +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json new file mode 100644 index 000000000..02c18c9cc --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c19-execution-checkpoint.json @@ -0,0 +1,1018 @@ +{ + "campaign_verdict": "INCONCLUSIVE", + "performance_acceptance": "No newly accepted gain", + "runs": [ + { + "trial": "c12-alignment-quality-5s-00-9bf5c373", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 471.80311120999977 + }, + "model_diagnostic": { + "source_head": "a29648e5acd3077b7b22739bc9522017782fe1ac", + "samples_ms": [ + 318280.59680899605 + ], + "latency_median_ms": 318280.59680899605, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-1\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-2\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-3\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c12-state-alignment-native-tests-0dc5859b", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/cache_state_alignment.py": "3340ec21a77aef78531c601be241bcfb1e44107c795e8c8649907d1a18ed377f", + "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "f4cd0e9aa9e824c92ec1acd02c57a8f7d8684c35d4edf92ffdfae3345ff44e99", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "436ec52cd6329cbc64e91d94b450bfa6f773a9a953b8f974db73a5eb986a98bb", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "4dec8f02a04bfe7e8d2ada24fc6d5ae40a0058671d8515c8a266ecf88824b7b0", + "tests/diffusion/models/minimax_h3/test_minimax_h3_state_cache.py": "89cf57482b35d8d8ad8d4978aec87e9ee3ef45330e5b911161adef4379950a65" + }, + "scope": "focused tests only; no full-model quality or performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 82.06077561300481 + }, + { + "trial": "c13-baseline-cuda-staging-e22d8651", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "vllm_omni/diffusion/offloader/sequential_backend.py": "d542935b656a0d9dced78de8fb086b8e1c5aade26fac6cee5316b32a49f0d569", + "tests/diffusion/offloader/test_sequential_backend.py": "dcf1d64e65bed572d65b095c1350950f55d7e8d9ddf8c7dfd72d0ea6e721685a" + }, + "scope": "Reproduce stock CUDA-graph staging expectation on unchanged migration owning source in the same Ascend test environment", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 50.234599059971515 + }, + { + "trial": "c13-c1-dlo-5s-audit-cead10d1", + "model_controller": { + "status": "failed", + "returncode": 2, + "elapsed_seconds": 0.11969929095357656 + } + }, + { + "trial": "c13-c1-dlo-5s-audit-v2-b0b86c6d", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 749.505351218977 + }, + "model_diagnostic": { + "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "samples_ms": [ + 315593.86175096733, + 284456.94282796467 + ], + "latency_median_ms": 300025.402289466, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 2, + "kind": "diagnostic", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c13-c1-dlo-5s-calibration-c98129d8", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 768.8735517460154 + }, + "model_diagnostic": { + "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "samples_ms": [ + 317177.1984040388, + 285534.089674009 + ], + "latency_median_ms": 301355.6440390239, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 2, + "kind": "calibration", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c13-dlo-gate-tests-bc92a71d", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", + "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" + }, + "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 47.3752507569734 + }, + { + "trial": "c13-dlo-regression-tests-95e9ed2a", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", + "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" + }, + "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 62.34616496198578 + }, + { + "trial": "c13-dlo-regression-v2-12c217d5", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", + "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" + }, + "scope": "Expanded offloader tests in Ascend environment; some stock accelerator fixtures use NPU, not a full model test", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 65.14432259398745 + }, + { + "trial": "c14-int8-prerequisites-fdd6633b", + "cleanup_passed": true, + "source_unchanged": true, + "helpers_unchanged": true, + "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 20.795623040001374, + "native_prerequisite": { + "verdict": "FAIL", + "scope": "TorchNPU block-INT8 attention prerequisites only", + "phase": "dynamic_block_int8_quant_query", + "versions": { + "torch": "2.10.0+cpu", + "torch_npu": "2.10.0.post4", + "mindiesd": "3.1.0" + }, + "apis": { + "float8_e4m3fn": true, + "npu_dynamic_block_quant": true, + "npu_fused_infer_attention_score_v2": true + }, + "device_name": "Ascend910_9392", + "input": { + "shape": [ + 1, + 14, + 256, + 128 + ], + "dtype": "torch.bfloat16", + "visible_devices": "2" + }, + "error_type": "RuntimeError", + "probe_sha256": "fab18252f860231f9b7fcaa54587afbaa96fce15987fd551922246c8964f7217", + "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 561002\n[ERROR] 2026-11-02-02:00:57 (PID:2662830, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2662830] 2026-11-02-02:00:57.039.986 Invalid_Argument(EZ0024): Parameter row_block_size of DynamicBlockQuant has incorrect value 128. It should be 1.\n Solution: Check whether the parameter value is correct.\nTraceBack (most recent call last):\n The attr get failed.[FUNC:CheckParams][FILE:dynamic_block_quant_i8_tiling.cpp][LINE:78]\n The params check failed.[FUNC:Tiling4DynamicBlockQuant][FILE:dynamic_block_quant_tiling.cpp][LINE:564]\n Failed to execute operator DynamicBlockQuant_0. Reason: Failed to execute tiling function.\n Check NnopbaseExecutorDoTiling(executor) failed\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" + } + }, + { + "trial": "c14-quant-prerequisites-358b9b28", + "cleanup_passed": true, + "source_unchanged": true, + "helpers_unchanged": true, + "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 20.143024684977718, + "native_prerequisite": { + "verdict": "FAIL", + "scope": "TorchNPU FP8 prerequisites only", + "phase": "dynamic_block_quant_query", + "versions": { + "torch": "2.10.0+cpu", + "torch_npu": "2.10.0.post4", + "mindiesd": "3.1.0" + }, + "apis": { + "float8_e4m3fn": true, + "npu_dynamic_block_quant": true, + "npu_fused_infer_attention_score_v2": true + }, + "device_name": "Ascend910_9392", + "input": { + "shape": [ + 1, + 14, + 256, + 128 + ], + "dtype": "torch.bfloat16", + "visible_devices": "2" + }, + "error_type": "RuntimeError", + "probe_sha256": "df6465956e6f73bbca140fb9880057e30869fd4b7976b2dc091f71a575dd1096", + "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 161002\n[ERROR] 2026-11-02-01:45:24 (PID:2491012, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2491012] 2026-11-02-01:45:24.732.576 Execution_Error(EZ1009): Failed to execute operator DynamicBlockQuant_0. Reason: The dtype or format of the actual input or output parameter of the operator is inconsistent with that defined in the operator prototype OpDef, details of parameter are x(DT_BFLOAT16, ND), yOut(DT_FLOAT8_E4M3FN, ND), scaleOut(DT_FLOAT, ND).\nTraceBack (most recent call last):\n Supported dtype and format list is:\n SupportInfo[0]: x(DT_FLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n SupportInfo[1]: x(DT_BFLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" + } + }, + { + "trial": "c15-video-row-native-066a9a40", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "e35b551b8fc0ec3b0ba1384b4df96a8daa0e5179c5ed39d641125310b2ee957f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "6f890fcf84047bc5bc084f7b756a2b02e72573888a3bcc12ba7d0211e1bb62ef", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "d2a5bed10620a337ff9aaf99cf4a3740d94a8962ef903bf2da75d65e3b50e21c" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 82.27795292000519 + }, + { + "trial": "c15-video-row-native-3a9721a6", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "8ec1b3ae4cd252f5d95ab23e0232b983900939e8cb15d1e64f4740f7204abf39", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 81.91829407500336 + }, + { + "trial": "c15-video-row-native-a2a224de", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "875d0e70621fc76906be130519d7bb1242de5dd58159e31025e26da37e60d98c" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 82.86056435096543 + }, + { + "trial": "c15-video-row-native-bb5084cc", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "fa1ac8fd375d754503b3507fae006381138d8875afdad2418f1ea5ae338b9c26" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 81.50971999001922 + }, + { + "trial": "c15-video-row-quality-5s-00-710315e0", + "model_controller": { + "status": "failed", + "returncode": 1, + "elapsed_seconds": 153.44603093498154 + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": {}, + "request-000\\rank-1\\attention_path_counts.json": {}, + "request-000\\rank-2\\attention_path_counts.json": {}, + "request-000\\rank-3\\attention_path_counts.json": {} + } + }, + { + "trial": "c15-video-row-v2-quality-5s-00-d6b921b9", + "model_controller": { + "status": "failed", + "returncode": 1, + "elapsed_seconds": 199.99670764501207 + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 2475, + "cache_bytes": 4019635200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 0, + "cache_bytes": 5330304000, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 251 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 251 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 251 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 251 + } + } + }, + { + "trial": "c15-video-row-v3-quality-5s-00-70703586", + "model_controller": { + "status": "failed", + "returncode": 1, + "elapsed_seconds": 191.09544755500974 + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 123750, + "cache_bytes": 4019635200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 0, + "cache_bytes": 5330304000, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 300 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 300 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 300 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 300 + } + } + }, + { + "trial": "c16-video-row-fp32-native-d3fe320c", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "c555d76f7bd6457a81238fca56100d11cb7ab043a4a649d55e23b6203d09bde2", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 78.8189791989862 + }, + { + "trial": "c16-video-row-fp32-quality-5s-00-8600757a", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 355.51310362201184 + }, + "model_diagnostic": { + "source_head": "e0a3462fcc44de698b5dedcf8ff2c99f9802f757", + "samples_ms": [ + 197716.36760496767 + ], + "latency_median_ms": 197716.36760496767, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 4083750, + "peak_cache_bytes": 8039270400, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10700390400, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10700390400, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10660608000, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c17-video-window-native-9b48cb56", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" + }, + "scope": "CPU window reference/routing and native batched BF16 FAS; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.878022204036824 + }, + { + "trial": "c17-video-window-operator-9bf7de3e", + "cleanup_passed": true, + "source_unchanged": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" + }, + "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.778123575029895, + "operator_benchmark": { + "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", + "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", + "fixture_tensor_sha256": { + "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", + "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", + "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" + }, + "query_shape": [ + 1, + 39808, + 14, + 128 + ], + "valid_kv": 39771, + "frames": 37, + "rows_per_frame": 1008, + "target_start": 2475, + "window": 3, + "finite": true, + "relative_l2_vs_dense": 0.5161040425300598, + "baseline_samples_ms": [ + 69.0619180095382, + 69.60304302629083, + 69.27597103640437, + 69.81903495034203, + 69.96654602698982 + ], + "candidate_samples_ms": [ + 25.191069988068193, + 25.301200977992266, + 25.325032009277493, + 25.26562102138996, + 25.219489994924515 + ], + "baseline_median_ms": 69.60304302629083, + "candidate_median_ms": 25.26562102138996, + "reduction_pct": 63.70040744936038, + "operator_gate_passed": true, + "warmups_per_operation": 2, + "measured_per_operation": 5, + "order": "AB/BA/AB/BA/AB", + "probe_sha256": "2465cf08173950a241ac3cc4713a4227d4c4b5a87f35358692a7ef6b60821824" + } + }, + { + "trial": "c17-video-window-quality-5s-00-a8c689e4", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 358.6752056070254 + }, + "model_diagnostic": { + "source_head": "c1589a2b5e4ada617e7cca38ef7c300fe9495c58", + "samples_ms": [ + 210397.12271699682 + ], + "latency_median_ms": 210397.12271699682, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + } + } + }, + { + "trial": "c18-video-pooled-context-native-1673db72", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" + }, + "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 49.97671151399845 + }, + { + "trial": "c18-video-pooled-context-operator-315db0e5", + "cleanup_passed": true, + "source_unchanged": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" + }, + "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.37963652302278, + "operator_benchmark": { + "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", + "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", + "fixture_tensor_sha256": { + "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", + "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", + "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" + }, + "query_shape": [ + 1, + 39808, + 14, + 128 + ], + "valid_kv": 39771, + "frames": 37, + "rows_per_frame": 1008, + "target_start": 2475, + "profile": "3x3-local1", + "finite": true, + "relative_l2_vs_dense": 0.21801254153251648, + "baseline_samples_ms": [ + 69.19049099087715, + 69.70348599134013, + 69.43478400353342, + 69.81050700414926, + 69.24652203451842 + ], + "candidate_samples_ms": [ + 68.29233502503484, + 64.26435202592984, + 71.05128600960597, + 64.39994304673746, + 64.39412303734571 + ], + "baseline_median_ms": 69.43478400353342, + "candidate_median_ms": 64.39994304673746, + "reduction_pct": 7.251179691924648, + "operator_gate_passed": false, + "warmups_per_operation": 2, + "measured_per_operation": 5, + "order": "AB/BA/AB/BA/AB", + "probe_sha256": "42852c0e4e693c5b3685981db617de00648feefe651f7060ad83c359b08dd81d" + } + }, + { + "trial": "c19-video-global-pool-native-f0f4c502", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ddfd6d6822cc6a5528fc95539f7f98dfdeed25d5489da3ed5cf76e6076a5ec94", + "tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py": "5457db550b59e3373be8f5f21873c1b78ac357a6ac170c2603ee531b50e91467" + }, + "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.528875903983135 + } + ], + "decisions": [ + "C12 fixed profile stopped: zero actual predictions.", + "C14 pinned block-FP8 and block-INT8 prerequisites failed; per-token INT8 is not excluded.", + "C13 numerical qualification and clean performance are separate.", + "C15 fixed FP16 video residual profile failed finite velocity at first reuse.", + "C16 FP32 residual completed but failed three frozen audio quality guards; fixed profile stopped.", + "C17 completed5s but failed11frozen quality checks; fixed window profile stopped.", + "C18 native17PASS but operation reduced only7.25%; no full model run.", + "C19 shares global pooledKV; native/operator/full model qualification are separate." + ] +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json new file mode 100644 index 000000000..cdaaac095 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c20-execution-checkpoint.json @@ -0,0 +1,1147 @@ +{ + "campaign_verdict": "INCONCLUSIVE", + "performance_acceptance": "No newly accepted gain", + "runs": [ + { + "trial": "c12-alignment-quality-5s-00-9bf5c373", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 471.80311120999977 + }, + "model_diagnostic": { + "source_head": "a29648e5acd3077b7b22739bc9522017782fe1ac", + "samples_ms": [ + 318280.59680899605 + ], + "latency_median_ms": 318280.59680899605, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-1\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-2\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-3\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c12-state-alignment-native-tests-0dc5859b", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/cache_state_alignment.py": "3340ec21a77aef78531c601be241bcfb1e44107c795e8c8649907d1a18ed377f", + "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "f4cd0e9aa9e824c92ec1acd02c57a8f7d8684c35d4edf92ffdfae3345ff44e99", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "436ec52cd6329cbc64e91d94b450bfa6f773a9a953b8f974db73a5eb986a98bb", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "4dec8f02a04bfe7e8d2ada24fc6d5ae40a0058671d8515c8a266ecf88824b7b0", + "tests/diffusion/models/minimax_h3/test_minimax_h3_state_cache.py": "89cf57482b35d8d8ad8d4978aec87e9ee3ef45330e5b911161adef4379950a65" + }, + "scope": "focused tests only; no full-model quality or performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 82.06077561300481 + }, + { + "trial": "c13-baseline-cuda-staging-e22d8651", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "vllm_omni/diffusion/offloader/sequential_backend.py": "d542935b656a0d9dced78de8fb086b8e1c5aade26fac6cee5316b32a49f0d569", + "tests/diffusion/offloader/test_sequential_backend.py": "dcf1d64e65bed572d65b095c1350950f55d7e8d9ddf8c7dfd72d0ea6e721685a" + }, + "scope": "Reproduce stock CUDA-graph staging expectation on unchanged migration owning source in the same Ascend test environment", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 50.234599059971515 + }, + { + "trial": "c13-c1-dlo-5s-audit-cead10d1", + "model_controller": { + "status": "failed", + "returncode": 2, + "elapsed_seconds": 0.11969929095357656 + } + }, + { + "trial": "c13-c1-dlo-5s-audit-v2-b0b86c6d", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 749.505351218977 + }, + "model_diagnostic": { + "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "samples_ms": [ + 315593.86175096733, + 284456.94282796467 + ], + "latency_median_ms": 300025.402289466, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 2, + "kind": "diagnostic", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c13-c1-dlo-5s-calibration-c98129d8", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 768.8735517460154 + }, + "model_diagnostic": { + "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "samples_ms": [ + 317177.1984040388, + 285534.089674009 + ], + "latency_median_ms": 301355.6440390239, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 2, + "kind": "calibration", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c13-dlo-gate-tests-bc92a71d", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", + "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" + }, + "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 47.3752507569734 + }, + { + "trial": "c13-dlo-regression-tests-95e9ed2a", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", + "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" + }, + "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 62.34616496198578 + }, + { + "trial": "c13-dlo-regression-v2-12c217d5", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", + "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" + }, + "scope": "Expanded offloader tests in Ascend environment; some stock accelerator fixtures use NPU, not a full model test", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 65.14432259398745 + }, + { + "trial": "c14-int8-prerequisites-fdd6633b", + "cleanup_passed": true, + "source_unchanged": true, + "helpers_unchanged": true, + "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 20.795623040001374, + "native_prerequisite": { + "verdict": "FAIL", + "scope": "TorchNPU block-INT8 attention prerequisites only", + "phase": "dynamic_block_int8_quant_query", + "versions": { + "torch": "2.10.0+cpu", + "torch_npu": "2.10.0.post4", + "mindiesd": "3.1.0" + }, + "apis": { + "float8_e4m3fn": true, + "npu_dynamic_block_quant": true, + "npu_fused_infer_attention_score_v2": true + }, + "device_name": "Ascend910_9392", + "input": { + "shape": [ + 1, + 14, + 256, + 128 + ], + "dtype": "torch.bfloat16", + "visible_devices": "2" + }, + "error_type": "RuntimeError", + "probe_sha256": "fab18252f860231f9b7fcaa54587afbaa96fce15987fd551922246c8964f7217", + "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 561002\n[ERROR] 2026-11-02-02:00:57 (PID:2662830, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2662830] 2026-11-02-02:00:57.039.986 Invalid_Argument(EZ0024): Parameter row_block_size of DynamicBlockQuant has incorrect value 128. It should be 1.\n Solution: Check whether the parameter value is correct.\nTraceBack (most recent call last):\n The attr get failed.[FUNC:CheckParams][FILE:dynamic_block_quant_i8_tiling.cpp][LINE:78]\n The params check failed.[FUNC:Tiling4DynamicBlockQuant][FILE:dynamic_block_quant_tiling.cpp][LINE:564]\n Failed to execute operator DynamicBlockQuant_0. Reason: Failed to execute tiling function.\n Check NnopbaseExecutorDoTiling(executor) failed\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" + } + }, + { + "trial": "c14-quant-prerequisites-358b9b28", + "cleanup_passed": true, + "source_unchanged": true, + "helpers_unchanged": true, + "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 20.143024684977718, + "native_prerequisite": { + "verdict": "FAIL", + "scope": "TorchNPU FP8 prerequisites only", + "phase": "dynamic_block_quant_query", + "versions": { + "torch": "2.10.0+cpu", + "torch_npu": "2.10.0.post4", + "mindiesd": "3.1.0" + }, + "apis": { + "float8_e4m3fn": true, + "npu_dynamic_block_quant": true, + "npu_fused_infer_attention_score_v2": true + }, + "device_name": "Ascend910_9392", + "input": { + "shape": [ + 1, + 14, + 256, + 128 + ], + "dtype": "torch.bfloat16", + "visible_devices": "2" + }, + "error_type": "RuntimeError", + "probe_sha256": "df6465956e6f73bbca140fb9880057e30869fd4b7976b2dc091f71a575dd1096", + "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 161002\n[ERROR] 2026-11-02-01:45:24 (PID:2491012, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2491012] 2026-11-02-01:45:24.732.576 Execution_Error(EZ1009): Failed to execute operator DynamicBlockQuant_0. Reason: The dtype or format of the actual input or output parameter of the operator is inconsistent with that defined in the operator prototype OpDef, details of parameter are x(DT_BFLOAT16, ND), yOut(DT_FLOAT8_E4M3FN, ND), scaleOut(DT_FLOAT, ND).\nTraceBack (most recent call last):\n Supported dtype and format list is:\n SupportInfo[0]: x(DT_FLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n SupportInfo[1]: x(DT_BFLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" + } + }, + { + "trial": "c15-video-row-native-066a9a40", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "e35b551b8fc0ec3b0ba1384b4df96a8daa0e5179c5ed39d641125310b2ee957f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "6f890fcf84047bc5bc084f7b756a2b02e72573888a3bcc12ba7d0211e1bb62ef", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "d2a5bed10620a337ff9aaf99cf4a3740d94a8962ef903bf2da75d65e3b50e21c" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 82.27795292000519 + }, + { + "trial": "c15-video-row-native-3a9721a6", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "8ec1b3ae4cd252f5d95ab23e0232b983900939e8cb15d1e64f4740f7204abf39", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 81.91829407500336 + }, + { + "trial": "c15-video-row-native-a2a224de", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "875d0e70621fc76906be130519d7bb1242de5dd58159e31025e26da37e60d98c" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 82.86056435096543 + }, + { + "trial": "c15-video-row-native-bb5084cc", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "fa1ac8fd375d754503b3507fae006381138d8875afdad2418f1ea5ae338b9c26" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 81.50971999001922 + }, + { + "trial": "c15-video-row-quality-5s-00-710315e0", + "model_controller": { + "status": "failed", + "returncode": 1, + "elapsed_seconds": 153.44603093498154 + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": {}, + "request-000\\rank-1\\attention_path_counts.json": {}, + "request-000\\rank-2\\attention_path_counts.json": {}, + "request-000\\rank-3\\attention_path_counts.json": {} + } + }, + { + "trial": "c15-video-row-v2-quality-5s-00-d6b921b9", + "model_controller": { + "status": "failed", + "returncode": 1, + "elapsed_seconds": 199.99670764501207 + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 2475, + "cache_bytes": 4019635200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 0, + "cache_bytes": 5330304000, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 251 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 251 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 251 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 251 + } + } + }, + { + "trial": "c15-video-row-v3-quality-5s-00-70703586", + "model_controller": { + "status": "failed", + "returncode": 1, + "elapsed_seconds": 191.09544755500974 + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 123750, + "cache_bytes": 4019635200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 0, + "cache_bytes": 5330304000, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 300 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 300 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 300 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 300 + } + } + }, + { + "trial": "c16-video-row-fp32-native-d3fe320c", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "c555d76f7bd6457a81238fca56100d11cb7ab043a4a649d55e23b6203d09bde2", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 78.8189791989862 + }, + { + "trial": "c16-video-row-fp32-quality-5s-00-8600757a", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 355.51310362201184 + }, + "model_diagnostic": { + "source_head": "e0a3462fcc44de698b5dedcf8ff2c99f9802f757", + "samples_ms": [ + 197716.36760496767 + ], + "latency_median_ms": 197716.36760496767, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 4083750, + "peak_cache_bytes": 8039270400, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10700390400, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10700390400, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10660608000, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c17-video-window-native-9b48cb56", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" + }, + "scope": "CPU window reference/routing and native batched BF16 FAS; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.878022204036824 + }, + { + "trial": "c17-video-window-operator-9bf7de3e", + "cleanup_passed": true, + "source_unchanged": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" + }, + "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.778123575029895, + "operator_benchmark": { + "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", + "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", + "fixture_tensor_sha256": { + "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", + "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", + "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" + }, + "query_shape": [ + 1, + 39808, + 14, + 128 + ], + "valid_kv": 39771, + "frames": 37, + "rows_per_frame": 1008, + "target_start": 2475, + "window": 3, + "finite": true, + "relative_l2_vs_dense": 0.5161040425300598, + "baseline_samples_ms": [ + 69.0619180095382, + 69.60304302629083, + 69.27597103640437, + 69.81903495034203, + 69.96654602698982 + ], + "candidate_samples_ms": [ + 25.191069988068193, + 25.301200977992266, + 25.325032009277493, + 25.26562102138996, + 25.219489994924515 + ], + "baseline_median_ms": 69.60304302629083, + "candidate_median_ms": 25.26562102138996, + "reduction_pct": 63.70040744936038, + "operator_gate_passed": true, + "warmups_per_operation": 2, + "measured_per_operation": 5, + "order": "AB/BA/AB/BA/AB", + "probe_sha256": "2465cf08173950a241ac3cc4713a4227d4c4b5a87f35358692a7ef6b60821824" + } + }, + { + "trial": "c17-video-window-quality-5s-00-a8c689e4", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 358.6752056070254 + }, + "model_diagnostic": { + "source_head": "c1589a2b5e4ada617e7cca38ef7c300fe9495c58", + "samples_ms": [ + 210397.12271699682 + ], + "latency_median_ms": 210397.12271699682, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + } + } + }, + { + "trial": "c18-video-pooled-context-native-1673db72", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" + }, + "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 49.97671151399845 + }, + { + "trial": "c18-video-pooled-context-operator-315db0e5", + "cleanup_passed": true, + "source_unchanged": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" + }, + "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.37963652302278, + "operator_benchmark": { + "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", + "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", + "fixture_tensor_sha256": { + "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", + "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", + "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" + }, + "query_shape": [ + 1, + 39808, + 14, + 128 + ], + "valid_kv": 39771, + "frames": 37, + "rows_per_frame": 1008, + "target_start": 2475, + "profile": "3x3-local1", + "finite": true, + "relative_l2_vs_dense": 0.21801254153251648, + "baseline_samples_ms": [ + 69.19049099087715, + 69.70348599134013, + 69.43478400353342, + 69.81050700414926, + 69.24652203451842 + ], + "candidate_samples_ms": [ + 68.29233502503484, + 64.26435202592984, + 71.05128600960597, + 64.39994304673746, + 64.39412303734571 + ], + "baseline_median_ms": 69.43478400353342, + "candidate_median_ms": 64.39994304673746, + "reduction_pct": 7.251179691924648, + "operator_gate_passed": false, + "warmups_per_operation": 2, + "measured_per_operation": 5, + "order": "AB/BA/AB/BA/AB", + "probe_sha256": "42852c0e4e693c5b3685981db617de00648feefe651f7060ad83c359b08dd81d" + } + }, + { + "trial": "c19-video-global-pool-native-f0f4c502", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ddfd6d6822cc6a5528fc95539f7f98dfdeed25d5489da3ed5cf76e6076a5ec94", + "tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py": "5457db550b59e3373be8f5f21873c1b78ac357a6ac170c2603ee531b50e91467" + }, + "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.528875903983135 + }, + { + "trial": "c19-video-global-pool-operator-933511c9", + "cleanup_passed": true, + "source_unchanged": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ddfd6d6822cc6a5528fc95539f7f98dfdeed25d5489da3ed5cf76e6076a5ec94", + "tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py": "5457db550b59e3373be8f5f21873c1b78ac357a6ac170c2603ee531b50e91467" + }, + "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.02866222098237, + "operator_benchmark": { + "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", + "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", + "fixture_tensor_sha256": { + "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", + "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", + "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" + }, + "query_shape": [ + 1, + 39808, + 14, + 128 + ], + "valid_kv": 39771, + "frames": 37, + "rows_per_frame": 1008, + "target_start": 2475, + "profile": "3x3-global", + "finite": true, + "relative_l2_vs_dense": 0.2588050067424774, + "baseline_samples_ms": [ + 69.95450798422098, + 69.79658600175753, + 69.58012498216704, + 70.22821099963039, + 69.11802099784836 + ], + "candidate_samples_ms": [ + 32.91054297005758, + 32.899541954975575, + 32.924533006735146, + 32.779561006464064, + 32.96928299823776 + ], + "baseline_median_ms": 69.79658600175753, + "candidate_median_ms": 32.91054297005758, + "reduction_pct": 52.847918708761966, + "operator_gate_passed": false, + "warmups_per_operation": 2, + "measured_per_operation": 5, + "order": "AB/BA/AB/BA/AB", + "probe_sha256": "a27906d8bb0b65e2f6ffb41ddeb4b513163b7bf23b9d2dbd33a5015fc93b2a08" + } + }, + { + "trial": "c20-video-variance-pool-native-50101063", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "4390450bc5324f7028125101cd4bb32e9de237b04bc12fa879b48110d7122e3c", + "tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py": "e871fb4b222d61d10b6c442a333a52c976cceaa0d406233502ecbcaddfeba8c9" + }, + "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 50.230675689992495 + }, + { + "trial": "c20-video-variance-pool-operator-209f263d", + "cleanup_passed": true, + "source_unchanged": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "4390450bc5324f7028125101cd4bb32e9de237b04bc12fa879b48110d7122e3c", + "tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py": "e871fb4b222d61d10b6c442a333a52c976cceaa0d406233502ecbcaddfeba8c9" + }, + "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 49.880220040970016, + "operator_benchmark": { + "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", + "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", + "fixture_tensor_sha256": { + "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", + "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", + "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" + }, + "query_shape": [ + 1, + 39808, + 14, + 128 + ], + "valid_kv": 39771, + "frames": 37, + "rows_per_frame": 1008, + "target_start": 2475, + "profile": "3x3-variance", + "finite": true, + "relative_l2_vs_dense": 0.26336145401000977, + "baseline_samples_ms": [ + 69.74446203093976, + 69.06059599714354, + 69.64961998164654, + 69.69038100214675, + 69.62426099926233 + ], + "candidate_samples_ms": [ + 35.81449302146211, + 36.30960697773844, + 35.925014002714306, + 35.99672397831455, + 35.80518299713731 + ], + "baseline_median_ms": 69.64961998164654, + "candidate_median_ms": 35.925014002714306, + "reduction_pct": 48.42037327385138, + "operator_gate_passed": false, + "warmups_per_operation": 2, + "measured_per_operation": 5, + "order": "AB/BA/AB/BA/AB", + "probe_sha256": "785e7a0c13719e37d5cc44fb66af05649e51e1f7e5d0b212e92ec44f79cb0184" + } + } + ], + "decisions": [ + "C12 fixed profile stopped: zero actual predictions.", + "C14 pinned block-FP8 and block-INT8 prerequisites failed; per-token INT8 is not excluded.", + "C13 numerical qualification and clean performance are separate.", + "C15 fixed FP16 video residual profile failed finite velocity at first reuse.", + "C16 FP32 residual completed but failed three frozen audio quality guards; fixed profile stopped.", + "C17 completed5s but failed11frozen quality checks; fixed window profile stopped.", + "C18 native17PASS but operation reduced only7.25%; no full model run.", + "C19 native17PASS; operation52.85%faster but relL2 .258805 exceeded fixed .258052screen; no model run.", + "C20 native17PASS; operation48.42%faster and relL2 .263361 failed both fixed primitive gates; no model run." + ] +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json new file mode 100644 index 000000000..353fac2a7 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c21-execution-checkpoint.json @@ -0,0 +1,1310 @@ +{ + "campaign_verdict": "INCONCLUSIVE", + "performance_acceptance": "No newly accepted gain", + "runs": [ + { + "trial": "c12-alignment-quality-5s-00-9bf5c373", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 471.80311120999977 + }, + "model_diagnostic": { + "source_head": "a29648e5acd3077b7b22739bc9522017782fe1ac", + "samples_ms": [ + 318280.59680899605 + ], + "latency_median_ms": 318280.59680899605, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-1\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-2\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-3\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c12-state-alignment-native-tests-0dc5859b", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/cache_state_alignment.py": "3340ec21a77aef78531c601be241bcfb1e44107c795e8c8649907d1a18ed377f", + "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "f4cd0e9aa9e824c92ec1acd02c57a8f7d8684c35d4edf92ffdfae3345ff44e99", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "436ec52cd6329cbc64e91d94b450bfa6f773a9a953b8f974db73a5eb986a98bb", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "4dec8f02a04bfe7e8d2ada24fc6d5ae40a0058671d8515c8a266ecf88824b7b0", + "tests/diffusion/models/minimax_h3/test_minimax_h3_state_cache.py": "89cf57482b35d8d8ad8d4978aec87e9ee3ef45330e5b911161adef4379950a65" + }, + "scope": "focused tests only; no full-model quality or performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 82.06077561300481 + }, + { + "trial": "c13-baseline-cuda-staging-e22d8651", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "vllm_omni/diffusion/offloader/sequential_backend.py": "d542935b656a0d9dced78de8fb086b8e1c5aade26fac6cee5316b32a49f0d569", + "tests/diffusion/offloader/test_sequential_backend.py": "dcf1d64e65bed572d65b095c1350950f55d7e8d9ddf8c7dfd72d0ea6e721685a" + }, + "scope": "Reproduce stock CUDA-graph staging expectation on unchanged migration owning source in the same Ascend test environment", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 50.234599059971515 + }, + { + "trial": "c13-c1-dlo-5s-audit-cead10d1", + "model_controller": { + "status": "failed", + "returncode": 2, + "elapsed_seconds": 0.11969929095357656 + } + }, + { + "trial": "c13-c1-dlo-5s-audit-v2-b0b86c6d", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 749.505351218977 + }, + "model_diagnostic": { + "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "samples_ms": [ + 315593.86175096733, + 284456.94282796467 + ], + "latency_median_ms": 300025.402289466, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 2, + "kind": "diagnostic", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c13-c1-dlo-5s-calibration-c98129d8", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 768.8735517460154 + }, + "model_diagnostic": { + "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "samples_ms": [ + 317177.1984040388, + 285534.089674009 + ], + "latency_median_ms": 301355.6440390239, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 2, + "kind": "calibration", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c13-dlo-gate-tests-bc92a71d", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", + "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" + }, + "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 47.3752507569734 + }, + { + "trial": "c13-dlo-regression-tests-95e9ed2a", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", + "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" + }, + "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 62.34616496198578 + }, + { + "trial": "c13-dlo-regression-v2-12c217d5", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", + "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" + }, + "scope": "Expanded offloader tests in Ascend environment; some stock accelerator fixtures use NPU, not a full model test", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 65.14432259398745 + }, + { + "trial": "c14-int8-prerequisites-fdd6633b", + "cleanup_passed": true, + "source_unchanged": true, + "helpers_unchanged": true, + "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 20.795623040001374, + "native_prerequisite": { + "verdict": "FAIL", + "scope": "TorchNPU block-INT8 attention prerequisites only", + "phase": "dynamic_block_int8_quant_query", + "versions": { + "torch": "2.10.0+cpu", + "torch_npu": "2.10.0.post4", + "mindiesd": "3.1.0" + }, + "apis": { + "float8_e4m3fn": true, + "npu_dynamic_block_quant": true, + "npu_fused_infer_attention_score_v2": true + }, + "device_name": "Ascend910_9392", + "input": { + "shape": [ + 1, + 14, + 256, + 128 + ], + "dtype": "torch.bfloat16", + "visible_devices": "2" + }, + "error_type": "RuntimeError", + "probe_sha256": "fab18252f860231f9b7fcaa54587afbaa96fce15987fd551922246c8964f7217", + "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 561002\n[ERROR] 2026-11-02-02:00:57 (PID:2662830, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2662830] 2026-11-02-02:00:57.039.986 Invalid_Argument(EZ0024): Parameter row_block_size of DynamicBlockQuant has incorrect value 128. It should be 1.\n Solution: Check whether the parameter value is correct.\nTraceBack (most recent call last):\n The attr get failed.[FUNC:CheckParams][FILE:dynamic_block_quant_i8_tiling.cpp][LINE:78]\n The params check failed.[FUNC:Tiling4DynamicBlockQuant][FILE:dynamic_block_quant_tiling.cpp][LINE:564]\n Failed to execute operator DynamicBlockQuant_0. Reason: Failed to execute tiling function.\n Check NnopbaseExecutorDoTiling(executor) failed\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" + } + }, + { + "trial": "c14-quant-prerequisites-358b9b28", + "cleanup_passed": true, + "source_unchanged": true, + "helpers_unchanged": true, + "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 20.143024684977718, + "native_prerequisite": { + "verdict": "FAIL", + "scope": "TorchNPU FP8 prerequisites only", + "phase": "dynamic_block_quant_query", + "versions": { + "torch": "2.10.0+cpu", + "torch_npu": "2.10.0.post4", + "mindiesd": "3.1.0" + }, + "apis": { + "float8_e4m3fn": true, + "npu_dynamic_block_quant": true, + "npu_fused_infer_attention_score_v2": true + }, + "device_name": "Ascend910_9392", + "input": { + "shape": [ + 1, + 14, + 256, + 128 + ], + "dtype": "torch.bfloat16", + "visible_devices": "2" + }, + "error_type": "RuntimeError", + "probe_sha256": "df6465956e6f73bbca140fb9880057e30869fd4b7976b2dc091f71a575dd1096", + "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 161002\n[ERROR] 2026-11-02-01:45:24 (PID:2491012, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2491012] 2026-11-02-01:45:24.732.576 Execution_Error(EZ1009): Failed to execute operator DynamicBlockQuant_0. Reason: The dtype or format of the actual input or output parameter of the operator is inconsistent with that defined in the operator prototype OpDef, details of parameter are x(DT_BFLOAT16, ND), yOut(DT_FLOAT8_E4M3FN, ND), scaleOut(DT_FLOAT, ND).\nTraceBack (most recent call last):\n Supported dtype and format list is:\n SupportInfo[0]: x(DT_FLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n SupportInfo[1]: x(DT_BFLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" + } + }, + { + "trial": "c15-video-row-native-066a9a40", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "e35b551b8fc0ec3b0ba1384b4df96a8daa0e5179c5ed39d641125310b2ee957f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "6f890fcf84047bc5bc084f7b756a2b02e72573888a3bcc12ba7d0211e1bb62ef", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "d2a5bed10620a337ff9aaf99cf4a3740d94a8962ef903bf2da75d65e3b50e21c" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 82.27795292000519 + }, + { + "trial": "c15-video-row-native-3a9721a6", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "8ec1b3ae4cd252f5d95ab23e0232b983900939e8cb15d1e64f4740f7204abf39", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 81.91829407500336 + }, + { + "trial": "c15-video-row-native-a2a224de", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "875d0e70621fc76906be130519d7bb1242de5dd58159e31025e26da37e60d98c" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 82.86056435096543 + }, + { + "trial": "c15-video-row-native-bb5084cc", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "fa1ac8fd375d754503b3507fae006381138d8875afdad2418f1ea5ae338b9c26" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 81.50971999001922 + }, + { + "trial": "c15-video-row-quality-5s-00-710315e0", + "model_controller": { + "status": "failed", + "returncode": 1, + "elapsed_seconds": 153.44603093498154 + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": {}, + "request-000\\rank-1\\attention_path_counts.json": {}, + "request-000\\rank-2\\attention_path_counts.json": {}, + "request-000\\rank-3\\attention_path_counts.json": {} + } + }, + { + "trial": "c15-video-row-v2-quality-5s-00-d6b921b9", + "model_controller": { + "status": "failed", + "returncode": 1, + "elapsed_seconds": 199.99670764501207 + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 2475, + "cache_bytes": 4019635200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 0, + "cache_bytes": 5330304000, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 251 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 251 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 251 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 251 + } + } + }, + { + "trial": "c15-video-row-v3-quality-5s-00-70703586", + "model_controller": { + "status": "failed", + "returncode": 1, + "elapsed_seconds": 191.09544755500974 + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 123750, + "cache_bytes": 4019635200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 0, + "cache_bytes": 5330304000, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 300 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 300 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 300 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 300 + } + } + }, + { + "trial": "c16-video-row-fp32-native-d3fe320c", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "c555d76f7bd6457a81238fca56100d11cb7ab043a4a649d55e23b6203d09bde2", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 78.8189791989862 + }, + { + "trial": "c16-video-row-fp32-quality-5s-00-8600757a", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 355.51310362201184 + }, + "model_diagnostic": { + "source_head": "e0a3462fcc44de698b5dedcf8ff2c99f9802f757", + "samples_ms": [ + 197716.36760496767 + ], + "latency_median_ms": 197716.36760496767, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 4083750, + "peak_cache_bytes": 8039270400, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10700390400, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10700390400, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10660608000, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c17-video-window-native-9b48cb56", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" + }, + "scope": "CPU window reference/routing and native batched BF16 FAS; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.878022204036824 + }, + { + "trial": "c17-video-window-operator-9bf7de3e", + "cleanup_passed": true, + "source_unchanged": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" + }, + "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.778123575029895, + "operator_benchmark": { + "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", + "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", + "fixture_tensor_sha256": { + "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", + "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", + "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" + }, + "query_shape": [ + 1, + 39808, + 14, + 128 + ], + "valid_kv": 39771, + "frames": 37, + "rows_per_frame": 1008, + "target_start": 2475, + "window": 3, + "finite": true, + "relative_l2_vs_dense": 0.5161040425300598, + "baseline_samples_ms": [ + 69.0619180095382, + 69.60304302629083, + 69.27597103640437, + 69.81903495034203, + 69.96654602698982 + ], + "candidate_samples_ms": [ + 25.191069988068193, + 25.301200977992266, + 25.325032009277493, + 25.26562102138996, + 25.219489994924515 + ], + "baseline_median_ms": 69.60304302629083, + "candidate_median_ms": 25.26562102138996, + "reduction_pct": 63.70040744936038, + "operator_gate_passed": true, + "warmups_per_operation": 2, + "measured_per_operation": 5, + "order": "AB/BA/AB/BA/AB", + "probe_sha256": "2465cf08173950a241ac3cc4713a4227d4c4b5a87f35358692a7ef6b60821824" + } + }, + { + "trial": "c17-video-window-quality-5s-00-a8c689e4", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 358.6752056070254 + }, + "model_diagnostic": { + "source_head": "c1589a2b5e4ada617e7cca38ef7c300fe9495c58", + "samples_ms": [ + 210397.12271699682 + ], + "latency_median_ms": 210397.12271699682, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + } + } + }, + { + "trial": "c18-video-pooled-context-native-1673db72", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" + }, + "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 49.97671151399845 + }, + { + "trial": "c18-video-pooled-context-operator-315db0e5", + "cleanup_passed": true, + "source_unchanged": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" + }, + "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.37963652302278, + "operator_benchmark": { + "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", + "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", + "fixture_tensor_sha256": { + "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", + "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", + "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" + }, + "query_shape": [ + 1, + 39808, + 14, + 128 + ], + "valid_kv": 39771, + "frames": 37, + "rows_per_frame": 1008, + "target_start": 2475, + "profile": "3x3-local1", + "finite": true, + "relative_l2_vs_dense": 0.21801254153251648, + "baseline_samples_ms": [ + 69.19049099087715, + 69.70348599134013, + 69.43478400353342, + 69.81050700414926, + 69.24652203451842 + ], + "candidate_samples_ms": [ + 68.29233502503484, + 64.26435202592984, + 71.05128600960597, + 64.39994304673746, + 64.39412303734571 + ], + "baseline_median_ms": 69.43478400353342, + "candidate_median_ms": 64.39994304673746, + "reduction_pct": 7.251179691924648, + "operator_gate_passed": false, + "warmups_per_operation": 2, + "measured_per_operation": 5, + "order": "AB/BA/AB/BA/AB", + "probe_sha256": "42852c0e4e693c5b3685981db617de00648feefe651f7060ad83c359b08dd81d" + } + }, + { + "trial": "c19-video-global-pool-native-f0f4c502", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ddfd6d6822cc6a5528fc95539f7f98dfdeed25d5489da3ed5cf76e6076a5ec94", + "tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py": "5457db550b59e3373be8f5f21873c1b78ac357a6ac170c2603ee531b50e91467" + }, + "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.528875903983135 + }, + { + "trial": "c19-video-global-pool-operator-933511c9", + "cleanup_passed": true, + "source_unchanged": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ddfd6d6822cc6a5528fc95539f7f98dfdeed25d5489da3ed5cf76e6076a5ec94", + "tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py": "5457db550b59e3373be8f5f21873c1b78ac357a6ac170c2603ee531b50e91467" + }, + "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.02866222098237, + "operator_benchmark": { + "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", + "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", + "fixture_tensor_sha256": { + "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", + "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", + "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" + }, + "query_shape": [ + 1, + 39808, + 14, + 128 + ], + "valid_kv": 39771, + "frames": 37, + "rows_per_frame": 1008, + "target_start": 2475, + "profile": "3x3-global", + "finite": true, + "relative_l2_vs_dense": 0.2588050067424774, + "baseline_samples_ms": [ + 69.95450798422098, + 69.79658600175753, + 69.58012498216704, + 70.22821099963039, + 69.11802099784836 + ], + "candidate_samples_ms": [ + 32.91054297005758, + 32.899541954975575, + 32.924533006735146, + 32.779561006464064, + 32.96928299823776 + ], + "baseline_median_ms": 69.79658600175753, + "candidate_median_ms": 32.91054297005758, + "reduction_pct": 52.847918708761966, + "operator_gate_passed": false, + "warmups_per_operation": 2, + "measured_per_operation": 5, + "order": "AB/BA/AB/BA/AB", + "probe_sha256": "a27906d8bb0b65e2f6ffb41ddeb4b513163b7bf23b9d2dbd33a5015fc93b2a08" + } + }, + { + "trial": "c20-video-variance-pool-native-50101063", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "4390450bc5324f7028125101cd4bb32e9de237b04bc12fa879b48110d7122e3c", + "tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py": "e871fb4b222d61d10b6c442a333a52c976cceaa0d406233502ecbcaddfeba8c9" + }, + "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 50.230675689992495 + }, + { + "trial": "c20-video-variance-pool-operator-209f263d", + "cleanup_passed": true, + "source_unchanged": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "4390450bc5324f7028125101cd4bb32e9de237b04bc12fa879b48110d7122e3c", + "tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py": "e871fb4b222d61d10b6c442a333a52c976cceaa0d406233502ecbcaddfeba8c9" + }, + "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 49.880220040970016, + "operator_benchmark": { + "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", + "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", + "fixture_tensor_sha256": { + "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", + "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", + "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" + }, + "query_shape": [ + 1, + 39808, + 14, + 128 + ], + "valid_kv": 39771, + "frames": 37, + "rows_per_frame": 1008, + "target_start": 2475, + "profile": "3x3-variance", + "finite": true, + "relative_l2_vs_dense": 0.26336145401000977, + "baseline_samples_ms": [ + 69.74446203093976, + 69.06059599714354, + 69.64961998164654, + 69.69038100214675, + 69.62426099926233 + ], + "candidate_samples_ms": [ + 35.81449302146211, + 36.30960697773844, + 35.925014002714306, + 35.99672397831455, + 35.80518299713731 + ], + "baseline_median_ms": 69.64961998164654, + "candidate_median_ms": 35.925014002714306, + "reduction_pct": 48.42037327385138, + "operator_gate_passed": false, + "warmups_per_operation": 2, + "measured_per_operation": 5, + "order": "AB/BA/AB/BA/AB", + "probe_sha256": "785e7a0c13719e37d5cc44fb66af05649e51e1f7e5d0b212e92ec44f79cb0184" + } + }, + { + "trial": "c21-video-pre-gate-native-3c5eebbe", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "8933b536317652d0dbc7a974ea7e0bca928e7b3bdf438988999ff5a498a6ec9d", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "019bf71106ba3263b55ad7b682064e3a176eecb81725cb48bbf178387356d5d9", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "04a9212994b5c8593f8d196820300ded077fed9af4274b64d87d9e6bf71eae07", + "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" + }, + "scope": "Legacy/pre-gate/DLO tests, two executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 127.31179964594776 + }, + { + "trial": "c21-video-pre-gate-quality-5s-00-7265c33e", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 318.92388185299933 + }, + "model_diagnostic": { + "source_head": "d48fa51d03dd748568cbcf57ed40f533c691c9bd", + "samples_ms": [ + 163805.5620979867 + ], + "latency_median_ms": 163805.5620979867, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 5197500, + "peak_cache_bytes": 8039270400, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10700390400, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10700390400, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10660608000, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + } + } + } + ], + "decisions": [ + "C12 fixed profile stopped: zero actual predictions.", + "C14 pinned block-FP8 and block-INT8 prerequisites failed; per-token INT8 is not excluded.", + "C13 numerical qualification and clean performance are separate.", + "C15 fixed FP16 video residual profile failed finite velocity at first reuse.", + "C16 FP32 residual completed but failed three frozen audio quality guards; fixed profile stopped.", + "C17 completed5s but failed11frozen quality checks; fixed window profile stopped.", + "C18 native17PASS but operation reduced only7.25%; no full model run.", + "C19 native17PASS; operation52.85%faster but relL2 .258805 exceeded fixed .258052screen; no model run.", + "C20 native17PASS; operation48.42%faster and relL2 .263361 failed both fixed primitive gates; no model run.", + "C21 native36PASS; current gates over cached BF16 video sublayers require separate frozen media and clean HTTP qualification." + ] +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json new file mode 100644 index 000000000..ad0f10fcd --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c22-execution-checkpoint.json @@ -0,0 +1,1351 @@ +{ + "campaign_verdict": "INCONCLUSIVE", + "performance_acceptance": "No newly accepted gain", + "runs": [ + { + "trial": "c12-alignment-quality-5s-00-9bf5c373", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 471.80311120999977 + }, + "model_diagnostic": { + "source_head": "a29648e5acd3077b7b22739bc9522017782fe1ac", + "samples_ms": [ + 318280.59680899605 + ], + "latency_median_ms": 318280.59680899605, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-1\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-2\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-3\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c12-state-alignment-native-tests-0dc5859b", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/cache_state_alignment.py": "3340ec21a77aef78531c601be241bcfb1e44107c795e8c8649907d1a18ed377f", + "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "f4cd0e9aa9e824c92ec1acd02c57a8f7d8684c35d4edf92ffdfae3345ff44e99", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "436ec52cd6329cbc64e91d94b450bfa6f773a9a953b8f974db73a5eb986a98bb", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "4dec8f02a04bfe7e8d2ada24fc6d5ae40a0058671d8515c8a266ecf88824b7b0", + "tests/diffusion/models/minimax_h3/test_minimax_h3_state_cache.py": "89cf57482b35d8d8ad8d4978aec87e9ee3ef45330e5b911161adef4379950a65" + }, + "scope": "focused tests only; no full-model quality or performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 82.06077561300481 + }, + { + "trial": "c13-baseline-cuda-staging-e22d8651", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "vllm_omni/diffusion/offloader/sequential_backend.py": "d542935b656a0d9dced78de8fb086b8e1c5aade26fac6cee5316b32a49f0d569", + "tests/diffusion/offloader/test_sequential_backend.py": "dcf1d64e65bed572d65b095c1350950f55d7e8d9ddf8c7dfd72d0ea6e721685a" + }, + "scope": "Reproduce stock CUDA-graph staging expectation on unchanged migration owning source in the same Ascend test environment", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 50.234599059971515 + }, + { + "trial": "c13-c1-dlo-5s-audit-cead10d1", + "model_controller": { + "status": "failed", + "returncode": 2, + "elapsed_seconds": 0.11969929095357656 + } + }, + { + "trial": "c13-c1-dlo-5s-audit-v2-b0b86c6d", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 749.505351218977 + }, + "model_diagnostic": { + "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "samples_ms": [ + 315593.86175096733, + 284456.94282796467 + ], + "latency_median_ms": 300025.402289466, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 2, + "kind": "diagnostic", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c13-c1-dlo-5s-calibration-c98129d8", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 768.8735517460154 + }, + "model_diagnostic": { + "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "samples_ms": [ + 317177.1984040388, + 285534.089674009 + ], + "latency_median_ms": 301355.6440390239, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 2, + "kind": "calibration", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c13-dlo-gate-tests-bc92a71d", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", + "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" + }, + "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 47.3752507569734 + }, + { + "trial": "c13-dlo-regression-tests-95e9ed2a", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", + "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" + }, + "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 62.34616496198578 + }, + { + "trial": "c13-dlo-regression-v2-12c217d5", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", + "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" + }, + "scope": "Expanded offloader tests in Ascend environment; some stock accelerator fixtures use NPU, not a full model test", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 65.14432259398745 + }, + { + "trial": "c14-int8-prerequisites-fdd6633b", + "cleanup_passed": true, + "source_unchanged": true, + "helpers_unchanged": true, + "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 20.795623040001374, + "native_prerequisite": { + "verdict": "FAIL", + "scope": "TorchNPU block-INT8 attention prerequisites only", + "phase": "dynamic_block_int8_quant_query", + "versions": { + "torch": "2.10.0+cpu", + "torch_npu": "2.10.0.post4", + "mindiesd": "3.1.0" + }, + "apis": { + "float8_e4m3fn": true, + "npu_dynamic_block_quant": true, + "npu_fused_infer_attention_score_v2": true + }, + "device_name": "Ascend910_9392", + "input": { + "shape": [ + 1, + 14, + 256, + 128 + ], + "dtype": "torch.bfloat16", + "visible_devices": "2" + }, + "error_type": "RuntimeError", + "probe_sha256": "fab18252f860231f9b7fcaa54587afbaa96fce15987fd551922246c8964f7217", + "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 561002\n[ERROR] 2026-11-02-02:00:57 (PID:2662830, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2662830] 2026-11-02-02:00:57.039.986 Invalid_Argument(EZ0024): Parameter row_block_size of DynamicBlockQuant has incorrect value 128. It should be 1.\n Solution: Check whether the parameter value is correct.\nTraceBack (most recent call last):\n The attr get failed.[FUNC:CheckParams][FILE:dynamic_block_quant_i8_tiling.cpp][LINE:78]\n The params check failed.[FUNC:Tiling4DynamicBlockQuant][FILE:dynamic_block_quant_tiling.cpp][LINE:564]\n Failed to execute operator DynamicBlockQuant_0. Reason: Failed to execute tiling function.\n Check NnopbaseExecutorDoTiling(executor) failed\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" + } + }, + { + "trial": "c14-quant-prerequisites-358b9b28", + "cleanup_passed": true, + "source_unchanged": true, + "helpers_unchanged": true, + "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 20.143024684977718, + "native_prerequisite": { + "verdict": "FAIL", + "scope": "TorchNPU FP8 prerequisites only", + "phase": "dynamic_block_quant_query", + "versions": { + "torch": "2.10.0+cpu", + "torch_npu": "2.10.0.post4", + "mindiesd": "3.1.0" + }, + "apis": { + "float8_e4m3fn": true, + "npu_dynamic_block_quant": true, + "npu_fused_infer_attention_score_v2": true + }, + "device_name": "Ascend910_9392", + "input": { + "shape": [ + 1, + 14, + 256, + 128 + ], + "dtype": "torch.bfloat16", + "visible_devices": "2" + }, + "error_type": "RuntimeError", + "probe_sha256": "df6465956e6f73bbca140fb9880057e30869fd4b7976b2dc091f71a575dd1096", + "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 161002\n[ERROR] 2026-11-02-01:45:24 (PID:2491012, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2491012] 2026-11-02-01:45:24.732.576 Execution_Error(EZ1009): Failed to execute operator DynamicBlockQuant_0. Reason: The dtype or format of the actual input or output parameter of the operator is inconsistent with that defined in the operator prototype OpDef, details of parameter are x(DT_BFLOAT16, ND), yOut(DT_FLOAT8_E4M3FN, ND), scaleOut(DT_FLOAT, ND).\nTraceBack (most recent call last):\n Supported dtype and format list is:\n SupportInfo[0]: x(DT_FLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n SupportInfo[1]: x(DT_BFLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" + } + }, + { + "trial": "c15-video-row-native-066a9a40", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "e35b551b8fc0ec3b0ba1384b4df96a8daa0e5179c5ed39d641125310b2ee957f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "6f890fcf84047bc5bc084f7b756a2b02e72573888a3bcc12ba7d0211e1bb62ef", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "d2a5bed10620a337ff9aaf99cf4a3740d94a8962ef903bf2da75d65e3b50e21c" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 82.27795292000519 + }, + { + "trial": "c15-video-row-native-3a9721a6", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "8ec1b3ae4cd252f5d95ab23e0232b983900939e8cb15d1e64f4740f7204abf39", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 81.91829407500336 + }, + { + "trial": "c15-video-row-native-a2a224de", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "875d0e70621fc76906be130519d7bb1242de5dd58159e31025e26da37e60d98c" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 82.86056435096543 + }, + { + "trial": "c15-video-row-native-bb5084cc", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "fa1ac8fd375d754503b3507fae006381138d8875afdad2418f1ea5ae338b9c26" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 81.50971999001922 + }, + { + "trial": "c15-video-row-quality-5s-00-710315e0", + "model_controller": { + "status": "failed", + "returncode": 1, + "elapsed_seconds": 153.44603093498154 + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": {}, + "request-000\\rank-1\\attention_path_counts.json": {}, + "request-000\\rank-2\\attention_path_counts.json": {}, + "request-000\\rank-3\\attention_path_counts.json": {} + } + }, + { + "trial": "c15-video-row-v2-quality-5s-00-d6b921b9", + "model_controller": { + "status": "failed", + "returncode": 1, + "elapsed_seconds": 199.99670764501207 + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 2475, + "cache_bytes": 4019635200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 0, + "cache_bytes": 5330304000, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 251 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 251 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 251 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 251 + } + } + }, + { + "trial": "c15-video-row-v3-quality-5s-00-70703586", + "model_controller": { + "status": "failed", + "returncode": 1, + "elapsed_seconds": 191.09544755500974 + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 123750, + "cache_bytes": 4019635200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 0, + "cache_bytes": 5330304000, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 300 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 300 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 300 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 300 + } + } + }, + { + "trial": "c16-video-row-fp32-native-d3fe320c", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "c555d76f7bd6457a81238fca56100d11cb7ab043a4a649d55e23b6203d09bde2", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 78.8189791989862 + }, + { + "trial": "c16-video-row-fp32-quality-5s-00-8600757a", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 355.51310362201184 + }, + "model_diagnostic": { + "source_head": "e0a3462fcc44de698b5dedcf8ff2c99f9802f757", + "samples_ms": [ + 197716.36760496767 + ], + "latency_median_ms": 197716.36760496767, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 4083750, + "peak_cache_bytes": 8039270400, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10700390400, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10700390400, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10660608000, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c17-video-window-native-9b48cb56", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" + }, + "scope": "CPU window reference/routing and native batched BF16 FAS; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.878022204036824 + }, + { + "trial": "c17-video-window-operator-9bf7de3e", + "cleanup_passed": true, + "source_unchanged": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" + }, + "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.778123575029895, + "operator_benchmark": { + "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", + "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", + "fixture_tensor_sha256": { + "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", + "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", + "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" + }, + "query_shape": [ + 1, + 39808, + 14, + 128 + ], + "valid_kv": 39771, + "frames": 37, + "rows_per_frame": 1008, + "target_start": 2475, + "window": 3, + "finite": true, + "relative_l2_vs_dense": 0.5161040425300598, + "baseline_samples_ms": [ + 69.0619180095382, + 69.60304302629083, + 69.27597103640437, + 69.81903495034203, + 69.96654602698982 + ], + "candidate_samples_ms": [ + 25.191069988068193, + 25.301200977992266, + 25.325032009277493, + 25.26562102138996, + 25.219489994924515 + ], + "baseline_median_ms": 69.60304302629083, + "candidate_median_ms": 25.26562102138996, + "reduction_pct": 63.70040744936038, + "operator_gate_passed": true, + "warmups_per_operation": 2, + "measured_per_operation": 5, + "order": "AB/BA/AB/BA/AB", + "probe_sha256": "2465cf08173950a241ac3cc4713a4227d4c4b5a87f35358692a7ef6b60821824" + } + }, + { + "trial": "c17-video-window-quality-5s-00-a8c689e4", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 358.6752056070254 + }, + "model_diagnostic": { + "source_head": "c1589a2b5e4ada617e7cca38ef7c300fe9495c58", + "samples_ms": [ + 210397.12271699682 + ], + "latency_median_ms": 210397.12271699682, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + } + } + }, + { + "trial": "c18-video-pooled-context-native-1673db72", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" + }, + "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 49.97671151399845 + }, + { + "trial": "c18-video-pooled-context-operator-315db0e5", + "cleanup_passed": true, + "source_unchanged": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" + }, + "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.37963652302278, + "operator_benchmark": { + "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", + "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", + "fixture_tensor_sha256": { + "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", + "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", + "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" + }, + "query_shape": [ + 1, + 39808, + 14, + 128 + ], + "valid_kv": 39771, + "frames": 37, + "rows_per_frame": 1008, + "target_start": 2475, + "profile": "3x3-local1", + "finite": true, + "relative_l2_vs_dense": 0.21801254153251648, + "baseline_samples_ms": [ + 69.19049099087715, + 69.70348599134013, + 69.43478400353342, + 69.81050700414926, + 69.24652203451842 + ], + "candidate_samples_ms": [ + 68.29233502503484, + 64.26435202592984, + 71.05128600960597, + 64.39994304673746, + 64.39412303734571 + ], + "baseline_median_ms": 69.43478400353342, + "candidate_median_ms": 64.39994304673746, + "reduction_pct": 7.251179691924648, + "operator_gate_passed": false, + "warmups_per_operation": 2, + "measured_per_operation": 5, + "order": "AB/BA/AB/BA/AB", + "probe_sha256": "42852c0e4e693c5b3685981db617de00648feefe651f7060ad83c359b08dd81d" + } + }, + { + "trial": "c19-video-global-pool-native-f0f4c502", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ddfd6d6822cc6a5528fc95539f7f98dfdeed25d5489da3ed5cf76e6076a5ec94", + "tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py": "5457db550b59e3373be8f5f21873c1b78ac357a6ac170c2603ee531b50e91467" + }, + "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.528875903983135 + }, + { + "trial": "c19-video-global-pool-operator-933511c9", + "cleanup_passed": true, + "source_unchanged": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ddfd6d6822cc6a5528fc95539f7f98dfdeed25d5489da3ed5cf76e6076a5ec94", + "tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py": "5457db550b59e3373be8f5f21873c1b78ac357a6ac170c2603ee531b50e91467" + }, + "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.02866222098237, + "operator_benchmark": { + "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", + "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", + "fixture_tensor_sha256": { + "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", + "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", + "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" + }, + "query_shape": [ + 1, + 39808, + 14, + 128 + ], + "valid_kv": 39771, + "frames": 37, + "rows_per_frame": 1008, + "target_start": 2475, + "profile": "3x3-global", + "finite": true, + "relative_l2_vs_dense": 0.2588050067424774, + "baseline_samples_ms": [ + 69.95450798422098, + 69.79658600175753, + 69.58012498216704, + 70.22821099963039, + 69.11802099784836 + ], + "candidate_samples_ms": [ + 32.91054297005758, + 32.899541954975575, + 32.924533006735146, + 32.779561006464064, + 32.96928299823776 + ], + "baseline_median_ms": 69.79658600175753, + "candidate_median_ms": 32.91054297005758, + "reduction_pct": 52.847918708761966, + "operator_gate_passed": false, + "warmups_per_operation": 2, + "measured_per_operation": 5, + "order": "AB/BA/AB/BA/AB", + "probe_sha256": "a27906d8bb0b65e2f6ffb41ddeb4b513163b7bf23b9d2dbd33a5015fc93b2a08" + } + }, + { + "trial": "c20-video-variance-pool-native-50101063", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "4390450bc5324f7028125101cd4bb32e9de237b04bc12fa879b48110d7122e3c", + "tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py": "e871fb4b222d61d10b6c442a333a52c976cceaa0d406233502ecbcaddfeba8c9" + }, + "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 50.230675689992495 + }, + { + "trial": "c20-video-variance-pool-operator-209f263d", + "cleanup_passed": true, + "source_unchanged": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "4390450bc5324f7028125101cd4bb32e9de237b04bc12fa879b48110d7122e3c", + "tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py": "e871fb4b222d61d10b6c442a333a52c976cceaa0d406233502ecbcaddfeba8c9" + }, + "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 49.880220040970016, + "operator_benchmark": { + "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", + "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", + "fixture_tensor_sha256": { + "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", + "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", + "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" + }, + "query_shape": [ + 1, + 39808, + 14, + 128 + ], + "valid_kv": 39771, + "frames": 37, + "rows_per_frame": 1008, + "target_start": 2475, + "profile": "3x3-variance", + "finite": true, + "relative_l2_vs_dense": 0.26336145401000977, + "baseline_samples_ms": [ + 69.74446203093976, + 69.06059599714354, + 69.64961998164654, + 69.69038100214675, + 69.62426099926233 + ], + "candidate_samples_ms": [ + 35.81449302146211, + 36.30960697773844, + 35.925014002714306, + 35.99672397831455, + 35.80518299713731 + ], + "baseline_median_ms": 69.64961998164654, + "candidate_median_ms": 35.925014002714306, + "reduction_pct": 48.42037327385138, + "operator_gate_passed": false, + "warmups_per_operation": 2, + "measured_per_operation": 5, + "order": "AB/BA/AB/BA/AB", + "probe_sha256": "785e7a0c13719e37d5cc44fb66af05649e51e1f7e5d0b212e92ec44f79cb0184" + } + }, + { + "trial": "c21-video-pre-gate-native-3c5eebbe", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "8933b536317652d0dbc7a974ea7e0bca928e7b3bdf438988999ff5a498a6ec9d", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "019bf71106ba3263b55ad7b682064e3a176eecb81725cb48bbf178387356d5d9", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "04a9212994b5c8593f8d196820300ded077fed9af4274b64d87d9e6bf71eae07", + "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" + }, + "scope": "Legacy/pre-gate/DLO tests, two executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 127.31179964594776 + }, + { + "trial": "c21-video-pre-gate-quality-5s-00-7265c33e", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 318.92388185299933 + }, + "model_diagnostic": { + "source_head": "d48fa51d03dd748568cbcf57ed40f533c691c9bd", + "samples_ms": [ + 163805.5620979867 + ], + "latency_median_ms": 163805.5620979867, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 5197500, + "peak_cache_bytes": 8039270400, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10700390400, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10700390400, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10660608000, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c22-video-pre-gate-secant-native-3521747e", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "b8394d22b054d60c1ab8e069e9600e1947cc73b5826e1f206e66c548123f632d", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_secant.py": "70a215fbb0dc63b90652d96f8f6d106a5785bd7d64dae75a2c4f6ec5c627ac1c", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "b991ebb5d43242357d5ab8cfb476808a76fc0658d6048db6f3b7131122c8a993", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "8933b536317652d0dbc7a974ea7e0bca928e7b3bdf438988999ff5a498a6ec9d", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "3904bc8ac6679bc51614b95f5510789331a41bf20a43a517f3a99d33505201a6", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "e21e0afe8f8948d805d374428a34f3b2c5c0b5041d63ea791f047af8f455cf16", + "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" + }, + "scope": "Legacy/pre-gate/DLO tests, three executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 154.04585235595005 + }, + { + "trial": "c22-video-pre-gate-secant-quality-5s-00-72202558", + "model_controller": { + "status": "failed", + "returncode": 1, + "elapsed_seconds": 5.625923136016354 + } + } + ], + "decisions": [ + "C12 fixed profile stopped: zero actual predictions.", + "C14 pinned block-FP8 and block-INT8 prerequisites failed; per-token INT8 is not excluded.", + "C13 numerical qualification and clean performance are separate.", + "C15 fixed FP16 video residual profile failed finite velocity at first reuse.", + "C16 FP32 residual completed but failed three frozen audio quality guards; fixed profile stopped.", + "C17 completed5s but failed11frozen quality checks; fixed window profile stopped.", + "C18 native17PASS but operation reduced only7.25%; no full model run.", + "C19 native17PASS; operation52.85%faster but relL2 .258805 exceeded fixed .258052screen; no model run.", + "C20 native17PASS; operation48.42%faster and relL2 .263361 failed both fixed primitive gates; no model run.", + "C21 native36PASS; full5s completed but failed7frozen quality checks; fixed zero-order profile stopped.", + "C22 native45PASS; same eight-anchor schedule with sigma secants requires independent media qualification.", + "First C22 model stopped before startup on duplicate controller aliases; audited canonicalization retains all attempts and time charges." + ] +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json new file mode 100644 index 000000000..459b9a3d3 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c12-through-c23-execution-checkpoint.json @@ -0,0 +1,2080 @@ +{ + "campaign_verdict": "INCONCLUSIVE", + "performance_acceptance": "No newly accepted gain", + "runs": [ + { + "trial": "c12-alignment-quality-5s-00-9bf5c373", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 471.80311120999977 + }, + "model_diagnostic": { + "source_head": "a29648e5acd3077b7b22739bc9522017782fe1ac", + "samples_ms": [ + 318280.59680899605 + ], + "latency_median_ms": 318280.59680899605, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-1\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-2\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-3\\alignment_cache_stats.json": { + "request_counters": { + "computed": 50, + "predicted": 0, + "vetoes": 45 + }, + "scope": "actual state-aligned compute/forecast/veto counts; not quality acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c12-state-alignment-native-tests-0dc5859b", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/cache_state_alignment.py": "3340ec21a77aef78531c601be241bcfb1e44107c795e8c8649907d1a18ed377f", + "vllm_omni/diffusion/models/minimax_h3/reference_kv_cache_dit.py": "f4cd0e9aa9e824c92ec1acd02c57a8f7d8684c35d4edf92ffdfae3345ff44e99", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "436ec52cd6329cbc64e91d94b450bfa6f773a9a953b8f974db73a5eb986a98bb", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "4dec8f02a04bfe7e8d2ada24fc6d5ae40a0058671d8515c8a266ecf88824b7b0", + "tests/diffusion/models/minimax_h3/test_minimax_h3_state_cache.py": "89cf57482b35d8d8ad8d4978aec87e9ee3ef45330e5b911161adef4379950a65" + }, + "scope": "focused tests only; no full-model quality or performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 82.06077561300481 + }, + { + "trial": "c13-baseline-cuda-staging-e22d8651", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "vllm_omni/diffusion/offloader/sequential_backend.py": "d542935b656a0d9dced78de8fb086b8e1c5aade26fac6cee5316b32a49f0d569", + "tests/diffusion/offloader/test_sequential_backend.py": "dcf1d64e65bed572d65b095c1350950f55d7e8d9ddf8c7dfd72d0ea6e721685a" + }, + "scope": "Reproduce stock CUDA-graph staging expectation on unchanged migration owning source in the same Ascend test environment", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 50.234599059971515 + }, + { + "trial": "c13-c1-dlo-5s-audit-cead10d1", + "model_controller": { + "status": "failed", + "returncode": 2, + "elapsed_seconds": 0.11969929095357656 + } + }, + { + "trial": "c13-c1-dlo-5s-audit-v2-b0b86c6d", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 749.505351218977 + }, + "model_diagnostic": { + "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "samples_ms": [ + 315593.86175096733, + 284456.94282796467 + ], + "latency_median_ms": 300025.402289466, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 2, + "kind": "diagnostic", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c13-c1-dlo-5s-calibration-c98129d8", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 768.8735517460154 + }, + "model_diagnostic": { + "source_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "samples_ms": [ + 317177.1984040388, + 285534.089674009 + ], + "latency_median_ms": 301355.6440390239, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 2, + "kind": "calibration", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-001\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c13-dlo-gate-tests-bc92a71d", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", + "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" + }, + "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 47.3752507569734 + }, + { + "trial": "c13-dlo-regression-tests-95e9ed2a", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", + "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" + }, + "scope": "CPU mock barrier/chunk tests; no actual NPU collective or full-model quality/performance acceptance", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 62.34616496198578 + }, + { + "trial": "c13-dlo-regression-v2-12c217d5", + "cleanup_passed": true, + "source_unchanged": true, + "native_test_executed": false, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "2b33f73dff9c79118cc2b1bd4d50e3542ebc45b7bf029b38c17abbef37e4b917", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/offloader/base.py": "533c793119e4cdc14a2ea3c6cb5bf97ffede673befeab3f0fb8dbbf51e655379", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "d441481513aba9e90a91265f4bba3f53f8766197be8c1302c3c714e690d5ea76", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "350461dfca565bc0b07087ac7af1e27b0e247f5eac717e350b637e37f62dd088", + "tests/diffusion/offloader/helpers.py": "729c73abba3b9a12911282d4bcd3cf8bfa148fbcbfdc2e2b7c86000e387fa687" + }, + "scope": "Expanded offloader tests in Ascend environment; some stock accelerator fixtures use NPU, not a full model test", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 65.14432259398745 + }, + { + "trial": "c14-int8-prerequisites-fdd6633b", + "cleanup_passed": true, + "source_unchanged": true, + "helpers_unchanged": true, + "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 20.795623040001374, + "native_prerequisite": { + "verdict": "FAIL", + "scope": "TorchNPU block-INT8 attention prerequisites only", + "phase": "dynamic_block_int8_quant_query", + "versions": { + "torch": "2.10.0+cpu", + "torch_npu": "2.10.0.post4", + "mindiesd": "3.1.0" + }, + "apis": { + "float8_e4m3fn": true, + "npu_dynamic_block_quant": true, + "npu_fused_infer_attention_score_v2": true + }, + "device_name": "Ascend910_9392", + "input": { + "shape": [ + 1, + 14, + 256, + 128 + ], + "dtype": "torch.bfloat16", + "visible_devices": "2" + }, + "error_type": "RuntimeError", + "probe_sha256": "fab18252f860231f9b7fcaa54587afbaa96fce15987fd551922246c8964f7217", + "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 561002\n[ERROR] 2026-11-02-02:00:57 (PID:2662830, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2662830] 2026-11-02-02:00:57.039.986 Invalid_Argument(EZ0024): Parameter row_block_size of DynamicBlockQuant has incorrect value 128. It should be 1.\n Solution: Check whether the parameter value is correct.\nTraceBack (most recent call last):\n The attr get failed.[FUNC:CheckParams][FILE:dynamic_block_quant_i8_tiling.cpp][LINE:78]\n The params check failed.[FUNC:Tiling4DynamicBlockQuant][FILE:dynamic_block_quant_tiling.cpp][LINE:564]\n Failed to execute operator DynamicBlockQuant_0. Reason: Failed to execute tiling function.\n Check NnopbaseExecutorDoTiling(executor) failed\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" + } + }, + { + "trial": "c14-quant-prerequisites-358b9b28", + "cleanup_passed": true, + "source_unchanged": true, + "helpers_unchanged": true, + "scope": "Bounded native prerequisites only; a failed probe is retained as negative evidence.", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 20.143024684977718, + "native_prerequisite": { + "verdict": "FAIL", + "scope": "TorchNPU FP8 prerequisites only", + "phase": "dynamic_block_quant_query", + "versions": { + "torch": "2.10.0+cpu", + "torch_npu": "2.10.0.post4", + "mindiesd": "3.1.0" + }, + "apis": { + "float8_e4m3fn": true, + "npu_dynamic_block_quant": true, + "npu_fused_infer_attention_score_v2": true + }, + "device_name": "Ascend910_9392", + "input": { + "shape": [ + 1, + 14, + 256, + 128 + ], + "dtype": "torch.bfloat16", + "visible_devices": "2" + }, + "error_type": "RuntimeError", + "probe_sha256": "df6465956e6f73bbca140fb9880057e30869fd4b7976b2dc091f71a575dd1096", + "error": "npu_dynamic_block_quant:../third_party/op-plugin/op_plugin/ops/opapi/DynamicBlockQuantNpuOpApi.cpp:61 NPU function error: call aclnnDynamicBlockQuantV2 failed, error code is 161002\n[ERROR] 2026-11-02-01:45:24 (PID:2491012, Device:0, RankID:-1) ERR00100 PTA call acl api failed.\n[PID: 2491012] 2026-11-02-01:45:24.732.576 Execution_Error(EZ1009): Failed to execute operator DynamicBlockQuant_0. Reason: The dtype or format of the actual input or output parameter of the operator is inconsistent with that defined in the operator prototype OpDef, details of parameter are x(DT_BFLOAT16, ND), yOut(DT_FLOAT8_E4M3FN, ND), scaleOut(DT_FLOAT, ND).\nTraceBack (most recent call last):\n Supported dtype and format list is:\n SupportInfo[0]: x(DT_FLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n SupportInfo[1]: x(DT_BFLOAT16, ND), yOut(DT_INT8, ND), scaleOut(DT_FLOAT, ND)\n Check NnopbaseExecutorTilingAndUpdateBinInfo(executor) failed\n Check NnopbaseExecutorMatchCache(executor) failed\n Check NnopbaseRunForWorkspace(*executor, workspaceSize) failed\n" + } + }, + { + "trial": "c15-video-row-native-066a9a40", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "e35b551b8fc0ec3b0ba1384b4df96a8daa0e5179c5ed39d641125310b2ee957f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "6f890fcf84047bc5bc084f7b756a2b02e72573888a3bcc12ba7d0211e1bb62ef", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "d2a5bed10620a337ff9aaf99cf4a3740d94a8962ef903bf2da75d65e3b50e21c" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 82.27795292000519 + }, + { + "trial": "c15-video-row-native-3a9721a6", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "8ec1b3ae4cd252f5d95ab23e0232b983900939e8cb15d1e64f4740f7204abf39", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 81.91829407500336 + }, + { + "trial": "c15-video-row-native-a2a224de", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "875d0e70621fc76906be130519d7bb1242de5dd58159e31025e26da37e60d98c" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "failed", + "returncode": 1, + "elapsed_seconds": 82.86056435096543 + }, + { + "trial": "c15-video-row-native-bb5084cc", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "118afcc28379dd73feec763fbb9cfc9f17c6a208afe5969a85b3c969b8059552", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "fa1ac8fd375d754503b3507fae006381138d8875afdad2418f1ea5ae338b9c26" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 81.50971999001922 + }, + { + "trial": "c15-video-row-quality-5s-00-710315e0", + "model_controller": { + "status": "failed", + "returncode": 1, + "elapsed_seconds": 153.44603093498154 + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": {}, + "request-000\\rank-1\\attention_path_counts.json": {}, + "request-000\\rank-2\\attention_path_counts.json": {}, + "request-000\\rank-3\\attention_path_counts.json": {} + } + }, + { + "trial": "c15-video-row-v2-quality-5s-00-d6b921b9", + "model_controller": { + "status": "failed", + "returncode": 1, + "elapsed_seconds": 199.99670764501207 + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 2475, + "cache_bytes": 4019635200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 1, + "subset_attention_calls": 1, + "subset_query_rows": 2475, + "omitted_query_rows": 37296, + "partial_mlp_rows": 0, + "cache_bytes": 5330304000, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 251 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 251 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 251 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 251 + } + } + }, + { + "trial": "c15-video-row-v3-quality-5s-00-70703586", + "model_controller": { + "status": "failed", + "returncode": 1, + "elapsed_seconds": 191.09544755500974 + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 123750, + "cache_bytes": 4019635200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 0, + "cache_bytes": 5350195200, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 5, + "partial_steps": 1, + "full_blocks": 250, + "partial_blocks": 50, + "subset_attention_calls": 50, + "subset_query_rows": 123750, + "omitted_query_rows": 1864800, + "partial_mlp_rows": 0, + "cache_bytes": 5330304000, + "closed": true + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 300 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 300 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 300 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 300 + } + } + }, + { + "trial": "c16-video-row-fp32-native-d3fe320c", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "c555d76f7bd6457a81238fca56100d11cb7ab043a4a649d55e23b6203d09bde2", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" + }, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 78.8189791989862 + }, + { + "trial": "c16-video-row-fp32-quality-5s-00-8600757a", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 355.51310362201184 + }, + "model_diagnostic": { + "source_head": "e0a3462fcc44de698b5dedcf8ff2c99f9802f757", + "samples_ms": [ + 197716.36760496767 + ], + "latency_median_ms": 197716.36760496767, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 4083750, + "peak_cache_bytes": 8039270400, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10700390400, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10700390400, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10660608000, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c17-video-window-native-9b48cb56", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" + }, + "scope": "CPU window reference/routing and native batched BF16 FAS; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.878022204036824 + }, + { + "trial": "c17-video-window-operator-9bf7de3e", + "cleanup_passed": true, + "source_unchanged": true, + "operator_gate_passed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" + }, + "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.778123575029895, + "operator_benchmark": { + "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", + "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", + "fixture_tensor_sha256": { + "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", + "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", + "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" + }, + "query_shape": [ + 1, + 39808, + 14, + 128 + ], + "valid_kv": 39771, + "frames": 37, + "rows_per_frame": 1008, + "target_start": 2475, + "window": 3, + "finite": true, + "relative_l2_vs_dense": 0.5161040425300598, + "baseline_samples_ms": [ + 69.0619180095382, + 69.60304302629083, + 69.27597103640437, + 69.81903495034203, + 69.96654602698982 + ], + "candidate_samples_ms": [ + 25.191069988068193, + 25.301200977992266, + 25.325032009277493, + 25.26562102138996, + 25.219489994924515 + ], + "baseline_median_ms": 69.60304302629083, + "candidate_median_ms": 25.26562102138996, + "reduction_pct": 63.70040744936038, + "operator_gate_passed": true, + "warmups_per_operation": 2, + "measured_per_operation": 5, + "order": "AB/BA/AB/BA/AB", + "probe_sha256": "2465cf08173950a241ac3cc4713a4227d4c4b5a87f35358692a7ef6b60821824" + } + }, + { + "trial": "c17-video-window-quality-5s-00-a8c689e4", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 358.6752056070254 + }, + "model_diagnostic": { + "source_head": "c1589a2b5e4ada617e7cca38ef7c300fe9495c58", + "samples_ms": [ + 210397.12271699682 + ], + "latency_median_ms": 210397.12271699682, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + } + } + }, + { + "trial": "c18-video-pooled-context-native-1673db72", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" + }, + "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 49.97671151399845 + }, + { + "trial": "c18-video-pooled-context-operator-315db0e5", + "cleanup_passed": true, + "source_unchanged": true, + "operator_gate_passed": false, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" + }, + "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.37963652302278, + "operator_benchmark": { + "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", + "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", + "fixture_tensor_sha256": { + "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", + "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", + "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" + }, + "query_shape": [ + 1, + 39808, + 14, + 128 + ], + "valid_kv": 39771, + "frames": 37, + "rows_per_frame": 1008, + "target_start": 2475, + "profile": "3x3-local1", + "finite": true, + "relative_l2_vs_dense": 0.21801254153251648, + "baseline_samples_ms": [ + 69.19049099087715, + 69.70348599134013, + 69.43478400353342, + 69.81050700414926, + 69.24652203451842 + ], + "candidate_samples_ms": [ + 68.29233502503484, + 64.26435202592984, + 71.05128600960597, + 64.39994304673746, + 64.39412303734571 + ], + "baseline_median_ms": 69.43478400353342, + "candidate_median_ms": 64.39994304673746, + "reduction_pct": 7.251179691924648, + "operator_gate_passed": false, + "warmups_per_operation": 2, + "measured_per_operation": 5, + "order": "AB/BA/AB/BA/AB", + "probe_sha256": "42852c0e4e693c5b3685981db617de00648feefe651f7060ad83c359b08dd81d" + } + }, + { + "trial": "c19-video-global-pool-native-f0f4c502", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ddfd6d6822cc6a5528fc95539f7f98dfdeed25d5489da3ed5cf76e6076a5ec94", + "tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py": "5457db550b59e3373be8f5f21873c1b78ac357a6ac170c2603ee531b50e91467" + }, + "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.528875903983135 + }, + { + "trial": "c19-video-global-pool-operator-933511c9", + "cleanup_passed": true, + "source_unchanged": true, + "operator_gate_passed": false, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ddfd6d6822cc6a5528fc95539f7f98dfdeed25d5489da3ed5cf76e6076a5ec94", + "tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py": "5457db550b59e3373be8f5f21873c1b78ac357a6ac170c2603ee531b50e91467" + }, + "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 51.02866222098237, + "operator_benchmark": { + "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", + "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", + "fixture_tensor_sha256": { + "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", + "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", + "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" + }, + "query_shape": [ + 1, + 39808, + 14, + 128 + ], + "valid_kv": 39771, + "frames": 37, + "rows_per_frame": 1008, + "target_start": 2475, + "profile": "3x3-global", + "finite": true, + "relative_l2_vs_dense": 0.2588050067424774, + "baseline_samples_ms": [ + 69.95450798422098, + 69.79658600175753, + 69.58012498216704, + 70.22821099963039, + 69.11802099784836 + ], + "candidate_samples_ms": [ + 32.91054297005758, + 32.899541954975575, + 32.924533006735146, + 32.779561006464064, + 32.96928299823776 + ], + "baseline_median_ms": 69.79658600175753, + "candidate_median_ms": 32.91054297005758, + "reduction_pct": 52.847918708761966, + "operator_gate_passed": false, + "warmups_per_operation": 2, + "measured_per_operation": 5, + "order": "AB/BA/AB/BA/AB", + "probe_sha256": "a27906d8bb0b65e2f6ffb41ddeb4b513163b7bf23b9d2dbd33a5015fc93b2a08" + } + }, + { + "trial": "c20-video-variance-pool-native-50101063", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "4390450bc5324f7028125101cd4bb32e9de237b04bc12fa879b48110d7122e3c", + "tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py": "e871fb4b222d61d10b6c442a333a52c976cceaa0d406233502ecbcaddfeba8c9" + }, + "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 50.230675689992495 + }, + { + "trial": "c20-video-variance-pool-operator-209f263d", + "cleanup_passed": true, + "source_unchanged": true, + "operator_gate_passed": false, + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "4390450bc5324f7028125101cd4bb32e9de237b04bc12fa879b48110d7122e3c", + "tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py": "e871fb4b222d61d10b6c442a333a52c976cceaa0d406233502ecbcaddfeba8c9" + }, + "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 49.880220040970016, + "operator_benchmark": { + "scope": "real captured 5s attention operation on logical2; no model quality or HTTP acceptance", + "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", + "fixture_tensor_sha256": { + "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", + "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", + "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" + }, + "query_shape": [ + 1, + 39808, + 14, + 128 + ], + "valid_kv": 39771, + "frames": 37, + "rows_per_frame": 1008, + "target_start": 2475, + "profile": "3x3-variance", + "finite": true, + "relative_l2_vs_dense": 0.26336145401000977, + "baseline_samples_ms": [ + 69.74446203093976, + 69.06059599714354, + 69.64961998164654, + 69.69038100214675, + 69.62426099926233 + ], + "candidate_samples_ms": [ + 35.81449302146211, + 36.30960697773844, + 35.925014002714306, + 35.99672397831455, + 35.80518299713731 + ], + "baseline_median_ms": 69.64961998164654, + "candidate_median_ms": 35.925014002714306, + "reduction_pct": 48.42037327385138, + "operator_gate_passed": false, + "warmups_per_operation": 2, + "measured_per_operation": 5, + "order": "AB/BA/AB/BA/AB", + "probe_sha256": "785e7a0c13719e37d5cc44fb66af05649e51e1f7e5d0b212e92ec44f79cb0184" + } + }, + { + "trial": "c21-video-pre-gate-native-3c5eebbe", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "8933b536317652d0dbc7a974ea7e0bca928e7b3bdf438988999ff5a498a6ec9d", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "019bf71106ba3263b55ad7b682064e3a176eecb81725cb48bbf178387356d5d9", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "04a9212994b5c8593f8d196820300ded077fed9af4274b64d87d9e6bf71eae07", + "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" + }, + "scope": "Legacy/pre-gate/DLO tests, two executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 127.31179964594776 + }, + { + "trial": "c21-video-pre-gate-quality-5s-00-7265c33e", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 318.92388185299933 + }, + "model_diagnostic": { + "source_head": "d48fa51d03dd748568cbcf57ed40f533c691c9bd", + "samples_ms": [ + 163805.5620979867 + ], + "latency_median_ms": 163805.5620979867, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 5197500, + "peak_cache_bytes": 8039270400, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10700390400, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10700390400, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10660608000, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c22-video-pre-gate-secant-native-3521747e", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "b8394d22b054d60c1ab8e069e9600e1947cc73b5826e1f206e66c548123f632d", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_secant.py": "70a215fbb0dc63b90652d96f8f6d106a5785bd7d64dae75a2c4f6ec5c627ac1c", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "b991ebb5d43242357d5ab8cfb476808a76fc0658d6048db6f3b7131122c8a993", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "8933b536317652d0dbc7a974ea7e0bca928e7b3bdf438988999ff5a498a6ec9d", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "3904bc8ac6679bc51614b95f5510789331a41bf20a43a517f3a99d33505201a6", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "e21e0afe8f8948d805d374428a34f3b2c5c0b5041d63ea791f047af8f455cf16", + "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" + }, + "scope": "Legacy/pre-gate/DLO tests, three executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 154.04585235595005 + }, + { + "trial": "c22-video-pre-gate-secant-quality-5s-00-72202558", + "model_controller": { + "status": "failed", + "returncode": 1, + "elapsed_seconds": 5.625923136016354 + } + }, + { + "trial": "c22-video-pre-gate-secant-v2-quality-5s-00-777147cb", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 317.8191881469684 + }, + "model_diagnostic": { + "source_head": "7b0d12bfefdf05868e93e74bc91f04c70ee34128", + "samples_ms": [ + 166858.99333201814 + ], + "latency_median_ms": 166858.99333201814, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 5197500, + "peak_cache_bytes": 12061056000, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate_secant", + "secant_forecasts": 4200, + "slope_codec": "int8_per_channel", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 0, + "peak_cache_bytes": 16052736000, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate_secant", + "secant_forecasts": 4200, + "slope_codec": "int8_per_channel", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 0, + "peak_cache_bytes": 16052736000, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate_secant", + "secant_forecasts": 4200, + "slope_codec": "int8_per_channel", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 0, + "peak_cache_bytes": 15993062400, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate_secant", + "secant_forecasts": 4200, + "slope_codec": "int8_per_channel", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c23-video-context-lse-components-783670a5", + "cleanup_passed": true, + "source_unchanged": true, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_context.py": "ac7ff16b789b7304add457cea82859d6ae21548ac355344a390125f2e2c36fbe", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "48533c941a50719995206d18fa9198330824b940ecba9399b46ba7615b969ccb", + "vllm_omni/diffusion/models/minimax_h3/attention/video_context.py": "f53c94e2ac9a8df8f4d7abd753393b2907170a086e92ad7b23818810868e02e1", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "73c96090e6fb0e13ce734bc96b23472fd5adb83bceb71f3eba2815d6343a33e4", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "577e290c2d92fc02822cd516cc499c3c4d1f465145ad2a837edfdfeaa09ed9fb", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cff17568fbd856c28394d104d393a806931613b7b057f288006ad8c6fb9aa05a", + "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" + }, + "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 52.930839737004135, + "component_diagnosis": { + "scope": "Synchronized stage diagnosis on real captured5s QKV; not clean model timing. Raw derived merge inputs/old output retained privately.", + "measurements": { + "nonvideo_query_full_kv": { + "samples_ms": [ + 5.2369519835338, + 5.142281006556004, + 5.150350974872708, + 5.160611995961517, + 5.198111000936478 + ], + "median_ms": 5.160611995961517 + }, + "video_query_prefix_kv_lse": { + "samples_ms": [ + 5.12795097893104, + 5.113500985316932, + 5.11934095993638, + 5.1073210197500885, + 5.115851003210992 + ], + "median_ms": 5.115851003210992 + }, + "video_query_video_kv_lse": { + "samples_ms": [ + 62.014786002691835, + 61.919544998090714, + 61.41817104071379, + 61.652702977880836, + 61.9437750428915 + ], + "median_ms": 61.919544998090714 + }, + "triton_lse_merge": { + "samples_ms": [ + 22.78909203596413, + 22.761480999179184, + 22.750200994778425, + 22.737870982382447, + 22.741902037523687 + ], + "median_ms": 22.750200994778425 + }, + "torch_lse_merge": { + "samples_ms": [ + 2.7590019744820893, + 2.740311960224062, + 2.7479419950395823, + 2.737232018262148, + 2.7427320019342005 + ], + "median_ms": 2.7427320019342005 + } + }, + "merge_fixture": { + "prefix_output": { + "shape": [ + 1, + 37296, + 14, + 128 + ], + "stride": [ + 66834432, + 1792, + 128, + 1 + ], + "dtype": "torch.bfloat16", + "bytes": 133668864, + "sha256": "27ca510cb9270c976fe10741d505e1e9542315efa0c09f3947e0ae109be6667f", + "file": "prefix_output.bin" + }, + "video_output": { + "shape": [ + 1, + 37296, + 14, + 128 + ], + "stride": [ + 66834432, + 1792, + 128, + 1 + ], + "dtype": "torch.bfloat16", + "bytes": 133668864, + "sha256": "61da35ea1aea5cfea779f630ea287675541d52adb28e60dcaaa2697d70d7d5cd", + "file": "video_output.bin" + }, + "prefix_lse": { + "shape": [ + 1, + 37296, + 14, + 1 + ], + "stride": [ + 522144, + 14, + 1, + 1 + ], + "dtype": "torch.float32", + "bytes": 2088576, + "sha256": "731027f3ec32dfe68302dc4198c46d06661362977106717425fbd751b4df2450", + "file": "prefix_lse.bin" + }, + "video_lse": { + "shape": [ + 1, + 37296, + 14, + 1 + ], + "stride": [ + 522144, + 14, + 1, + 1 + ], + "dtype": "torch.float32", + "bytes": 2088576, + "sha256": "eacaded11a805bf19f94a18b25fdbd90243f71760b7de9071b3b3bbd8ae6e987", + "file": "video_lse.bin" + }, + "original_merge": { + "shape": [ + 1, + 37296, + 14, + 128 + ], + "stride": [ + 66834432, + 1792, + 128, + 1 + ], + "dtype": "torch.bfloat16", + "bytes": 133668864, + "sha256": "b1fa215f72bdfbddf11431ddec28cd2a06049e4bbbbb9b0857b6312713139919", + "file": "original_merge.bin" + }, + "torch_merge": { + "shape": [ + 1, + 37296, + 14, + 128 + ], + "stride": [ + 66834432, + 1792, + 128, + 1 + ], + "dtype": "torch.bfloat16", + "bytes": 133668864, + "sha256": "b1fa215f72bdfbddf11431ddec28cd2a06049e4bbbbb9b0857b6312713139919", + "file": "torch_merge.bin" + } + }, + "original_merge_bit_equal_torch": true, + "probe_sha256": "5e53eff1cb66ec478d1b04eb39510f253927614679499065d7deb0d1a80950ea" + } + }, + { + "trial": "c23-video-context-lse-native-ef09a725", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_context.py": "ac7ff16b789b7304add457cea82859d6ae21548ac355344a390125f2e2c36fbe", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "48533c941a50719995206d18fa9198330824b940ecba9399b46ba7615b969ccb", + "vllm_omni/diffusion/models/minimax_h3/attention/video_context.py": "f53c94e2ac9a8df8f4d7abd753393b2907170a086e92ad7b23818810868e02e1", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "73c96090e6fb0e13ce734bc96b23472fd5adb83bceb71f3eba2815d6343a33e4", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "577e290c2d92fc02822cd516cc499c3c4d1f465145ad2a837edfdfeaa09ed9fb", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cff17568fbd856c28394d104d393a806931613b7b057f288006ad8c6fb9aa05a", + "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" + }, + "scope": "Legacy/pre-gate/DLO tests, three executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 158.8542353429948 + }, + { + "trial": "c23-video-context-lse-operator-66d6ce78", + "cleanup_passed": true, + "source_unchanged": true, + "operator_gate_passed": false, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_context.py": "ac7ff16b789b7304add457cea82859d6ae21548ac355344a390125f2e2c36fbe", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "48533c941a50719995206d18fa9198330824b940ecba9399b46ba7615b969ccb", + "vllm_omni/diffusion/models/minimax_h3/attention/video_context.py": "f53c94e2ac9a8df8f4d7abd753393b2907170a086e92ad7b23818810868e02e1", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "73c96090e6fb0e13ce734bc96b23472fd5adb83bceb71f3eba2815d6343a33e4", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "577e290c2d92fc02822cd516cc499c3c4d1f465145ad2a837edfdfeaa09ed9fb", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cff17568fbd856c28394d104d393a806931613b7b057f288006ad8c6fb9aa05a", + "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" + }, + "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 94.22810779500287, + "operator_benchmark": { + "scope": "One captured static5s attention input over50calls/8refresh42reuse; includes packing/LSE merge/lifecycle, excludes FFN/model/HTTP and changing-input quality.", + "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", + "fixture_tensor_sha256": { + "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", + "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", + "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" + }, + "query_shape": [ + 1, + 39808, + 14, + 128 + ], + "valid_kv": 39771, + "target_start": 2475, + "finite": true, + "relative_l2_vs_dense": 0.0020166223403066397, + "baseline_cycle_samples_ms": [ + 3497.467301960569, + 3494.0786649822257, + 3493.1092370534316, + 3494.0861849463545, + 3495.386554975994 + ], + "candidate_cycle_samples_ms": [ + 2157.6043039676733, + 2157.4149020016193, + 2156.61749598803, + 2157.2416909621097, + 2159.0202350053005 + ], + "baseline_median_ms": 3494.0861849463545, + "candidate_median_ms": 2157.4149020016193, + "reduction_pct": 38.25524649917178, + "actual_context_calls": { + "video_context_refreshes": 8, + "video_context_reuses": 42, + "fresh_video_prefix_calls": 50, + "peak_cache_bytes": 135757440 + }, + "untimed_validation": { + "finite_all50": true, + "max_relative_l2_all50": 0.0020166223403066397 + }, + "operator_gate_passed": false, + "warmup_cycles_per_operation": 2, + "measured_cycles_per_operation": 5, + "order": "AB/BA/AB/BA/AB", + "probe_sha256": "4c494437eeaeda7721cb7b4e170297dbc49ba595c66e75a4afbae02895bcbf3f" + } + }, + { + "trial": "c23-video-context-lse-quality-5s-00-704b6bb0", + "model_controller": { + "status": "completed", + "returncode": 0, + "elapsed_seconds": 351.4251744679641 + }, + "model_diagnostic": { + "source_head": "6eb7bb094effb40853431208d1631effaf452a3f", + "samples_ms": [ + 199064.918508986 + ], + "latency_median_ms": 199064.918508986, + "scope": "http_complete_mp4", + "warmup": 0, + "repetitions": 1, + "kind": "quality", + "diagnostic": "audit", + "clean_timing": false, + "cleanup_passed": true + }, + "actual_counters": { + "request-000\\rank-0\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 0, + "subset_query_rows": 0, + "omitted_query_rows": 0, + "partial_mlp_rows": 0, + "peak_cache_bytes": 6787872000, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "video_context_lse", + "video_context_refreshes": 400, + "video_context_reuses": 2100, + "fresh_video_prefix_calls": 2500, + "full_row_mlp_blocks": 2500, + "fresh_attention_gates": 0, + "fresh_mlp_gates": 0, + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-1\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 0, + "subset_query_rows": 0, + "omitted_query_rows": 0, + "partial_mlp_rows": 0, + "peak_cache_bytes": 6787872000, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "video_context_lse", + "video_context_refreshes": 400, + "video_context_reuses": 2100, + "fresh_video_prefix_calls": 2500, + "full_row_mlp_blocks": 2500, + "fresh_attention_gates": 0, + "fresh_mlp_gates": 0, + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-2\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 0, + "subset_query_rows": 0, + "omitted_query_rows": 0, + "partial_mlp_rows": 0, + "peak_cache_bytes": 6787872000, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "video_context_lse", + "video_context_refreshes": 400, + "video_context_reuses": 2100, + "fresh_video_prefix_calls": 2500, + "full_row_mlp_blocks": 2500, + "fresh_attention_gates": 0, + "fresh_mlp_gates": 0, + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-3\\video_row_cache_stats.json": { + "request_counters": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 0, + "subset_query_rows": 0, + "omitted_query_rows": 0, + "partial_mlp_rows": 0, + "peak_cache_bytes": 6787872000, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "video_context_lse", + "video_context_refreshes": 400, + "video_context_reuses": 2100, + "fresh_video_prefix_calls": 2500, + "full_row_mlp_blocks": 2500, + "fresh_attention_gates": 0, + "fresh_mlp_gates": 0, + "live_cache_bytes": 0 + }, + "scope": "actual full/partial blocks, subset queries and cache bytes; not quality/performance acceptance" + }, + "request-000\\rank-0\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-1\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-2\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "request-000\\rank-3\\attention_path_counts.json": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + } + } + }, + { + "trial": "c23-video-context-lse-v2-native-dc375564", + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_context.py": "1e76074b71f0ea4e070c435db623add702d1414a351427fd86ebb40e8dfd67fd", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "48533c941a50719995206d18fa9198330824b940ecba9399b46ba7615b969ccb", + "vllm_omni/diffusion/models/minimax_h3/attention/video_context.py": "f4cfd2225514b6f99d36567879f06fb4a42f9d235ac0114fc643001b345cb22e", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "73c96090e6fb0e13ce734bc96b23472fd5adb83bceb71f3eba2815d6343a33e4", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "577e290c2d92fc02822cd516cc499c3c4d1f465145ad2a837edfdfeaa09ed9fb", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cff17568fbd856c28394d104d393a806931613b7b057f288006ad8c6fb9aa05a", + "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" + }, + "scope": "Legacy/pre-gate/DLO tests, three executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 157.24341464199824 + }, + { + "trial": "c23-video-context-lse-v2-operator-5614bf46", + "cleanup_passed": true, + "source_unchanged": true, + "operator_gate_passed": true, + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_context.py": "1e76074b71f0ea4e070c435db623add702d1414a351427fd86ebb40e8dfd67fd", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "48533c941a50719995206d18fa9198330824b940ecba9399b46ba7615b969ccb", + "vllm_omni/diffusion/models/minimax_h3/attention/video_context.py": "f4cfd2225514b6f99d36567879f06fb4a42f9d235ac0114fc643001b345cb22e", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "73c96090e6fb0e13ce734bc96b23472fd5adb83bceb71f3eba2815d6343a33e4", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "577e290c2d92fc02822cd516cc499c3c4d1f465145ad2a837edfdfeaa09ed9fb", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cff17568fbd856c28394d104d393a806931613b7b057f288006ad8c6fb9aa05a", + "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" + }, + "scope": "Real captured H3 attention operator benchmark; full-model quality and HTTP are separate; no full-model quality/performance acceptance", + "status": "completed", + "returncode": 0, + "elapsed_seconds": 85.06819563696627, + "operator_benchmark": { + "scope": "One captured static5s attention input over50calls/8refresh42reuse; includes packing/LSE merge/lifecycle, excludes FFN/model/HTTP and changing-input quality.", + "fixture_sha256": "00767e6afb2ba777bb2677eb9b60dc2f00fb649f5006ae8c1d77191214fbf912", + "fixture_tensor_sha256": { + "query": "313095dd143adf0636d57c90d77b689405dee510df3ebd6385a49d071c3323e9", + "key": "6638aee85f0039ee814206e7c976a2d1a9a2b185723c7bbb74c69b1e68c5bad4", + "value": "101eede996c8b13d61d487019d47f0b5fd2193664d8c178665063675cb66fdca" + }, + "query_shape": [ + 1, + 39808, + 14, + 128 + ], + "valid_kv": 39771, + "target_start": 2475, + "finite": true, + "relative_l2_vs_dense": 0.0020166223403066397, + "baseline_cycle_samples_ms": [ + 3477.4728709599003, + 3477.3380900151096, + 3478.938751970418, + 3477.9253639862873, + 3477.4123289971612 + ], + "candidate_cycle_samples_ms": [ + 1166.7012969846837, + 1168.3354200213216, + 1166.2831630092114, + 1165.7003490254283, + 1166.387434990611 + ], + "baseline_median_ms": 3477.4728709599003, + "candidate_median_ms": 1166.387434990611, + "reduction_pct": 66.45876249011114, + "actual_context_calls": { + "video_context_refreshes": 8, + "video_context_reuses": 42, + "fresh_video_prefix_calls": 50, + "peak_cache_bytes": 135757440 + }, + "untimed_validation": { + "finite_all50": true, + "max_relative_l2_all50": 0.0020166223403066397 + }, + "real_merge_replay": { + "bit_equal_v1_real_merge": true, + "output_sha256": "b1fa215f72bdfbddf11431ddec28cd2a06049e4bbbbb9b0857b6312713139919", + "elements": 66834432, + "bytes": 133668864 + }, + "operator_gate_passed": true, + "warmup_cycles_per_operation": 2, + "measured_cycles_per_operation": 5, + "order": "AB/BA/AB/BA/AB", + "probe_sha256": "6f57858263f0e9e0ddd32724b2276224081645cab6149b06e221df2d441cb886" + } + } + ], + "decisions": [ + "C12 fixed profile stopped: zero actual predictions.", + "C14 pinned block-FP8 and block-INT8 prerequisites failed; per-token INT8 is not excluded.", + "C13 numerical qualification and clean performance are separate.", + "C15 fixed FP16 video residual profile failed finite velocity at first reuse.", + "C16 FP32 residual completed but failed three frozen audio quality guards; fixed profile stopped.", + "C17 completed5s but failed11frozen quality checks; fixed window profile stopped.", + "C18 native17PASS but operation reduced only7.25%; no full model run.", + "C19 native17PASS; operation52.85%faster but relL2 .258805 exceeded fixed .258052screen; no model run.", + "C20 native17PASS; operation48.42%faster and relL2 .263361 failed both fixed primitive gates; no model run.", + "C21 native36PASS; full5s completed but failed7frozen quality checks; fixed zero-order profile stopped.", + "C22 native45PASS; full5s completed but failed9frozen quality checks; fixed secant profile stopped.", + "C23 V1 native44PASS; weighted attention cycle reduced38.255%, below unchanged50%operator gate; no model run for V1.", + "C23 component diagnosis found a bit-equal native Torch merge faster than manual Triton on real tensors; V2 native44PASS and weighted operator66.459%reduction passed, full-model quality and HTTP remain separate.", + "C23 full5s completed with134input checks and expected counters but failed8frozen quality checks; fixed profile stopped, no15s or clean timing expansion.", + "First C22 model stopped before startup on duplicate controller aliases; audited canonicalization retains all attempts and time charges." + ] +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c13-5s-strict-comparison.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c13-5s-strict-comparison.json new file mode 100644 index 000000000..9280a6dad --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c13-5s-strict-comparison.json @@ -0,0 +1,9 @@ +{ + "verdict": "PASS", + "checks": 439, + "failures": [], + "missing_count": 0, + "scope": "specified latent points, schedule, decoded RGB24/PCM; no performance acceptance", + "source_commit": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "performance_acceptance": "NOT_EVALUATED" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c15-native-qualification.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c15-native-qualification.json new file mode 100644 index 000000000..4e1d3d786 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c15-native-qualification.json @@ -0,0 +1,19 @@ +{ + "trial": "c15-video-row-native-3a9721a6", + "tests_passed": 17, + "source_base_head": "0917a23dac44fee3b4d156de5cfa5addc329cc4a", + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cf9fb13c5f24c7595d62d13cae6e283cafb2178612b20028d35357f0cafaac83", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "8ec1b3ae4cd252f5d95ab23e0232b983900939e8cb15d1e64f4740f7204abf39", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" + }, + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "scope": "Focused state/block/request tests, real Ref2VA/FL2VA packing and native four-rank subset attention. Full-model quality/performance remain separate.", + "first_model_attempt": "Failed before step0: image references are not reference video spans. Task propagation corrected; original failure retained." +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c16-5s-00-execution-proof.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c16-5s-00-execution-proof.json new file mode 100644 index 000000000..14308ba8c --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c16-5s-00-execution-proof.json @@ -0,0 +1,100 @@ +{ + "run": "c16-video-row-fp32-quality-5s-00-8600757a", + "source_commit": "e0a3462fcc44de698b5dedcf8ff2c99f9802f757", + "latency_diagnostic_ms": 197716.36760496767, + "clean_timing": false, + "cleanup_passed": true, + "helpers_unchanged": true, + "input_checks": { + "verdict": "PASS", + "checks": 134, + "failures": [], + "scope": "input/schedule/geometry/finite-state checks only; perceptual quality and latency acceptance are separate" + }, + "counters_by_rank": { + "0": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 4083750, + "peak_cache_bytes": 8039270400, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "1": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10700390400, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "2": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10700390400, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + }, + "3": { + "full_steps": 17, + "partial_steps": 33, + "full_blocks": 850, + "partial_blocks": 1650, + "subset_attention_calls": 1650, + "subset_query_rows": 4083750, + "omitted_query_rows": 61538400, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10660608000, + "closed": true, + "storage_dtype": "torch.float32", + "live_cache_bytes": 0 + } + }, + "step4_range": { + "records": 200, + "elements": 10025164800, + "fp32_nonfinite": 0, + "counterfactual_fp16_nonfinite": 166071, + "max_abs_delta": 2232320.0 + }, + "quality_verdict": "FAIL", + "observed_checks": 39, + "failures": [ + "5s-00:clap: similarity", + "5s-00: clap segment", + "5s-00: audio temporal envelope" + ], + "missing_samples": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "media_sha256": "eeb8b2dca5c467f90680986a545d874ae6df8fa558ebd1816974af5e692f2951", + "features_sha256": "170179e1c4916ae9317456206091d9822496bad04b941c42bfe74abd82db0bd2", + "decision": "Reject fixed C16 profile on three observed audio violations; no5s01/15s expansion or clean performance qualification." +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c16-native-qualification.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c16-native-qualification.json new file mode 100644 index 000000000..d69798b07 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c16-native-qualification.json @@ -0,0 +1,18 @@ +{ + "trial": "c16-video-row-fp32-native-d3fe320c", + "tests_passed": 18, + "source_base_head": "81c1beb6e47ecfa7602e4309a7342a38108aabc8", + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "c555d76f7bd6457a81238fca56100d11cb7ab043a4a649d55e23b6203d09bde2", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "1258655fcdca4692c2306100846761a2b0b149bbcaa5cde48a9abaa559913bc3", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "a8257f98c2e30c7bc7978bf645966d2eaa98cd49a42a8a923c062b5808dcdcdf", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849" + }, + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "scope": "Focused state/block tests and native four-rank subset attention; no full-model quality/performance acceptance" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c17-5s-00-execution-proof.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c17-5s-00-execution-proof.json new file mode 100644 index 000000000..81a3fd2c6 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c17-5s-00-execution-proof.json @@ -0,0 +1,75 @@ +{ + "run": "c17-video-window-quality-5s-00-a8c689e4", + "source_commit": "c1589a2b5e4ada617e7cca38ef7c300fe9495c58", + "latency_diagnostic_ms": 210397.12271699682, + "clean_timing": false, + "cleanup_passed": true, + "helpers_unchanged": true, + "input_checks": { + "verdict": "PASS", + "checks": 134, + "failures": [], + "scope": "input/schedule/geometry/finite-state checks only; perceptual quality and latency acceptance are separate" + }, + "counters_by_rank": { + "0": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + }, + "1": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + }, + "2": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + }, + "3": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 4900, + "_video_temporal_window_bnsd": 2400 + } + }, + "quality_verdict": "FAIL", + "observed_checks": 39, + "failures": [ + "5s-00:dino: similarity", + "5s-00:clip: similarity", + "5s-00:clap: similarity", + "5s-00: dino lower tail", + "5s-00: clap segment", + "5s-00:reference_clip_cosine: adherence", + "5s-00:reference_dino_cosine: adherence", + "5s-00:prompt_clip_cosine: adherence", + "5s-00: audio level", + "5s-00: audio temporal envelope", + "5s-00: motion energy" + ], + "missing_samples": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "decision": "Reject fixed C17 window profile; no15s or clean performance expansion." +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c17-native-qualification.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c17-native-qualification.json new file mode 100644 index 000000000..9a5138353 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c17-native-qualification.json @@ -0,0 +1,13 @@ +{ + "trial": "c17-video-window-native-9b48cb56", + "tests_passed": 15, + "source_base_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "66ca27991ce42ce302e5de078be6395c9d1ab53752378babef25089f3f75c3aa", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_window.py": "0bc59f35eff41164e125a5070e034c94ea60bf0112733ebe59eb6378eccbfed8" + }, + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "scope": "CPU window reference/routing and native batched BF16 FAS; no full-model quality/performance acceptance" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c18-native-qualification.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c18-native-qualification.json new file mode 100644 index 000000000..cc7a0daa9 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c18-native-qualification.json @@ -0,0 +1,13 @@ +{ + "trial": "c18-video-pooled-context-native-1673db72", + "tests_passed": 17, + "source_base_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "1febc030a7917b6e10fe287014fe2aacef76ade7bf58ebb604fdedc36fb7c21d", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pooled_context.py": "45a9ae5ba56858daee6a1088a8c8b1ed612ea9afd3c229c90a9b6db3c80a06b2" + }, + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c19-native-qualification.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c19-native-qualification.json new file mode 100644 index 000000000..d3a307edb --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c19-native-qualification.json @@ -0,0 +1,13 @@ +{ + "trial": "c19-video-global-pool-native-f0f4c502", + "tests_passed": 17, + "source_base_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ddfd6d6822cc6a5528fc95539f7f98dfdeed25d5489da3ed5cf76e6076a5ec94", + "tests/diffusion/models/minimax_h3/test_minimax_h3_global_pool.py": "5457db550b59e3373be8f5f21873c1b78ac357a6ac170c2603ee531b50e91467" + }, + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c20-native-qualification.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c20-native-qualification.json new file mode 100644 index 000000000..fc85c5389 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c20-native-qualification.json @@ -0,0 +1,13 @@ +{ + "trial": "c20-video-variance-pool-native-50101063", + "tests_passed": 17, + "source_base_head": "e89b1d40d37f7f72767892a56fa7980ea572ba8e", + "files_before": { + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "4390450bc5324f7028125101cd4bb32e9de237b04bc12fa879b48110d7122e3c", + "tests/diffusion/models/minimax_h3/test_minimax_h3_variance_pool.py": "e871fb4b222d61d10b6c442a333a52c976cceaa0d406233502ecbcaddfeba8c9" + }, + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "scope": "Independent weighted global-context reference and native padded-head BF16 FAS; no full-model quality/performance acceptance" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c21-5s-00-execution-proof.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c21-5s-00-execution-proof.json new file mode 100644 index 000000000..b657b2f8b --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c21-5s-00-execution-proof.json @@ -0,0 +1,139 @@ +{ + "run": "c21-video-pre-gate-quality-5s-00-7265c33e", + "source_commit": "d48fa51d03dd748568cbcf57ed40f533c691c9bd", + "latency_diagnostic_ms": 163805.5620979867, + "clean_timing": false, + "cleanup_passed": true, + "helpers_unchanged": true, + "input_checks": { + "verdict": "PASS", + "checks": 134, + "failures": [], + "scope": "input/schedule/geometry/finite-state checks only; perceptual quality and latency acceptance are separate" + }, + "counters_by_rank": { + "0": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 5197500, + "peak_cache_bytes": 8039270400, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + }, + "1": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10700390400, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + }, + "2": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10700390400, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + }, + "3": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 0, + "peak_cache_bytes": 10660608000, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + } + }, + "actual_paths_by_rank": { + "0": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "1": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "2": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "3": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + } + }, + "quality_verdict": "FAIL", + "observed_checks": 39, + "failures": [ + "5s-00:dino: similarity", + "5s-00:clip: similarity", + "5s-00:clap: similarity", + "5s-00: dino lower tail", + "5s-00: clap segment", + "5s-00:reference_dino_cosine: adherence", + "5s-00: audio temporal envelope" + ], + "missing_samples": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "media_sha256": "c03567a778ed6aae6a8b8e05674e6d83065178871751603ba3ffe2f7a363a4ce", + "features_sha256": "71638d355adff149f27d135fceb8b0b0e2de34557a26344113218adcb2a6ea75", + "decision": "Reject fixed C21 eight-anchor zero-order pre-gate profile on seven observed quality violations; no15s or clean timing expansion." +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c21-c22-direction-identity-correction.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c21-c22-direction-identity-correction.json new file mode 100644 index 000000000..284057775 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c21-c22-direction-identity-correction.json @@ -0,0 +1,79 @@ +{ + "checked_at": "2026-11-02T06:55:00.775781+00:00", + "before_count": 24, + "after_count": 23, + "before_directions": [ + "C0-upstream-migration", + "C1-BF16-attention-execution", + "C2-HSDP2-replicate2", + "C3-Cache-DiT-residual-reuse", + "C4-dense-BF16-attention-alternatives", + "C5-H3-Laser-video-queries-BF16-context", + "C6-Laser-input-range-scaling", + "C7-native-RainFusion-BF16", + "C8-reference-KV-device-refresh2", + "C9-TP4-resident-DiT", + "C10-H3-audio-cache-veto", + "C11-H3-modality-sigma-residual", + "C12-H3-state-aligned-residual", + "C13-C1-DLO-QKV-order", + "C14-A3-quantized-attention-prerequisites", + "C15-H3-target-video-row-cache", + "C16-H3-video-row-FP32-residual", + "C17-H3-video-temporal-window", + "C18-H3-video-global-pooled-context", + "C19-H3-video-shared-global-pool", + "C20-H3-video-variance-pool", + "C21-H3-video-row-FP32-residual", + "C21-H3-video-pre-gate-cache", + "C22-H3-video-row-FP32-residual" + ], + "canonical_directions": [ + "C0-upstream-migration", + "C1-BF16-attention-execution", + "C2-HSDP2-replicate2", + "C3-Cache-DiT-residual-reuse", + "C4-dense-BF16-attention-alternatives", + "C5-H3-Laser-video-queries-BF16-context", + "C6-Laser-input-range-scaling", + "C7-native-RainFusion-BF16", + "C8-reference-KV-device-refresh2", + "C9-TP4-resident-DiT", + "C10-H3-audio-cache-veto", + "C11-H3-modality-sigma-residual", + "C12-H3-state-aligned-residual", + "C13-C1-DLO-QKV-order", + "C14-A3-quantized-attention-prerequisites", + "C15-H3-target-video-row-cache", + "C16-H3-video-row-FP32-residual", + "C17-H3-video-temporal-window", + "C18-H3-video-global-pooled-context", + "C19-H3-video-shared-global-pool", + "C20-H3-video-variance-pool", + "C21-H3-video-pre-gate-cache", + "C22-H3-video-pre-gate-sigma-secant" + ], + "aliases": { + "C21-H3-video-row-FP32-residual": "C21-H3-video-pre-gate-cache", + "C22-H3-video-row-FP32-residual": "C22-H3-video-pre-gate-sigma-secant" + }, + "scope": "Controller label correction for two existing mechanisms; no new direction/budget approval inferred.", + "cost_policy": "Retain all raw attempts, failures and retries. Original start/deadline/device-hour ceiling unchanged; wall-clock accounting includes this repair.", + "source_binding": { + "C21": "d48fa51d03dd748568cbcf57ed40f533c691c9bd", + "C22": "7b0d12bfefdf05868e93e74bc91f04c70ee34128" + }, + "retained_failed_model": "c22-video-pre-gate-secant-quality-5s-00-72202558", + "controllers": { + "C21": { + "before_sha256": "1ca526f2040b5c53d091068766dac0346c79268c4f5ae0b10674e8d2a97b3a40", + "after_sha256": "a3ab34ba341fad6b1e3041c8b06716cc669a3fe0171bad2b6e4a6693e54d0780", + "original_bytes": "run_c21_native_checks-before-family-alias.py" + }, + "C22": { + "before_sha256": "3e58beffbe688a15cfa73a00ec8fbf20c6e78a25235a002422898f6506cae1d2", + "after_sha256": "4a9175a8f8db782df06f14af6a0329368da27ccb5ea776a02ce97ad86df0dd33", + "original_bytes": "run_c22_native_checks-before-family-alias.py" + } + } +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c21-native-qualification.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c21-native-qualification.json new file mode 100644 index 000000000..f29e129c6 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c21-native-qualification.json @@ -0,0 +1,29 @@ +{ + "trial": "c21-video-pre-gate-native-3c5eebbe", + "tests_passed": 36, + "source_base_head": "d027b4d0c53bf40876215d4e3856bc17450a4c39", + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "8933b536317652d0dbc7a974ea7e0bca928e7b3bdf438988999ff5a498a6ec9d", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "019bf71106ba3263b55ad7b682064e3a176eecb81725cb48bbf178387356d5d9", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "04a9212994b5c8593f8d196820300ded077fed9af4274b64d87d9e6bf71eae07", + "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" + }, + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "scope": "Legacy/pre-gate/DLO tests, two executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c22-5s-00-execution-proof.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c22-5s-00-execution-proof.json new file mode 100644 index 000000000..287869077 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c22-5s-00-execution-proof.json @@ -0,0 +1,151 @@ +{ + "run": "c22-video-pre-gate-secant-v2-quality-5s-00-777147cb", + "source_commit": "7b0d12bfefdf05868e93e74bc91f04c70ee34128", + "latency_diagnostic_ms": 166858.99333201814, + "clean_timing": false, + "cleanup_passed": true, + "helpers_unchanged": true, + "input_checks": { + "verdict": "PASS", + "checks": 134, + "failures": [], + "scope": "input/schedule/geometry/finite-state checks only; perceptual quality and latency acceptance are separate" + }, + "counters_by_rank": { + "0": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 5197500, + "peak_cache_bytes": 12061056000, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate_secant", + "secant_forecasts": 4200, + "slope_codec": "int8_per_channel", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + }, + "1": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 0, + "peak_cache_bytes": 16052736000, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate_secant", + "secant_forecasts": 4200, + "slope_codec": "int8_per_channel", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + }, + "2": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 0, + "peak_cache_bytes": 16052736000, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate_secant", + "secant_forecasts": 4200, + "slope_codec": "int8_per_channel", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + }, + "3": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 2100, + "subset_query_rows": 5197500, + "omitted_query_rows": 78321600, + "partial_mlp_rows": 0, + "peak_cache_bytes": 15993062400, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "pre_gate_secant", + "secant_forecasts": 4200, + "slope_codec": "int8_per_channel", + "fresh_attention_gates": 2100, + "fresh_mlp_gates": 2100, + "live_cache_bytes": 0 + } + }, + "actual_paths_by_rank": { + "0": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "1": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "2": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "3": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + } + }, + "quality_verdict": "FAIL", + "observed_checks": 39, + "failures": [ + "5s-00:dino: similarity", + "5s-00:clip: similarity", + "5s-00:clap: similarity", + "5s-00: dino lower tail", + "5s-00: clap segment", + "5s-00:reference_clip_cosine: adherence", + "5s-00:reference_dino_cosine: adherence", + "5s-00:audio_caption_clap_cosine: adherence", + "5s-00: motion energy" + ], + "missing_samples": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "media_sha256": "d0f6bb62948680a171e7368ec9d6b3ad916fb856b9e0969fc437b292aae7d1e5", + "features_sha256": "c3121d4fe81f3eadd50c4b720cf161b5ba6fa9f6596828395520ede1d296bcba", + "decision": "Reject fixed C22 sigma-secant pre-gate profile on nine observed violations; no15s or clean timing expansion.", + "prior_failed_attempt": "c22-video-pre-gate-secant-quality-5s-00-72202558", + "direction_identity_correction": "c21-c22-direction-identity-correction.json" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c22-native-qualification.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c22-native-qualification.json new file mode 100644 index 000000000..b2271d113 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c22-native-qualification.json @@ -0,0 +1,30 @@ +{ + "trial": "c22-video-pre-gate-secant-native-3521747e", + "tests_passed": 45, + "source_base_head": "d48fa51d03dd748568cbcf57ed40f533c691c9bd", + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "b8394d22b054d60c1ab8e069e9600e1947cc73b5826e1f206e66c548123f632d", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_secant.py": "70a215fbb0dc63b90652d96f8f6d106a5785bd7d64dae75a2c4f6ec5c627ac1c", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "ca87df1936fd2694fcc768079e9e8f926b6d45969ddc61ebf353f5e726d70b8b", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "b991ebb5d43242357d5ab8cfb476808a76fc0658d6048db6f3b7131122c8a993", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "8933b536317652d0dbc7a974ea7e0bca928e7b3bdf438988999ff5a498a6ec9d", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "3904bc8ac6679bc51614b95f5510789331a41bf20a43a517f3a99d33505201a6", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "e21e0afe8f8948d805d374428a34f3b2c5c0b5041d63ea791f047af8f455cf16", + "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" + }, + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "scope": "Legacy/pre-gate/DLO tests, three executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c23-5s-00-execution-proof.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c23-5s-00-execution-proof.json new file mode 100644 index 000000000..1858c95cf --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c23-5s-00-execution-proof.json @@ -0,0 +1,157 @@ +{ + "run": "c23-video-context-lse-quality-5s-00-704b6bb0", + "source_commit": "6eb7bb094effb40853431208d1631effaf452a3f", + "latency_diagnostic_ms": 199064.918508986, + "clean_timing": false, + "cleanup_passed": true, + "helpers_unchanged": true, + "source_unchanged": true, + "input_checks": { + "verdict": "PASS", + "checks": 134, + "failures": [], + "scope": "input/schedule/geometry/finite-state checks only; perceptual quality and latency acceptance are separate" + }, + "counters_by_rank": { + "0": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 0, + "subset_query_rows": 0, + "omitted_query_rows": 0, + "partial_mlp_rows": 0, + "peak_cache_bytes": 6787872000, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "video_context_lse", + "video_context_refreshes": 400, + "video_context_reuses": 2100, + "fresh_video_prefix_calls": 2500, + "full_row_mlp_blocks": 2500, + "fresh_attention_gates": 0, + "fresh_mlp_gates": 0, + "live_cache_bytes": 0 + }, + "1": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 0, + "subset_query_rows": 0, + "omitted_query_rows": 0, + "partial_mlp_rows": 0, + "peak_cache_bytes": 6787872000, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "video_context_lse", + "video_context_refreshes": 400, + "video_context_reuses": 2100, + "fresh_video_prefix_calls": 2500, + "full_row_mlp_blocks": 2500, + "fresh_attention_gates": 0, + "fresh_mlp_gates": 0, + "live_cache_bytes": 0 + }, + "2": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 0, + "subset_query_rows": 0, + "omitted_query_rows": 0, + "partial_mlp_rows": 0, + "peak_cache_bytes": 6787872000, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "video_context_lse", + "video_context_refreshes": 400, + "video_context_reuses": 2100, + "fresh_video_prefix_calls": 2500, + "full_row_mlp_blocks": 2500, + "fresh_attention_gates": 0, + "fresh_mlp_gates": 0, + "live_cache_bytes": 0 + }, + "3": { + "full_steps": 8, + "partial_steps": 42, + "full_blocks": 400, + "partial_blocks": 2100, + "subset_attention_calls": 0, + "subset_query_rows": 0, + "omitted_query_rows": 0, + "partial_mlp_rows": 0, + "peak_cache_bytes": 6787872000, + "closed": true, + "storage_dtype": "torch.bfloat16", + "cache_stage": "video_context_lse", + "video_context_refreshes": 400, + "video_context_reuses": 2100, + "fresh_video_prefix_calls": 2500, + "full_row_mlp_blocks": 2500, + "fresh_attention_gates": 0, + "fresh_mlp_gates": 0, + "live_cache_bytes": 0 + } + }, + "actual_attention_counts": { + "0": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "1": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "2": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + }, + "3": { + "dlo._issue_chunked_allgather.ungated": 100, + "dlo._allgather_after_attn_a2a": 2600, + "dlo._issue_chunked_allgather.gated": 2600, + "dlo._flush_deferred_allgather": 2600, + "_prefix_bnsd": 2500 + } + }, + "quality_verdict": "FAIL", + "observed_checks": 39, + "failures": [ + "5s-00:dino: similarity", + "5s-00:clip: similarity", + "5s-00:clap: similarity", + "5s-00: clap segment", + "5s-00:reference_dino_cosine: adherence", + "5s-00:audio_caption_clap_cosine: adherence", + "5s-00: audio level", + "5s-00: audio temporal envelope" + ], + "missing_samples": [ + "5s-01", + "5s-02", + "5s-03", + "5s-04", + "15s-00", + "15s-01", + "15s-02", + "15s-03", + "15s-04" + ], + "media_sha256": "f3a34a7bf46a2049b154edacab0bd0c9ae254c6dcdee02c50133a25c987ab5c5", + "features_sha256": "e10cfd296ebc7d38a8d36c0686b94b5ad7e2732751268006a61517f731c8fbfa", + "decision": "Reject fixed C23 profile on observed frozen quality violations; no15s or clean timing expansion." +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c23-native-qualification.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c23-native-qualification.json new file mode 100644 index 000000000..c389bb8c2 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c23-native-qualification.json @@ -0,0 +1,31 @@ +{ + "trial": "c23-video-context-lse-v2-native-dc375564", + "tests_passed": 44, + "source_base_head": "d48fa51d03dd748568cbcf57ed40f533c691c9bd", + "files_before": { + "tests/diffusion/distributed/test_collective_gate.py": "6a02f32882d85eca1e3bb9a023ce3adb3d5b59c213845e11e85c96a89432f477", + "tests/diffusion/models/minimax_h3/test_minimax_h3_pre_gate_cache.py": "a4924d0b3dd58ae413259edba6cf2487b3e41d391f943fb74358934be72eb53b", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_context.py": "1e76074b71f0ea4e070c435db623add702d1414a351427fd86ebb40e8dfd67fd", + "tests/diffusion/models/minimax_h3/test_minimax_h3_video_row_cache.py": "e09b872f6900e821d42632a0ca09595f2affe6f9794049dbc74d69ba6a52b2b0", + "vllm_omni/config/omni_config.py": "6894120a330474f718d43bf2bf7928892eb996152839ce6469c34d18a7df09d1", + "vllm_omni/config/stage_config.py": "0752e8cc1c6f1021f737830d08663cfc118923a83fd909c607f9ec3ab2a3d2fb", + "vllm_omni/diffusion/attention/layer.py": "76c5fce3dabd23e39a8890316e1691c03bc203066cb60af5611ec4790eefc632", + "vllm_omni/diffusion/data.py": "5db83d5c71cb27cc7810d35fcfc891da80f0b003adbb16bdecc3816a63305e11", + "vllm_omni/diffusion/distributed/collective_gate.py": "01c6317cfa615840681b8bb8b8513fa3736d26e845a24ad9e1ab7f87489b962f", + "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "48533c941a50719995206d18fa9198330824b940ecba9399b46ba7615b969ccb", + "vllm_omni/diffusion/models/minimax_h3/attention/video_context.py": "f4cfd2225514b6f99d36567879f06fb4a42f9d235ac0114fc643001b345cb22e", + "vllm_omni/diffusion/models/minimax_h3/denoise_loop.py": "c5fab405ee82bc553e9a902fa27e6dfbc5213ec18311e17b6415396a69b50f43", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "73c96090e6fb0e13ce734bc96b23472fd5adb83bceb71f3eba2815d6343a33e4", + "vllm_omni/diffusion/models/minimax_h3/minimax_h3_transformer.py": "577e290c2d92fc02822cd516cc499c3c4d1f465145ad2a837edfdfeaa09ed9fb", + "vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py": "e8e2e74fb8edb05e02d3e2328ea1f3ad5ddcfd7a10ed09c5cf3911b1d063e849", + "vllm_omni/diffusion/models/minimax_h3/video_row_cache.py": "cff17568fbd856c28394d104d393a806931613b7b057f288006ad8c6fb9aa05a", + "vllm_omni/diffusion/offloader/base.py": "23922136deaf54e2d98e827b007b34c7e3ad36171b71bfe9060d390efc0b679f", + "vllm_omni/diffusion/offloader/distributed_layerwise_backend.py": "19fbce0f1036e4784d68cab798c499526a9099101909e41ce96dd0caa033b6a4", + "vllm_omni/engine/arg_utils.py": "52569e3688cc708729a48ca0a2c875c28193733c2f934f6ff89f6c815e4009c7", + "vllm_omni/entrypoints/cli/serve.py": "f8284ad6b751ba4a9e4f7254d78132cc2cfd49871a0f9af7fde49c6789d79cb0" + }, + "cleanup_passed": true, + "source_unchanged": true, + "required_native_test_executed": true, + "scope": "Legacy/pre-gate/DLO tests, three executed native four-rank HCCL/subsetFA suites and fresh current gates; no full-model quality/performance acceptance" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c23-retired-proof.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c23-retired-proof.json new file mode 100644 index 000000000..c80ca79fd --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c23-retired-proof.json @@ -0,0 +1,31 @@ +{ + "checked_at": "2026-11-02T15:48:59.537140+08:00", + "selected_logical_devices": [ + 2, + 3, + 4, + 5 + ], + "selected_devices_idle": true, + "ports_free": { + "19193": true, + "19194": true, + "29597": true + }, + "source_head": "6eb7bb094effb40853431208d1631effaf452a3f", + "source_clean": true, + "tested_files_unchanged": true, + "helper_and_validators": { + "scripts/h3_diagnostics.py": "514d1c735f204693be2af7e4f4fbbba064e2655a59b3c59ea62c35755802fae7", + "validator/strict_compare.py": "b83f2bff553ff6ca2970a16e25525cac5d9523612e9c05ea563852c814019028", + "quality-tools/score_media.py": "99f5a76436a3c8d37462fab83f63d24eae74b4458c30e5a00b60c19fde139f3e", + "quality-tools/compare_quality.py": "99e8e23463f8ce83b52a22de9cdc697ef4bbf14d910a0c67e76bfd01e94c4b3d" + }, + "runtime_diagnostic_version": 17, + "charged_direction_families": 24, + "direction_ceiling": 24, + "deadline": "2026-11-03T01:19:00+08:00", + "candidate_quality_verdict": "FAIL", + "accepted_50pct_http_gain": false, + "scope": "Retire rejected C23 fixed profile; preserve evidence and original budget. No new search direction launched." +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/c4-pfa-operator.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c4-pfa-operator.json new file mode 100644 index 000000000..4994b1ceb --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/c4-pfa-operator.json @@ -0,0 +1,71 @@ +{ + "scope": "Single real5s first-long-attention fixture,one logical device; not model performance", + "cases": [ + { + "name": "pfa_bnsd_prefix", + "bit_exact": false, + "unequal_elements": 1577651, + "compared_elements": 71269632, + "finite": true, + "reference_sha256": "7cabc1030b41d2e42c77aac352c20a2b676e194d1f9883b54d5c9b8f72b38c21", + "candidate_sha256": "8487659367fd0776592f548aa9142d5a9631ab4b8c2877df692de76f1166c23c", + "relative_l2": 0.0005370235303416848, + "maximum_absolute_error": 0.03125, + "baseline_ms": [ + 72.55231699673459, + 71.770221053157, + 71.50708901463076, + 71.71859999652952, + 71.54928898671642, + 72.0424129976891, + 71.68136001564562 + ], + "candidate_ms": [ + 85.81408398458734, + 85.6127529987134, + 85.21084999665618, + 85.61692299554124, + 85.85116500034928, + 84.94565699947998, + 85.60653298627585 + ], + "baseline_median_ms": 71.71859999652952, + "candidate_median_ms": 85.6127529987134, + "reduction_pct": -19.37315145981129 + }, + { + "name": "pfa_bnsd_head_batches", + "bit_exact": false, + "unequal_elements": 1577651, + "compared_elements": 71269632, + "finite": true, + "reference_sha256": "7cabc1030b41d2e42c77aac352c20a2b676e194d1f9883b54d5c9b8f72b38c21", + "candidate_sha256": "8487659367fd0776592f548aa9142d5a9631ab4b8c2877df692de76f1166c23c", + "relative_l2": 0.0005370235303416848, + "maximum_absolute_error": 0.03125, + "baseline_ms": [ + 72.57743796799332, + 71.2658169795759, + 71.57570996787399, + 71.21048599947244, + 70.94532495830208, + 71.41773804323748, + 71.54709001770243 + ], + "candidate_ms": [ + 86.40253898920491, + 85.96450602635741, + 86.10950701404363, + 86.288328981027, + 85.96854598727077, + 85.93391597969458, + 86.74239204265177 + ], + "baseline_median_ms": 71.41773804323748, + "candidate_median_ms": 86.10950701404363, + "reduction_pct": -20.57159659958918 + } + ], + "campaign_verdict": "INCONCLUSIVE", + "decision": "Both nativePFA layouts are non-bit-exact and slower by19.4/20.6%; reject,do not integrate." +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/installed-foca-source-check.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/installed-foca-source-check.json new file mode 100644 index 000000000..58af866da --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/installed-foca-source-check.json @@ -0,0 +1,22 @@ +{ + "verified_utc": "2026-11-01T18:28:58Z", + "module": "cache_dit/caching/cache_contexts/calibrators/foca.py", + "sha256": "258119cffacf83da942d278a29e9a40a1b1f3ffbbf30359bda41bc4af0b0a804", + "bytes": 792, + "distribution_metadata": [ + { + "distribution_directory": "cache_dit-1.5.0.dist-info", + "fields": [ + "Name: cache_dit", + "Version: 1.5.0" + ] + } + ], + "not_implemented_methods": [ + "reset_cache", + "approximate", + "mark_step_begin", + "update" + ], + "scope": "static source check only; not an available H3 configuration or a device experiment" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/mindie-quant-api-source-check.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/mindie-quant-api-source-check.json new file mode 100644 index 000000000..a7320bf69 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/mindie-quant-api-source-check.json @@ -0,0 +1,10 @@ +{ + "installed_label": "3.1.0", + "installed_source_revision": "UNKNOWN", + "installed_quant_attention_export_and_file": false, + "private_source_commit": "ee3920f9cd61030b90e5dcb1a0a432e52810251d", + "public_api_introduction_commit": "76f8761794a0837f4fae3b5efbe52ccdd9847816", + "public_api_introduction_date": "2026-09-14", + "source": "https://github.com/Ascend/MindIE-SD/commit/76f8761794a0837f4fae3b5efbe52ccdd9847816", + "decision": "The API exists in dev; a version label alone does not establish wheel contents or 910C native support." +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/native-cli-doctor.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/native-cli-doctor.json new file mode 100644 index 000000000..29e669a44 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/native-cli-doctor.json @@ -0,0 +1,15 @@ +{ + "doctor_passed": true, + "native_cli_returncode": 0, + "parsed_record_count": 16, + "unique_physical_npu_count": 8, + "requested_physical_npu_ids": [ + "1", + "2" + ], + "execution_visible_logical_ids": "2,3,4,5", + "native_inventory_unchanged": true, + "compatibility": "unverified", + "first_wrapper_failure": "Own check incorrectly expected8rawrecords; nativeCLIitselfreturned0. DuplicatedphysicalIDsare retained in the raw16chiprows.", + "scope": "CLI discovery/targetPythonmetadata only; no pairedbenchmark or backendcompatibility acceptance" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/publication-integrity.json b/evidence/h3-ref2va-910c-4logical-20261101-297/results/publication-integrity.json new file mode 100644 index 000000000..eeb70ac92 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/publication-integrity.json @@ -0,0 +1,97 @@ +{ + "patch/c1-h3-bf16-prefix.patch": "afecd661c342860a563ad7c45af81560f16d4ccf10ab71f5ca3b33cdb2691d00", + "patch/c1-source-identity.json": "c6b90d9e91040a96c8761af2823110134593c5aa883847ef7c91594cd8025dec", + "patch/c10-h3-audio-cache-guard.patch": "626cd6267e7eec35efabb53a67d281303b05dc1f978188085521a35fbeba02fc", + "patch/c10-source-identity.json": "12bc7d5c39482cae653c8db9a43766cb9b63662565172562a84e057a5364425d", + "patch/c11-fp32-source-identity.json": "ea6fbce617f372dae895b08f6bfa35012ba4835c0e747762ba20b693a2a108f7", + "patch/c11-modality-sigma-residual.patch": "b2458c0dad78e479d17887cd86be7800181fbbe4751f4426d17587e41046ddea", + "patch/c11-sigma-fp32-residual.patch": "f8308af2c3f0cccc3b8c48f0a3d584dd6fd68ee4c8c87c0bc3fb8f67e239bd95", + "patch/c11-source-identity.json": "b8475b5e02073e4dfccf893d852d4139efdad10b5505b620e4782ede4b94032d", + "patch/c12-candidate.patch": "cb54ff0e336903290a9af8a48a6f06c0893747c7ba2be763886a9100f8cbe9d8", + "patch/c12-source-identity.json": "da6a05d52f3dfdff204d77e34ef3448f38f21cfe8496fd08c9be2a7e88f2e6fe", + "patch/c13-candidate.patch": "79c836a3919b9bcf7d2d01f3c1d496a193bf72484c01d4284daaa967b622b3e2", + "patch/c13-source-identity.json": "6ecfb0b0f9e5f3279e71b0c28ebdbbd46ab75b45ea965e717328042963cbba0e", + "patch/c15-candidate.patch": "6b98e4dd9938ed387c5d07180470233f5172c87fdd6c5636bd9a7b5141ebf7d5", + "patch/c15-source-identity.json": "8ac49340c964e3925542fc7c192024095d6983ab3508c3006bf6b192bc6b9557", + "patch/c16-candidate.patch": "6d9b2d42d43f47bc0185ffbdd6aca79405664b10f62652733ea74c2b4c07eee7", + "patch/c16-source-identity.json": "4884d8cfc3799651a96d84a315a64040fb90eeb7975168f13119ee84d00ce217", + "patch/c17-candidate.patch": "1fcfc4d1fdae4b6492ccd6556cb61e1f361ebba33b0c7defeab922d0797f6f16", + "patch/c17-source-identity.json": "bf663a65aad387fe6c962284a7d95de063295827981c63e2d1234399448bfa47", + "patch/c18-candidate.patch": "c338d2efe83ab6c6508849a1abe1df0f11d55c7d28753835a45dc996472d7daa", + "patch/c18-source-identity.json": "56c70cc3d09bc7c4478b67e6e629b3bf6987de3ad96941a23f0a110075c5836b", + "patch/c19-candidate.patch": "5a4b0e2083bbbd1a74eb12e480d2d18aff600b7baff7b41f43034522b25cc328", + "patch/c19-source-identity.json": "0d52e081037ab827ddebefe20b1517845edaaa498b2b6308bdd4fe3176dbd71e", + "patch/c20-candidate.patch": "da86f4c6ebbf5005f6daa8c8ecadcb77cbfa4f0c7cf39cb096fea4dd427a2a4d", + "patch/c20-source-identity.json": "85e44ab21bbb463f35e6211b472d8405ed278302ed3cd0c3edb8d17aa712671d", + "patch/c21-candidate.patch": "fe6bce768eb90fcd8f5b5f49327c2ea6ee3f2c6078dcffd2f58d3934ec74a6c6", + "patch/c21-source-identity.json": "0cdca3f7764c05abdcd4b7cf2fb3349fd12a82da927baf479c59bab0b049a25c", + "patch/c22-candidate.patch": "c4e2988f91057f896a0ac297a507dc8fc7853eaf5da112d1a066aa4c344e9967", + "patch/c22-source-identity.json": "b44d3b8d6e285618831b1e25101d49ad16188bb08e86559595c33c70c7eeedea", + "patch/c23-candidate.patch": "cd1db1acacf5fe41afedb775ec618f369d4960d7deda9478eaffd51752a6e7c4", + "patch/c23-source-identity.json": "c85034206b2e544efd455249ab2add7f41ea41bc27241f36f605053fe0493f96", + "patch/README.md": "d3ebf520d18b3aa305ab3c3e74f4b175888d85270cd1293662fa43e0b217f1cd", + "plan/c11-modality-sigma-residual.json": "4a52e72c1b48f4bbd09be742148720c64f528bb09068dcb0c5f4b50930621eb2", + "plan/c11-sigma-fp32-residual.json": "7a3104c1b23c3a89d97ec2a398ad703e31f050f41fd80efe4a0f46e209493c0f", + "plan/execution-checkpoint.json": "8decf6192d1e99d6665efff3af375909ac8f223ac6dc81a0d4077ca43c920416", + "plan/frozen-lossless-contract-v1.json": "b3ec4b2f58d1f9de4475977ed4bbdd75e4d38cad3857244fe0cfd842f7469cab", + "plan/frozen-lossy-contract-v1.json": "b8fd9e634e6db5ebfbce3842f3360f1b2241a08119953ee71bfc26553dbbafed", + "plan/native-cli-910c-device-map.json": "bfe453f783c51b747920255113fbc7abbf4f6a1523e10b708e703ca1448e8eb3", + "plan/README.md": "930282bf6d990c86930d8a11595dcdbb33995fd1c927060215507830ca5ba00e", + "plan/reproduction.md": "5afacd7b430a292af56322543d583da4d3db52369b5ca2fcef2f3434ee8a4601", + "plan/requester-decisions.json": "e62c5398fb1a5cabf4c51e2609d890d1d68b15a84fb00d08167e7ccefe341ccb", + "README.md": "d32ccc8a727e6587a54c002d8234033f00cd773aff0687009335a100f1b53bb4", + "results/automatic-quality-checkpoint.json": "19f896ca61ad692082ac8c714b8aaa05f85ae96e525c25cfa1696fc59f22596e", + "results/c1-15s-execution.json": "ba790d7d9e352fd27848104e235a82dad25e758b2801578e0576eaf93417c8b0", + "results/c1-15s-strict-comparison.json": "72b196500c0f045eb78c4e994279189c36b59c48f103e0ab4f94ff51968c62be", + "results/c1-5s-strict-comparison.json": "f333a0cf8d65d59e92d67a3882608a344a00f57542ab5a22e832c6800d81a4cb", + "results/c1-clean-5s-screening.json": "11bf3c1c71fc78d1c83c62e5244cf02f14d2d5eebefb18dbf85477211070d277", + "results/c1-queryfold-operator.json": "273cdcab38055d28925c1209f5b3b8ea59264a674b089abee586ba005d8cdb5b", + "results/c1-upstream-main-native.json": "3b9e08044b52b0b7e477225d7ab6fecfc4db50e7411cf974a83eb3b0bf129b09", + "results/c10-5s-00-audit.json": "9248fac9a87d76e01d9eb512a35dce7794496f6baa5598643bc3c8a3f14acde1", + "results/c10-5s-01-audit.json": "e96f01f455f42363deb3921be5332bc60694148e5ae12915f351713c16b7b3fd", + "results/c10-native-tests.json": "023c10a332567bf98507fd9ae91acee1cfb498aa43896f5f58e9edc17ab016cd", + "results/c11-fp32-15s-00-audit.json": "d18b0a6cb1492f815747eb4c8bce664db9062527676f8ac8c999e6d479e95da8", + "results/c11-fp32-15s-first-step.json": "f8d5d696dcbf3b725b132188325644688adc8ff1b0c95267585ef3a31bfe4b11", + "results/c11-fp32-5s-00-audit.json": "c1062b404ad9c82430aecac4fcad42fc7bbffc4e38ca33ca470dfadacb3ff804", + "results/c11-fp32-5s-first-step.json": "706cc16402428315e0d8a3d1312f9d40b02250a56c606321c27374de707a6e0c", + "results/c11-fp32-native-execution.json": "74bd1021b2fcadf00ac5a72503dc5d1ad8e388419faf58b5693376c0d8018567", + "results/c11-full-compute-first-step.json": "10f847c6eb4fc446853708148596445e2340129dd5acfebf8d18b32966f889ab", + "results/c11-native-execution.json": "e226442b31c54acbd8a1dcc1921598ae353b7839fd9e9cee771414d748165562", + "results/c11-original-5s-00-audit.json": "1f8d96141fa286655a3c6224853fd59408a7847fbf1eb16e2b299168f8637973", + "results/c12-through-c15-execution-checkpoint.json": "ceb8db8f9dbb4154db8b3b2ec3ac62c02e7dc8b0e59afdf581b12e3241b58d4d", + "results/c12-through-c17-execution-checkpoint.json": "c9ecb3988ed19ff2abfec58c87b46b69f86eda7571c35ce9c85dfc04505cd3cf", + "results/c12-through-c19-execution-checkpoint.json": "db77306a076f5118df38106c3971d0217f92cf369f7ca8905a5552d47806a71b", + "results/c12-through-c20-execution-checkpoint.json": "124cdd33a46aad0d81a32dc6c0f26d14315ea62d0387af9f045888b68ecf28ca", + "results/c12-through-c21-execution-checkpoint.json": "9f4d5d4a091dfedc24addc36bc0e0f2a6a71486c1ad8bfcd3a7ad84a2ad1f0f3", + "results/c12-through-c22-execution-checkpoint.json": "37edc3c7503a175b2be2d410c199f23e9bbde00fa88b6cc970199fdab8662bf0", + "results/c12-through-c23-execution-checkpoint.json": "f87cc234efc10adb703d01124c76d2076b46d5251d142ad178b3dc6590079f6b", + "results/c13-5s-strict-comparison.json": "563d1ac0411ed3fa78b78556f521f11bfc510fb1fdef67291d09b9b9e2fa3884", + "results/c15-native-qualification.json": "148f26ac90d2fc151f94efc819f4463da1d3afc2cfc48337eb35ebd14d2a59fa", + "results/c16-5s-00-execution-proof.json": "3d113052ae2584fa22cb6a5e1b566715e22c57a89453469e8a96eb4dbd630b40", + "results/c16-native-qualification.json": "70291bdde95ca6edb2bdf5b33e0612c4b6f0605529c02083ff97efe6adccbac0", + "results/c17-5s-00-execution-proof.json": "2fb3367b8baff791c8be3a5ad6bd64a08631074828862f176353b7147a859488", + "results/c17-native-qualification.json": "e109f84ff1c718a339cbc42e2bf3eb6b5a0381be21ac761bf20f7ecd8d4f4ee5", + "results/c18-native-qualification.json": "dfd17be0be798c1586f9aa343a55aa441bde667254ad77571762904476af4216", + "results/c19-native-qualification.json": "946eaec18d2104c1c12cd468d8abec3f0b6a36a7f84635300d8c44db268de262", + "results/c20-native-qualification.json": "23f1ff63ff080dbc11ea44ce5e4abb6e8eeb27aa6555f612fbcf744f5e4b2403", + "results/c21-5s-00-execution-proof.json": "a1295ce47e68ec2abb623dc3ab34b2ceb9c583a00bf79a11d54c10d4c37b1b44", + "results/c21-c22-direction-identity-correction.json": "42fbf61b4cd487fffc55361d53f3399f51ea00d75a4232755bddd5e07f4f0c12", + "results/c21-native-qualification.json": "280d086e2f3d04fc9e86bd92eee9c15df65298349a97260d3d2bde9d920469a4", + "results/c22-5s-00-execution-proof.json": "4f3044a8892d33a5d62f597dc6210149195b1e5416924cd6078d821779066f93", + "results/c22-native-qualification.json": "d259bbde6732f8a3b560eee64af417335afe46a7cf1d78cbd8d4d45b446e5ee0", + "results/c23-5s-00-execution-proof.json": "02ceb72f9c125b842bf89a6654e24816da496c5487498e6115462b37f961c72a", + "results/c23-native-qualification.json": "f5538b528d9b18b3a11f7bdb3a7c8e3d94a7ed2876a35f44c647030149a34850", + "results/c23-retired-proof.json": "be82f9834955b02739cc08f1576345c74ba89431faa4f733ab5d9bbfe4769e8a", + "results/c4-pfa-operator.json": "37c4e36ba947da1054846b07648bcbd11adbabe6bb0c9357928bf6b785befda2", + "results/installed-foca-source-check.json": "bd2ac184c76d9bf9b1122e2ea2179ad6b51f5be4996e16813a0b9a50951cfdc6", + "results/mindie-quant-api-source-check.json": "4b26ec103344405b3fca2ceed6ed2a7d86831abf765ecadf5171b1277b939123", + "results/native-cli-doctor.json": "ac99ef541ce0febd6b018210de333e38f459c48c3e949c45f66f977c11512499", + "results/README.md": "07fbbe5dff05c1bd0499077cbbdebab1516c9ad7ddd7b3ddfb193555769890de", + "results/report.md": "3d46aba9dbf938bb3ae34eee3e972afc952c8e1e7e6bcacaad49516259d784f8", + "validator/check_media.py": "617c79d3c55dd74b5ccd2a10ba51f70241e36fe95be28a08e3f714fddcdbfa38", + "validator/frozen-snapshots.json": "f52b54bc0a6eb872991611af1368793d25631797389d09380857b2ef228ca283", + "validator/quality-tools/compare_quality.py": "99e8e23463f8ce83b52a22de9cdc697ef4bbf14d910a0c67e76bfd01e94c4b3d", + "validator/quality-tools/score_media.py": "99f5a76436a3c8d37462fab83f63d24eae74b4458c30e5a00b60c19fde139f3e", + "validator/README.md": "3af6ed22e5e3ecb4ef5d9f6587069b4b6c79e54a9ba19da417f8c4ed8c0082a3", + "validator/strict_compare.py": "b83f2bff553ff6ca2970a16e25525cac5d9523612e9c05ea563852c814019028" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/results/report.md b/evidence/h3-ref2va-910c-4logical-20261101-297/results/report.md new file mode 100644 index 000000000..e5aa46019 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/results/report.md @@ -0,0 +1,146 @@ +# MiniMax-H3 Ref2VA on four logical Ascend 910C devices + +Evidence cutoff: 2026-11-02 15:52 +08. Governing acceptance: [#297](https://github.com/InferMatrixLab/InferRSI/issues/297). +**Campaign verdict: INCONCLUSIVE. No accepted 50% complete-HTTP latency reduction.** +This is a tracking snapshot of completed, failed and incomplete work. + +## Workload and identities + +| Item | Tested scope | +| --- | --- | +| Hardware | Four logical 910C devices, two dual-die physical NPUs, 64 GiB per logical device | +| Model | Original MiniMax-H3 Ref2VA snapshot `6818f6c32d12b210915e44ad56a4228c2608f160` | +| Inputs | 1344×768, 24 FPS, 124/362 frames for 5/15 s, 50 solver steps | +| Reference | Original BF16 dense B0 `df1c1b05936381dbaa3891c1ae4c4b21ab375a5d` | +| Migration source | vLLM-Omni snapshot `922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb`; separate from original B0 | +| Execution harness | InferRSI `50c19da43e0b358e6f990c00719891ad308a8ac1` | +| Stack | Private Ubuntu 22.04 namespace, GLIBC 2.35, CANN 9.1, Torch 2.10.0+cpu, TorchNPU 2.10.0.post4, MindIE-SD 3.1.0, Cache-DiT 1.5.0 | +| Parallelism | USP4/ring1, TE TP4, tiled VAE parallel4, DLO enabled in the reference; exceptions recorded per candidate | +| Quality | Frozen strict bit-exact contract and automatic lossy gates; no human media inspection | + +The MindIE wheel's source revision is UNKNOWN. Model bytes were SHA/LFS checked +privately; weights, input image, raw media, tensors and host logs are external. +This campaign is separate from the eight-device 910B1 evidence in +[PR #294](https://github.com/InferMatrixLab/InferRSI/pull/294). Its results do not transfer here. + +## Four-track target status + +| Track | Numerical or model-quality evidence | Clean complete-HTTP evidence | Target status | +| --- | --- | --- | --- | +| 5 s lossless | C1 direct-B0 strict comparison: 439 PASS, zero failures/missing; C13 also 439 PASS on the fixed request | One C1 fresh-server screening pair: 291.247763→286.666760 s, 1.572888689% reduction | Performance FAIL at the 50% gate; final confirmation incomplete | +| 15 s lossless | C1 direct-B0 strict comparison: 439 PASS, zero failures/missing on the fixed request | Not run | INCONCLUSIVE | +| 5 s lossy | Tested fast profiles have observed frozen quality failures; no selected complete quality cohort | No qualified candidate's clean pair | INCONCLUSIVE overall; rejected configurations remain FAIL | +| 15 s lossy | Tested Taylor/C11 configurations have independent visual/audio failures at this duration | No qualified candidate's clean pair | INCONCLUSIVE overall; rejected configurations remain FAIL | + +Strict qualification covers fixed requests, not a universal bit-exact guarantee. +The original B0 repeated exactly under the contract at each duration (219 checks +each). The default dense migration's 15 s qualification is separate and incomplete. + +## Clean screening and timing boundaries + +[C1 clean screening](c1-clean-5s-screening.json) used one fresh server per arm, +two same-shape warmups and three measured complete MP4 responses. Audit and +profiling were disabled. Source integrity, cleanup and measured RGB24/PCM checks +passed; the paired strict check had 35 PASS, with the separate audit's 439 PASS. + +The measured delta compares original B0 with migration+C1. There is no isolated +migration-only clean arm. Only one screening pair completed; statistical +significance and final acceptance are not claimed. The frozen final protocol +requires three fresh pairs in AB/BA/AB order for each track/duration, two warmups +and three measurements per service, and at least 50% reduction in every pair +and the aggregate. No final-confirmation pair completed. + +Audited request times below use zero warmups and one request unless stated. +They are diagnostic observations, not accepted speedups against this clean B0. + +## Direction ledger + +This index summarizes carried device evidence and points to the published +projections. Full private executor archives remain external. A native-test PASS +or primitive-gate PASS does not establish model quality or service performance. + +| Direction | Mechanism and origin | Observed disposition | +| --- | --- | --- | +| C0 | Pinned migration and profiling | Fixed 5 s strict PASS; a two-step trace attributed 7148.150 of 11332.154 ms to FA union. This is not the full HTTP fraction. Raw trace is external. | +| C1 | H3-owned BF16 attention layout and valid-KV prefix | Fixed 5/15 s strict PASS; clean 5 s reduction 1.5729%, below target. Query-fold variants preserved fixture bits but were slower. | +| C2 | HSDP2/replicate2 topology | Four-step bit-exact smoke; cold request about 85.649 s. No complete 50-step performance qualification. | +| C3 | Cache-DiT Taylor2/Taylor3 profiles | Frozen audio or visual gates failed across independent samples/durations; a first passing clip did not qualify a profile. | +| C4 | Native dense FAS/FIA/FIA2 alternatives | FIA2 fixture improvement about 5.4%/5.0% at 5/15 s, non-bit-exact; PFA and query/layout alternatives failed their useful-gain tests. | +| C5 | MindIE Laser for video queries, BF16 conditioning queries | Complete 5 s diagnostic 279.751 s; frozen audio-caption adherence failed. | +| C6 | Laser input-range scaling | Tested scales did not materially improve about 0.002713 relative L2; stopped. | +| C7 | Native RainFusion BF16 | Actual leaf execution observed; fixed sparsity profiles failed frozen quality checks. | +| C8 | Reference-KV device residency and refresh2 | Complete request failed CLAP; creation alone did not prove reuse counts in the older diagnostic. | +| C9 | TP4 with resident DiT | Complete 5 s diagnostic 327.206 s; audio-caption adherence failed. An earlier offload attempt failed before step0. | +| C10 | Local audio modality-veto guard | Native 15 PASS; independent 5 s clips exposed different audio failures. Fixed variants rejected. | +| C11 | Local dual-sigma residual forecasting, including FP32 anchors | Native original 20/FP32 revision 23 PASS. Fixed 5 s and independent 15 s profiles failed frozen quality guards. | +| C12 | Local state-aligned residual prediction | Native 14 PASS; full 5 s executed 50 computes/zero predictions/45 vetoes. No acceleration; cohort incomplete. | +| C13 | Adapted upstream vLLM-Omni PR #9351 ordering to current chunked DLO, over C1 | Focused 10 PASS; expanded309 PASS/1 FAIL, reproduced on the migration baseline. Full 5 s strict 439 PASS; clean timing and 15 s incomplete. | +| C14 | Quantized-attention prerequisites on the actual A3 stack | Block-FP8 and block-INT8 probes failed before FIA2. Per-token INT8 and other SDKs are not disproved. | +| C15 | Local target-video block-residual cache, FP16 storage | Corrected native17 PASS; complete-path attempt failed finite video velocity at first reuse. | +| C16 | FP32 storage repair for C15 | Native 18 PASS; complete 5 s input 134 PASS, but three frozen audio checks failed. | +| C17 | Local three-latent-frame attention window | Native 15 PASS; real-input operation 63.7004% lower. Full-model quality failed 11 checks. | +| C18 | Current-frame detail plus global 3×3 pooled KV | Native 17 PASS; operation 7.2512% lower, below the fixed 50% primitive gate. No full model. | +| C19 | Shared global 3×3 pooled KV | Native 17 PASS; operation 52.8479% lower, but relative L2 0.258805 exceeded the fixed 0.258052 bound. No full model. | +| C20 | C19 plus isotropic key-variance correction | Native 17 PASS; operation 48.4204% lower and relative L2 0.263361; both primitive gates failed. No full model. | +| C21 | Local pre-gate video activation cache | Native 36 PASS; full 5 s diagnostic 163.806 s, input 134 PASS; seven frozen quality failures. | +| C22 | Local sigma-secant pre-gate cache, BF16 anchor/INT8 per-channel delta | Native 45 PASS; full 5 s diagnostic 166.859 s, input 134 PASS; nine frozen quality failures. | +| C23 | Local fresh conditioning attention with cached video-to-video output/LSE | V2 native 44 PASS, static attention cycle 66.4588% lower; full 5 s diagnostic 199.065 s, input 134 PASS; eight frozen quality failures. | + +Primary result indexes: [automatic quality comparisons](automatic-quality-checkpoint.json), +[C12–C23 run projections](c12-through-c23-execution-checkpoint.json), +[C1 query folding](c1-queryfold-operator.json), [C4 PFA](c4-pfa-operator.json), +[C11 15 s audit](c11-fp32-15s-00-audit.json), and +[source identities and patches](../patch/README.md). +The ledger summarizes 24 direction families, not 24 successful candidates. + +## C23: primitive improvement rejected by full-model quality + +Private engine commit `6eb7bb094effb40853431208d1631effaf452a3f` retains all +50 solver steps, fresh QKV/SP/projections/FFN/current gates and original weights. +Each step refreshes nonvideo-to-full and video-to-conditioning attention. +Only video-to-video BF16 attention output and FP32 LSE are reused between full +steps 0,1,4,9,19,29,39,49. This is an intentional approximation for changing Q/KV. + +The first manual Triton merge passed 44 native tests but the full 50-call static +attention cycle improved only 38.2552%, below the prospective 50% gate. Component +diagnosis measured 22.750201 ms for that merge and 2.742732 ms for a native Torch +expression, with all 66,834,432 real BF16 output elements bit-equal. V1 failures +and source hashes are retained in the run projection. + +V2 adopted that measured Torch expression, keeping the algorithm and gates fixed. +It passed 44 focused tests, including three executed four-rank suites. On one +captured real 5 s input, 50 attention calls with 8 refresh/42 reuse, 2 warm cycles and +5 measured cycles in AB/BA/AB/BA/AB order, cycle medians were 3477.472871 and +1166.387435 ms: 66.458762% lower. All 50 validation outputs were finite, max relative +L2 was 0.00201662234 < 0.005, and saved real-merge replay was bit-equal to V1. +This excludes FFN, changing inputs, complete-model timing and HTTP. + +[Full 5s-00 execution](c23-5s-00-execution-proof.json) then completed all 50 steps +with 134 input checks and expected per-rank counters: 400 context refreshes, +2100 reuses, 2500 fresh conditioning-attention calls and 2500 full-row FFN blocks. +Persistent cache peaked at 6787872000 bytes/rank and closed to zero live bytes. +Diagnostic complete HTTP was 199064.918509 ms. The frozen scorer rejected 8 of 39 +checks: DINO/CLIP/CLAP similarity, segmented CLAP, reference-DINO adherence, +audio-caption adherence, audio level and audio envelope. The other 9 missing +samples do not erase these violations. No 15 s or clean timing extension follows. + +## Limits and continuation + +All 24 direction families are charged. The requester extended the original budget +to 32 wall hours/128 logical-device-hours, ending 2026-11-03 01:19+08; preparation, +failures and retries remain charged. The controller-alias repair groups two +duplicate labels under their existing source-bound families, preserving the +raw budget and failed attempts: [audit receipt](c21-c22-direction-identity-correction.json). +No additional final-confirmation budget is authorized. + +[Retirement readback](c23-retired-proof.json) verified selected devices idle, +experiment ports free and tested source/helpers/validators unchanged at +2026-11-02 15:48:59+08. No device job is scheduled by this tracking PR. +Further mechanism search requires a requester extension beyond 24 directions. +Four-track final confirmation, full lossy corpus coverage and independent review +remain incomplete. Goal acceptance belongs to the requester. + +Published files are sanitized field projections and patch snapshots. They do +not contain model/scorer weights, media, raw tensors, full executor archives or +the private namespace/bootstrap environment. [Replay and availability notes](../plan/reproduction.md) +describe what can be verified locally and what still needs external artifacts. diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/validator/README.md b/evidence/h3-ref2va-910c-4logical-20261101-297/validator/README.md new file mode 100644 index 000000000..d3f944753 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/validator/README.md @@ -0,0 +1,20 @@ +# Validator status + +The requester selected **strict bit-exact** for the lossless tracks. The +[lossless contract](../plan/frozen-lossless-contract-v1.json) was frozen before +search. It fixes video/audio FP32 checkpoints, schedule and all decoded RGB24/PCM +bytes. Original B0 repeatability passed separately at both durations. Equal MP4 +hashes alone do not establish this contract. + +The [lossy contract](../plan/frozen-lossy-contract-v1.json) was frozen before real +lossy outputs. It specifies ten prompts, pinned scorers/weights, reduced official +VBench dimensions and paired video/audio/reference/temporal guards. The source +requirement's human inspection is excluded from this RSI loop and is not claimed +as completed. These proxies do not certify absolute quality or semantic AV synchrony. + +The [snapshot manifest](frozen-snapshots.json) identifies exact original bytes. +Archived scorer paths assume the original execution-tree layout and pinned +dependencies; this evidence folder alone is not a self-contained runtime. No +weights or raw media are bundled. Keep numerical qualification, automatic model +quality and clean timing separate. Missing samples remain INCONCLUSIVE; an observed +gate violation remains FAIL even when other samples are missing. diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/validator/check_media.py b/evidence/h3-ref2va-910c-4logical-20261101-297/validator/check_media.py new file mode 100644 index 000000000..4e7f70252 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/validator/check_media.py @@ -0,0 +1,60 @@ +"""Decode every frame/sample and hash canonical RGB24 and interleaved PCM.""" + +import argparse +import hashlib +import json +from pathlib import Path + +import av +import numpy as np + + +def inspect(path, frames): + video_hash, audio_hash = hashlib.sha256(), hashlib.sha256() + frame_hashes = [] + with av.open(str(path)) as media: + assert len(media.streams.video) == 1 and len(media.streams.audio) == 1 + stream = media.streams.video[0] + assert (stream.width, stream.height, float(stream.average_rate)) == (1344, 768, 24.) + for frame in media.decode(video=0): + value = frame.to_ndarray(format='rgb24') + payload = value.tobytes(order='C') + video_hash.update(payload) + frame_hashes.append(hashlib.sha256(payload).hexdigest()) + assert len(frame_hashes) == frames, len(frame_hashes) + samples, sum_squares, peak = 0, 0., 0. + with av.open(str(path)) as media: + stream = media.streams.audio[0] + assert stream.rate == 32000 and stream.codec_context.channels == 2 + for frame in media.decode(audio=0): + assert frame.format.name == 'fltp' and frame.layout.nb_channels == 2 + value = frame.to_ndarray() + assert value.dtype == np.float32 and np.isfinite(value).all() + samples += value.shape[1] + sum_squares += np.square(value.astype(np.float64)).sum().item() + peak = max(peak, float(np.abs(value).max())) + audio_hash.update(value.T.copy(order='C').tobytes()) + assert samples > 0 and sum_squares > 0 + assert abs(samples / 32000 - frames / 24) <= 1 / 24 + return {'video_frames': frames, 'width': 1344, 'height': 768, 'fps': 24, + 'rgb24_sha256': video_hash.hexdigest(), 'frame_rgb24_sha256': frame_hashes, + 'audio_sample_rate': 32000, 'audio_channels': 2, 'audio_samples': samples, + 'pcm_f32_interleaved_sha256': audio_hash.hexdigest(), + 'finite_audio': True, 'audio_rms': (sum_squares / (samples * 2)) ** .5, + 'audio_peak': peak, 'media_integrity': 'PASS', + 'model_quality': 'not_evaluated'} + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument('--media', required=True) + parser.add_argument('--frames', type=int, required=True) + parser.add_argument('--output', required=True) + args = parser.parse_args() + result = inspect(Path(args.media), args.frames) + Path(args.output).write_text(json.dumps(result, indent=2)) + print(json.dumps({k: v for k, v in result.items() if k != 'frame_rgb24_sha256'})) + + +if __name__ == '__main__': + main() diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/validator/frozen-snapshots.json b/evidence/h3-ref2va-910c-4logical-20261101-297/validator/frozen-snapshots.json new file mode 100644 index 000000000..562419337 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/validator/frozen-snapshots.json @@ -0,0 +1,10 @@ +{ + "plan/frozen-lossy-contract-v1.json": "b8fd9e634e6db5ebfbce3842f3360f1b2241a08119953ee71bfc26553dbbafed", + "plan/frozen-lossless-contract-v1.json": "b3ec4b2f58d1f9de4475977ed4bbdd75e4d38cad3857244fe0cfd842f7469cab", + "validator/strict_compare.py": "b83f2bff553ff6ca2970a16e25525cac5d9523612e9c05ea563852c814019028", + "validator/check_media.py": "617c79d3c55dd74b5ccd2a10ba51f70241e36fe95be28a08e3f714fddcdbfa38", + "validator/quality-tools/compare_quality.py": "99e8e23463f8ce83b52a22de9cdc697ef4bbf14d910a0c67e76bfd01e94c4b3d", + "validator/quality-tools/score_media.py": "99f5a76436a3c8d37462fab83f63d24eae74b4458c30e5a00b60c19fde139f3e", + "results/c1-5s-strict-comparison.json": "f333a0cf8d65d59e92d67a3882608a344a00f57542ab5a22e832c6800d81a4cb", + "results/c1-15s-strict-comparison.json": "72b196500c0f045eb78c4e994279189c36b59c48f103e0ab4f94ff51968c62be" +} diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/validator/quality-tools/compare_quality.py b/evidence/h3-ref2va-910c-4logical-20261101-297/validator/quality-tools/compare_quality.py new file mode 100644 index 000000000..1dbc94626 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/validator/quality-tools/compare_quality.py @@ -0,0 +1,128 @@ +"""Apply the prospective relative-quality gates; missing samples never pass.""" + +import argparse +import hashlib +import json +from pathlib import Path + +import numpy as np + + +def digest(path): + return hashlib.sha256(path.read_bytes()).hexdigest() + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument('--baseline-index', required=True) + parser.add_argument('--candidate-index', required=True) + parser.add_argument('--contract', required=True) + parser.add_argument('--output', required=True) + args = parser.parse_args() + path = Path(args.contract) + contract = json.loads(path.read_text()) + contract_hash = digest(path) + assert digest(Path(__file__)) == contract['comparison_sha256'] + gates = contract['gates'] + failures, missing, observed, checks = [], [], {}, 0 + + def check(condition, label): + nonlocal checks + checks += 1 + if not condition: + failures.append(label) + + def read_index(name): + records = json.loads(Path(name).read_text())['scores'] + result = {item['sample_id']: Path(item['path']) for item in records} + check(len(result) == len(records), name + ': unique sample IDs') + return result + + baseline_index, candidate_index = read_index(args.baseline_index), read_index(args.candidate_index) + for sample in contract['samples']: + key = sample['id'] + if key not in baseline_index or key not in candidate_index: + missing.append(key) + continue + left, right = baseline_index[key], candidate_index[key] + baseline, candidate = json.loads(left.read_text()), json.loads(right.read_text()) + for label, record, score_path in [('baseline', baseline, left), ('candidate', candidate, right)]: + check(record['sample_id'] == key, key + ':' + label + ': ID') + check(record['contract_sha256'] == contract_hash, key + ':' + label + ': contract') + check(record['cpu_only'], key + ':' + label + ': scorer backend') + check(digest(score_path.parent / 'features.npz') == record['features_sha256'], + key + ':' + label + ': descriptor hash') + check(record['media']['media_integrity'] == 'PASS', key + ':' + label + ': media') + arrays = [] + for score_path in [left, right]: + with np.load(score_path.parent / 'features.npz', allow_pickle=False) as data: + arrays.append({name: data[name] for name in data.files}) + a, b = arrays + for name in ['dino', 'clip', 'clap', 'audio_rms_envelope', 'motion']: + check(a[name].shape == b[name].shape, key + ':' + name + ': shape') + check(bool(np.isfinite(a[name]).all() and np.isfinite(b[name]).all()), key + ':' + name + ': finite') + similarities = {} + for name in ['dino', 'clip', 'clap']: + cosine = np.sum(a[name] * b[name], axis=-1) + similarities[name] = {'mean': float(cosine.mean()), 'min': float(cosine.min()), + 'p05': float(np.quantile(cosine, 0.05))} + check(float(cosine.mean()) >= gates[name + '_paired_mean_cosine_min'], key + ':' + name + ': similarity') + check(similarities['dino']['p05'] >= gates['dino_paired_p05_cosine_min'], key + ': dino lower tail') + check(similarities['clap']['min'] >= gates['clap_paired_segment_cosine_min'], key + ': clap segment') + for metric in ['reference_clip_cosine', 'reference_dino_cosine', 'prompt_clip_cosine', + 'audio_caption_clap_cosine']: + check(candidate[metric] >= baseline[metric] - gates['adherence_max_absolute_cosine_drop'], + key + ':' + metric + ': adherence') + rms_db = 20 * np.log10(candidate['media']['audio_rms'] / baseline['media']['audio_rms']) + check(abs(rms_db) <= gates['audio_rms_max_abs_db'], key + ': audio level') + check(candidate['clipping_fraction'] <= baseline['clipping_fraction'] + gates['clipping_max_fraction_increase'], + key + ': audio clipping') + envelope_error = np.linalg.norm(b['audio_rms_envelope'] - a['audio_rms_envelope']) / \ + max(np.linalg.norm(a['audio_rms_envelope']), 1e-12) + check(envelope_error <= gates['audio_envelope_relative_l2_max'], key + ': audio temporal envelope') + motion_a, motion_b = float(a['motion'].mean()), float(b['motion'].mean()) + if motion_a < gates['motion_floor']: + check(motion_b < 2 * gates['motion_floor'], key + ': near-static motion') + motion_ratio = None + else: + motion_ratio = motion_b / motion_a + check(gates['motion_ratio_min'] <= motion_ratio <= gates['motion_ratio_max'], key + ': motion energy') + check(baseline['media']['audio_samples'] == candidate['media']['audio_samples'], key + ': audio coverage') + check(abs(candidate['pts_start_difference_seconds']) <= 1 / 24, key + ': media timestamp alignment') + reduced_a = float(np.mean(list(baseline['vbench'].values()))) + reduced_b = float(np.mean(list(candidate['vbench'].values()))) + check(reduced_a > 0, key + ': positive baseline score') + error = abs(reduced_b - reduced_a) / max(abs(reduced_a), 1e-12) + check(error < gates['per_clip_reduced_vbench_relative_error_max'], key + ': clip score') + observed[key] = {'seconds': sample['request']['seconds'], 'baseline': baseline['vbench'], + 'candidate': candidate['vbench'], 'cosines': similarities, + 'reduced_relative_error': error, 'audio_rms_db': float(rms_db), + 'audio_envelope_relative_l2': float(envelope_error), 'motion_ratio': motion_ratio} + tracks = {} + for duration in ['5', '15']: + required = [item['id'] for item in contract['samples'] if item['request']['seconds'] == duration] + available = [key for key in required if key in observed] + if len(available) != len(required): + tracks[duration] = {'verdict': 'INCONCLUSIVE', 'missing': list(set(required) - set(available))} + continue + values = {side: {dimension: float(np.mean([observed[key][side][dimension] for key in required])) + for dimension in ['subject_consistency', 'background_consistency']} + for side in ['baseline', 'candidate']} + total = {side: float(np.mean(list(values[side].values()))) for side in values} + relative_error = abs(total['candidate'] - total['baseline']) / abs(total['baseline']) + check(relative_error < gates['reduced_vbench_relative_error_max'], duration + ': reduced VBench total') + for dimension in values['baseline']: + error = abs(values['candidate'][dimension] - values['baseline'][dimension]) / abs(values['baseline'][dimension]) + check(error < gates['dimension_relative_error_max'], duration + ':' + dimension) + tracks[duration] = {'dimension_means': values, 'reduced_total': total, 'relative_error': relative_error} + verdict = 'FAIL' if failures else 'INCONCLUSIVE' if missing else 'PASS' + result = {'verdict': verdict, 'contract_sha256': contract_hash, 'checks': checks, + 'failures': failures, 'missing': missing, 'observed': observed, 'tracks': tracks, + 'scope': contract['scope'], 'performance_acceptance': 'separate'} + Path(args.output).write_text(json.dumps(result, indent=2, allow_nan=False)) + print(json.dumps({'verdict': verdict, 'checks': checks, 'failures': failures, 'missing': missing}), flush=True) + return {'PASS': 0, 'FAIL': 1, 'INCONCLUSIVE': 2}[verdict] + + +if __name__ == '__main__': + raise SystemExit(main()) diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/validator/quality-tools/score_media.py b/evidence/h3-ref2va-910c-4logical-20261101-297/validator/quality-tools/score_media.py new file mode 100644 index 000000000..ed6325e56 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/validator/quality-tools/score_media.py @@ -0,0 +1,173 @@ +"""CPU-only, source-pinned descriptors for the H3 relative-quality contract.""" + +import argparse +import hashlib +import json +import os +from pathlib import Path +import sys + +ROOT = Path(__file__).resolve().parents[1] +os.environ['TORCH_DEVICE_BACKEND_AUTOLOAD'] = '0' +sys.path[:0] = [str(ROOT / 'quality-adapter'), str(ROOT / 'source/clip'), + str(ROOT / 'source/dino'), str(ROOT / 'validator')] +import av +import cv2 +import numpy as np +from PIL import Image +import torch +import torch.nn.functional as F +import clip +import vision_transformer +from scipy.signal import resample_poly +from vbench.subject_consistency import subject_consistency +from vbench.background_consistency import background_consistency +from vbench.utils import dino_transform_Image +from check_media import inspect + + +def digest(path): + return hashlib.sha256(path.read_bytes()).hexdigest() + + +def normalize(array): + return array / np.linalg.norm(array, axis=-1, keepdims=True).clip(1e-12) + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument('--media', required=True) + parser.add_argument('--request', required=True) + parser.add_argument('--sample-id', required=True) + parser.add_argument('--audio-caption', required=True) + parser.add_argument('--output', required=True) + parser.add_argument('--contract') + args = parser.parse_args() + torch.set_num_threads(8) + request = json.loads(Path(args.request).read_text()) + assert int(request['num_inference_steps']) == 50 + duration = int(request['seconds']) + expected_frames = {5: 124, 15: 362}[duration] + output = Path(args.output) + output.mkdir(parents=True, exist_ok=False) + source_media = Path(args.media).resolve() + media_info = inspect(source_media, expected_frames) + contract_hash = None + if args.contract: + path = Path(args.contract) + contract = json.loads(path.read_text()) + contract_hash = digest(path) + assert digest(Path(__file__)) == contract['scorer_sha256'] + sample = next(item for item in contract['samples'] if item['id'] == args.sample_id) + assert sample['audio_caption'] == args.audio_caption + for key in ['model', 'prompt', 'width', 'height', 'aspect_ratio', 'fps', 'seconds', + 'num_inference_steps', 'seed', 'flow_shift', 'image_reference']: + assert request[key] == sample['request'][key] + for key in ['task', 'duration', 'audio_flow_shift']: + assert json.loads(request['extra_params'])[key] == json.loads(sample['request']['extra_params'])[key] + for entry in contract['weight_files']: + assert digest(ROOT / entry['path']) == entry['sha256'] + for entry in contract['source_files']: + assert digest(ROOT / entry['path']) == entry['sha256'] + assert digest(ROOT / 'assets/reference.png') == contract['reference_image_sha256'] + videos = output / 'videos' + videos.mkdir() + (videos / 'clip.mp4').symlink_to(source_media) + frames = output / 'frames/clip' + frames.mkdir(parents=True) + pts, motion, previous = [], [], None + decoded_hash = hashlib.sha256() + with av.open(str(source_media)) as reader: + for index, frame in enumerate(reader.decode(video=0)): + pixels = frame.to_ndarray(format='rgb24') + decoded_hash.update(pixels.tobytes()) + assert frame.time is not None + pts.append(float(frame.time)) + Image.fromarray(pixels).save(frames / f'{index:05d}.png', compress_level=1) + small = cv2.resize(pixels, (128, 72), interpolation=cv2.INTER_AREA).astype(np.float32) / 255 + grey = small @ np.array([0.299, 0.587, 0.114], dtype=np.float32) + if previous is not None: + motion.append(float(np.abs(grey - previous).mean())) + previous = grey + assert decoded_hash.hexdigest() == media_info['rgb24_sha256'] + assert len(pts) == expected_frames and np.all(np.diff(pts) > 0) + assert np.max(np.abs(np.diff(pts) - 1 / 24)) < 1e-6 + weights = ROOT / 'quality-cache/weights' + clip_model, transform = clip.load(str(weights / 'ViT-B-32.pt'), device='cpu', jit=False) + dino = vision_transformer.vit_base(patch_size=16, num_classes=0) + dino.load_state_dict(torch.load(weights / 'dino_vitbase16_pretrain.pth', + map_location='cpu', weights_only=True), strict=True) + dino.eval() + dino_features, clip_features = [], [] + dino_hook = dino.register_forward_hook(lambda module, inputs, result: + dino_features.append(result.detach().cpu().numpy())) + clip_hook = clip_model.visual.register_forward_hook(lambda module, inputs, result: + clip_features.append(result.detach().cpu().numpy())) + with torch.inference_mode(): + subject = subject_consistency(dino, [str(videos / 'clip.mp4')], 'cpu', True) + background = background_consistency(clip_model, transform, [str(videos / 'clip.mp4')], 'cpu', True) + dino_hook.remove() + clip_hook.remove() + dino_features = normalize(np.concatenate(dino_features)) + clip_features = normalize(np.concatenate(clip_features)) + assert dino_features.shape[0] == clip_features.shape[0] == expected_frames + reference = Image.open(ROOT / 'assets/reference.png').convert('RGB') + with torch.inference_mode(): + ref_clip = F.normalize(clip_model.encode_image(transform(reference).unsqueeze(0)), dim=-1).numpy()[0] + ref_dino = F.normalize(dino(dino_transform_Image(224)(reference).unsqueeze(0)), dim=-1).numpy()[0] + text_clip = F.normalize(clip_model.encode_text(clip.tokenize([request['prompt']])), dim=-1).numpy()[0] + from transformers import ClapModel, ClapFeatureExtractor, RobertaTokenizerFast + clap_path = ROOT / 'quality-cache/clap/8fa0f1c6d0433df6e97c127f64b2a1d6c0dcda8a' + extractor = ClapFeatureExtractor.from_pretrained(clap_path, local_files_only=True) + tokenizer = RobertaTokenizerFast.from_pretrained(clap_path, local_files_only=True) + clap_model = ClapModel.from_pretrained(clap_path, local_files_only=True, dtype=torch.float32).cpu().eval() + assert all(parameter.device.type == 'cpu' for parameter in clap_model.parameters()) + audio, audio_pts = [], [] + with av.open(str(source_media)) as reader: + for frame in reader.decode(audio=0): + assert frame.time is not None + audio_pts.append(float(frame.time)) + audio.append(frame.to_ndarray()) + audio = np.concatenate(audio, axis=1) + assert np.all(np.diff(audio_pts) > 0) + assert np.max(np.diff(audio_pts)) <= 1025 / 32000 + assert abs(pts[0] - audio_pts[0]) <= 1 / 24 + mono = resample_poly(audio.mean(axis=0), 3, 2) + boundaries = np.linspace(0, len(mono), {5: 1, 15: 3}[duration] + 1).astype(int) + clap_features = [] + with torch.inference_mode(): + for start, end in zip(boundaries[:-1], boundaries[1:]): + np.random.seed(0) + inputs = extractor(mono[start:end], sampling_rate=48000, return_tensors='pt') + clap_features.append(clap_model.get_audio_features(**inputs).pooler_output.numpy()[0]) + audio_text = clap_model.get_text_features(**tokenizer([args.audio_caption], return_tensors='pt', + padding=True)).pooler_output.numpy()[0] + clap_features = normalize(np.stack(clap_features)) + envelope = [] + for index in range(expected_frames): + start, end = int(index * 32000 / 24), min(int((index + 1) * 32000 / 24), audio.shape[1]) + assert end > start + envelope.append(float(np.sqrt(np.mean(audio[:, start:end].astype(np.float64) ** 2)))) + features = {'dino': dino_features, 'clip': clip_features, 'clap': clap_features, + 'audio_rms_envelope': np.array(envelope), 'motion': np.array(motion)} + assert all(np.isfinite(value).all() for value in features.values()) + assert 'torch_npu' not in sys.modules + np.savez_compressed(output / 'features.npz', **features) + result = {'sample_id': args.sample_id, 'request': request, 'contract_sha256': contract_hash, + 'media_sha256': digest(source_media), 'media': media_info, + 'features_sha256': digest(output / 'features.npz'), + 'vbench': {'subject_consistency': subject[0], 'background_consistency': background[0]}, + 'reference_clip_cosine': float((clip_features @ ref_clip).mean()), + 'reference_dino_cosine': float((dino_features @ ref_dino).mean()), + 'prompt_clip_cosine': float((clip_features @ text_clip).mean()), + 'audio_caption_clap_cosine': float((clap_features @ audio_text).mean()), + 'clipping_fraction': float(np.mean(np.abs(audio) >= 0.999)), + 'pts_start_difference_seconds': pts[0] - audio_pts[0], + 'cpu_only': True, 'quality_verdict': 'INCONCLUSIVE; descriptors require paired validation', + 'scope': 'relative BF16 quality proxies; timestamps do not establish semantic AV synchrony'} + (output / 'score.json').write_text(json.dumps(result, indent=2, allow_nan=False)) + print(json.dumps({key: value for key, value in result.items() if key not in ('media', 'request')}), flush=True) + + +if __name__ == '__main__': + main() diff --git a/evidence/h3-ref2va-910c-4logical-20261101-297/validator/strict_compare.py b/evidence/h3-ref2va-910c-4logical-20261101-297/validator/strict_compare.py new file mode 100644 index 000000000..ee6a1eb95 --- /dev/null +++ b/evidence/h3-ref2va-910c-4logical-20261101-297/validator/strict_compare.py @@ -0,0 +1,121 @@ +"""Compare complete H3 audit evidence without importing either inference arm.""" + +import argparse +import hashlib +import json +from pathlib import Path + +import numpy as np + +POINTS = ['initial_noise', 'post_step_000', 'post_step_009', 'post_step_024', + 'post_step_049', 'vae_inputs'] +MEDIA_KEYS = ['video_frames', 'width', 'height', 'fps', 'rgb24_sha256', + 'audio_sample_rate', 'audio_channels', 'audio_samples', + 'pcm_f32_interleaved_sha256', 'finite_audio'] + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument('--baseline', required=True) + parser.add_argument('--candidate') + parser.add_argument('--output', required=True) + args = parser.parse_args() + missing, failures, checks = [], [], 0 + + def load(path): + try: + return json.loads(path.read_text()) + except (OSError, ValueError) as failure: + missing.append({'path': str(path), 'error': str(failure)}) + return None + + def same(before, after, label): + nonlocal checks + checks += 1 + if before != after: + failures.append({'check': label, 'baseline': before, 'observed': after}) + + def audits(directory): + summary, cleanup = load(directory / 'summary.json'), load(directory / 'cleanup.json') + request = load(directory / 'request.json') + if summary is None or cleanup is None or request is None: + return None + same(cleanup.get('passed'), True, str(directory) + ':cleanup') + same(int(request['num_inference_steps']), 50, str(directory) + ':50_steps') + same(summary['diagnostic'], 'audit', str(directory) + ':audit_mode') + if summary['repetitions'] < 2: + missing.append({'path': str(directory), 'error': 'Two measured audit requests are required'}) + records = [] + for sample in range(summary['repetitions']): + index = summary['warmup'] + sample + path = directory / 'diagnostics' / f'request-{index:03d}' + value = {'media': load(directory / f'sample-{sample}.media.json'), 'ranks': {}} + for rank in range(4): + folder = path / f'rank-{rank}' + config = load(folder / 'configuration.json') + if config is not None: + same(config.get('steps'), 50, str(folder) + ':steps') + same(config.get('cache_backend') in (None, 'none'), True, str(folder) + ':cache_backend') + same(config.get('kv_offload_config'), None, str(folder) + ':kv_config') + same(config.get('reference_kv_state_created'), False, str(folder) + ':actual_kv_state') + tensors = {} + for point in POINTS: + pair = load(folder / (point + '.json')) + if pair is None: + continue + tensors[point] = {} + for stream in ['video', 'audio']: + data = pair[stream] + same(data['dtype'], 'torch.float32', str(folder) + ':' + point + ':' + stream + ':dtype') + tensors[point][stream] = {k: data[k] for k in ['shape', 'dtype', 'bytes', 'sha256']} + if rank == 0: + payload = data.get('tensor_file') + if payload is None or not (folder / payload).exists(): + missing.append({'path': str(folder), 'error': point + ':' + stream + ':raw tensor missing'}) + continue + raw = (folder / payload).read_bytes() + same(len(raw), data['bytes'], str(folder) + ':' + point + ':raw_length') + same(hashlib.sha256(raw).hexdigest(), data['sha256'], str(folder) + ':' + point + ':raw_hash') + same(bool(np.isfinite(np.frombuffer(raw, dtype='