iChubai · Oct 20, 2026 · Oct 20, 2026
diff --git a/Makefile b/Makefile
@@ -223,12 +223,16 @@ INFER_TENSOR_CONTRACTS = \
 	tests/eval_core/test_catalog_integration_status.py \
 	$(EVALUATION_CPU_CONTRACTS)
 EVALUATION_CPU_CONTRACTS = \
+	tests/eval_core/test_worldmembench_official_runner.py \
+	tests/eval_core/test_worldmembench_runtime.py \
 	tests/evaluation/execution/test_resume_options.py \
 	tests/evaluation/execution/test_result_integrity.py \
 	tests/evaluation/execution/test_helper_reuse.py \
 	tests/evaluation/execution/test_pipeline_boundaries.py \
 	tests/evaluation/execution/test_generation_artifact_materialization.py \
 	tests/evaluation/execution/test_shared_scoring_backend.py \
+	tests/evaluation/execution/test_shared_quality.py \
+	tests/evaluation/execution/test_worldmembench_camera.py \
 	tests/evaluation/execution/test_vbench_appearance_reduction.py \
 	tests/evaluation/execution/test_worldscore_evaluator_lifecycle.py \
 	tests/evaluation/metrics/test_scalar_aggregation.py \

diff --git a/THIRD-PARTY-NOTICES b/THIRD-PARTY-NOTICES
@@ -6331,3 +6331,38 @@ Wan2.2 support assets retain their Apache-2.0 license.
 
 Original-File: WorldPlay2/WorldPlay2/LICENSE.txt
 license: cc-by-nc-4.0
+
+
+WorldMemBench evaluation protocols
+---------------------------------
+Source: https://github.com/ZJU-REAL/WorldMemBench
+Source revision: 1b79840143a98e017891e893b5d3ef75b93078a3
+License: MIT
+Files: worldfoundry/evaluation/tasks/execution/runners/worldmembench/runtime/
+
+WorldFoundry retains the official sampling, revisit, alignment, aggregation,
+VLM judging and result protocols. CLIP, LAION aesthetic heads and DINO model
+loading use shared WorldFoundry components. Human and object masked DINO
+features share one implementation. Model weights are distributed separately.
+
+MIT License
+
+Copyright (c) 2026 WorldMemBench
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
diff --git a/docs/fumadocs/content/docs/evaluation/benchmark-hub/meta.json b/docs/fumadocs/content/docs/evaluation/benchmark-hub/meta.json
@@ -63,6 +63,7 @@
     "rbench",
     "---World Models---",
     "worldmodelbench",
+    "worldmembench",
     "worldscore",
     "worldbench",
     "wbench",

diff --git a/docs/fumadocs/content/docs/evaluation/benchmark-hub/meta.zh.json b/docs/fumadocs/content/docs/evaluation/benchmark-hub/meta.zh.json
@@ -63,6 +63,7 @@
     "rbench",
     "---World Models---",
     "worldmodelbench",
+    "worldmembench",
     "worldscore",
     "worldbench",
     "wbench",

diff --git a/docs/fumadocs/content/docs/evaluation/benchmark-hub/wbench.mdx b/docs/fumadocs/content/docs/evaluation/benchmark-hub/wbench.mdx
@@ -17,6 +17,12 @@ WorldFoundry **checks in the metric runtime** (HPSv3, TransNetV2, PAVRM, DreamSi
 
 <BenchmarkPageSections id="wbench" locale="en" />
 
+Imaging quality uses the official PyIQA `musiq` variant (KonIQ), with its
+checkpoint resolved explicitly through `wbench_quality_metric_assets`.
+`WORLDFOUNDRY_WBENCH_MUSIQ_KONIQ_CKPT` can select an already staged checkpoint.
+Its preprocessing and per-frame normalization retain the official protocol;
+WorldMemBench's MUSIQ-SPAQ is a different metric backend profile.
+
 ## How to run
 
 ```bash

diff --git a/docs/fumadocs/content/docs/evaluation/benchmark-hub/wbench.zh.mdx b/docs/fumadocs/content/docs/evaluation/benchmark-hub/wbench.zh.mdx
@@ -17,6 +17,10 @@ WorldFoundry **检入指标 runtime**（HPSv3、TransNetV2、PAVRM、DreamSim、
 
 <BenchmarkPageSections id="wbench" locale="zh" />
 
+成像质量采用官方 PyIQA `musiq` 版本（KonIQ），通过 `wbench_quality_metric_assets`
+显式解析权重。可用 `WORLDFOUNDRY_WBENCH_MUSIQ_KONIQ_CKPT` 指向已准备的权重。
+预处理和逐帧归一化保留官方协议；WorldMemBench 的 MUSIQ-SPAQ 属于不同的后端配置。
+
 ## 如何运行
 
 ```bash

diff --git a/docs/fumadocs/content/docs/evaluation/benchmark-hub/worldmembench.mdx b/docs/fumadocs/content/docs/evaluation/benchmark-hub/worldmembench.mdx
@@ -0,0 +1,104 @@
+---
+title: WorldMemBench
+description: In-tree visual-memory and camera-control evaluation with shared model components.
+---
+
+<BenchmarkPageSections id="worldmembench" locale="en" />
+
+## Scope
+
+[WorldMemBench](https://github.com/ZJU-REAL/WorldMemBench) evaluates human,
+object, environment and dynamic memory, camera control, and video quality.
+WorldFoundry runs all twelve official metric protocols in tree, adapted from
+revision `1b79840143a98e017891e893b5d3ef75b93078a3` under MIT. An upstream
+WorldMemBench checkout or Python package is not required.
+
+| Metric ID | Official score field | Direction |
+| --- | --- | --- |
+| `imaging_quality` | `score` | Higher |
+| `aesthetic_quality` | `score` | Higher |
+| `human_identity` | `score` | Higher |
+| `human_appearance` | `score` | Higher |
+| `object_geometry` | `score` | Higher |
+| `object_appearance` | `score` | Higher |
+| `environment_visible_memory` | `score` | Higher |
+| `environment_depth_fidelity` | `depth_spearman` | Higher |
+| `environment_scene_reconstruction` | `reconstruction_score` | Higher |
+| `dynamic_memory` | `score` | Higher |
+| `camera_rotation` | `camera_rotation_rpe_path` | Lower |
+| `camera_translation` | `camera_translation_rpe` | Lower |
+
+Scores retain their official units, including negative depth correlations and
+unbounded camera errors. There is no cross-metric average or composite score.
+
+## Shared components and protocol
+
+OpenAI CLIP, LAION aesthetic heads and DINO checkpoint loading use canonical
+WorldFoundry components. Human and object appearance share one masked DINOv2
+extractor. Within a run, identical DINO checkpoint/device pairs share the model;
+preprocessing and batch-size state remain protocol-specific.
+WorldScore and MemoBench use the shared LAION MLP; MIND, WBench and
+WorldArena use the shared linear head. These heads are not interchangeable.
+WorldScore and 4DWorldBench share their 512-square IQA preprocessing and base
+implementation; that transform is not applied to WorldMemBench.
+
+WorldMemBench retains MUSIQ-SPAQ, its aspect-preserving OpenCV resize, stride-4
+quality sampling, score normalization, and equal-weight video aggregation.
+Revisit segmentation, frozen target association, masked crops, camera alignment,
+scene reconstruction and Dynamic Memory's event-gated VLM checklist retain
+the official protocols.
+
+## Evaluate generated outputs
+
+Use the official `model/category/trajectory/case/video.mp4` layout or an official
+JSONL manifest. Keep annotations, reference images, trajectory files and target
+poses with each case. The dataset is
+[`rescuerz/WorldMemBench`](https://huggingface.co/datasets/rescuerz/WorldMemBench).
+
+Install the selected metric dependencies into the prepared GPU environment;
+`worldfoundry/data/benchmarks/requirements/worldmembench.txt` lists the complete
+set without replacing its Torch/CUDA installation. Stage the official
+checkpoints explicitly. `--weights-dir` accepts the upstream metric asset
+layout; YAML checkpoint overrides can point to already staged shared assets.
+
+```bash
+python -m worldfoundry.evaluation.tasks.execution.runners.worldmembench.run_worldmembench_official_runner \
+  --run-official --input-path <manifest.jsonl> \
+  --weights-dir <metric-assets> --config <worldmembench.yaml> \
+  --output-dir <fresh-out> --json
+```
+
+Repeat `--metric` to select metrics. `--gpus 0,1` shards videos across visible
+GPUs. `--expected-revisits 1|2|3` selects the official fixed revisit protocol;
+otherwise manifest values or official return detection apply.
+
+DA3-Streaming and ViPE preparation reuse existing `da3/results.npz`,
+`da3/scene_geometry.npz` and `vipe/results.npz` artifacts. To generate missing
+artifacts, configure `da3.source`, `da3.dinov2_source` and `vipe.source` for the
+official model runtimes, with optional `python` and `weights` overrides in each
+section. DA3-Streaming is not replaced with single-batch DA3, nor ViPE's default
+pipeline with WorldFoundry's reduced pose-only profile. Dynamic Memory requires
+the official VLM configuration; API keys stay in its configured environment
+variable, not CLI arguments.
+
+## Import official results
+
+```bash
+worldfoundry-eval zoo benchmark-run --benchmark-id worldmembench \
+  --mode official-validation --official-results-path <official-eval-output> \
+  --result-model-id <model-id> --output-dir <out> --json
+```
+
+Accepts an evaluation output root, its `metrics/` directory, or a metric's
+`summary.json`. Multiple model groups require explicit selection. Summary
+coverage, group breakdowns and adjacent `cases.jsonl` records are retained in
+`scorecard.json`, `raw_metric_table.jsonl` and `per_sample_scores.jsonl`.
+
+## Limitations
+
+GPU/checkpoint and complete-dataset replay remain pending; in-tree execution
+and weight-free tests do not establish full-suite or leaderboard validity.
+DA3-Streaming and full-profile ViPE inference require their configured model
+runtimes unless artifacts are already available. `model_failure` retains its
+official scored result; `not_applicable` is not zero-filled. `eval_failed` and
+`data_invalid` retain coverage and cause a nonzero exit, even with partial scores.
diff --git a/docs/fumadocs/content/docs/evaluation/benchmark-hub/worldmembench.zh.mdx b/docs/fumadocs/content/docs/evaluation/benchmark-hub/worldmembench.zh.mdx
@@ -0,0 +1,89 @@
+---
+title: WorldMemBench
+description: 复用共享模型组件的仓内视觉记忆与相机控制评测。
+---
+
+<BenchmarkPageSections id="worldmembench" locale="zh" />
+
+## 范围
+
+[WorldMemBench](https://github.com/ZJU-REAL/WorldMemBench) 评测人物、物体、环境和
+动态记忆，以及相机控制和视频质量。WorldFoundry 的 12 项官方指标协议均在仓内执行，
+基于 MIT 许可的上游版本 `1b79840143a98e017891e893b5d3ef75b93078a3` 适配，
+不需要额外检出或安装 WorldMemBench。
+
+| 指标 ID | 官方分数字段 | 方向 |
+| --- | --- | --- |
+| `imaging_quality` | `score` | 越高越好 |
+| `aesthetic_quality` | `score` | 越高越好 |
+| `human_identity` | `score` | 越高越好 |
+| `human_appearance` | `score` | 越高越好 |
+| `object_geometry` | `score` | 越高越好 |
+| `object_appearance` | `score` | 越高越好 |
+| `environment_visible_memory` | `score` | 越高越好 |
+| `environment_depth_fidelity` | `depth_spearman` | 越高越好 |
+| `environment_scene_reconstruction` | `reconstruction_score` | 越高越好 |
+| `dynamic_memory` | `score` | 越高越好 |
+| `camera_rotation` | `camera_rotation_rpe_path` | 越低越好 |
+| `camera_translation` | `camera_translation_rpe` | 越低越好 |
+
+分数保留官方单位，包括负的深度相关系数和无上界的相机误差，不构造跨指标平均分或综合分。
+
+## 共享组件与协议
+
+OpenAI CLIP、LAION 美学头和 DINO 权重加载复用 WorldFoundry 的规范组件；人物和物体
+外观共用一份掩码 DINOv2 特征提取实现。同次评测中，相同权重路径与设备共享模型，
+预处理和批大小状态保持独立。WorldScore、MemoBench 使用共享 LAION MLP，
+MIND、WBench、WorldArena 使用共享线性头，两种头不能互换。WorldScore 与
+4DWorldBench 共用其 512 方形 IQA 预处理和基类，该变换不应用于 WorldMemBench。
+
+WorldMemBench 保留 MUSIQ-SPAQ、保持宽高比的 OpenCV 缩放、每 4 帧采样、分数归一化
+和视频等权聚合。重访分段、冻结目标关联、掩码裁剪、相机对齐、场景重建，以及动态记忆
+按事件门控的 VLM 问答均遵循官方协议。
+
+## 评测生成输出
+
+使用官方 `model/category/trajectory/case/video.mp4` 目录结构或官方 JSONL 清单。
+每个 case 保留标注、参考图、轨迹文件及目标相机位姿。数据集为
+[`rescuerz/WorldMemBench`](https://huggingface.co/datasets/rescuerz/WorldMemBench)。
+
+在已准备的 GPU 环境中安装所选指标的依赖；
+`worldfoundry/data/benchmarks/requirements/worldmembench.txt` 列出完整依赖，
+不替换环境中的 Torch/CUDA。显式准备官方权重；`--weights-dir` 接受上游指标资产布局，
+YAML 中的 checkpoint 覆盖项可以指向已经准备的共享资产。
+
+```bash
+python -m worldfoundry.evaluation.tasks.execution.runners.worldmembench.run_worldmembench_official_runner \
+  --run-official --input-path <manifest.jsonl> \
+  --weights-dir <metric-assets> --config <worldmembench.yaml> \
+  --output-dir <fresh-out> --json
+```
+
+重复 `--metric` 可选择指标，`--gpus 0,1` 按视频划分到可见 GPU。
+`--expected-revisits 1|2|3` 选择官方固定重访协议；否则使用清单中的值或官方返回检测。
+
+DA3-Streaming 和 ViPE 预处理复用已有的 `da3/results.npz`、
+`da3/scene_geometry.npz` 和 `vipe/results.npz`。如需生成缺失资产，配置
+`da3.source`、`da3.dinov2_source` 和 `vipe.source` 指向官方模型运行时，
+各节还可指定 `python` 和 `weights`。不将 DA3-Streaming 替换为单批 DA3，
+也不将 ViPE 默认流水线替换为 WorldFoundry 的精简位姿配置。动态记忆需要官方 VLM
+配置；API 密钥保留在配置指定的环境变量中，不通过命令行传递。
+
+## 导入官方结果
+
+```bash
+worldfoundry-eval zoo benchmark-run --benchmark-id worldmembench \
+  --mode official-validation --official-results-path <official-eval-output> \
+  --result-model-id <model-id> --output-dir <out> --json
+```
+
+接受评测输出根目录、其中的 `metrics/` 目录或单指标的 `summary.json`。
+多模型结果必须显式选择模型。汇总覆盖率、分组明细和相邻的 `cases.jsonl` 记录
+保留在 `scorecard.json`、`raw_metric_table.jsonl` 和 `per_sample_scores.jsonl` 中。
+
+## 限制
+
+GPU/真实权重和完整数据集回放尚未完成；仓内执行与免权重测试不代表全套验证或排行榜
+有效性。没有预计算资产时，DA3-Streaming 和完整配置 ViPE 推理仍需要配置对应模型
+运行时。`model_failure` 保留官方赋分，`not_applicable` 不填零。
+`eval_failed` 和 `data_invalid` 保留覆盖率，并使运行器返回非零，即使已有部分分数。
diff --git a/docs/fumadocs/lib/benchmark-catalog-status.json b/docs/fumadocs/lib/benchmark-catalog-status.json
@@ -1810,6 +1810,27 @@
     "summaryZh": "仓内 evaluator 按 task protocol 评估 artifact bundle",
     "verificationStatus": "bounded_fixture_verified"
   },
+  "worldmembench": {
+    "aliases": [],
+    "badges": ["integrated"],
+    "category": "World Models",
+    "categoryZh": "世界模型",
+    "domains": ["human-memory", "object-memory", "environment-memory", "dynamic-memory"],
+    "integrationStatus": "integrated",
+    "leaderboardValid": false,
+    "metrics": [
+      "imaging_quality", "aesthetic_quality", "human_identity", "human_appearance",
+      "object_geometry", "object_appearance", "environment_visible_memory",
+      "environment_depth_fidelity", "environment_scene_reconstruction", "dynamic_memory",
+      "camera_rotation", "camera_translation"
+    ],
+    "name": "WorldMemBench",
+    "requiresApi": false,
+    "runtimeKind": "in_tree_official_runtime",
+    "summary": "Runs twelve in-tree memory and camera metrics with shared model components",
+    "summaryZh": "复用共享模型组件，在仓内执行十二项记忆与相机指标",
+    "verificationStatus": "official_runtime_ready_real_data_validation_pending"
+  },
   "worldmodelbench": {
     "aliases": [],
     "badges": [