From 6e130fc6bb5bff59deb4f7032bc17ee098b87475 Mon Sep 17 00:00:00 2001
From: iChubai <149308795+iChubai@users.noreply.github.com>
Date: Tue, 20 Oct 2026 18:46:29 +0800
Subject: [PATCH] [docs]Align WonderWorld notes with native Gaussian
compositing
Signed-off-by: iChubai <149308795+iChubai@users.noreply.github.com>
---
.../docs/guides/supported-models/wonderworld.mdx | 8 ++++----
.../guides/supported-models/wonderworld.zh.mdx | 8 ++++----
docs/fumadocs/lib/model-recipes-data.json | 15 ++++++++-------
.../catalog/three_d_four_d/wonderworld.yaml | 6 +++---
.../data/models/runtime/profiles/wonderworld.yaml | 3 ++-
5 files changed, 21 insertions(+), 19 deletions(-)
diff --git a/docs/fumadocs/content/docs/guides/supported-models/wonderworld.mdx b/docs/fumadocs/content/docs/guides/supported-models/wonderworld.mdx
index ddeecfffc..d03783d2c 100644
--- a/docs/fumadocs/content/docs/guides/supported-models/wonderworld.mdx
+++ b/docs/fumadocs/content/docs/guides/supported-models/wonderworld.mdx
@@ -21,7 +21,7 @@ The pipeline starts at the identity seed camera, then visits each supplied camer
## Environment and assets
-Use the [unified GPU environment](/docs/maintainers/contributing). `gsplat` needs a complete CUDA development toolkit for its first extension build, including `nvcc` and `cuda_runtime.h`; set `CUDA_HOME` to that toolkit if auto-discovery selects an incomplete installation. Triton computes opacity-crossing median depth from the same gsplat projections and tile lists.
+Use the [unified GPU environment](/docs/maintainers/contributing). `gsplat` needs a complete CUDA development toolkit for its first extension build, including `nvcc` and `cuda_runtime.h`; set `CUDA_HOME` to that toolkit if auto-discovery selects an incomplete installation. gsplat supplies differentiable projections and sorted tile lists. Triton composites RGB/alpha and computes opacity-crossing median depth with a shared alpha cap of 0.99. A surface contributes before its transmission triggers early termination.
WonderWorld uses shared released component weights rather than a separate trained WonderWorld generator checkpoint. The default checkpoint layout is:
@@ -82,7 +82,7 @@ worldfoundry-eval run wonderworld --print-config --json \
## Validation
-Component contracts cover FP32/BF16 DDIM and unscaled-Euler trajectories, schedule boundaries, stateless guidance probes, real small U-Net channel ordering, VAE posterior/RNG ordering, depth decoding, pinned recipes, and shared codec ownership. Analytic geometry contracts cover perspective pixel covariance, tilted ground planes, off-axis pixels, and camera rotation. Reconstruction contracts cover equal-depth compositing, bounded trainable ranges, frozen-prefix/suffix preservation, pruning-state migration, nonmutation of input scenes, and median-depth traversal after exactly 50% transmission. Public contracts exercise standard evaluation invocation, image aliases, camera controls, generation-parameter precedence, lazy loading, input preflight, and failed/cancelled initialization cleanup.
+Component contracts cover FP32/BF16 DDIM and unscaled-Euler trajectories, schedule boundaries, stateless guidance probes, real small U-Net channel ordering, VAE posterior/RNG ordering, depth decoding, pinned recipes, and shared codec ownership. Analytic geometry contracts cover perspective pixel covariance, tilted ground planes, off-axis pixels, and camera rotation. Reconstruction contracts cover equal-depth compositing, bounded trainable ranges, frozen-prefix/suffix preservation, pruning-state migration, nonmutation of input scenes, median-depth traversal after exactly 50% transmission, opaque-tail compositing, subpixel tile-boundary support, and RGB/alpha gradients. Continuation contracts cover fully covered views, current-camera conditioning, known-depth anchors, sky handling, and generation masks. Public contracts exercise standard evaluation invocation, image aliases, camera controls, generation-parameter precedence, lazy loading, input preflight, and failed/cancelled initialization cleanup.
The checkpoint-backed component contract checks every loaded tensor against its BF16-cast safetensors value, then compares native conditioning, initialization, U-Net prediction, scheduler update, and decoding against the corresponding network/schedule primitives. It also exercises decoded-depth gradients with frozen model weights:
@@ -116,8 +116,8 @@ The replay resolves the catalog loader and executes the standard evaluation invo
## Limitations
- Unrestricted CUDA decoded-depth gradients and photometric fitting do not guarantee bitwise repeatability. Small geometry differences can change disocclusion masks and subsequent generated regions. The exact A/B/A replay uses the documented MATH/deterministic-preference configuration and `refinement_steps=0`.
-- The translated/rolled room replay has visible boundary holes and fragmented surfaces; visual quality is not accepted. Native artifact validation, released-stage numerical comparison, independent official full-pipeline parity and performance acceptance are separate gates.
-- Native reconstruction uses gsplat RGB, opacity-crossing median depth, `0.8 L1 + 0.2 DSSIM`, and opacity/extent pruning. The pinned FLAGS fork regularizes covariance differently in forward and backward and can miss a later median-depth crossing after exactly 50% transmission. Native gradients are checked by finite differences and median depth by opacity-compositing fixtures, not fitted-output bitwise equivalence. Panorama generation, GPT prompts and the upstream browser server are outside this offline API. `stream()` is unsupported.
+- The translated/rolled room replay has visible boundary holes and fragmented surfaces. Scene-quality observations are separate from implementation validation; native request/artifact contracts and released-stage numerical behavior are tested independently of visual preferences. Independent official full-pipeline parity and performance remain unvalidated.
+- Native reconstruction uses gsplat projections, native RGB/alpha compositing, opacity-crossing median depth, `0.8 L1 + 0.2 DSSIM`, and opacity/extent pruning. The pinned FLAGS fork regularizes covariance differently in forward and backward and can miss a later median-depth crossing after exactly 50% transmission. Its subpixel tile bounds can also omit valid boundary contributions. Native gradients are checked by finite differences and median depth by opacity-compositing fixtures, not fitted-output bitwise equivalence. Panorama generation, GPT prompts and the upstream browser server are outside this offline API. `stream()` is unsupported.
- The pinned upstream repository publishes inference source but does not identify a root software license. Public availability is not an OSI-license claim. SD2 weights retain Open RAIL++-M terms; Marigold, OneFormer, RepViT-SAM, and gsplat retain their component terms. The catalog records component-specific licensing.
diff --git a/docs/fumadocs/content/docs/guides/supported-models/wonderworld.zh.mdx b/docs/fumadocs/content/docs/guides/supported-models/wonderworld.zh.mdx
index 37d19c55a..7f631c4c6 100644
--- a/docs/fumadocs/content/docs/guides/supported-models/wonderworld.zh.mdx
+++ b/docs/fumadocs/content/docs/guides/supported-models/wonderworld.zh.mdx
@@ -21,7 +21,7 @@ WorldFoundry 已将 **WonderWorld** 接入仓库内的离线图像到三维世
## 环境与权重
-使用[统一 GPU 环境](/docs/maintainers/contributing)。`gsplat` 首次构建扩展需要完整 CUDA 开发工具链，包括 `nvcc` 和 `cuda_runtime.h`；若自动发现的安装不完整，通过 `CUDA_HOME` 指向完整工具链。Triton 从同一份 gsplat 投影和瓦片列表计算透明度越过 50% 时的中位深度。
+使用[统一 GPU 环境](/docs/maintainers/contributing)。`gsplat` 首次构建扩展需要完整 CUDA 开发工具链，包括 `nvcc` 和 `cuda_runtime.h`；若自动发现的安装不完整，通过 `CUDA_HOME` 指向完整工具链。gsplat 提供可微投影和排序后的瓦片列表，Triton 合成 RGB/透明度并计算透明度中位深度，共用 0.99 的透明度上限。触发透射率提前终止的表面仍参与本次合成。
WonderWorld 使用已经公开的共享组件权重，不需要单独训练的 WonderWorld 生成器检查点。默认目录布局：
@@ -82,7 +82,7 @@ worldfoundry-eval run wonderworld --print-config --json \
## 验证
-组件契约覆盖 FP32/BF16 的 DDIM 与不缩放输入的 Euler 轨迹、排程边界、无状态引导探测、真实小 U-Net 的通道拼接、VAE 后验与随机数顺序、深度解码、固定配方和共享编解码器所有权。解析几何契约覆盖透视像素协方差、倾斜地面、离轴像素和相机旋转。重建契约覆盖同深度合成顺序、限定可训练区间、冻结前后缀保留、剪枝状态迁移、输入场景不变性，以及恰好 50% 透射率后的中位深度遍历。公共契约执行标准评估 invocation，检查图像别名、相机控制、生成参数优先级、延迟加载、输入预检及加载失败或取消时的清理。
+组件契约覆盖 FP32/BF16 的 DDIM 与不缩放输入的 Euler 轨迹、排程边界、无状态引导探测、真实小 U-Net 的通道拼接、VAE 后验与随机数顺序、深度解码、固定配方和共享编解码器所有权。解析几何契约覆盖透视像素协方差、倾斜地面、离轴像素和相机旋转。重建契约覆盖同深度合成顺序、限定可训练区间、冻结前后缀保留、剪枝状态迁移、输入场景不变性，以及恰好 50% 透射率后的中位深度遍历、高不透明度尾部合成、亚像素瓦片边界覆盖和 RGB/透明度梯度。续建契约覆盖完全可见视角、当前相机的条件图、已有深度锚点、天空处理和生成掩码。公共契约执行标准评估 invocation，检查图像别名、相机控制、生成参数优先级、延迟加载、输入预检及加载失败或取消时的清理。
真实权重组件契约逐张量核对加载值与 safetensors 转换为 BF16 后的值，并将原生条件编码、初始化、U-Net 预测、调度更新和解码与对应网络及调度原语比较，同时执行冻结模型权重下的解码深度梯度检查：
@@ -116,8 +116,8 @@ python tests/manual/wonderworld_inference.py \
## 限制
- 不约束后端时，CUDA 解码深度梯度和光度拟合不保证逐位重复；几何的细微差异可能改变显露区域掩码，进而改变后续生成内容。精确 A/B/A 回放使用上述 MATH/确定性优先配置及 `refinement_steps=0`。
-- 平移/滚转房间回放仍有可见的边界孔洞和表面碎裂，视觉质量未验收。原生产物验证、发布阶段数值对照、独立官方端到端对齐与性能验收属于不同门禁。
-- 原生重建使用 gsplat RGB、透明度中位深度、`0.8 L1 + 0.2 DSSIM` 及透明度/尺度剪枝。固定版本 FLAGS 的协方差正向与反向正则项不一致，且可能漏掉恰好 50% 透射率之后的中位深度跨越。原生梯度通过有限差分校验，中位深度通过透明度合成样例验证，不以拟合产物逐位一致作为证明。全景生成、GPT 提示词和上游浏览器服务器不属于此离线 API。暂不支持 `stream()`。
+- 平移/滚转房间回放存在可见的边界孔洞和表面碎裂。场景质量观察与实现验证分开记录；原生请求/产物契约及发布阶段数值行为独立于视觉偏好进行测试。独立官方端到端对齐与性能尚未验证。
+- 原生重建使用 gsplat 投影、原生 RGB/透明度合成、透明度中位深度、`0.8 L1 + 0.2 DSSIM` 及透明度/尺度剪枝。固定版本 FLAGS 的协方差正向与反向正则项不一致，且可能漏掉恰好 50% 透射率之后的中位深度跨越；其亚像素瓦片包围盒也可能漏掉边界上的有效贡献。原生梯度通过有限差分校验，中位深度通过透明度合成样例验证，不以拟合产物逐位一致作为证明。全景生成、GPT 提示词和上游浏览器服务器不属于此离线 API。暂不支持 `stream()`。
- 固定版本的上游仓库公开了推理源码，但没有标明根目录软件许可证。公开可获取不等于采用 OSI 许可证。SD2 权重保留 Open RAIL++-M 条款；Marigold、OneFormer、RepViT-SAM、gsplat 保留各自条款，目录按组件记录许可。
diff --git a/docs/fumadocs/lib/model-recipes-data.json b/docs/fumadocs/lib/model-recipes-data.json
index 007eff3d6..e648e82e5 100644
--- a/docs/fumadocs/lib/model-recipes-data.json
+++ b/docs/fumadocs/lib/model-recipes-data.json
@@ -111484,13 +111484,13 @@
"Camera paths use row-major OpenCV world-to-camera matrices; named upstream viewer matrices use column-major 16-vectors.",
"Exported scene depth equals Marigold scene depth times 1000; camera translations use these exported units.",
"Guided depth uses the final eight Euler steps and known scene/ground anchors; latent noise and U-Net input remain unscaled as in the pinned source.",
- "gsplat RGB rendering, opacity-crossing median depth, and 0.8 L1 plus 0.2 DSSIM fitting are native implementations; full official rasterizer and quality parity remain pending.",
+ "gsplat supplies differentiable projections and sorted tile lists; native Triton RGB/alpha and median depth share alpha cap 0.99 and retain the surface crossing the termination threshold.",
+ "Fitting uses 0.8 L1 plus 0.2 DSSIM. Integration contracts and numerical comparison are separate from scene-quality observations; full official pipeline parity remains pending.",
"New Gaussians precede the frozen scene at equal depth; only their bounded particle range is fitted and pruned.",
"Upstream reports 48 GB VRAM. Local component weights and a CUDA-compatible gsplat installation are required.",
"No separate WonderWorld generator checkpoint is needed. SD2 uses CreativeML Open RAIL++-M; retain all component licensing terms.",
"Uses the unified WorldFoundry GPU environment and an ABI-compatible gsplat build.",
- "All inference orchestration is in-tree; no external checkout, PyTorch3D, Flask server, or OpenAI credential is needed.",
- "Stage the public SD2 Inpainting, Marigold depth/normals, OneFormer ADE20K metadata/weights, and RepViT-SAM weights outside the source tree."
+ "All inference orchestration is in-tree; no external checkout, PyTorch3D, Flask server, or OpenAI credential is needed."
]
},
"sources": [
@@ -111709,7 +111709,7 @@
}
],
"notes": [
- "Native SD2 inpainting and Marigold recipes, canonical OneFormer/RepViT-SAM perception, guided depth, occlusion layers, and gsplat Gaussian reconstruction.",
+ "Native SD2 inpainting and Marigold recipes, canonical OneFormer/RepViT-SAM perception, guided depth, occlusion layers, and Gaussian reconstruction with gsplat projections and native compositing.",
"Finite offline image/camera-path requests export standard .splat artifacts without an external checkout or upstream server.",
"Full official FLAGS rasterizer/panorama parity and interactive browser acceptance remain separate validation scopes.",
"Native standard-runner recipes own SD2/Marigold diffusion; no upstream checkout, Diffusers pipeline, Flask server, or PyTorch3D extension is required.",
@@ -111717,7 +111717,8 @@
"Camera paths use row-major OpenCV world-to-camera matrices; named upstream viewer matrices use column-major 16-vectors.",
"Exported scene depth equals Marigold scene depth times 1000; camera translations use these exported units.",
"Guided depth uses the final eight Euler steps and known scene/ground anchors; latent noise and U-Net input remain unscaled as in the pinned source.",
- "gsplat RGB rendering, opacity-crossing median depth, and 0.8 L1 plus 0.2 DSSIM fitting are native implementations; full official rasterizer and quality parity remain pending.",
+ "gsplat supplies differentiable projections and sorted tile lists; native Triton RGB/alpha and median depth share alpha cap 0.99 and retain the surface crossing the termination threshold.",
+ "Fitting uses 0.8 L1 plus 0.2 DSSIM. Integration contracts and numerical comparison are separate from scene-quality observations; full official pipeline parity remains pending.",
"New Gaussians precede the frozen scene at equal depth; only their bounded particle range is fitted and pruned.",
"Upstream reports 48 GB VRAM. Local component weights and a CUDA-compatible gsplat installation are required.",
"No separate WonderWorld generator checkpoint is needed. SD2 uses CreativeML Open RAIL++-M; retain all component licensing terms."
@@ -111736,10 +111737,10 @@
"WonderWorld 从单张图像创建分层高斯世界，并在指定相机视角扩展场景。WorldFoundry 使用原生扩散配方与树内场景重建，导出标准 .splat 场景和视角预览。"
],
"architecture": [
- "Guided Marigold depth diffusion aligns generated views to existing geometry and a ground plane. RepViT-SAM refines region disparity, OneFormer identifies sky and occluders, SD2 completes hidden/new regions, and normal-oriented Gaussians are reconstructed and fitted with gsplat."
+ "Guided Marigold depth diffusion aligns generated views to existing geometry and a ground plane. RepViT-SAM refines region disparity, OneFormer identifies sky and occluders, SD2 completes hidden/new regions, and normal-oriented Gaussians use gsplat projections and native differentiable RGB/alpha compositing for reconstruction and fitting."
],
"architectureZh": [
- "引导式 Marigold 深度扩散将新视角对齐已有几何与地面。RepViT-SAM 细化区域视差，OneFormer 识别天空与遮挡物，SD2 补全隐藏和新出现区域，再通过 gsplat 重建并拟合法线定向的高斯。"
+ "引导式 Marigold 深度扩散将新视角对齐已有几何与地面。RepViT-SAM 细化区域视差，OneFormer 识别天空与遮挡物，SD2 补全隐藏和新出现区域，再通过 gsplat 投影与原生可微 RGB/透明度合成重建并拟合法线定向的高斯。"
],
"usageNotes": [
"Supply an RGB image and local SD2 Inpainting, Marigold depth/normals, OneFormer, and RepViT-SAM assets. Optional camera_path contains OpenCV world-to-camera matrices in exported scene units. Default image size is 512x512; calls own independent scenes while retaining weights."
diff --git a/worldfoundry/data/models/catalog/three_d_four_d/wonderworld.yaml b/worldfoundry/data/models/catalog/three_d_four_d/wonderworld.yaml
index a00d1dec8..02c26c7cc 100644
--- a/worldfoundry/data/models/catalog/three_d_four_d/wonderworld.yaml
+++ b/worldfoundry/data/models/catalog/three_d_four_d/wonderworld.yaml
@@ -33,7 +33,7 @@ integration:
kind: pipeline_runner
status: integrated
notes:
- - Native SD2 inpainting and Marigold recipes, canonical OneFormer/RepViT-SAM perception, guided depth, occlusion layers, and gsplat Gaussian reconstruction.
+ - Native SD2 inpainting and Marigold recipes, canonical OneFormer/RepViT-SAM perception, guided depth, occlusion layers, and Gaussian reconstruction with gsplat projections and native compositing.
- Finite offline image/camera-path requests export standard .splat artifacts without an external checkout or upstream server.
runtime_profile: runtime-profile:wonderworld
runner_target: worldfoundry.evaluation.models.runners.pipeline:WorldFoundryPipelineRunner
@@ -53,8 +53,8 @@ docs:
publisher_zh: 斯坦福大学 / 麻省理工学院
overview: WonderWorld creates a layered Gaussian world from one image and expands it at requested camera views. WorldFoundry runs native diffusion recipes and in-tree scene reconstruction, exporting a standard .splat scene and view previews.
overview_zh: WonderWorld 从单张图像创建分层高斯世界，并在指定相机视角扩展场景。WorldFoundry 使用原生扩散配方与树内场景重建，导出标准 .splat 场景和视角预览。
- architecture: Guided Marigold depth diffusion aligns generated views to existing geometry and a ground plane. RepViT-SAM refines region disparity, OneFormer identifies sky and occluders, SD2 completes hidden/new regions, and normal-oriented Gaussians are reconstructed and fitted with gsplat.
- architecture_zh: 引导式 Marigold 深度扩散将新视角对齐已有几何与地面。RepViT-SAM 细化区域视差，OneFormer 识别天空与遮挡物，SD2 补全隐藏和新出现区域，再通过 gsplat 重建并拟合法线定向的高斯。
+ architecture: Guided Marigold depth diffusion aligns generated views to existing geometry and a ground plane. RepViT-SAM refines region disparity, OneFormer identifies sky and occluders, SD2 completes hidden/new regions, and normal-oriented Gaussians use gsplat projections and native differentiable RGB/alpha compositing for reconstruction and fitting.
+ architecture_zh: 引导式 Marigold 深度扩散将新视角对齐已有几何与地面。RepViT-SAM 细化区域视差，OneFormer 识别天空与遮挡物，SD2 补全隐藏和新出现区域，再通过 gsplat 投影与原生可微 RGB/透明度合成重建并拟合法线定向的高斯。
usage_notes: Supply an RGB image and local SD2 Inpainting, Marigold depth/normals, OneFormer, and RepViT-SAM assets. Optional camera_path contains OpenCV world-to-camera matrices in exported scene units. Default image size is 512x512; calls own independent scenes while retaining weights.
usage_notes_zh: 提供 RGB 图像以及本地 SD2 Inpainting、Marigold 深度/法线、OneFormer 和 RepViT-SAM 资产。可选 camera_path 使用导出场景单位下的 OpenCV 世界到相机矩阵。默认图像为 512x512；每次调用独立生成场景并复用常驻权重。
limitations:
diff --git a/worldfoundry/data/models/runtime/profiles/wonderworld.yaml b/worldfoundry/data/models/runtime/profiles/wonderworld.yaml
index 4a2c6f936..e8c55111f 100644
--- a/worldfoundry/data/models/runtime/profiles/wonderworld.yaml
+++ b/worldfoundry/data/models/runtime/profiles/wonderworld.yaml
@@ -55,7 +55,8 @@ notes:
- Camera paths use row-major OpenCV world-to-camera matrices; named upstream viewer matrices use column-major 16-vectors.
- Exported scene depth equals Marigold scene depth times 1000; camera translations use these exported units.
- Guided depth uses the final eight Euler steps and known scene/ground anchors; latent noise and U-Net input remain unscaled as in the pinned source.
-- gsplat RGB rendering, opacity-crossing median depth, and 0.8 L1 plus 0.2 DSSIM fitting are native implementations; full official rasterizer and quality parity remain pending.
+- gsplat supplies differentiable projections and sorted tile lists; native Triton RGB/alpha and median depth share alpha cap 0.99 and retain the surface crossing the termination threshold.
+- Fitting uses 0.8 L1 plus 0.2 DSSIM. Integration contracts and numerical comparison are separate from scene-quality observations; full official pipeline parity remains pending.
- New Gaussians precede the frozen scene at equal depth; only their bounded particle range is fitted and pruned.
- Upstream reports 48 GB VRAM. Local component weights and a CUDA-compatible gsplat installation are required.
- No separate WonderWorld generator checkpoint is needed. SD2 uses CreativeML Open RAIL++-M; retain all component licensing terms.