Skip to content

Add async texture load hook to glTF extension handlers - #25669

Closed
jiangheng90-opensource wants to merge 1 commit into
bevyengine:mainfrom
jiangheng90-opensource:gltf-texture-load-hook
Closed

Add async texture load hook to glTF extension handlers#25669
jiangheng90-opensource wants to merge 1 commit into
bevyengine:mainfrom
jiangheng90-opensource:gltf-texture-load-hook

Conversation

@jiangheng90-opensource

@jiangheng90-opensource jiangheng90-opensource commented Sep 3, 2026

Copy link
Copy Markdown

Objective

Fixes #19104. Related: #24207, an earlier draft of a similar hook. This PR instead extends the on_gltf_primitive pattern (established in #23376) to texture loading.

bevy's glTF loader currently does not support the KHR_texture_basisu extension. Decoding it requires FFI (the Basis Universal transcoder), which is unlikely to live in the bevy_gltf crate itself, so this PR adds an extension point that lets third-party handlers take over texture decoding (ETC1S/UASTC KTX2 → BC7/ETC2/ASTC).

The same decoding has been running in my own project (a mini Cesium implementation in Rust) via a customized bevy_gltf, successfully loading KTX2 + Draco encoded 3D Tiles services from Cesium ion. I would like to share this solution upstream: by moving just the interface into bevy's glTF module, a plugin based on basis_transcoder can do the same on both wasm and native through one API — the same pattern as bevy_gltf_draco.

A working KHR_texture_basisu plugin built on this hook exists here: bevy_gltf_basisu.

Disclosure: this PR was developed with substantial AI assistance for implementation and test drafting under my direction; the design decisions, code review and verification are my own. (AI was also used to translate this description; the original Chinese text is attached below.)

Solution

Add an async on_texture_load hook to GltfExtensionHandler, following the same out-parameter pattern as on_gltf_primitive:

fn on_texture_load(
    &mut self,
    load_context: &mut LoadContext<'_>,
    gltf_document: &gltf::Gltf,
    gltf_texture: &gltf::Texture,
    buffer_data: &[Vec<u8>],
    gltf_path: &AssetPath<'_>,
    is_srgb: bool,
    sampler: ImageSamplerDescriptor,
    supported_compressed_formats: CompressedImageFormats,
    render_asset_usages: RenderAssetUsages,
    user_image: &mut Option<Image>,
) -> impl ConditionalSendFuture<Output = ()>

Unlike #24207, the hook is async (matching on_gltf_primitive) and receives the raw gltf::Texture, so handlers can resolve image references stored in extension data (e.g. KHR_texture_basisu, which may omit the standard texture.source).

Implementation: the first loop calls every handler for each texture; a handler claims a texture by setting user_image. The second loop loads all unclaimed textures through the existing default path — the parallel IoTaskPool texture loading is preserved unchanged, so assets without handlers see no behavior change.

Note: textures that omit source fail glTF validation, so handlers for such extensions require GltfLoaderSettings::validate = false (documented on the hook).

Testing

  • Added gltf_extension_texture_load_hook test: two textures that omit texture.source and carry a test extension (the KHR_texture_basisu shape). Asserts that a bypass-only handler sees all textures without interfering, that the loading handler claims both, and that the materials end up referencing the handler-provided images.
  • cargo test -p bevy_gltf --all-features passes.
  • cargo clippy -p bevy_gltf --all-targets --all-features -- -D warnings is clean.
  • The bevy_gltf_basisu plugin has an integration test loading a real Cesium ion ETC1S 3D Tiles asset through this hook.

Implementation details

Loading flow

Texture loading now happens in three passes:

  1. Handler pass: for each texture, every registered handler is called sequentially (handlers get exclusive &mut LoadContext access). A handler claims the texture by setting user_image; the image is wrapped as ImageOrPath::Image with the usual TextureN label.
  2. Default pass: textures no handler claimed go through the existing load_image unchanged — parallel via IoTaskPool on native, serial on wasm / single-texture, exactly as before.
  3. Finalize pass: in original texture order, each result is registered via process_loaded_texture and the existing on_texture hook fires, so texture_handles indices keep matching glTF texture indices for material lookup.

Design decisions

  • Out-parameter pattern: user_image: &mut Option<Image> mirrors on_gltf_primitive (Async handler hook mesh #23376) instead of introducing a new result enum. Handlers that fail log the error and leave user_image unset, which falls back to the default loading — the same convention the draco handler uses.
  • Raw glTF access: handlers receive the gltf::Texture, the gltf::Gltf document and buffer_data, because extensions like KHR_texture_basisu keep the image reference in the extension JSON. The hook runs before Texture::source(), which panics when source is omitted (the gltf crate is compiled without allow_empty_texture).
  • Sequential handlers, parallel default: the sequential handler pass is the price of exclusive load-context access (same trade-off as on_gltf_primitive). The default path keeps its parallelism, so assets without handlers see no behavior or performance change.
  • Pre-resolved parameters: is_srgb, sampler and render_asset_usages are resolved by the loader (linear texture set, override_sampler, loader settings) and passed in, so handlers don't re-derive them.
  • Validation: textures omitting source fail glTF validation, so such assets require GltfLoaderSettings::validate = false (documented on the hook).

Real-world validation

The bevy_gltf_basisu plugin implements KHR_texture_basisu on top of this hook. Its integration tests cover ETC1S and UASTC, sRGB and linear textures, bufferView / data URI / external URI image sources, and zstd-supercompressed KTX2 levels, using real assets: an ETC1S texture from a 3D Tiles service and the full Khronos StainedGlassLamp glTF-KTX-BasisU model (19/19 textures transcoded).

KTX2 (BasisLZ) + Draco encoded 3D Tiles from Cesium ion rendered in the author's Cesium project

The solution running in my Cesium project: KTX2 (BasisLZ) + Draco encoded 3D Tiles streamed from Cesium ion (Japan 3D Buildings).

Original Chinese text (AI-assisted translation above)

Objective

由于bevy目前并不支持gltf KHR_texture_basisu拓展,该拓展的解码还是需要依赖ffi,而这个功能不太可能在bevy gltf crate中实现,所以我希望增设一个接口来拓展解码 ETC1S/UASTC KTX2 → BC7/ETC2/ASTC的情况。这部分功能已在我自己的开发中项目 rust cesium 中通过定制修改的 bevy_gltf 实现,能够成功解析 ion 上的 ktx2+draco 编码的 3dtiles 服务。我希望把这个解决方案分享出来:只把接口搬进 bevy 的 gltf 模块,基于 basis-transcoder 的插件就能和 bevy-gltf-draco 一样,用一套 api 完成 wasm 和 native 的解码。该pr 深度使用 AI 辅助,但是设计上经过人类的审查与决策

Solution

Add an async on_texture_load hook to GltfExtensionHandler,采用与 on_gltf_primitive 相同的 out 参数模式(user_image: &mut Option<Image>)。

#24207 的关系:那是更早之前一个类似 hook 的草案;本 PR 是把 #23376 确立的 on_gltf_primitive 模式延伸到纹理加载。和#24207不同的是,它是一个异步接口,它先接管 image 解码是否需要在拓展中执行,假如过滤失败则退回到现有默认的图片解码。 实现上第一遍循环对每个纹理依次调用 handlers,handler 通过设置 user_image 认领;第二遍对无人认领的纹理走原有默认解码(保持并行)。

Testing

  • Added gltf_extension_texture_load_hook test: two textures that omit texture.source and carry a test extension (the KHR_texture_basisu shape). Asserts that a bypass-only handler sees all textures without interfering, that the loading handler claims both, and that the materials end up referencing the handler-provided images.
  • cargo test -p bevy_gltf --all-features passes.
  • cargo clippy -p bevy_gltf --all-targets --all-features -- -D warnings is clean.

A real-world handler built on this hook (ETC1S/UASTC KTX2 → BC7/ETC2/ASTC transcoding for KHR_texture_basisu in 3D Tiles / Cesium ion assets) already exists downstream and can be upstreamed or published as a plugin once the hook lands.

Implementation details(实现详情)

加载流程

纹理由三个 pass 处理:

  1. handler pass:对每个纹理按注册顺序串行调用所有 handler(handler 独占 &mut LoadContext)。handler 通过设置 user_image 认领纹理,认领的 image 包装为 ImageOrPath::Image,沿用 TextureN 标签。
  2. 默认 pass:无人认领的纹理走原有 load_image,native 上保持 IoTaskPool 并行、wasm/单纹理串行,与改动前完全一致。
  3. 收尾 pass:按纹理原始顺序统一 process_loaded_texture 并触发已有的 on_texture hook,保证 texture_handles 的下标与 glTF 纹理下标一致,材质引用不错位。

设计决策

  • out 参数模式user_image: &mut Option<Image> 对齐 on_gltf_primitiveAsync handler hook mesh #23376),不引入新的结果枚举。handler 失败时自行记录错误并保持 user_image 未设置,即回退默认路径——与 draco handler 的惯例一致。
  • 原始 glTF 访问:handler 拿到 gltf::Texturegltf::Gltf 文档和 buffer_data,因为 KHR_texture_basisu 这类扩展把 image 引用放在扩展 JSON 里。hook 在 Texture::source() 之前执行——gltf crate 未开 allow_empty_texture 时,省略 source 会在 source() 里 panic。
  • handler 串行、默认并行:串行是 &mut LoadContext 独占访问的代价(与 on_gltf_primitive 同一取舍);默认路径保持并行,没有 handler 的资产行为和性能零变化。
  • 预解析参数is_srgbsamplerrender_asset_usages 由 loader 解析(线性纹理集合、override_sampler、加载设置)后传入,handler 不必重复推导。
  • 校验:省略 source 的纹理过不了 glTF 校验,此类资产需要 GltfLoaderSettings::validate = false(已写进 hook 文档)。

真实场景验证

bevy_gltf_basisu 插件基于此 hook 实现了 KHR_texture_basisu。其集成测试覆盖 ETC1S/UASTC、sRGB/线性、bufferView/data URI/外部 URI 三种图片来源、zstd 超压缩 KTX2,全部使用真实资产:来自 3D Tiles 服务的 ETC1S 纹理和完整的 Khronos StainedGlassLamp glTF-KTX-BasisU 模型(19/19 纹理全部转码)。

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Welcome, new contributor!

Please make sure you've read our contributing guide, as well as our policy regarding AI usage, and we look forward to reviewing your pull request shortly ✨

Add `GltfExtensionHandler::on_texture_load`, letting extension handlers
take over the loading of individual glTF textures before the default
loading runs. The hook follows the same pattern as `on_gltf_primitive`:
handlers receive the raw `gltf::Texture` and `gltf::Gltf` so they can
resolve image references stored in extension data, and provide the
loaded image through a `&mut Option<Image>` out parameter. If no
handler sets it, the default texture loading is used.

This enables third-party support for extensions like
`KHR_texture_basisu` (bevyengine#19104), which moves the image reference into the
extension and may omit the standard `texture.source`.

Handled textures are loaded sequentially since handlers get exclusive
access to the load context, while textures without a handler keep going
through the existing parallel default loading.

This is a rework of the approach in bevyengine#24207: the hook is async (matching
`on_gltf_primitive`), receives the raw glTF texture instead of a parsed
intermediate, and the parallel texture loading is preserved.
@jiangheng90-opensource

jiangheng90-opensource commented Sep 3, 2026

Copy link
Copy Markdown
Author

A live WebAssembly demo of the consumer plugin is available here:

https://jiangheng90-opensource.github.io/bevy_gltf_basisu/

it needs a little long time to load textures

截屏2026-09-03 21 59 03

It loads the Khronos StainedGlassLamp glTF-KTX-BasisU model — all 19 textures use KHR_texture_basisu — through bevy_gltf_basisu, a plugin built entirely on the on_texture_load hook proposed in this PR (no bevy fork). The same plugin runs natively and on wasm (in a Web Worker), transcoding to BC7/ETC2/ASTC where supported, with an RGBA8 fallback.

The demo page is rebuilt and deployed by CI on every push to the plugin
repo's main branch.

Original Chinese text (AI-assisted translation above)

基于本 PR 的 on_texture_load hook 实现的消费端插件,其 WebAssembly 在线演示见:

https://jiangheng90-opensource.github.io/bevy_gltf_basisu/

演示加载 Khronos StainedGlassLamp glTF-KTX-BasisU 模型(19 个纹理全部使用 KHR_texture_basisu),插件 bevy_gltf_basisu 完全构建于本 hook 之上(无需 fork bevy)。同一插件在 native 和 wasm(Web Worker 中)都按 GPU 支持转码到 BC7/ETC2/ASTC,RGBA8 兜底。

演示页面由插件仓库 main 分支的 CI 自动构建部署。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support KHR_texture_basisu in bevy_gltf

2 participants