Add async texture load hook to glTF extension handlers - #25669
Add async texture load hook to glTF extension handlers#25669jiangheng90-opensource wants to merge 1 commit into
Conversation
|
Welcome, new contributor! Please make sure you've read our contributing guide, as well as our policy regarding AI usage, and we look forward to reviewing your pull request shortly ✨ |
Add `GltfExtensionHandler::on_texture_load`, letting extension handlers take over the loading of individual glTF textures before the default loading runs. The hook follows the same pattern as `on_gltf_primitive`: handlers receive the raw `gltf::Texture` and `gltf::Gltf` so they can resolve image references stored in extension data, and provide the loaded image through a `&mut Option<Image>` out parameter. If no handler sets it, the default texture loading is used. This enables third-party support for extensions like `KHR_texture_basisu` (bevyengine#19104), which moves the image reference into the extension and may omit the standard `texture.source`. Handled textures are loaded sequentially since handlers get exclusive access to the load context, while textures without a handler keep going through the existing parallel default loading. This is a rework of the approach in bevyengine#24207: the hook is async (matching `on_gltf_primitive`), receives the raw glTF texture instead of a parsed intermediate, and the parallel texture loading is preserved.
c60c1bb to
2cad45d
Compare
|
A live WebAssembly demo of the consumer plugin is available here: https://jiangheng90-opensource.github.io/bevy_gltf_basisu/ it needs a little long time to load textures
It loads the Khronos StainedGlassLamp The demo page is rebuilt and deployed by CI on every push to the plugin Original Chinese text (AI-assisted translation above)基于本 PR 的 https://jiangheng90-opensource.github.io/bevy_gltf_basisu/ 演示加载 Khronos StainedGlassLamp 演示页面由插件仓库 main 分支的 CI 自动构建部署。 |

Objective
Fixes #19104. Related: #24207, an earlier draft of a similar hook. This PR instead extends the
on_gltf_primitivepattern (established in #23376) to texture loading.bevy's glTF loader currently does not support the
KHR_texture_basisuextension. Decoding it requires FFI (the Basis Universal transcoder), which is unlikely to live in the bevy_gltf crate itself, so this PR adds an extension point that lets third-party handlers take over texture decoding (ETC1S/UASTC KTX2 → BC7/ETC2/ASTC).The same decoding has been running in my own project (a mini Cesium implementation in Rust) via a customized bevy_gltf, successfully loading KTX2 + Draco encoded 3D Tiles services from Cesium ion. I would like to share this solution upstream: by moving just the interface into bevy's glTF module, a plugin based on basis_transcoder can do the same on both wasm and native through one API — the same pattern as bevy_gltf_draco.
A working
KHR_texture_basisuplugin built on this hook exists here: bevy_gltf_basisu.Disclosure: this PR was developed with substantial AI assistance for implementation and test drafting under my direction; the design decisions, code review and verification are my own. (AI was also used to translate this description; the original Chinese text is attached below.)
Solution
Add an async
on_texture_loadhook toGltfExtensionHandler, following the same out-parameter pattern ason_gltf_primitive:Unlike #24207, the hook is async (matching
on_gltf_primitive) and receives the rawgltf::Texture, so handlers can resolve image references stored in extension data (e.g.KHR_texture_basisu, which may omit the standardtexture.source).Implementation: the first loop calls every handler for each texture; a handler claims a texture by setting
user_image. The second loop loads all unclaimed textures through the existing default path — the parallelIoTaskPooltexture loading is preserved unchanged, so assets without handlers see no behavior change.Note: textures that omit
sourcefail glTF validation, so handlers for such extensions requireGltfLoaderSettings::validate = false(documented on the hook).Testing
gltf_extension_texture_load_hooktest: two textures that omittexture.sourceand carry a test extension (theKHR_texture_basisushape). Asserts that a bypass-only handler sees all textures without interfering, that the loading handler claims both, and that the materials end up referencing the handler-provided images.cargo test -p bevy_gltf --all-featurespasses.cargo clippy -p bevy_gltf --all-targets --all-features -- -D warningsis clean.Implementation details
Loading flow
Texture loading now happens in three passes:
&mut LoadContextaccess). A handler claims the texture by settinguser_image; the image is wrapped asImageOrPath::Imagewith the usualTextureNlabel.load_imageunchanged — parallel viaIoTaskPoolon native, serial on wasm / single-texture, exactly as before.process_loaded_textureand the existingon_texturehook fires, sotexture_handlesindices keep matching glTF texture indices for material lookup.Design decisions
user_image: &mut Option<Image>mirrorson_gltf_primitive(Async handler hook mesh #23376) instead of introducing a new result enum. Handlers that fail log the error and leaveuser_imageunset, which falls back to the default loading — the same convention the draco handler uses.gltf::Texture, thegltf::Gltfdocument andbuffer_data, because extensions likeKHR_texture_basisukeep the image reference in the extension JSON. The hook runs beforeTexture::source(), which panics whensourceis omitted (the gltf crate is compiled withoutallow_empty_texture).on_gltf_primitive). The default path keeps its parallelism, so assets without handlers see no behavior or performance change.is_srgb,samplerandrender_asset_usagesare resolved by the loader (linear texture set,override_sampler, loader settings) and passed in, so handlers don't re-derive them.sourcefail glTF validation, so such assets requireGltfLoaderSettings::validate = false(documented on the hook).Real-world validation
The bevy_gltf_basisu plugin implements
KHR_texture_basisuon top of this hook. Its integration tests cover ETC1S and UASTC, sRGB and linear textures, bufferView / data URI / external URI image sources, and zstd-supercompressed KTX2 levels, using real assets: an ETC1S texture from a 3D Tiles service and the full Khronos StainedGlassLampglTF-KTX-BasisUmodel (19/19 textures transcoded).The solution running in my Cesium project: KTX2 (BasisLZ) + Draco encoded 3D Tiles streamed from Cesium ion (Japan 3D Buildings).
Original Chinese text (AI-assisted translation above)
Objective
由于bevy目前并不支持gltf KHR_texture_basisu拓展,该拓展的解码还是需要依赖ffi,而这个功能不太可能在bevy gltf crate中实现,所以我希望增设一个接口来拓展解码 ETC1S/UASTC KTX2 → BC7/ETC2/ASTC的情况。这部分功能已在我自己的开发中项目 rust cesium 中通过定制修改的 bevy_gltf 实现,能够成功解析 ion 上的 ktx2+draco 编码的 3dtiles 服务。我希望把这个解决方案分享出来:只把接口搬进 bevy 的 gltf 模块,基于 basis-transcoder 的插件就能和 bevy-gltf-draco 一样,用一套 api 完成 wasm 和 native 的解码。该pr 深度使用 AI 辅助,但是设计上经过人类的审查与决策
Solution
Add an async
on_texture_loadhook toGltfExtensionHandler,采用与on_gltf_primitive相同的 out 参数模式(user_image: &mut Option<Image>)。与 #24207 的关系:那是更早之前一个类似 hook 的草案;本 PR 是把 #23376 确立的
on_gltf_primitive模式延伸到纹理加载。和#24207不同的是,它是一个异步接口,它先接管 image 解码是否需要在拓展中执行,假如过滤失败则退回到现有默认的图片解码。 实现上第一遍循环对每个纹理依次调用 handlers,handler 通过设置user_image认领;第二遍对无人认领的纹理走原有默认解码(保持并行)。Testing
gltf_extension_texture_load_hooktest: two textures that omittexture.sourceand carry a test extension (theKHR_texture_basisushape). Asserts that a bypass-only handler sees all textures without interfering, that the loading handler claims both, and that the materials end up referencing the handler-provided images.cargo test -p bevy_gltf --all-featurespasses.cargo clippy -p bevy_gltf --all-targets --all-features -- -D warningsis clean.A real-world handler built on this hook (ETC1S/UASTC KTX2 → BC7/ETC2/ASTC transcoding for
KHR_texture_basisuin 3D Tiles / Cesium ion assets) already exists downstream and can be upstreamed or published as a plugin once the hook lands.Implementation details(实现详情)
加载流程
纹理由三个 pass 处理:
&mut LoadContext)。handler 通过设置user_image认领纹理,认领的 image 包装为ImageOrPath::Image,沿用TextureN标签。load_image,native 上保持IoTaskPool并行、wasm/单纹理串行,与改动前完全一致。process_loaded_texture并触发已有的on_texturehook,保证texture_handles的下标与 glTF 纹理下标一致,材质引用不错位。设计决策
user_image: &mut Option<Image>对齐on_gltf_primitive(Async handler hook mesh #23376),不引入新的结果枚举。handler 失败时自行记录错误并保持user_image未设置,即回退默认路径——与 draco handler 的惯例一致。gltf::Texture、gltf::Gltf文档和buffer_data,因为KHR_texture_basisu这类扩展把 image 引用放在扩展 JSON 里。hook 在Texture::source()之前执行——gltf crate 未开allow_empty_texture时,省略source会在source()里 panic。&mut LoadContext独占访问的代价(与on_gltf_primitive同一取舍);默认路径保持并行,没有 handler 的资产行为和性能零变化。is_srgb、sampler、render_asset_usages由 loader 解析(线性纹理集合、override_sampler、加载设置)后传入,handler 不必重复推导。source的纹理过不了 glTF 校验,此类资产需要GltfLoaderSettings::validate = false(已写进 hook 文档)。真实场景验证
bevy_gltf_basisu 插件基于此 hook 实现了
KHR_texture_basisu。其集成测试覆盖 ETC1S/UASTC、sRGB/线性、bufferView/data URI/外部 URI 三种图片来源、zstd 超压缩 KTX2,全部使用真实资产:来自 3D Tiles 服务的 ETC1S 纹理和完整的 Khronos StainedGlassLampglTF-KTX-BasisU模型(19/19 纹理全部转码)。