Skip to content

expose complex-scripts feature (#24094) - #25674

Open
unclepomedev wants to merge 1 commit into
bevyengine:mainfrom
unclepomedev:main
Open

expose complex-scripts feature (#24094)#25674
unclepomedev wants to merge 1 commit into
bevyengine:mainfrom
unclepomedev:main

Conversation

@unclepomedev

Copy link
Copy Markdown

Objective

Fixes #24094

(Rebase of #24683, closed due to merge conflicts on main. Re-verified the fix still works on current main. That PR had received a community review approval.)

Text containing complex scripts that lack explicit word boundaries (CJK, Thai, Khmer, Lao, Myanmar) is segmented by parley's lightweight segmenter, which emits a data error per layout:

ICU4X data error: No segmentation model for language: ja

parley added a complex-scripts Cargo feature that switches to dictionary-based segmentation, but it was not reachable from Bevy.

Solution

  • Add an opt-in complex_script_segmentation feature to bevy_text that forwards to parley/complex-scripts, propagated through bevy_internal and bevy. It is off by default because the bundled segmentation dictionary increases binary size (see below).

This supplies the segmentation data and enables dictionary-based word segmentation; it does not change CJK line-break positions, which ICU keeps permissive by design.

Testing

  • cargo build -p bevy_text -p bevy_ui -p bevy_ui_widgets, with default features and with --features complex_script_segmentation.
  • cargo tree -e features -i parley confirms parley's complex-scripts feature is only enabled when complex_script_segmentation is requested (not pulled in by any default/collection feature).
  • Ran a Text2d example with a Japanese string in a width-bounded box (TextBounds, LineBreak::WordBoundary) under RUST_LOG=warn:
    • default: the No segmentation model for language: ja error is logged.
    • --features complex_script_segmentation: the error is gone.
  • cargo run -p ci (lints + compile + tests). One unrelated local failure: bevy_ecs ... filtered_backtrace_test, which also fails on main on my machine (an environment-dependent backtrace-symbolization test, unrelated to this PR).
  • Tested on macOS (Apple Silicon).

Binary size impact

Measured with a stripped release build of the text_debug example:

  • without complex_script_segmentation: 57,018,696 bytes
  • with complex_script_segmentation: 60,832,984 bytes
  • delta: ~3.6 MiB / ~6.7%

It seems the feature should be an opt-in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ICU4X data error: No segmentation model for language: ja when updating CJK characters

1 participant