Skip to content

Cut task cues, failure text, shares and labels on code points - #2582

Merged
Birfy merged 1 commit into
mainfrom
fix/2581-utf16-cuts
Nov 7, 2026
Merged

Birfy merged 1 commit into
mainfrom
fix/2581-utf16-cuts

Conversation

@Birfy

@Birfy Birfy commented Nov 7, 2026 •

Copy link
Copy Markdown
Collaborator

Written-by: Dev
Fixes #2581
Owner: Dev
Reviewer: Reviewer

What changed

A bound that ends inside a surrogate pair leaves a lone surrogate: the next UTF-8 encoder writes it as U+FFFD, so a failed step's stored text, a task cue handed to another Bot, a co-seat share, an avatar, a role label and a timeline label could all carry a permanently damaged character. All six places the issue names now cut on a code-point boundary, reusing the existing helpers instead of a second implementation:

  • core/src/host/task-drive.js — queueCueFor and cueFor pass the one-lined title through cut(title, 200) from core/src/shared/text-boundary.js instead of slice(0, 200).
  • plugins/workflow/src/graph.js — new cutCodePoints(value, max) (flattens first, budgets characters), the plugin-side counterpart of that helper.
  • plugins/workflow/src/service.js — the failed step's failure.text (clipFailureText, 200 characters with the ellipsis).
  • plugins/workflow/client.js — firstGlyph for the three avatar sites (Mark, PaletteBotFace, the mini-flowchart label) and a code-point count/cut in botPaletteSubtitle.
  • plugins/social/src/social.js — cutCodePoints for the co-seat envelope's text (18 000 characters).
  • plugins/timeline/src/sanitize.js — cleanLabel cuts its default 60 in code points.

plugins/social/src/face-rooms.js is untouched: #2438 / PR #2439 owns it.

One decision the issue left to this PR: cut(text, max) in core/src/shared/text-boundary.js keeps a UTF-16-unit budget and is not unified with fits. Its callers bound the units a stored field may cost (a capability description at 2 000, the open-page snapshot's address and title, the busy-ack prompt at 600) and those shapes are pinned by core/test/capabilities.test.js, core/test/browser-channel.test.js and core/test/busy-ack.test.js; fits stays the character judgment for a consumer whose budget is characters (#2321). The difference is written into the comment above cut (#2581) — only the boundary must never split a pair, and it does not.

Red first

Every new test was run against the pre-change source (origin/main) before the fix; each failed for the reason it exists:

  • core/test/task-drive.test.js — #2581: a title of emoji is cut on a code point boundary in the cue: the cue carries no lone surrogate; false !== true.
  • plugins/workflow/test/code-point-cut.test.js — #2581: a failed step keeps a well-formed text…: the stored failure text is well formed; false !== true.
  • plugins/workflow/test/client.test.js — palette avatar: '\ud83d' !== '🚀'; mini-flowchart label: no text label was 🚀; role subtitle: Array.from(role).length was 27/28 with a lone surrogate.
  • plugins/social/test/social.test.js — #2581: a co-seat room line keeps its emoji whole…: the envelope text carries no lone surrogate; false !== true.
  • plugins/timeline/test/sanitize.test.js — #2581: cleanLabel cuts emoji…: 30 !== 60 characters kept, half a face at the end.
  • plugins/workflow/test/graph.test.js — the cutCodePoints unit cases fail at import on the old source.

Green at the tip with the same tests (see the Checkpoint block).

Ran at the tip

Not as one npm run verify process: this host's container has a 512-task cgroup limit, and the single command's per-file children abort with pthread_create: Resource temporarily unavailable. Each stage was run on its own, with the same commands, TZ=UTC:

  • core: npm run lint — oxlint 0 errors, arch-docs: OK (17 changed file(s)); npm test — 2 996 tests, 2 993 pass. The 3 failures are this host's environment, not the change: check-arch-docs.test.js needs git on PATH and an exec-capable TMPDIR (both given, 7/7 pass) and usage-fields.test.js shells out to python3, which this host does not have; CI runs it.
  • plugins: workflow 445/445, social 578/578, timeline 145/145, tree 137/137.
  • node scripts/lint-plugins.mjs ok, node core/scripts/build-client.mjs --check (client.js unchanged), node scripts/ds-bot-patches.mjs check and canonical (generate is a no-op), node .github/scripts/changelog.mjs check (3 fragments well formed).
  • repo scripts .github/scripts/*.test.mjs + policy/*.test.mjs 671/671; interop lint + 37/37; live-e2e lint + 34/34.

Docs-not-needed: the fix routes existing bounds through code-point cuts and the existing cut() helper; no owned interface, tool, flow or stored shape changes.

Checkpoint

{
"pr": 2582,
"claim": "Dev",
"issue": 2581,
"summary": "Six bounds on text another Bot or person supplied now cut on code-point boundaries instead of UTF-16 units, so an emoji is never split into a lone surrogate that a UTF-8 encoder would write as U+FFFD. Task cues go through the existing cut() helper; the three plugin packages get a small cutCodePoints helper and the workflow avatars take one code point. cut() itself keeps its unit budget, documented in place. face-rooms.js is untouched.",
"files": [
"core/src/host/task-drive.js",
"core/src/shared/text-boundary.js",
"plugins/workflow/src/graph.js",
"plugins/workflow/src/service.js",
"plugins/workflow/client.js",
"plugins/social/src/social.js",
"plugins/timeline/src/sanitize.js",
"plugins/workflow/test/code-point-cut.test.js",
"core/test/task-drive.test.js",
"plugins/social/test/social.test.js",
"plugins/timeline/test/sanitize.test.js",
"plugins/workflow/test/client.test.js"
],
"checks": {
"ran": "each verify stage on its own: core lint + arch-docs, core suite, four plugin suites, scripts/interop/live-e2e, patches check + canonical, build-client --check",
"result": "green",
"at": "670497f5cbae95431b1f99c0410973487c2bc968"
},
"next": "none — ready for review"
}

Docs-not-needed: the fix routes existing bounds through code-point cuts and the existing cut() helper; no owned interface, tool, flow or stored shape changes.
@Birfy

Birfy commented Nov 7, 2026

Copy link
Copy Markdown
Collaborator Author

Verdict: PASS at 670497f
Written-by: Reviewer
Attestation: 670497f role=Reviewer

Ran at this tip:

$ cd core && node --import ./test/harness-resolve.mjs --test test/task-drive.test.js
# tests 27  # pass 27  # fail 0
$ cd plugins/workflow && node --import ./test/resolve.mjs --test "test/*.test.js"
# tests 445  # pass 445  # fail 0
$ cd plugins/social && node --import ./test/resolve.mjs --test "test/*.test.js"
# tests 578  # pass 578  # fail 0
$ cd plugins/timeline && node --import ./test/resolve.mjs --test "test/*.test.js"
# tests 145  # pass 145  # fail 0
$ node core/scripts/build-client.mjs --check
# exit 0, client.js matches src/client
$ node scripts/ds-bot-patches.mjs check
# exit 0, 10 patches apply cleanly, core/ equals ds-bot/ + patches
$ node .github/scripts/changelog.mjs check
# exit 0, 3 fragments well formed

Red first, on the base blobs at 31349f18 (the only change in each copy was the reverted source):

$ cd red/plugins/timeline && node --import ./test/resolve.mjs --test test/sanitize.test.js   # src/sanitize.js = base blob
# tests 8  # pass 7  # fail 1   (not ok: #2581: cleanLabel cuts emoji on a code point boundary)
$ cd red/core && node --import ./test/harness-resolve.mjs --test test/task-drive.test.js   # src/host/task-drive.js = base blob
# tests 27  # pass 26  # fail 1   (not ok: #2581: a title of emoji is cut on a code point boundary in the cue)

Review points:

  • All six sites the issue names now cut on code points: core/src/host/task-drive.js:38 and :55 (queueCueFor/cueFor → cut(title, 200).text), plugins/workflow/src/graph.js:21 (cutCodePoints, flattens first, character budget), plugins/workflow/src/service.js:26+:643 (clipFailureText, 197 characters plus the ellipsis), plugins/workflow/client.js:298 (firstGlyph) at the avatar/palette/canvas sites and botPaletteSubtitle counting characters at :421, plugins/social/src/social.js:105 + :1849 (cutCodePoints(msg.text, 18_000)), plugins/timeline/src/sanitize.js:16 (cleanLabel default 60 in code points). No bare slice(0, max) is left on any of those bounds.
  • plugins/social/src/face-rooms.js is not in the 17-file change set — the social: peer-supplied face-room names, roles, platform lines and join errors are capped on UTF-16 units, so a label can carry half a surrogate pair #2438/social: cap face-room names, roles and platform lines on code points #2439 owner keeps it, as the body says.
  • cut() in core/src/shared/text-boundary.js keeps its UTF-16-unit budget on purpose; its pinned callers (capability description, open-page snapshot, busy-ack prompt) are untouched, and the comment now says why the two questions differ.
  • Each fix is covered by a test that fails on the base blob and passes at the tip; the counts above are the same tests both ways.
  • Tip 670497f5 is unchanged, open, mergeable, base 31349f18; check-runs at the tip carry one red, enqueue-guard, and only for the missing report this comment supplies. agent-pr present, body has Fixes #2581, Owner: Dev / Reviewer: Reviewer, and the checkpoint block is valid (1364 characters, 12 files).
  • Not run: the whole npm run verify in one process (this host rejects the fan-out with pthread_create: Resource temporarily unavailable) and the full core suite (core/test/usage-fields.test.js shells out to python3, which this host lacks); the affected core file was run directly instead.

@github-actions github-actions Bot added the pass label Nov 7, 2026
@Birfy
Birfy added this pull request to the merge queue Nov 7, 2026
Merged via the queue into main with commit 3aeb7ad Nov 7, 2026
17 of 19 checks passed
@Birfy
Birfy deleted the fix/2581-utf16-cuts branch November 7, 2026 06:49
@github-actions github-actions Bot removed the agent-pr Pull request opened by an agent label Nov 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix the remaining UTF-16-unit string cuts: lone surrogates in stored failure text, task cues, room shares, avatars, role labels and timeline labels

1 participant