E2E test for text formatting shortcuts (Notion test case 364) - #8333
E2E test for text formatting shortcuts (Notion test case 364)#8333JohnThomson wants to merge 3 commits into
Conversation
The new spec drives the Edit tab's CKEditor formatting toolbar and its shortcuts on a two-paragraph text box: bold, italic, underline, superscript and text color go on words through the toolbar buttons and Ctrl+B/I/U, singly and in layers; Ctrl+A then Ctrl+Space takes every formatting off and the top bar's Undo button puts it back; the Remove Formatting button clears only the selected text and undo restores it. helpers/textFormatting.ts is the new surface module: keyboard selection of a stretch of text, the toolbar buttons and shortcuts, the text color palette, and reading a box back as formatted runs. bookMaking.ts gains typeParagraphsInGroup and workspace.ts gains clickUndoButton, the UI route beside the existing production-path undo. Bloom production code: EditingControlButton in editTopBarControls.tsx gets a data-testid per command so the Undo button can be found in any UI language. AUTOMATION-DEBT.md records that Ctrl+Z is a WinForms accelerator no test can press, and a new entry for the text color palette, which Bloom's selection-check hiding closes under the click about one run in two; the helper clicks again as a person would. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The Undo button is a React top-bar button that clickUndoButton clicks; only Ctrl+Z is the WinForms accelerator a test cannot press. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
| for (let attempt = 1; attempt <= 6; attempt++) { | ||
| await clickFormatButton(page, "textColor"); | ||
| const opened = await panel | ||
| .waitFor({ state: "visible", timeout: 2000 }) | ||
| .then(() => true) | ||
| .catch(() => false); | ||
| if (!opened) continue; | ||
| // The palette is showing. Click the swatch straight away, before the selection check can | ||
| // hide the panel; if the panel goes anyway, the click times out and the loop tries again. | ||
| try { | ||
| await swatch.click({ timeout: 2000 }); | ||
| await panel.waitFor({ state: "hidden", timeout: 15000 }); | ||
| return; |
There was a problem hiding this comment.
[Devin] Investigate: Color retry masks first-click failures
pickTextColorFromToolbar clicks the Text Color button up to six times to get around a race in which Bloom hides the palette right after it opens. That race is something a person can hit too (the palette appears not to open). With the retry in place the suite stays green while that user-facing behavior stays broken, so nothing in the nightly run would tell anyone if it got worse.
There was a problem hiding this comment.
Assessment (Claude Fable 5.1): agreed that the retry hides the product-side flakiness from the nightly run, and that is a deliberate trade-off rather than an oversight. The behavior under test on this card is the character formatting, not whether the palette opens on the first click, and a test that failed one run in two on that would be red for the wrong reason. The product problem is recorded in src/BloomE2E/AUTOMATION-DEBT.md ("The text color palette sometimes does not open on the first click"), with what causes it: Bloom hides every CKEditor panel on each selection check, so a palette that CKEditor thinks is open ends up hidden.
Leaving this thread open for the developer to decide whether that gets its own YouTrack card, or whether the test should instead fail when the first click does not open the palette.
… their result selectTextInGroup now presses Shift+ArrowRight once per grapheme (Intl.Segmenter) rather than per UTF-16 code unit, so an emoji or a combining sequence is one caret step as the browser counts it. clickFormatButton and pressFormatShortcut wait for the command to land: a style command until its toolbar button toggles, Remove Formatting until the selection is plain. pickTextColorFromToolbar waits out CKEditor's selection-check throttle after the palette opens before clicking a swatch, because a click into a panel that closes at that moment landed on the text and moved the selection; it now fails if the selection moved. AUTOMATION-DEBT.md records that. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
| // that has just opened. A swatch click delivered into a panel that closes at that moment | ||
| // lands on the text beneath it and moves the selection, which is worse than a retry. So | ||
| // wait out that one check, and click only a panel that is still showing afterwards. | ||
| await page.waitForTimeout(kSelectionCheckMs); |
There was a problem hiding this comment.
[Devin] Investigate: Fixed sleep bypasses state synchronization
pickTextColorFromToolbar waits a fixed 250 milliseconds after the palette opens instead of observing CKEditor's selection check. On a slow machine the check could still arrive after the wait, so the race is narrowed rather than closed, and every successful run pays the delay.
There was a problem hiding this comment.
Assessment (Claude Fable 5.1): this wait is deliberate, and it is the one place in the helper where a state to wait on does not exist. CKEditor runs its selection check from a private timer (a 200 ms throttle in its selection plugin); nothing on the page says whether one is still pending, and it is that check in which Bloom hides the palette. Without the wait, a swatch click delivered into a panel that closes at that moment landed on the text underneath and moved the selection, which is what the previous run did (the color went onto "liq" instead of "liquor"). The wait is named after the throttle it waits out, and the helper still checks the panel is showing afterwards and fails if the selection moved, so a slow machine gets a clear failure rather than a corrupted test.
The proper fix is in Bloom, not the helper: stop hiding a palette CKEditor has just opened (see the "text color palette" entry in src/BloomE2E/AUTOMATION-DEBT.md). Leaving this open with the related thread above for the developer to decide.
|
Consulted Devin on 2026-09-08 up to commit 4144d98 (two rounds: 663018c, then the fix commit). Round 1 raised one bug and two flags, all in the new e2e helper |
1 participant
Bloom production code changes
src/BloomBrowserUI/bookEdit/topbar/editTopBarControls.tsx:EditingControlButtonnow puts adata-testidof the formedit-top-bar-<command>-buttonon its button (Paste, Cut, Copy, Undo), so an e2e test can find the Undo button in any UI language. No behavior change.Purpose
Automates Notion test case 364 (Text Formatting Shortcuts): https://app.notion.com/p/Text-Formatting-Shortcuts-3904bb19df1281dab22bd07b14769b38
The test protects character formatting in a text box: Bold, Italic, Underline, Superscript and text color applied through the CKEditor formatting toolbar and through Ctrl+B / Ctrl+I / Ctrl+U, singly and layered; clearing it with Ctrl+Space and with the Remove Formatting button; and undo putting it back.
Steps
src/BloomE2E/tests/text-formatting-shortcuts.spec.ts, four serial tests on a Basic Book made at run time:How it verifies the result
A new helper,
getFormattedRuns, reads the box back as runs of text with{bold, italic, underline, superscript, color}derived from the<strong>/<em>/<u>/<sup>/<span style=color>ancestors CKEditor writes, so assertions name formatting rather than markup. Every assertion is polled. The test also checks that the paragraph texts are unchanged after each operation, and that two never-formatted words stay plain.Three green runs in a row locally (about 20 s each after the first launch); no
Bloom.exesurvived any run.Test Steps covered / not covered
Covered: every step of the card. The "CTRL + Z" step is covered with a caveat: Ctrl+Z is a WinForms accelerator the shell handles before the browser sees it, so no test can press it. For that step the test calls
helpers/workspace.ts undo, which runs the sameworkspaceBundle.handleUndo()the shell calls for the key, so what undo does is verified and only the key press itself is not. The Undo button step is a real click.Helpers added
helpers/textFormatting.ts(new):selectTextInGroup,selectAllInGroup,clickFormatButton,pressFormatShortcut,pickTextColorFromToolbar,getFormattedRuns,formattingOf,expectFormatting,paragraphTextsOf,describeRuns,PLAIN.helpers/bookMaking.ts:typeParagraphsInGroup.helpers/workspace.ts:clickUndoButton(the UI route, beside the existingundo).Automation debt
e2e/hook that runs the shell's accelerator handling for a named key.🤖 Generated with Claude Code
Devin review
This change is