Skip to content

convert Block Editor HTML to ProseMirror JSON on save (server-side)#36473

Open
hassandotcms wants to merge 4 commits into
mainfrom
36470-html-to-prosemirror-json-on-save-server-side
Open

convert Block Editor HTML to ProseMirror JSON on save (server-side)#36473
hassandotcms wants to merge 4 commits into
mainfrom
36470-html-to-prosemirror-json-on-save-server-side

Conversation

@hassandotcms

@hassandotcms hassandotcms commented Jul 8, 2026

Copy link
Copy Markdown
Member

What

Converts Story Block (Block Editor) field values sent as HTML to Tiptap/ProseMirror JSON server-side, on the shared content save path — completing the HTML criterion deferred from #36002. Non-interactive clients (AI agents, headless imports) no longer need a human to open and re-save the contentlet for an HTML value to read back as structured content.

How

  • New TiptapHtml converter (jsoup DOM walk) — same node/mark shapes and primitive whitelist as TiptapMarkdown, so both paths produce documents the Block Editor loads identically. Sanitizes untrusted HTML: drops script/style/iframe/… subtrees, allow-lists attributes, scheme-checks href/src.
  • Wired into MapToContentletPopulator: { → JSON passthrough, an HTML tag → TiptapHtml, else → TiptapMarkdown. Reuses Workflow fire: convert Block Editor Markdown to ProseMirror JSON on save (server-side) #36002's rich-content overwrite guard and never-block-the-save fallback.
  • Adds org.jsoup:jsoup 1.21.1; OpenAPI fire note regenerated ("Markdown or HTML").

Behavior change

An HTML value is now converted and stored as structured JSON instead of verbatim — only when a client resends the field; stored values are never rewritten on their own. Additive and rollback-safe.

Testing

27 unit (TiptapHtmlTest) + 9 integration (StoryBlockMarkdownPopulatorTest, including HTML → GraphQL read-back and an editor-saved primitive doc updated via HTML end to end. Existing TiptapMarkdown tests unchanged.

Closes #36470

@github-actions github-actions Bot added the Area : Backend PR changes Java/Maven backend code label Jul 8, 2026
@hassandotcms hassandotcms changed the title Workflow fire: convert Block Editor HTML to ProseMirror JSON on save (server-side) convert Block Editor HTML to ProseMirror JSON on save (server-side) Jul 8, 2026
@claude

claude Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Claude finished @hassandotcms's task in 4m 34s —— View job


Code Review — PR #36473

  • Read the diff and changed files
  • Review TiptapHtml.java (converter + sanitization)
  • Review MapToContentletPopulator.java wiring
  • Review pom / dependency changes
  • Check test coverage
  • Post review

This is a well-constructed PR. The converter mirrors TiptapMarkdown's node/mark shapes, the sanitization is thorough (subtree drop-list, attribute allow-listing, URL scheme + breakout-char validation), depth is capped against stack overflow, and the save-path wiring reuses #36002's overwrite guard and never-block-the-save fallback. Test coverage is strong on both legs (27 unit + integration round-trips incl. GraphQL read-back and the autolink routing regression guard).

I found no blocking (Critical/High) issues. A few non-blocking notes below.

New Issues

  • 🟡 Medium: dotCMS/src/main/java/com/dotcms/rest/MapToContentletPopulator.java:326 — When an HTML value converts to an empty document (all content dropped by sanitization, e.g. <script>...</script> with no other body), the field is silently overwritten with {"type":"doc","content":[]}. The rich-content overwrite guard (line 317) only protects existing docs that are not markdown-representable; a primitive existing doc gets clobbered with an empty doc, and a client that sent "content" sees the field cleared with no warning. Consider logging (or preserving the prior value) when conversion yields an empty doc but the input was non-blank. Low severity — it's sanitization behaving correctly — but the silent clear is surprising.

  • 🟡 Medium: dotCMS/src/main/java/com/dotcms/tiptap/TiptapHtml.java:595sanitizeUrl rejects control/quote/angle/backtick/whitespace chars but not backslash. A value like \\evil.com has no colon, so it's returned as a "relative" URL; some renderers normalize \\//, treating it as protocol-relative to an external host.

    • Assumption: the render surface may treat \ as /.
    • What to verify: this is not meaningfully worse than the already-intended //host protocol-relative allowance (line 602), and it's a link the user must click — not script execution (breakout chars are already rejected). Flagging only for awareness; likely acceptable given the existing //host policy.
  • 🟡 Medium: dotCMS/src/main/java/com/dotcms/rest/MapToContentletPopulator.java:92 — The HTML_START regex requires an ASCII-only tag name terminated by [\s/>], so a hyphenated tag (<custom-el>, <my-widget>) fails to match and routes to the Markdown converter instead of HTML. Harmless for the primitive whitelist (Markdown treats it as raw HTML and drops the wrapper), but worth a one-line comment noting custom elements are intentionally not HTML-routed, so it doesn't read as an oversight.

Notes (non-issues, verified)

  • URL scheme allow-list, tab-smuggling (java\tscript:), event-handler stripping, and depth-cap are all correctly handled and tested. ✅
  • Dependency version correctly added to bom/application/pom.xml only; dotCMS/pom.xml references it without a version. ✅
  • TiptapHtml.java:349 uses fully-qualified java.util.List/ArrayList despite the import java.util.List at line 14 — pure style, not flagging.
    · 36470-html-to-prosemirror-json-on-save-server-side

Adds org.jsoup:jsoup (HTML5-conformant parser) for converting Story Block
HTML values to ProseMirror JSON on save (#36470). Version managed via a
jsoup.version property in bom/application/pom.xml; the dotCMS module gets a
version-less dependency entry.

This is distinct from the legacy shaded com.dotcms.lib:dot.jsoup (jsoup 1.6.1,
package com.dotcms.repackage.org.jsoup) still used by a few callers — the two
live in different packages and coexist. New code uses org.jsoup only; retiring
the legacy repackage is a separate follow-up.
Pure, server-side converter from an HTML fragment to a Tiptap/ProseMirror
document node, without a browser DOM. Companion to TiptapMarkdown and emits the
identical node/mark shapes, so both ingestion paths produce documents the Block
Editor loads the same way.

- Output restricted to the same primitive whitelist TiptapMarkdown produces;
  unknown-but-safe elements are transparent (text kept), dangerous ones dropped.
- Content-model valid by construction: list items always open with a paragraph,
  cells always hold a block, empty lists/tables/rows/blockquotes are elided,
  block descendants of inline-only nodes are flattened, and no empty text node
  is ever emitted (an empty text node hard-fails the editor and wipes the doc).
- Sanitization: script/style/iframe/object/embed/... subtrees dropped entirely,
  attributes copied by allow-list (event handlers never read), href/src checked
  against a scheme allow-list with attribute-breakout characters rejected.
- Never throws; a recursion depth cap guards against pathological nesting.

Covered by TiptapHtmlTest: conversion fidelity, content-model validity,
whitespace, a security/sanitization matrix, and mechanical invariants asserted
across a battery of inputs (valid representable doc, no empty text, no overflow).

Refs #36470
Wires TiptapHtml into the shared content-save seam so a Story Block field value
supplied as HTML is converted to Tiptap/ProseMirror JSON server-side, completing
the HTML acceptance criterion deferred from #36002. Non-interactive clients (AI
agents, headless imports) no longer need a human to open and re-save the field.

- MapToContentletPopulator.toStoryBlockJson now routes a Story Block value:
  '{' (or blank) -> stored unchanged (editor JSON); a value opening with a real
  HTML tag/comment -> TiptapHtml; anything else -> TiptapMarkdown. A first-tag
  regex keeps CommonMark autolinks (<https://x>) and text like "<3" on the
  Markdown path rather than mis-parsing them as HTML.
- The rich-content overwrite guard and the graceful-degrade fallback (never
  block the save) now cover the HTML branch too. On an update the seam sees the
  prior stored value (the fire/content resources copy the existing map before
  populating), so a rich document is preserved rather than overwritten by HTML.
- WorkflowResource Block Editor note updated to "Markdown or HTML"; openapi.yaml
  regenerated from the annotation.

Behavior change: a Story Block value sent as HTML is now converted and stored as
structured JSON instead of stored verbatim. This only triggers when a client
resends the field; stored values are never rewritten on their own.

Covered by StoryBlockMarkdownPopulatorTest (integration): HTML converted + read
back through checkin and GraphQL, an editor-saved primitive document updated via
HTML and read back valid end to end, HTML overwrite of rich content ignored,
script sanitized on save, and an autolink value routed to the Markdown converter.

Closes #36470
…-to-prosemirror-json-on-save-server-side
@hassandotcms hassandotcms force-pushed the 36470-html-to-prosemirror-json-on-save-server-side branch from 37c9a68 to 043b8b7 Compare July 9, 2026 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI: Safe To Rollback Area : Backend PR changes Java/Maven backend code

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

Workflow fire: convert Block Editor HTML to ProseMirror JSON on save (server-side)

1 participant