feat!: bound untrusted-input cost and parse inlines on a delimiter stack

[plimeor]

Summary

Bounds the cost of parsing untrusted Markdown and moves the inline parser onto CommonMark's delimiter-stack algorithm. SemVer-breaking: parse output changes for crossing marks, BOM/NUL input, bracket precedence, and input nested past the limits.

Commits

• fix: bound parse and serialize cost on untrusted input: removes the quadratic, cubic, and exponential paths. Raw-content scans (code spans, autolinks, raw HTML, MDX, wikilinks, directive labels) go through per-input memo tables. MDX JSX tags are matched through a tag index. Nesting limits keep stack use bounded.
• docs: adopt spec-flow layout and plan inline delimiter stack: moves the docs to the overview / specs / plans / archive / decisions layout and adds the plan.
• feat!: parse inline marks and brackets on a delimiter stack:
  • All emphasis-like marks (*, _, ~~, ~, ^, ++, ==, ||, underline __) now pair on one delimiter stack. Links, images, and footnotes resolve on a bracket stack when ] arrives. The closer-search budget, the link-label verdict memo, and link recursion are gone.
  • All inline containers share one 32-level nesting limit.
  • A leading U+FEFF is skipped and U+0000 is read as U+FFFD, as CommonMark requires; spans stay in original-input coordinates.
• fix: correct emphasis spans, reference fallback, and table spoiler splitting:
  • Emphasis/Strong spans now cover exactly their own delimiters and content.
  • [foo][bar falls back to a shortcut reference.
  • Validation accepts a hard line break that ends link text, alt text, an inline footnote, or a text directive label.
  • One table-row scanner, shared by the parser and the serializer, replaces three diverging spoiler predictions.
  • The plan is archived and its spec changes are merged into docs/specs/.

Behavior changes reviewers should know

• Crossing marks resolve in closing order. **a ==b** c== now gives Strong(a ==b) + c==. Marks that don't cross parse as before.
• BOM / NUL handling.
  • "\u{feff}# t" is now a heading.
  • NUL is written as U+FFFD in every node value.
• Inputs past the nesting limits. Inner levels form and the outer delimiters stay literal.
• Autolinks inside brackets follow GitHub (micromark), not comrak. The two conformance suites contradict each other on [https://…].
• Table spoiler prediction is lexical. A spoiler that a link, a crossing mark, inline math, raw HTML, or an autolink breaks inside a cell leaves its bars as text (see docs/specs/block-syntax.md).

Design and trade-offs:

• docs/decisions/0006-bounded-cost-on-untrusted-input.md
• docs/archive/2026-10-05-inline-delimiter-stack/plan.md

Validation

• Tests and builds: cargo test passes with 225 tests, and with --features html 244 pass. cargo fmt --check, RUSTDOCFLAGS='-D warnings' cargo doc --no-deps, the wasm32 build, and cargo +1.82 build all pass. Clippy has no new warning kinds.
• Conformance: 2233/2236. The remaining 3 are the comrak cases for autolinks inside brackets, which conflict with micromark.
• Linear time: adversarial growth sweeps across 6 dialects, plus serialize/render/validate, show no superlinear growth. A table-row growth check is also linear.
• Stack: 98 worst-case inputs × operations stay within 615 KiB (debug) and 199 KiB (release), under a 2 MiB thread stack.
• Diff against the previous commit (fixture corpus, CommonMark examples, and 4k generated inputs): only the intended changes. No input newly fails to serialize or to round-trip stably.
• Independent review: the two problems it found (escaped pipes merging table cells; escaped backticks read as code-span delimiters) are fixed and covered by tests.
• Generated table rows: the spoilers the scanner predicts match the spoilers the inline parser forms in each cell.

Known gaps (not addressed here)

• Indented lines: inline spans on lines that carry leading whitespace inside their block are offset by that whitespace. This predates the change and is listed under Next in docs/overview.md.
• Table speed: rows where every cell holds a code span and a spoiler parse about 12–20% slower and serialize about 15% slower. Plain tables and ordinary documents are unchanged.

🤖 Generated with Claude Code