Pilot snapshot tests with Nullean.Snapshots in the API Explorer tests by akira28 · Pull Request #4654 · elastic/docs-builder

akira28 · 2026-11-02T16:32:29Z

Adds snapshot testing with Nullean.Snapshots to Elastic.ApiExplorer.Tests as a small pilot, and shares one RenderAsync across the operation rendering tests.

Affects: Internals only

Prompt summary: Pilot snapshot testing in one test project to see whether it cuts the many html.Should().Contain(...) checks without making diffs harder to read. Convert one file, keep targeted assertions that express intent, and report numbers so the team can decide whether to expand. The pilot started on Verify and now uses Nullean.Snapshots, because Verify's maintenance fee blocks adopting current releases.

Why

Rendering tests assert on HTML with long chains of Contain checks. Each check covers one fragment, so a reviewer never sees the page as a whole. Five test files also carry an identical private RenderAsync that builds an ApiRenderContext and round-trips through a mock file system.

Verify's maintenance fee covers every release published after 1 September 2026, so the team cannot adopt it without a licensing decision. Nullean.Snapshots is MIT licensed, built on AwesomeAssertions, and has a TUnit adapter.

What

Snapshot library

Nullean.Snapshots.Html and Nullean.Snapshots.TUnit 0.2.0 go in Directory.Packages.props, and only the API Explorer test project references them. Their dependency floors (AngleSharp 1.7.2, AngleSharp.Diffing 1.1.1, AwesomeAssertions 9.6.0, TUnit.Core 1.78.0) match what the repo already pins. Verified snapshots live under .snapshots/. Received files go to .artifacts/, which is already ignored, so no .gitignore entry is needed. The library never accepts a snapshot on its own, and per its documentation never when CI=true. .gitattributes keeps the snapshot folder on LF line endings.

Shared render helper

ApiExplorerFixture gains RenderAsync(ILeafNavigationItem<IApiModel>) and RenderContextFor(INavigationItem). The five tests with a private copy (OperationBaseUrl, OperationDeprecatedBadge, OperationStateBadge, OperationHeadingLevel, EndpointPathLink) call it instead.

Converted test

OperationHeadingLevelTests keeps its heading-order assertion and replaces eight Contain checks with one MatchHtmlSnapshot call that selects main#content-container. The page chrome around that element embeds the frontend bundle name and the CI git branch, commit and ref, which differ on every build and failed the first CI run of the Verify version.

Decision gate

Test count: the PR adds and removes no tests; the eight merged checks sit in one existing test. The project runs 1422 tests (1421 passed, 1 skipped).
Lines in .cs files: 112 removed and 38 added. The snapshot adds one 896-line .verified.html file.
Readability: a changed string fails with the DOM path, the expected and actual text, and a line diff with context. Changing Response headers to Response header printed expected: Response headers, actual: Response header and the surrounding lines.
File name: the snapshot is OperationHeadingLevelTests/Render_SearchOperation_UsesOneH1ThenH2SectionsAndH3Nested.verified.html, with no hash, even though the fixture reaches the test through the class constructor. With 0.1.0-alpha.1 the constructor argument added an eight-digit hash to the name. I reported that as nullean/snapshots#13, and 0.2.0 fixes it by leaving fixtures injected with ClassDataSource out of the hash (nullean/snapshots#19). The test class did not change.
Maturity: 0.2.0 is a regular release, not a prerelease, but the library is still 0.x, so its API can change. The snapshot is plain pretty-printed HTML, so moving to another tool later is cheap.
Setup: no module initializer is needed any more, since the library handles CI behaviour itself.

Verify

dotnet test --project tests/Elastic.ApiExplorer.Tests

github-actions · 2026-11-02T16:44:30Z

Docs preview (local build)

Handbook preview: https://docs-v3-preview.elastic.dev/elastic/docs-builder/pull/4654/

Replace eight HTML Contain checks in the heading-level test with one Verify snapshot, keeping the heading-order assertion. The five operation rendering tests now share RenderAsync from ApiExplorerFixture instead of five private copies. Verify.TUnit 31.28.0 is the newest release that needs a TUnit.Core no newer than the repo's TUnit 1.65.0. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

The full-page snapshot embeds the frontend bundle name and the CI git branch, commit and ref, so it failed on every build that differed from the one that recorded it. The test now snapshots the main content element, which holds none of those values. The asset-hash and year scrubbers have nothing left to scrub and are removed. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

Verify.TUnit 32.0.1 is the newest release before the Verify 33 sponsorship check, and it works with the TUnit 1.78.0 now on main. ApiExplorerFixture.DisposeAsync now calls GC.SuppressFinalize, which clears the CA1816 warning that the PR's diff surfaced on that line. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

Verify's maintenance fee covers every release published after 1 September 2026. Verify.TUnit 32.0.0 was published on 2026-08-26 and 32.0.1 on 2026-09-12, so 32.0.1 is covered. 32.0.0 needs TUnit.Core 1.65.63 or later, which the TUnit 1.78.0 on main satisfies. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

The shared test helpers PR (#4660) makes the same DisposeAsync change. Keeping it in both PRs made them conflict in ApiExplorerFixture.cs, so this PR no longer touches that method. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

Nullean.Snapshots is MIT licensed, so the pilot no longer depends on Verify's maintenance fee. It is built on AwesomeAssertions and has a TUnit adapter, and its dependency floors match the versions this repo already pins. The pilot test now calls MatchHtmlSnapshot with a Select on the operation content, which replaces the manual AngleSharp extraction and the module initializer. Verified snapshots live under .snapshots, and received files go to .artifacts, which is already ignored, so the *.received.* rule is gone. The packages are 0.1.0-alpha.1. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

The pattern .snapshots/** only matches at the repository root, so git applied no line-ending rule to tests/Elastic.ApiExplorer.Tests/.snapshots and a Windows checkout with core.autocrlf could convert the verified file to CRLF. **/.snapshots/** matches the folder wherever a test project keeps it. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

… name Nullean.Snapshots 0.2.0 leaves fixtures injected with ClassDataSource out of the arguments hash, which fixes the report from this pilot (nullean/snapshots#13). The pilot test now writes OperationHeadingLevelTests/Render_SearchOperation_UsesOneH1ThenH2SectionsAndH3Nested.verified.html with no change to the test class. The snapshot content is byte for byte identical, so the file is renamed. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

The deleted Contain checks in OperationHeadingLevelTests stay deleted. Main edited one of them for the renamed section, and the snapshot now carries that change. The snapshot is updated for two rendering changes that came in with main: operation sections are expanded by default, and an operation page has a row of links to its sections. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

Mpdreamz

Reviewed the code and the PR's decision gate. The refactor is clean: the five private RenderAsync copies become one ApiExplorerFixture.RenderAsync, and OperationHeadingLevelTests keeps its heading-order assertion while one MatchHtmlSnapshot replaces eight Contain checks. Selecting main#content-container avoids the per-build bundle name and git metadata in the page chrome. The new packages are MIT, test-only, and pinned at 0.2.0 (still 0.x, as the description says). One coordination point, not a blocker: any change to the operation page markup now has to update .snapshots/OperationHeadingLevelTests/*.verified.html, so the open API PRs that change that markup will need a snapshot refresh when they merge.

The only conflict was .gitignore. This branch no longer changes it, since received snapshots go to .artifacts, which is already ignored, so the file is taken from main as it is. Main's latest commit, which adds an unformatted CdnChangelogEntryFetcherTests.cs that fails the lint hook until #4680 lands, is left out of this merge. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

#4311 stopped the popover from emitting `additionalInfo` when every lifecycle is negative. #4654 added this snapshot at the same time, so main shipped with a stale copy and the Linux build failed on it. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

The snapshot from #4654 was recorded before #4311 landed. #4311 hides the popover's additional availability info when every lifecycle is negative, and the deprecated badge on the search operation is such a case. `main` fails the same test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…hange (#4683) #4311 sets `additionalInfo` to null in the applies-to popover when all lifecycles are negative. The snapshot added in #4654 still held the old text, so the ApiExplorer test failed on main. Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>

* Index the direct children of each API property The exporter already walks the property tree, so each indexed row now stores the suffixes that extend its path to its direct children, such as `.query`, `[].query` or `.<string>.terms`. The list is an unindexed keyword. `Flatten` and the child lists both derive from one `DirectChildren` definition, so the flat index and the suggestions cannot drift apart. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Return a property's children from API search and read a trailing dot `retriever.standard.` is a child field not yet typed, so the query now reads it as `retriever.standard` instead of a bare name that matches nothing. Each property in the response carries `children`, always an array and capped at 100 entries. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * List a field's children under its row in API search results A property row now shows the fields nested under it as one muted line in code font, so typing `retriever.standard` tells the reader that `filter`, `min_score`, `query` and the rest come next. The line ellipsizes and the row's title carries the full list. Nothing is interactive; the reader types the next segment. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Document the child field list in API search Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Use the page's tooltip on API search property rows The operation page shows a property's JSON path in a tippy tooltip, and the search row showed its own hover text through native title attributes, which look different and appear a second later. The row's path, type, description and child list now carry data-tippy-content, which the delegate in tooltip.ts already covers, so one hover style serves both surfaces. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Put a field's path and description on separate lines in API search One line held the path, type, location and description, so at the dropdown's width almost all of it ellipsized. The row now takes two lines: the path with its leaf in full weight, cut at the start so the leaf stays visible, then the type chip and the description. The location chip shows only when the same path appears in two locations. This is direction A of the three mockups from the search redesign exploration, kept in the same grouping and keyboard order. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Mark API search field rows with a braces icon on a subdued tint The indent and guide line that set a field row apart from its operation are replaced by EUI's tokenObject icon in a gutter, and the row sits on a subdued background that brightens when selected, as in direction A of the search redesign mockups. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Keep API search field rows white and show the braces icon at 16px A field row is white like the operation row and turns grey only on hover or when selected. The gutter icon goes from 12px to 16px, the size the mockup drew it at. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Draw the field icon as strokes, delay its tooltips, and link-style the path Three fixes from reviewing the two-line field row in the dropdown. EUI's tokenObject is a filled token glyph that blurs at 16px, so the gutter draws the braces as strokes instead and the icon cache goes back to what main has. The row's tooltips wait 500ms, since the page's 80ms opened one on every row the mouse crossed. The path turns link-blue and underlined on hover and on the selected row, the way the operation title does. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Show a field's children as chips that complete the typed path One ellipsized line hid most children of a wide field. The children now wrap as chips under the row, at most twelve with a `+N more` note that holds the rest in its tooltip. A click on a chip replaces the typed field word with the child's path and searches again, so `retriever` then `knn` reads `retriever.knn` with its own children. The chips sit beside the row's link rather than inside it, since a button inside an anchor is not valid HTML. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Document the child field chips in API search Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Show a field's children as informational tokens A click on a child rewrote the query, and the re-ranked results could put a different operation first, which read as the row changing under the reader. The children are now plain tokens inside the row's link, styled like the type chip, wrapping so every child shows, twelve at most with a `+N more` note. Typing the next segment is how the search narrows. The fill handler and the container that kept buttons out of the anchor go away with it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Label the child tokens of an API search field row A row of bare tokens under a field did not say what they were. The line now starts with "Child properties", and the group and the row's accessible name say the same. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Always show an operation's route in API search results An operation row showed its route only when a query word named a path, and its description otherwise. The description mostly repeated the title, while the route is what tells two search operations apart. The row now shows the route a query word names, else the page's primary route, and keeps the description only for a row without routes, such as a docs page. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Put the child tokens in a labelled column The inline label made wrapped token lines start at different x positions. The block is now a two-column grid with "Contains" on the left and the wrapping tokens on the right, stacked on small screens. "Contains" names the relation, so it cannot be misread under a row that is itself a field; the accessible name keeps "Child properties". Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * List the children of a typed path as rows in API search results Each indexed property now stores the path of the row it is listed under, with array and map markers removed so that `[].retriever` (an `msearch` body) is listed under `retriever`, the way a reader types it. The child suffix list goes away with the token block that used it. In the API lookup the nested property query gains a `parent` option at a lower weight than any name match, so the typed path's direct children come back as ordinary rows: up to five after their parent, or twenty when the field ends with a dot, in page order. A reading also lifts an operation that lists children of the typed path above one that merely contains a field of that name; every exact name match otherwise ties and the tie broke towards `get-script`'s `script.source.retriever`. Name matches and children are told apart by named clauses, and inner hits grow from 5 to 20. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Show child rows flat with the location of every field The children of a typed path arrive from the server as ordinary properties and render as ordinary rows; the dimmed parent segment of the path shows the relation. Every property row now carries a chip saying where the field lives (path, query, header, request body, response body) instead of only when a path appears twice. The token block, its "Contains" column and the "+N more" note are gone. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Document child rows and field locations in API search Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * List a field's children only when the query ends with a dot Five children under every exact path blurred the line between "find this field" and "show me what is under it". A query without a trailing dot is the name match it always was, with no children clause sent; a query ending with a dot lists every child the inner-hits window returns, up to 100, after the parent row. The lift that ranks an operation listing children of the typed path first applies either way, so the operation whose row is the typed path still comes first. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Document the trailing dot that lists a field's children Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Make API search results plain boosted links that close on click A result row went through a per-row hook that normalised its href and processed it for htmx, and the dropdown closed and blurred its input only when a later `htmx:afterSwap` reached the component. That event reaches a same-page target differently from a cross-page one, since the search element survives one swap and is replaced by the other, and after a same-page result the box would not reopen on focus. Rows are now `<a href>` as given, the list runs one `htmx.process` over itself after each render, and a click or Enter closes the dropdown and blurs the input on the spot. A ref keeps that blur from logging a second close for the same navigation. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Reposition the API search dropdown when the page scrolls under it The dropdown stays mounted while the box has text and is hidden with CSS. EUI positions the portaled panel in page coordinates when it opens, so a result that jumps to an anchor scrolled the page about 3,700px while the sticky box stayed put. Focusing the box again showed the panel at its old coordinates, far above the viewport, which looked like the dropdown not opening. `repositionOnScroll` recomputes the position on scroll. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Simplify the API property search plumbing The child cap becomes a flag, since it only ever took "none" or "the whole window". Name options are composed into one dis_max instead of lists mutated across helpers, the boosts that must stay ordered are named together, and the ranking lift gets a name that says what it lifts. The array and map path markers move into the contract so the exporter and `ParentKey` share them, and the response drops a `parent` field no client reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Simplify the API search dropdown Highlight terms are computed once per list instead of once per row, the slow tooltip delay lives next to the delegate's own delay, and a single close helper that logs only when the dropdown was open replaces the flag that suppressed a second close log. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: fetch 5 inner hits for the docs-wide property lookup (per review by @infra-vault-gh-plugin-prod) The docs-wide search keeps only three property rows and never lists children, so fetching 20 per document wasted payload; the API lookup keeps 20 and an expanded one 100. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: normalise the typed path before matching api.properties.parent (per review by @infra-vault-gh-plugin-prod) The index stores each row's parent as ApiPropertyPath.ParentKey, so a typed path with [] or <string> markers matched no children and lifted no operation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: list children under the parent row of their own location (per review by @infra-vault-gh-plugin-prod) A query parameter and a body field can share a path, so matching the parent row on path alone put body children under the query parameter and mixed request and response children. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: highlight nothing on the fallback API route (per review by @infra-vault-gh-plugin-prod) When no query word names a route the row shows the page's primary route, so marking query words inside it pointed at a match that is not there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: keep the search open when a click opens a result elsewhere (per review by @infra-vault-gh-plugin-prod) A Cmd, Ctrl or Shift click opens the result in a new tab or window and leaves the reader on this page, so closing and blurring the search threw away what they were browsing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: destroy row tooltips when search results unmount or change (per review by @infra-vault-gh-plugin-prod) The page's tippy delegate mounts tooltips on the body and only drops them on an htmx swap, so a tooltip shown over a row stayed on screen after Escape removed the row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Update the search operation snapshot for the deprecated badge popover The snapshot from #4654 was recorded before #4311 landed. #4311 hides the popover's additional availability info when every lifecycle is negative, and the deprecated badge on the search operation is such a case. `main` fails the same test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Leave modifier clicks on boosted links to htmx and the browser htmx 2 already skips a Ctrl or Cmd click on a boosted link, so the browser opens it in a new tab. The hook also cancelled Shift-clicks and opened them with `window.open`; a Shift-click is now an ordinary htmx navigation like any other click. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: close the API search dropdown on Shift-click (per review by @infra-vault-gh-plugin-prod) A Shift-click on a result is now an ordinary htmx navigation, so it closes the dropdown like a plain click; only Ctrl and Cmd clicks, which htmx leaves to the browser, keep it open. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Revert "fix: close the API search dropdown on Shift-click" This reverts commit a8b4011. The site hook opens a Shift-click in a new tab again, so the dropdown stays open for it like for Cmd and Ctrl. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Revert "Leave modifier clicks on boosted links to htmx and the browser" This reverts commit 27663a2. Measured against production, Cmd and Ctrl clicks already open a new tab either way, so the only effect was that a Shift-click on any boosted link navigated in place instead of opening a new tab. The site-wide hook is out of scope for this PR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Jan Calanog <reakaleek@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>

* Wire Nullean.Snapshots into every test project The ApiExplorer pilot (#4654) used Nullean.Snapshots.Html and .TUnit 0.2.0 in one project. This makes the library available to all unit and integration test projects at 0.4.0 and adds the shared pieces the next migration steps build on: - tests-shared/Elastic.Documentation.Testing: DocsHtml (page section without the fixture H1 and htmx attributes), DiagnosticsSnapshot, NavigationDump, MockFileSystemDump. A scrubber for temp and repo paths is compiled into each test project. - The snap CLI as a local tool; CI uploads received snapshots when a test run fails; the build fails on orphaned snapshots after a unit run. - AGENTS.md describes the workflow. The PR review prompt no longer claims the repo uses xUnit. The five TestDiagnosticsCollector copies stay: they differ in lifecycle (Write vs HandleItem), so merging them is a behaviour change. DiagnosticsSnapshot works on any of them. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Scrub paths the same way on every operating system The repository and temp folder scrubbers replaced only the root, so a Windows run printed {repo}\docs\index.md where the verified text says {repo}/docs/index.md, and about 160 diagnostics snapshots failed there. The path that follows the root is now written with forward slashes. The temp scrubber also swallowed everything up to the next whitespace, including the :line after a file name. A path now ends at a colon too, and the repository root is tried before the temp root because the repository can sit under the temp folder. The doc comment claimed it scrubbed git SHAs; it does not, and no longer says so. Tested with a Windows-style root, which also runs on Linux and macOS. Removing the separator replacement makes that test fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Scrub random folder names under the temp directory A test that works in a Guid-named folder in the temp directory printed that name in its diagnostics, so a snapshot of them could not be stable. The temp scrubber now replaces a run of 32 hex digits in the path with {id}. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Make diagnostics snapshots and snapshot paths the same on every operating system CI on Windows failed five of the stacked PRs for reasons a laptop never shows: - DiagnosticsSnapshot now writes paths with forward slashes and drops a leading drive letter, so `C:\docs\toc.yml`, `C:/docs/toc.yml` and `/docs/toc.yml` read the same, and `testing\req.md` inside a message reads `testing/req.md`. - ToSnapshotText(firstLineOnly: true) keeps only the first line of each message, for diagnostics that carry an exception: its stack trace has source line numbers and the exception type differs per platform. - SnapshotPathLengthTests fails when a verified snapshot's path, with the received file's longer folder and the CI checkout root, would pass Windows' 260 characters. git on the runner cannot check such a file out, so the whole job failed before any test ran. Each has a test. Lowering the length limit makes the guard fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Match the repository root in the form DiagnosticsSnapshot writes it DiagnosticsSnapshot drops a leading drive letter and writes forward slashes, so on Windows D:\a\docs-builder\docs-builder\docs\index.md reaches the {repo} scrubber as /a/docs-builder/docs-builder/docs/index.md, which it no longer recognised. Every diagnostics snapshot that names a file failed on the Windows runner. The scrubber now also matches the root without its drive letter. Removing that variant makes the new test fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>

* Wire Nullean.Snapshots into every test project The ApiExplorer pilot (#4654) used Nullean.Snapshots.Html and .TUnit 0.2.0 in one project. This makes the library available to all unit and integration test projects at 0.4.0 and adds the shared pieces the next migration steps build on: - tests-shared/Elastic.Documentation.Testing: DocsHtml (page section without the fixture H1 and htmx attributes), DiagnosticsSnapshot, NavigationDump, MockFileSystemDump. A scrubber for temp and repo paths is compiled into each test project. - The snap CLI as a local tool; CI uploads received snapshots when a test run fails; the build fails on orphaned snapshots after a unit run. - AGENTS.md describes the workflow. The PR review prompt no longer claims the repo uses xUnit. The five TestDiagnosticsCollector copies stay: they differ in lifecycle (Write vs HandleItem), so merging them is a behaviour change. DiagnosticsSnapshot works on any of them. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Compare rendered HTML with Nullean.Snapshots.Html ShouldBeHtml and ShouldContainHtml (Elastic.Markdown.Tests) and ConvertsToHtml, ConvertsToContainingHtml and the per-file ToHtml and ContainsHtml (Elastic.Authoring.Tests) keep their names and call sites, and now run on BeHtml and ContainHtml with the shared DocsHtml options. That removes the second copy of the AngleSharp.Diffing diff code and the AngleSharp.Diffing package. DiffPlex stays in Authoring for the plain text assertions. One test passed without testing anything. "marks exactly two options as a fork" gave the bare text `is-fork` to ConvertsToContainingHtml. The old normaliser printed only element children, so a text-only fragment became an empty string and every page contains an empty string. It now uses ConvertsToContainingRawHtml, like the test above it. Checked by corrupting one ShouldContainHtml and one ConvertsToHtml expectation: both fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Scrub paths the same way on every operating system The repository and temp folder scrubbers replaced only the root, so a Windows run printed {repo}\docs\index.md where the verified text says {repo}/docs/index.md, and about 160 diagnostics snapshots failed there. The path that follows the root is now written with forward slashes. The temp scrubber also swallowed everything up to the next whitespace, including the :line after a file name. A path now ends at a colon too, and the repository root is tried before the temp root because the repository can sit under the temp folder. The doc comment claimed it scrubbed git SHAs; it does not, and no longer says so. Tested with a Windows-style root, which also runs on Linux and macOS. Removing the separator replacement makes that test fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Scrub random folder names under the temp directory A test that works in a Guid-named folder in the temp directory printed that name in its diagnostics, so a snapshot of them could not be stable. The temp scrubber now replaces a run of 32 hex digits in the path with {id}. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Make diagnostics snapshots and snapshot paths the same on every operating system CI on Windows failed five of the stacked PRs for reasons a laptop never shows: - DiagnosticsSnapshot now writes paths with forward slashes and drops a leading drive letter, so `C:\docs\toc.yml`, `C:/docs/toc.yml` and `/docs/toc.yml` read the same, and `testing\req.md` inside a message reads `testing/req.md`. - ToSnapshotText(firstLineOnly: true) keeps only the first line of each message, for diagnostics that carry an exception: its stack trace has source line numbers and the exception type differs per platform. - SnapshotPathLengthTests fails when a verified snapshot's path, with the received file's longer folder and the CI checkout root, would pass Windows' 260 characters. git on the runner cannot check such a file out, so the whole job failed before any test ran. Each has a test. Lowering the length limit makes the guard fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Match the repository root in the form DiagnosticsSnapshot writes it DiagnosticsSnapshot drops a leading drive letter and writes forward slashes, so on Windows D:\a\docs-builder\docs-builder\docs\index.md reaches the {repo} scrubber as /a/docs-builder/docs-builder/docs/index.md, which it no longer recognised. Every diagnostics snapshot that names a file failed on the Windows runner. The scrubber now also matches the root without its drive letter. Removing that variant makes the new test fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>

* Wire Nullean.Snapshots into every test project The ApiExplorer pilot (#4654) used Nullean.Snapshots.Html and .TUnit 0.2.0 in one project. This makes the library available to all unit and integration test projects at 0.4.0 and adds the shared pieces the next migration steps build on: - tests-shared/Elastic.Documentation.Testing: DocsHtml (page section without the fixture H1 and htmx attributes), DiagnosticsSnapshot, NavigationDump, MockFileSystemDump. A scrubber for temp and repo paths is compiled into each test project. - The snap CLI as a local tool; CI uploads received snapshots when a test run fails; the build fails on orphaned snapshots after a unit run. - AGENTS.md describes the workflow. The PR review prompt no longer claims the repo uses xUnit. The five TestDiagnosticsCollector copies stay: they differ in lifecycle (Write vs HandleItem), so merging them is a behaviour change. DiagnosticsSnapshot works on any of them. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Compare rendered HTML with Nullean.Snapshots.Html ShouldBeHtml and ShouldContainHtml (Elastic.Markdown.Tests) and ConvertsToHtml, ConvertsToContainingHtml and the per-file ToHtml and ContainsHtml (Elastic.Authoring.Tests) keep their names and call sites, and now run on BeHtml and ContainHtml with the shared DocsHtml options. That removes the second copy of the AngleSharp.Diffing diff code and the AngleSharp.Diffing package. DiffPlex stays in Authoring for the plain text assertions. One test passed without testing anything. "marks exactly two options as a fork" gave the bare text `is-fork` to ConvertsToContainingHtml. The old normaliser printed only element children, so a text-only fragment became an empty string and every page contains an empty string. It now uses ConvertsToContainingRawHtml, like the test above it. Checked by corrupting one ShouldContainHtml and one ConvertsToHtml expectation: both fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Authoring tests: expected text, JSON and HTML as inline snapshots ConvertsToHtml, ToHtml, ConvertsToPlainText, ConvertsToNewLlm, ConvertsToLlmWithMetadata and ConvertsToJson keep their names and call sites, but now compare through Nullean.Snapshots inline snapshots, so the expected literal is accepted with snap instead of hand-edited. The wrappers forward the caller's file, member and line and name themselves as the method snap looks for in the source (InlineSnapshots.cs). The expected literals passed unchanged except for these, which snap rewrote: - LLM markdown literals started with a blank line that the old comparison trimmed; they are now the trimmed text. - Two ToHtml literals (IncludeBlocks) listed an <h1> that the old BeHtml stripped from both sides; the snapshot no longer has it. - The old text comparison ignored whitespace (DiffPlex default), so indentation mistakes in four literals went unnoticed: mermaid code lost its indent in two, and nested list items and a fenced block in a nested list lose their indentation in the exporter output. The snapshots record the output as it is today and the tests carry a note. These comparisons are therefore stricter than before. ConvertsToJson compares as a tree, so key order no longer matters. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Authoring tests: whole-output snapshots instead of fragment assertions Where a test class asserted a cluster of ConvertsToContainingHtml, ConvertsToContainingRawHtml and DoesNotConvertToContainingHtml fragments on one page, the cluster is now one snapshot of the page content: inline (ConvertsToHtml) up to about 45 lines, a verified .html file (ConvertsToHtmlSnapshot) beyond that, 7 in all. Classes asserting HasError, HasWarning or HasHint now snapshot every diagnostic as 'severity file:line message' (HasDiagnostics), which also absorbs their HasNoErrors and HasNoWarnings. Other HasNoErrors/HasNoWarnings stay. These tests became stricter: they used to look at the first diagnostic's substring or at fragments, and now cover the whole output. Before accepting, every old fragment was checked to appear in the snapshot (whitespace and entities normalised) and every negative fragment to be absent, and every old diagnostic substring to appear in the diagnostic text; the only mismatches were valueless attributes that AngleSharp prints as name="" (alt, open, checked, hidden, data-*). Kept as targeted assertions, because a section snapshot cannot show them: - the newline between <code> elements inside <pre> (ConsoleCodeBlockSegments); - <head> meta tags (ProductsFrontMatter); - the page shell, header, footer and layouts (DesignSystem), whose whole page carries the sprite sheet and the checkout directory name. Dropped the helpers nothing uses any more (ConvertsToContainingHtml, ContainsHtml, DoesNotContainHtml on the result task, the DiffPlex line diff) and the DiffPlex package reference. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Scrub paths the same way on every operating system The repository and temp folder scrubbers replaced only the root, so a Windows run printed {repo}\docs\index.md where the verified text says {repo}/docs/index.md, and about 160 diagnostics snapshots failed there. The path that follows the root is now written with forward slashes. The temp scrubber also swallowed everything up to the next whitespace, including the :line after a file name. A path now ends at a colon too, and the repository root is tried before the temp root because the repository can sit under the temp folder. The doc comment claimed it scrubbed git SHAs; it does not, and no longer says so. Tested with a Windows-style root, which also runs on Linux and macOS. Removing the separator replacement makes that test fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Scrub random folder names under the temp directory A test that works in a Guid-named folder in the temp directory printed that name in its diagnostics, so a snapshot of them could not be stable. The temp scrubber now replaces a run of 32 hex digits in the path with {id}. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Make diagnostics snapshots and snapshot paths the same on every operating system CI on Windows failed five of the stacked PRs for reasons a laptop never shows: - DiagnosticsSnapshot now writes paths with forward slashes and drops a leading drive letter, so `C:\docs\toc.yml`, `C:/docs/toc.yml` and `/docs/toc.yml` read the same, and `testing\req.md` inside a message reads `testing/req.md`. - ToSnapshotText(firstLineOnly: true) keeps only the first line of each message, for diagnostics that carry an exception: its stack trace has source line numbers and the exception type differs per platform. - SnapshotPathLengthTests fails when a verified snapshot's path, with the received file's longer folder and the CI checkout root, would pass Windows' 260 characters. git on the runner cannot check such a file out, so the whole job failed before any test ran. Each has a test. Lowering the length limit makes the guard fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Match the repository root in the form DiagnosticsSnapshot writes it DiagnosticsSnapshot drops a leading drive letter and writes forward slashes, so on Windows D:\a\docs-builder\docs-builder\docs\index.md reaches the {repo} scrubber as /a/docs-builder/docs-builder/docs/index.md, which it no longer recognised. Every diagnostics snapshot that names a file failed on the Windows runner. The scrubber now also matches the root without its drive letter. Removing that variant makes the new test fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>

* Wire Nullean.Snapshots into every test project The ApiExplorer pilot (#4654) used Nullean.Snapshots.Html and .TUnit 0.2.0 in one project. This makes the library available to all unit and integration test projects at 0.4.0 and adds the shared pieces the next migration steps build on: - tests-shared/Elastic.Documentation.Testing: DocsHtml (page section without the fixture H1 and htmx attributes), DiagnosticsSnapshot, NavigationDump, MockFileSystemDump. A scrubber for temp and repo paths is compiled into each test project. - The snap CLI as a local tool; CI uploads received snapshots when a test run fails; the build fails on orphaned snapshots after a unit run. - AGENTS.md describes the workflow. The PR review prompt no longer claims the repo uses xUnit. The five TestDiagnosticsCollector copies stay: they differ in lifecycle (Write vs HandleItem), so merging them is a behaviour change. DiagnosticsSnapshot works on any of them. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Compare rendered HTML with Nullean.Snapshots.Html ShouldBeHtml and ShouldContainHtml (Elastic.Markdown.Tests) and ConvertsToHtml, ConvertsToContainingHtml and the per-file ToHtml and ContainsHtml (Elastic.Authoring.Tests) keep their names and call sites, and now run on BeHtml and ContainHtml with the shared DocsHtml options. That removes the second copy of the AngleSharp.Diffing diff code and the AngleSharp.Diffing package. DiffPlex stays in Authoring for the plain text assertions. One test passed without testing anything. "marks exactly two options as a fork" gave the bare text `is-fork` to ConvertsToContainingHtml. The old normaliser printed only element children, so a text-only fragment became an empty string and every page contains an empty string. It now uses ConvertsToContainingRawHtml, like the test above it. Checked by corrupting one ShouldContainHtml and one ConvertsToHtml expectation: both fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Directives tests: snapshot the rendered admonition and dropdown HTML The dropdown title and applies_to tests each asserted a few Contain and NotContain fragments on the rendered HTML. They are now one whole-output snapshot per class (inline for the small dropdown outputs, a file for the nested-content dropdown and the applies_to admonitions, whose popover payload makes them long). The tests became stricter: every old fragment was checked to appear in (or, for NotContain, be absent from) the accepted snapshot before it was accepted. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Directives tests: snapshot the rendered agent skill HTML The Contain and NotContain fragment tests became whole-output snapshots per class, so the tests are stricter: every old fragment was checked to appear in (or, for NotContain, be absent from) the accepted snapshot before it was accepted. Small outputs are inline snapshots, the rest are files. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Directives tests: snapshot the rendered changelog HTML The Contain and NotContain fragment tests became whole-output snapshots per class, so the tests are stricter: every old fragment was checked to appear in (or, for NotContain, be absent from) the accepted snapshot before it was accepted. Small outputs are inline snapshots, the rest are files. The changelog tests that only asserted NotContain on an otherwise empty output (empty bundles, filtered-out types) now assert the empty output and an empty diagnostics list, which a failing directive no longer satisfies. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Directives tests: snapshot the rendered csv, sub-pages, stepper, storybook and table HTML The Contain and NotContain fragment tests became whole-output snapshots per class, so the tests are stricter: every old fragment was checked to appear in (or, for NotContain, be absent from) the accepted snapshot before it was accepted. Small outputs are inline snapshots, the rest are files. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Directives tests: finish the agent skill HTML snapshots The remaining single-fragment Contain and NotContain tests of these classes became the class's whole-output snapshot, so the tests are stricter: every old fragment was checked to appear in (or, for NotContain, be absent from) the accepted snapshot before it was accepted. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Directives tests: finish the changelog HTML snapshots The remaining single-fragment Contain and NotContain tests of these classes became the class's whole-output snapshot, so the tests are stricter: every old fragment was checked to appear in (or, for NotContain, be absent from) the accepted snapshot before it was accepted. The NotContain-only tests of changelog output that ends up empty also assert an empty diagnostics list now. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Directives tests: snapshot the rendered button, mermaid, math, vector sizing and applicability switch HTML The remaining single-fragment Contain and NotContain tests of these classes became the class's whole-output snapshot, so the tests are stricter: every old fragment was checked to appear in (or, for NotContain, be absent from) the accepted snapshot before it was accepted. Mermaid snapshots scrub the diagram file name hash, which tracks the renderer output rather than the markup. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Directives tests: finish the csv, sub-pages, storybook and table HTML snapshots The remaining single-fragment Contain and NotContain tests of these classes became the class's whole-output snapshot, so the tests are stricter: every old fragment was checked to appear in (or, for NotContain, be absent from) the accepted snapshot before it was accepted. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Directives tests: snapshot the HTML of changelog, button and storybook tests that also assert on the block Tests that asserted on the parsed block and on fragments of the rendered HTML keep the block assertions and check the HTML against a snapshot, so they are stricter: every old fragment is in the accepted snapshot, and the NotContain fragments are absent. The button group's Contain(Primary).And.Contain(Secondary) test is covered by the group snapshot. Tests whose HTML expectation is derived from the block (TOC slugs, catalog titles and URLs, element order and counts) keep their assertions. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Directives tests: snapshot the changelog diagnostics The Collector assertions (empty, counts, severity and message-substring predicates) became a snapshot of the whole diagnostics text (error|warning|hint file:line message), so the tests are stricter: they now also pin the file and line and fail on any additional diagnostic. Every old message fragment and severity was checked to appear in the accepted text, and the empty-diagnostics tests are MatchInlineSnapshot(""). Tests that became identical were merged. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Directives tests: snapshot the diagnostics of the other directives The Collector assertions (empty, counts, severity and message-substring predicates) became a snapshot of the whole diagnostics text (error|warning|hint file:line message), so the tests are stricter: they now also pin the file and line and fail on any additional diagnostic. Every old message fragment and severity was checked to appear in the accepted text, and the empty-diagnostics tests are MatchInlineSnapshot(""). Tests that became identical were merged. The unsupported directive tests differ per derived class, so they use a file snapshot each. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Scrub paths the same way on every operating system The repository and temp folder scrubbers replaced only the root, so a Windows run printed {repo}\docs\index.md where the verified text says {repo}/docs/index.md, and about 160 diagnostics snapshots failed there. The path that follows the root is now written with forward slashes. The temp scrubber also swallowed everything up to the next whitespace, including the :line after a file name. A path now ends at a colon too, and the repository root is tried before the temp root because the repository can sit under the temp folder. The doc comment claimed it scrubbed git SHAs; it does not, and no longer says so. Tested with a Windows-style root, which also runs on Linux and macOS. Removing the separator replacement makes that test fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Scrub random folder names under the temp directory A test that works in a Guid-named folder in the temp directory printed that name in its diagnostics, so a snapshot of them could not be stable. The temp scrubber now replaces a run of 32 hex digits in the path with {id}. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Make diagnostics snapshots and snapshot paths the same on every operating system CI on Windows failed five of the stacked PRs for reasons a laptop never shows: - DiagnosticsSnapshot now writes paths with forward slashes and drops a leading drive letter, so `C:\docs\toc.yml`, `C:/docs/toc.yml` and `/docs/toc.yml` read the same, and `testing\req.md` inside a message reads `testing/req.md`. - ToSnapshotText(firstLineOnly: true) keeps only the first line of each message, for diagnostics that carry an exception: its stack trace has source line numbers and the exception type differs per platform. - SnapshotPathLengthTests fails when a verified snapshot's path, with the received file's longer folder and the CI checkout root, would pass Windows' 260 characters. git on the runner cannot check such a file out, so the whole job failed before any test ran. Each has a test. Lowering the length limit makes the guard fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Keep exception internals out of the directive diagnostics snapshots Three snapshots held a .NET stack trace: the storybook registry error and the two malformed applies-switch YAML errors. A trace has source line numbers and library frames, so it breaks when StorybookBlock.cs is edited or SharpYaml is upgraded, and on Windows the storybook case throws a different exception altogether (a path with a colon is not a legal path there). They now snapshot the first line of each diagnostic. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Match the repository root in the form DiagnosticsSnapshot writes it DiagnosticsSnapshot drops a leading drive letter and writes forward slashes, so on Windows D:\a\docs-builder\docs-builder\docs\index.md reaches the {repo} scrubber as /a/docs-builder/docs-builder/docs/index.md, which it no longer recognised. Every diagnostics snapshot that names a file failed on the Windows runner. The scrubber now also matches the root without its drive letter. Removing that variant makes the new test fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>

* Wire Nullean.Snapshots into every test project The ApiExplorer pilot (#4654) used Nullean.Snapshots.Html and .TUnit 0.2.0 in one project. This makes the library available to all unit and integration test projects at 0.4.0 and adds the shared pieces the next migration steps build on: - tests-shared/Elastic.Documentation.Testing: DocsHtml (page section without the fixture H1 and htmx attributes), DiagnosticsSnapshot, NavigationDump, MockFileSystemDump. A scrubber for temp and repo paths is compiled into each test project. - The snap CLI as a local tool; CI uploads received snapshots when a test run fails; the build fails on orphaned snapshots after a unit run. - AGENTS.md describes the workflow. The PR review prompt no longer claims the repo uses xUnit. The five TestDiagnosticsCollector copies stay: they differ in lifecycle (Write vs HandleItem), so merging them is a behaviour change. DiagnosticsSnapshot works on any of them. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Compare rendered HTML with Nullean.Snapshots.Html ShouldBeHtml and ShouldContainHtml (Elastic.Markdown.Tests) and ConvertsToHtml, ConvertsToContainingHtml and the per-file ToHtml and ContainsHtml (Elastic.Authoring.Tests) keep their names and call sites, and now run on BeHtml and ContainHtml with the shared DocsHtml options. That removes the second copy of the AngleSharp.Diffing diff code and the AngleSharp.Diffing package. DiffPlex stays in Authoring for the plain text assertions. One test passed without testing anything. "marks exactly two options as a fork" gave the bare text `is-fork` to ConvertsToContainingHtml. The old normaliser printed only element children, so a text-only fragment became an empty string and every page contains an empty string. It now uses ConvertsToContainingRawHtml, like the test above it. Checked by corrupting one ShouldContainHtml and one ConvertsToHtml expectation: both fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Markdown tests: inline HTML snapshots where ShouldBeHtml/ShouldContainHtml were used The 17 ShouldBeHtml/ShouldContainHtml call sites outside Directives become MatchInlineHtmlSnapshot with DocsHtml.Body(). The ShouldBeHtml sites kept their literals unchanged and pass as they are. The ShouldContainHtml sites compared a fragment of the page; the snapshot is the whole rendered output, so those tests became stricter. Every old fragment appears in the accepted snapshot (the htmx attributes are removed by DocsHtml.Body(), as ShouldContainHtml already did). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Markdown tests (Inline): whole-output snapshots instead of Contain fragments Output of the Inline fixtures now lives in inline snapshots. Where a test (or a cluster of tests of one class) asserted Contain/NotContain fragments on the rendered HTML, it is one MatchInlineHtmlSnapshot of the whole output; where it asserted a count or message on Collector.Diagnostics, it is a MatchInlineSnapshot of the diagnostics text ("" when the run is clean). These tests became stricter: they now fail on any unexpected change in the output, not only on the fragments they named. Before accepting, every old Contain fragment was checked to appear in the accepted snapshot (and no NotContain fragment to appear); the htmx attributes on links are removed by DocsHtml.Body(), as ShouldContainHtml already did. Tests of one class that only asserted fragments of the same output are merged into one test; their explanatory comments moved onto it. Kept as they were: the footnote index/regex-count tests, the raw-output nested anchor check (the HTML parser would split nested anchors in a snapshot), and BeNullOrWhiteSpace. Found: MutationOperatorsWorkInLinks asserted Contain("ELASTICSEARCH 9.0"), which a literal "[text](ELASTICSEARCH 9.0)" in the output satisfied, so the test passed while the link did not render as a link. The snapshot now shows it. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Markdown tests (Inline): footnote outputs over 25 lines move to file snapshots Same expectations as before (the accepted inline snapshots, moved to .snapshots/Inline/*.verified.html) so the test file stays readable. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Markdown tests (CodeBlocks): whole-output snapshots instead of fragments Contain/NotContain fragments on the rendered HTML become one MatchInlineHtmlSnapshot per test and diagnostics counts/prefix checks become MatchInlineSnapshot of the diagnostics text, so these tests became stricter. Every old Contain fragment was checked to appear in the accepted snapshot. Caveat: the HTML snapshot collapses whitespace inside pre, so a change in the line structure of a code block alone is not caught by it. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Markdown tests (FileInclusion): whole-output snapshots instead of fragments Fragment checks on the included output become HTML snapshots (files for the larger ones) and the diagnostics assertions become diagnostics snapshots. These tests became stricter; every old Contain fragment appears in the accepted snapshot. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Markdown tests (SettingsInclusion): HTML snapshots of the included settings The settings fixtures rendered several hundred lines each, so those snapshots are files. The tests of one fixture that asserted fragments of the same output are one snapshot test now; they became stricter. Every old Contain fragment appears in the accepted snapshot. RandomFileEmitsAnError keeps its targeted assertions: the diagnostic message embeds a YAML parser stack trace, which a snapshot should not pin. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Markdown tests (TaskList, Assembler, Codex): whole-output snapshots The Assembler and Codex tests are about the htmx attributes and the link target on cross links, so their snapshots keep those attributes (no DocsHtml.Body() normalisation). Fragment checks became whole-output snapshots, so the tests became stricter. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Markdown tests (AppliesTo, DocSet): JSON snapshots of the serialised output The nine Should().Be("""...""") JSON comparisons (AppliesTo serialisation, RepositoryLinks) become MatchInlineJsonSnapshot with their literals unchanged, and they pass as they were. SerializeAllLifecycles, SerializeComplex and SerializeValidatesJsonStructure checked fragments or walked the parsed JSON; they are whole-output snapshots now, so they became stricter (every old fragment and property value appears in the snapshots). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Markdown tests (Search): snapshot the serialised DocumentationDocument The applies_to tests walked the parsed JSON property by property; they are one JSON snapshot of the whole serialised document each, so they became stricter. Every property the old tests asserted appears in the snapshots. SerializeDocumentWithNullAppliesToOmitsField was vacuous: its else branch asserted true.Should().BeTrue(), so it passed whether the field was omitted or null. The snapshot pins that the field is omitted. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Markdown tests (Exporters): snapshot OKF index content The four RenderIndexContent tests checked Contain/StartWith fragments of a few lines of text; they are inline text snapshots now and became stricter. Every old fragment appears in the snapshots. The pure-function tests and the YAML round-trip tests stay as they are. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Markdown tests (FrontMatter and root): diagnostics snapshots Count and message-substring assertions on the collector become snapshots of the diagnostics text, so they pin the severity, location and whole message and became stricter. The four products-suggestion tests keep their old assertions: their diagnostic embeds an exception stack trace with source line numbers, which a snapshot should not pin. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Markdown tests (Tracking): diagnostics snapshots for the redirect validation errors The three tests that checked an error by substring now snapshot the diagnostics text (the temp path is scrubbed to {temp}), so they pin the severity and the whole message and became stricter. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Scrub paths the same way on every operating system The repository and temp folder scrubbers replaced only the root, so a Windows run printed {repo}\docs\index.md where the verified text says {repo}/docs/index.md, and about 160 diagnostics snapshots failed there. The path that follows the root is now written with forward slashes. The temp scrubber also swallowed everything up to the next whitespace, including the :line after a file name. A path now ends at a colon too, and the repository root is tried before the temp root because the repository can sit under the temp folder. The doc comment claimed it scrubbed git SHAs; it does not, and no longer says so. Tested with a Windows-style root, which also runs on Linux and macOS. Removing the separator replacement makes that test fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Scrub random folder names under the temp directory A test that works in a Guid-named folder in the temp directory printed that name in its diagnostics, so a snapshot of them could not be stable. The temp scrubber now replaces a run of 32 hex digits in the path with {id}. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Make diagnostics snapshots and snapshot paths the same on every operating system CI on Windows failed five of the stacked PRs for reasons a laptop never shows: - DiagnosticsSnapshot now writes paths with forward slashes and drops a leading drive letter, so `C:\docs\toc.yml`, `C:/docs/toc.yml` and `/docs/toc.yml` read the same, and `testing\req.md` inside a message reads `testing/req.md`. - ToSnapshotText(firstLineOnly: true) keeps only the first line of each message, for diagnostics that carry an exception: its stack trace has source line numbers and the exception type differs per platform. - SnapshotPathLengthTests fails when a verified snapshot's path, with the received file's longer folder and the CI checkout root, would pass Windows' 260 characters. git on the runner cannot check such a file out, so the whole job failed before any test ran. Each has a test. Lowering the length limit makes the guard fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Match the repository root in the form DiagnosticsSnapshot writes it DiagnosticsSnapshot drops a leading drive letter and writes forward slashes, so on Windows D:\a\docs-builder\docs-builder\docs\index.md reaches the {repo} scrubber as /a/docs-builder/docs-builder/docs/index.md, which it no longer recognised. Every diagnostics snapshot that names a file failed on the Windows runner. The scrubber now also matches the root without its drive letter. Removing that variant makes the new test fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>

* Wire Nullean.Snapshots into every test project The ApiExplorer pilot (#4654) used Nullean.Snapshots.Html and .TUnit 0.2.0 in one project. This makes the library available to all unit and integration test projects at 0.4.0 and adds the shared pieces the next migration steps build on: - tests-shared/Elastic.Documentation.Testing: DocsHtml (page section without the fixture H1 and htmx attributes), DiagnosticsSnapshot, NavigationDump, MockFileSystemDump. A scrubber for temp and repo paths is compiled into each test project. - The snap CLI as a local tool; CI uploads received snapshots when a test run fails; the build fails on orphaned snapshots after a unit run. - AGENTS.md describes the workflow. The PR review prompt no longer claims the repo uses xUnit. The five TestDiagnosticsCollector copies stay: they differ in lifecycle (Write vs HandleItem), so merging them is a behaviour change. DiagnosticsSnapshot works on any of them. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Compare rendered HTML with Nullean.Snapshots.Html ShouldBeHtml and ShouldContainHtml (Elastic.Markdown.Tests) and ConvertsToHtml, ConvertsToContainingHtml and the per-file ToHtml and ContainsHtml (Elastic.Authoring.Tests) keep their names and call sites, and now run on BeHtml and ContainHtml with the shared DocsHtml options. That removes the second copy of the AngleSharp.Diffing diff code and the AngleSharp.Diffing package. DiffPlex stays in Authoring for the plain text assertions. One test passed without testing anything. "marks exactly two options as a fork" gave the bare text `is-fork` to ConvertsToContainingHtml. The old normaliser printed only element children, so a text-only fragment became an empty string and every page contains an empty string. It now uses ConvertsToContainingRawHtml, like the test above it. Checked by corrupting one ShouldContainHtml and one ConvertsToHtml expectation: both fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Changelog render tests: snapshot the rendered files The render tests read index.md, highlights.md and the other generated files back from the mock file system and checked a few Contain and NotContain fragments. Each one now snapshots every file the service wrote (MockFileSystemDump), inline when the output is short and as a verified file otherwise, so a change to any generated line shows up as a diff and the absence of a file is part of the expectation. The tests became stricter: before accepting, every old Contain and NotContain fragment was checked against the received output, and the ordering checks in the GFM tests (sections, highlights first) are covered by the snapshot. Warnings and errors the tests looked for with Diagnostics.Should().Contain(...) are inline diagnostics snapshots now, so any other diagnostic would fail them too. Guids in the paths of the mock file system are numbered by a scrubber registered for this project. The invalid-bundle test accepted either of two messages; it now asserts the single error that is actually produced. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Changelog create tests: snapshot the changelog files and diagnostics The create tests read the YAML file the service wrote and checked a few Contain fragments (title, type, a PR url...). Each one now snapshots the whole file inline, and the multi-file tests snapshot every entry under its file name. The snapshots are text, not YAML trees, so the quoting and key order of the written YAML stay pinned (several old checks were about exactly that, e.g. feature-id: "feature:foo"). Every changelog file starts with the same 78 lines documenting the fields; ChangelogEntryYaml.WithoutSchemaComments strips them so the snapshots are about the entry, and the Unpack test keeps its check that the block is there. The bundle's release-date is scrubbed. Warnings and errors that were looked up with Diagnostics.Should().Contain are inline diagnostics snapshots, so the exact message and any additional diagnostic is now part of the expectation. The tests became stricter: before accepting, every old Contain and NotContain fragment was checked against the received output. One old check was vacuous: the multiple-PRs-with-failing-fetch test accepted either PR url and only one is written; the snapshot pins the first. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Changelog bundle tests: snapshot the bundle files and diagnostics The bundling tests read the bundle (or amend sidecar) the service wrote and checked a few Contain fragments, or split the text to count entries. Each one now snapshots the whole file, inline up to 35 lines and as a verified file beyond that. Errors and warnings looked up with Diagnostics.Should().Contain(...) are inline diagnostics snapshots. The tests became stricter: before accepting, every old Contain and NotContain fragment was checked against the received output (top-level product checks that applied only to the part before entries: still hold, the snapshot shows the entries with their own products). The snapshots also pin the deprecation and repository-name warnings several profile tests used to ignore. A bundle is stamped with the day it was created, so the project scrubs a release-date that equals today (a release-date a test sets stays visible). Also two comments describing the create tests are put back. The two checksum comparison tests (two bundles from the same data) and the calls that need two outputs in one test are left as they were. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Changelog amend tests: snapshot the CDN amend sidecars The four CDN amend tests checked a few Contain fragments of the sidecar that ReadSingleAmendAsync returns. They snapshot it inline now. The tests became stricter; the old fragments (exclude-entries, the entry names, the title) were checked against the received output first. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Scrub paths the same way on every operating system The repository and temp folder scrubbers replaced only the root, so a Windows run printed {repo}\docs\index.md where the verified text says {repo}/docs/index.md, and about 160 diagnostics snapshots failed there. The path that follows the root is now written with forward slashes. The temp scrubber also swallowed everything up to the next whitespace, including the :line after a file name. A path now ends at a colon too, and the repository root is tried before the temp root because the repository can sit under the temp folder. The doc comment claimed it scrubbed git SHAs; it does not, and no longer says so. Tested with a Windows-style root, which also runs on Linux and macOS. Removing the separator replacement makes that test fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Scrub random folder names under the temp directory A test that works in a Guid-named folder in the temp directory printed that name in its diagnostics, so a snapshot of them could not be stable. The temp scrubber now replaces a run of 32 hex digits in the path with {id}. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Make diagnostics snapshots and snapshot paths the same on every operating system CI on Windows failed five of the stacked PRs for reasons a laptop never shows: - DiagnosticsSnapshot now writes paths with forward slashes and drops a leading drive letter, so `C:\docs\toc.yml`, `C:/docs/toc.yml` and `/docs/toc.yml` read the same, and `testing\req.md` inside a message reads `testing/req.md`. - ToSnapshotText(firstLineOnly: true) keeps only the first line of each message, for diagnostics that carry an exception: its stack trace has source line numbers and the exception type differs per platform. - SnapshotPathLengthTests fails when a verified snapshot's path, with the received file's longer folder and the CI checkout root, would pass Windows' 260 characters. git on the runner cannot check such a file out, so the whole job failed before any test ran. Each has a test. Lowering the length limit makes the guard fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Strip the changelog schema comments whatever the line ending WithoutSchemaComments looks for the blank line that ends the field documentation, "\n\n". The writer uses the platform's line ending, so on Windows it is "\r\n\r\n", the header stayed in, and 161 tests failed there. The helper now normalises line endings first. The new test runs both endings and fails for "\r\n" without the fix. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Match the repository root in the form DiagnosticsSnapshot writes it DiagnosticsSnapshot drops a leading drive letter and writes forward slashes, so on Windows D:\a\docs-builder\docs-builder\docs\index.md reaches the {repo} scrubber as /a/docs-builder/docs-builder/docs/index.md, which it no longer recognised. Every diagnostics snapshot that names a file failed on the Windows runner. The scrubber now also matches the root without its drive letter. Removing that variant makes the new test fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Scrub entry checksums from the changelog bundle snapshots A bundle records the SHA-1 of each entry file it was made from. The test inputs are raw string literals and the product writes files with the platform's line ending, so on Windows the same entry has other bytes and another checksum: 111 bundle snapshots failed there on the checksum line alone. The value is now replaced by {sha1} in the snapshots. What a checksum must be is still checked by the tests that compute the expected one from the file content. All 112 changed snapshots differ only in their checksum lines (reviewed before accepting). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>

…4726) * Wire Nullean.Snapshots into every test project The ApiExplorer pilot (#4654) used Nullean.Snapshots.Html and .TUnit 0.2.0 in one project. This makes the library available to all unit and integration test projects at 0.4.0 and adds the shared pieces the next migration steps build on: - tests-shared/Elastic.Documentation.Testing: DocsHtml (page section without the fixture H1 and htmx attributes), DiagnosticsSnapshot, NavigationDump, MockFileSystemDump. A scrubber for temp and repo paths is compiled into each test project. - The snap CLI as a local tool; CI uploads received snapshots when a test run fails; the build fails on orphaned snapshots after a unit run. - AGENTS.md describes the workflow. The PR review prompt no longer claims the repo uses xUnit. The five TestDiagnosticsCollector copies stay: they differ in lifecycle (Write vs HandleItem), so merging them is a behaviour change. DiagnosticsSnapshot works on any of them. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Compare rendered HTML with Nullean.Snapshots.Html ShouldBeHtml and ShouldContainHtml (Elastic.Markdown.Tests) and ConvertsToHtml, ConvertsToContainingHtml and the per-file ToHtml and ContainsHtml (Elastic.Authoring.Tests) keep their names and call sites, and now run on BeHtml and ContainHtml with the shared DocsHtml options. That removes the second copy of the AngleSharp.Diffing diff code and the AngleSharp.Diffing package. DiffPlex stays in Authoring for the plain text assertions. One test passed without testing anything. "marks exactly two options as a fork" gave the bare text `is-fork` to ConvertsToContainingHtml. The old normaliser printed only element children, so a text-only fragment became an empty string and every page contains an empty string. It now uses ConvertsToContainingRawHtml, like the test above it. Checked by corrupting one ShouldContainHtml and one ConvertsToHtml expectation: both fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Navigation tests: snapshot navigation trees instead of walking them The isolated and assembler navigation tests found nodes with ElementAt/First and asserted Url, type and child count one node at a time, so a wrong URL elsewhere in the tree went unnoticed (several only checked StartWith on a prefix). They now snapshot the whole tree inline as indented text. The shared NavigationDump leaves out each node's index page, where most URLs live, so NavigationTreeDump in this project adds an index line per node and flags a child whose Parent is not the node it is listed under. These tests became stricter: every URL, title, type, hidden/island flag and child count of the tree is now pinned, which includes every fact the removed assertions checked. Tests about behaviour (HomeProvider changes, island rendering, parent and root identity) are untouched. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Navigation rendering tests: snapshot the rendered component The secondary nav, nav tree, table of contents, footer, docs bar and nav side tests checked fragments with Contain/NotContain and IndexOf ordering. They now write the rendered component to a verified HTML file (htmx attributes on links removed), so an unexpected extra, missing or reordered element shows in the diff. These tests became stricter. Before accepting, every Contain/NotContain literal of the old assertions was checked against the snapshot text with a script (the scoped negatives, such as 'no chevron inside the page link' and 'no <a> before the title', were checked by reading the HTML). Several tests rendered the same input and asserted different fragments of it (top nav bar: 5 tests into 1, built-in bar: 2 into 1, mobile drawer: 2 into 1, docs bar: 4 into 1), which would have produced identical snapshot files. They are merged into one test per render, so the test count drops by 8 with no coverage lost. Left as Contain checks: the landing page and style guide pages (whole pages of several hundred lines), the head asset ordering tests and the examples' generic invariants. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Configuration tests: snapshot diagnostics and deserialised tables of contents Error and warning tests asserted a count (Errors == 1, Warnings == 1) or that one message contained a substring, so a test named EmitsError_WhenProductMissing passed on any single error. They now snapshot every diagnostic as 'severity file:line message' (UnknownYamlKeyTests, the API configuration errors, the CTA default errors, the Storybook registry warning, the docset/toc unknown key errors). The Column assertions stay as explicit checks because the snapshot text has no column. The parameterised repository-form error uses a file snapshot, as its output differs per case. DocumentationSetFileTests, SiteNavigationFileTests and CreateNavigationFileTests walked deserialised refs element by element, or checked the generated navigation.yml with Contain/NotContain. They now snapshot the TOC (TocDump), the site navigation file (SiteNavDump) and the generated YAML text. These tests became stricter: each fact the removed assertions checked (types, paths, hidden/island flags, titles, child counts, removed private entries) is visible in the snapshots. Default-value and scalar property tests, the Arguments tables, CsCheck properties and Throw assertions are untouched. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Use the shared NavigationDump in Navigation.Tests NavigationDump now prints each node's index page before its children and flags a child whose Parent is not the node it is listed under. That is what Navigation.Tests had in a local NavigationTreeDump, so the local copy goes and the snapshots stay as they were. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Scrub paths the same way on every operating system The repository and temp folder scrubbers replaced only the root, so a Windows run printed {repo}\docs\index.md where the verified text says {repo}/docs/index.md, and about 160 diagnostics snapshots failed there. The path that follows the root is now written with forward slashes. The temp scrubber also swallowed everything up to the next whitespace, including the :line after a file name. A path now ends at a colon too, and the repository root is tried before the temp root because the repository can sit under the temp folder. The doc comment claimed it scrubbed git SHAs; it does not, and no longer says so. Tested with a Windows-style root, which also runs on Linux and macOS. Removing the separator replacement makes that test fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Scrub random folder names under the temp directory A test that works in a Guid-named folder in the temp directory printed that name in its diagnostics, so a snapshot of them could not be stable. The temp scrubber now replaces a run of 32 hex digits in the path with {id}. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Make diagnostics snapshots and snapshot paths the same on every operating system CI on Windows failed five of the stacked PRs for reasons a laptop never shows: - DiagnosticsSnapshot now writes paths with forward slashes and drops a leading drive letter, so `C:\docs\toc.yml`, `C:/docs/toc.yml` and `/docs/toc.yml` read the same, and `testing\req.md` inside a message reads `testing/req.md`. - ToSnapshotText(firstLineOnly: true) keeps only the first line of each message, for diagnostics that carry an exception: its stack trace has source line numbers and the exception type differs per platform. - SnapshotPathLengthTests fails when a verified snapshot's path, with the received file's longer folder and the CI checkout root, would pass Windows' 260 characters. git on the runner cannot check such a file out, so the whole job failed before any test ran. Each has a test. Lowering the length limit makes the guard fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Match the repository root in the form DiagnosticsSnapshot writes it DiagnosticsSnapshot drops a leading drive letter and writes forward slashes, so on Windows D:\a\docs-builder\docs-builder\docs\index.md reaches the {repo} scrubber as /a/docs-builder/docs-builder/docs/index.md, which it no longer recognised. Every diagnostics snapshot that names a file failed on the Windows runner. The scrubber now also matches the root without its drive letter. Removing that variant makes the new test fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>

* Wire Nullean.Snapshots into every test project The ApiExplorer pilot (#4654) used Nullean.Snapshots.Html and .TUnit 0.2.0 in one project. This makes the library available to all unit and integration test projects at 0.4.0 and adds the shared pieces the next migration steps build on: - tests-shared/Elastic.Documentation.Testing: DocsHtml (page section without the fixture H1 and htmx attributes), DiagnosticsSnapshot, NavigationDump, MockFileSystemDump. A scrubber for temp and repo paths is compiled into each test project. - The snap CLI as a local tool; CI uploads received snapshots when a test run fails; the build fails on orphaned snapshots after a unit run. - AGENTS.md describes the workflow. The PR review prompt no longer claims the repo uses xUnit. The five TestDiagnosticsCollector copies stay: they differ in lifecycle (Write vs HandleItem), so merging them is a behaviour change. DiagnosticsSnapshot works on any of them. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Compare rendered HTML with Nullean.Snapshots.Html ShouldBeHtml and ShouldContainHtml (Elastic.Markdown.Tests) and ConvertsToHtml, ConvertsToContainingHtml and the per-file ToHtml and ContainsHtml (Elastic.Authoring.Tests) keep their names and call sites, and now run on BeHtml and ContainHtml with the shared DocsHtml options. That removes the second copy of the AngleSharp.Diffing diff code and the AngleSharp.Diffing package. DiffPlex stays in Authoring for the plain text assertions. One test passed without testing anything. "marks exactly two options as a fork" gave the bare text `is-fork` to ConvertsToContainingHtml. The old normaliser printed only element children, so a text-only fragment became an empty string and every page contains an empty string. It now uses ConvertsToContainingRawHtml, like the test above it. Checked by corrupting one ShouldContainHtml and one ConvertsToHtml expectation: both fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Run the MCP handshake test from an .http file McpHandshakeTests posted one initialize request, parsed the SSE data line by hand and asserted three fields. The request now lives in http/mcp-handshake.http (initialize, then tools/list) and the whole response is the snapshot, so a change to the capabilities, the instructions or any tool definition shows up as a diff. Stricter than before: the full initialize result and the complete tools/list response are compared, not just protocolVersion and serverInfo. The SSE body is quoted text in the HTTP format, so a small scrubber pretty-prints each data: line. The server version is replaced by a placeholder only when it equals the assembly's informational version, which keeps the old check that the wiring reports the right version. Adds the Nullean.Snapshots.Http packages (0.4.0) to central package management and to the two HTTP-facing test projects. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Run the page-feedback endpoint through an .http file and snapshot the stored records http/page-feedback.http sends a valid PUT with an euid cookie (204), an invalid PUT (400) and a DELETE (204) through the shared test host, and the responses are the snapshot. The three tests that checked the PageFeedbackRecord the endpoint handed to the service field by field now compare the whole record as an inline JSON snapshot. The feedback id is replaced by a placeholder only where it equals the id in the request URL, so storing the record under another id still fails. Stricter than before: the reaction-only and multiple-reasons tests used to check only some fields and now check all of them. The other 400/503 tests stay: they check the service was not called, which a response snapshot cannot show. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Run the URL normaliser tests with the unit tests DocumentUrlNormalizerTests needs no Elasticsearch, but it sat in the integration project because DocumentGateway.NormalizeUrl is internal and only that project could see it. The Mcp.Remote assembly now also shows its internals to Mcp.Remote.Tests, so the test moves to the unit project and runs in CI, which the integration tests do not. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Scrub paths the same way on every operating system The repository and temp folder scrubbers replaced only the root, so a Windows run printed {repo}\docs\index.md where the verified text says {repo}/docs/index.md, and about 160 diagnostics snapshots failed there. The path that follows the root is now written with forward slashes. The temp scrubber also swallowed everything up to the next whitespace, including the :line after a file name. A path now ends at a colon too, and the repository root is tried before the temp root because the repository can sit under the temp folder. The doc comment claimed it scrubbed git SHAs; it does not, and no longer says so. Tested with a Windows-style root, which also runs on Linux and macOS. Removing the separator replacement makes that test fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Scrub random folder names under the temp directory A test that works in a Guid-named folder in the temp directory printed that name in its diagnostics, so a snapshot of them could not be stable. The temp scrubber now replaces a run of 32 hex digits in the path with {id}. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Make diagnostics snapshots and snapshot paths the same on every operating system CI on Windows failed five of the stacked PRs for reasons a laptop never shows: - DiagnosticsSnapshot now writes paths with forward slashes and drops a leading drive letter, so `C:\docs\toc.yml`, `C:/docs/toc.yml` and `/docs/toc.yml` read the same, and `testing\req.md` inside a message reads `testing/req.md`. - ToSnapshotText(firstLineOnly: true) keeps only the first line of each message, for diagnostics that carry an exception: its stack trace has source line numbers and the exception type differs per platform. - SnapshotPathLengthTests fails when a verified snapshot's path, with the received file's longer folder and the CI checkout root, would pass Windows' 260 characters. git on the runner cannot check such a file out, so the whole job failed before any test ran. Each has a test. Lowering the length limit makes the guard fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Match the repository root in the form DiagnosticsSnapshot writes it DiagnosticsSnapshot drops a leading drive letter and writes forward slashes, so on Windows D:\a\docs-builder\docs-builder\docs\index.md reaches the {repo} scrubber as /a/docs-builder/docs-builder/docs/index.md, which it no longer recognised. Every diagnostics snapshot that names a file failed on the Windows runner. The scrubber now also matches the root without its drive letter. Removing that variant makes the new test fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>

* Wire Nullean.Snapshots into every test project The ApiExplorer pilot (#4654) used Nullean.Snapshots.Html and .TUnit 0.2.0 in one project. This makes the library available to all unit and integration test projects at 0.4.0 and adds the shared pieces the next migration steps build on: - tests-shared/Elastic.Documentation.Testing: DocsHtml (page section without the fixture H1 and htmx attributes), DiagnosticsSnapshot, NavigationDump, MockFileSystemDump. A scrubber for temp and repo paths is compiled into each test project. - The snap CLI as a local tool; CI uploads received snapshots when a test run fails; the build fails on orphaned snapshots after a unit run. - AGENTS.md describes the workflow. The PR review prompt no longer claims the repo uses xUnit. The five TestDiagnosticsCollector copies stay: they differ in lifecycle (Write vs HandleItem), so merging them is a behaviour change. DiagnosticsSnapshot works on any of them. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Compare rendered HTML with Nullean.Snapshots.Html ShouldBeHtml and ShouldContainHtml (Elastic.Markdown.Tests) and ConvertsToHtml, ConvertsToContainingHtml and the per-file ToHtml and ContainsHtml (Elastic.Authoring.Tests) keep their names and call sites, and now run on BeHtml and ContainHtml with the shared DocsHtml options. That removes the second copy of the AngleSharp.Diffing diff code and the AngleSharp.Diffing package. DiffPlex stays in Authoring for the plain text assertions. One test passed without testing anything. "marks exactly two options as a fork" gave the bare text `is-fork` to ConvertsToContainingHtml. The old normaliser printed only element children, so a text-only fragment became an empty string and every page contains an empty string. It now uses ConvertsToContainingRawHtml, like the test above it. Checked by corrupting one ShouldContainHtml and one ConvertsToHtml expectation: both fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * LegacyDocs.Migration.Tests: inline snapshots of emitted markdown EmitterTests checked ~70 Contain/NotContain fragments per output. Each test now snapshots the whole emitted markdown inline. The tests became stricter: every old fact (no raw << , no include::/ifeval::, fences, links) is visible in the literals, and any other output change now fails too. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * SiteSearch.Tests: JSON snapshots of mapped documents and ES mappings ContentStackMappingTests asserted a handful of properties per fixture; each fixture test now snapshots the whole mapped SiteDocument (the two clock-stamped fields are removed first). MappingStructureTests walked the mapping JSON property by property; the five document mappings are now snapshotted whole. The tests became stricter. Dropped the old tautology Body.Should().Be(Body) and if-guarded checks that passed vacuously when the field was absent. Random AutoBogus roundtrips and the small inline-JSON mapper cases stay. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Api.Tests: snapshot the built ES queries and API mappings SearchQueryBuilderTests and ApiPropertySearchTests walked the query JSON with GetProperty chains and Contain/NotContain. The queries and the api.* mapping (lexical and semantic) are now JSON snapshots, so the tests became stricter. The existing one-line expectations are kept as inline JSON snapshots. Dropped an unused 'prefix' argument that both rows set to false. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Api.Infrastructure.Tests: snapshot the SSE output of the stream transformers StreamTransformerTests parsed the transformer output back into events and asserted counts and a few fields. They now snapshot the SSE text the client receives, with generated ids and clock timestamps scrubbed. The tests became stricter: every event, its order and its content is pinned. The three parameterised ConversationStart checks (event present, id, timestamp) collapse into one snapshot per transformer. The multi-line data test asserted HaveCountGreaterThanOrEqualTo(0), which cannot fail; it now pins the output. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Build.Tests: snapshot the sitemap XML and the sitemap search body SitemapTests parsed the XML and checked element by element, and checked the PIT search body with Contain. The generated sitemap.xml and the search body are now inline snapshots, so the tests became stricter (namespace, declaration, lastmod format, size, sort and all). The ordering and API-exclusion tests used the current time for lastmod; they use a fixed date now. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Mcp.Remote.Tests: one inline snapshot per profile's server instructions McpServerInstructionTests had ~30 Contain/NotContain checks on the composed instruction text, plus two tests that already compared the whole text with Be. The Contain tests are subsumed by the whole-text comparison, so they are removed and the two exact tests became inline snapshots with the same literals (every fact the Contain/NotContain checks asserted is visible in them). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Scrub paths the same way on every operating system The repository and temp folder scrubbers replaced only the root, so a Windows run printed {repo}\docs\index.md where the verified text says {repo}/docs/index.md, and about 160 diagnostics snapshots failed there. The path that follows the root is now written with forward slashes. The temp scrubber also swallowed everything up to the next whitespace, including the :line after a file name. A path now ends at a colon too, and the repository root is tried before the temp root because the repository can sit under the temp folder. The doc comment claimed it scrubbed git SHAs; it does not, and no longer says so. Tested with a Windows-style root, which also runs on Linux and macOS. Removing the separator replacement makes that test fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Scrub random folder names under the temp directory A test that works in a Guid-named folder in the temp directory printed that name in its diagnostics, so a snapshot of them could not be stable. The temp scrubber now replaces a run of 32 hex digits in the path with {id}. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Make diagnostics snapshots and snapshot paths the same on every operating system CI on Windows failed five of the stacked PRs for reasons a laptop never shows: - DiagnosticsSnapshot now writes paths with forward slashes and drops a leading drive letter, so `C:\docs\toc.yml`, `C:/docs/toc.yml` and `/docs/toc.yml` read the same, and `testing\req.md` inside a message reads `testing/req.md`. - ToSnapshotText(firstLineOnly: true) keeps only the first line of each message, for diagnostics that carry an exception: its stack trace has source line numbers and the exception type differs per platform. - SnapshotPathLengthTests fails when a verified snapshot's path, with the received file's longer folder and the CI checkout root, would pass Windows' 260 characters. git on the runner cannot check such a file out, so the whole job failed before any test ran. Each has a test. Lowering the length limit makes the guard fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Shorten the stream transformer test so its snapshots fit a Windows path The common-behaviour test's two verified files were 230 characters from the repository root. With the runner's checkout folder in front, and the longer received path under .artifacts when a run fails, they passed Windows' 260 characters, and git on the runner could not check the pull request out at all. The method is now NewConversationEmitsConversationStart. The files are renamed, with the same content. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Match the repository root in the form DiagnosticsSnapshot writes it DiagnosticsSnapshot drops a leading drive letter and writes forward slashes, so on Windows D:\a\docs-builder\docs-builder\docs\index.md reaches the {repo} scrubber as /a/docs-builder/docs-builder/docs/index.md, which it no longer recognised. Every diagnostics snapshot that names a file failed on the Windows runner. The scrubber now also matches the root without its drive letter. Removing that variant makes the new test fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Give the stream transformer test cases the same line endings on every platform The SSE text is an argument of NewConversationEmitsConversationStart, and the name of its snapshot files carries a hash of the arguments. A checkout on Windows turns this file's line endings into "\r\n", so the text, and with it the hash, differed there: the runner looked for `...AgentBuilderStreamTransformer.ea024bf4.verified.txt` where macOS and Linux write `.084fb69f`. The text is now normalised to "\n". Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Update the API search snapshots for the version-tree scoping #4732 scoped API navigation search to the reader's version tree. It added the `key` and `trees` fields to the API property mapping and moved the filter that excludes bare /docs roots and hidden documents to wrap the whole query, lexical and semantic. These snapshots caught both, which the old Contain checks would not have. Ten snapshots changed and nothing else in them: two mapping snapshots gain the `key` and `trees` fields (15 lines each), and eight query snapshots lose the same 23-line filter block, which is now applied around the whole query. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Update the SiteSearch mapping snapshots for the version-tree scoping The two whole-mapping snapshots (DocumentationDocument and WebsiteSearchDocument) hold the API property mapping too, so #4732's new `key` and `trees` fields show up there as well: the only change in either file. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>

* Wire Nullean.Snapshots into every test project The ApiExplorer pilot (#4654) used Nullean.Snapshots.Html and .TUnit 0.2.0 in one project. This makes the library available to all unit and integration test projects at 0.4.0 and adds the shared pieces the next migration steps build on: - tests-shared/Elastic.Documentation.Testing: DocsHtml (page section without the fixture H1 and htmx attributes), DiagnosticsSnapshot, NavigationDump, MockFileSystemDump. A scrubber for temp and repo paths is compiled into each test project. - The snap CLI as a local tool; CI uploads received snapshots when a test run fails; the build fails on orphaned snapshots after a unit run. - AGENTS.md describes the workflow. The PR review prompt no longer claims the repo uses xUnit. The five TestDiagnosticsCollector copies stay: they differ in lifecycle (Write vs HandleItem), so merging them is a behaviour change. DiagnosticsSnapshot works on any of them. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Compare rendered HTML with Nullean.Snapshots.Html ShouldBeHtml and ShouldContainHtml (Elastic.Markdown.Tests) and ConvertsToHtml, ConvertsToContainingHtml and the per-file ToHtml and ContainsHtml (Elastic.Authoring.Tests) keep their names and call sites, and now run on BeHtml and ContainHtml with the shared DocsHtml options. That removes the second copy of the AngleSharp.Diffing diff code and the AngleSharp.Diffing package. DiffPlex stays in Authoring for the plain text assertions. One test passed without testing anything. "marks exactly two options as a fork" gave the bare text `is-fork` to ConvertsToContainingHtml. The old normaliser printed only element children, so a text-only fragment became an empty string and every page contains an empty string. It now uses ConvertsToContainingRawHtml, like the test above it. Checked by corrupting one ShouldContainHtml and one ConvertsToHtml expectation: both fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * ApiExplorer tests: snapshot small rendered components Breadcrumbs, catalog cards, code blocks, the design-system nav, the TOC, the copy buttons, parameter section headers and the description pipeline now compare the whole rendered component to a snapshot (inline when a few lines, a file otherwise) instead of a cluster of Contain/NotContain fragments. These tests became stricter: every old fragment appears in the accepted snapshots, and the NotContain checks are covered by the snapshot showing the full markup. The line-number gutter keeps its own Contain, because the HTML snapshot collapses the newlines the gutter is made of. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * ApiExplorer tests: snapshot the page actions, union options and API nav Each test renders one component and now compares its whole markup to a snapshot instead of asserting fragments. These tests became stricter: the old fragments (and the ordering and count checks on aria-current, border-b and the switcher) are all visible in the accepted snapshots, and the NotContain checks are covered by the complete markup. The reasons the old assertions carried are kept as comments. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * ApiExplorer tests: snapshot the examples panel and carousel The scenario panel, scenario content and response caption tests compare the rendered markup to snapshots, scoped with Remove/Select so the card bodies do not repeat in every file. These tests became stricter: each old Contain fragment is visible in the accepted snapshots. The positional checks (IndexOf ordering) are now the order of the snapshot. Three tests that rendered the same two-scenario panel and asserted different fragments (chips, more menu, heading) are one test with one snapshot, since their snapshots were identical. Structural counts on the generated-media test and the description toggle regex stay, as cheap precise assertions. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * ApiExplorer tests: snapshot badges, security badges and property rows Document security, state and deprecated badges, overview row lifecycle badges, the property reference link and the schema type constraint compare rendered markup to snapshots, scoped to the badge strip or the authentication section with Select. These tests became stricter: every old fragment is in the accepted snapshots (the section checks now see the whole authentication panel, so the Bearer, Basic and Api key panels are pinned too). The where-it-sits ordering checks that need the whole page stay. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * ApiExplorer tests: snapshot responses block and parameter rows The remaining responses block tests and the parameter row render tests compare the rendered markup (scoped to the parameter list with Select where the test is about one section) to snapshots. These tests became stricter: all old fragments appear in the accepted snapshots. The button/anchor position checks are now the nesting in the snapshot. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * ApiExplorer tests: snapshot the generated CommonMark pages The landing, operation and catalog markdown the generator writes, and the wrapped authored markdown page, are file snapshots instead of about forty Contain/NotContain checks. These tests became stricter: every old fragment is in the accepted files, and the NotContain checks (no applies_to, no DOCTYPE, no merged status line) are covered by the whole text. The file snapshots also pin the section order of the operation page, which the fragments did not. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * ApiExplorer tests: snapshot the generated catalog page The catalog page tests (combined entries, groups, declared categories, the title band) compare the page's main container, minus its inline SVG icons, to a snapshot instead of IndexOf/Split counting. These tests became stricter: all old fragments are in the accepted snapshots, and the ordering and tag-count checks are the order and content of the snapshot. The main-container class and a few absence checks of whole-page furniture stay as targeted assertions. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * ApiExplorer tests: inline JSON snapshots for the generated example bodies The request and response example tests that compared a body's JSON text to a one-line literal use MatchInlineJsonSnapshot with the same literals. The literals pass unchanged, which shows the conversion keeps the expectation. Key order and number spelling stay significant (MapOrder.Significant and StrictNumbers), so the strictness is the same as the string comparison it replaces. The [Arguments] table keeps its string comparison, since an inline snapshot cannot take a per-row literal. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * ApiExplorer tests: inline snapshots for front matter, sanitizer and structural pages The front matter writer, the HTML sanitizer and the StripHtml helper, and the authentication and servers pages, compare their whole output to an inline snapshot instead of Contain/NotContain fragments. These tests became stricter: the sanitizer tests now pin the exact output, so a tag or attribute that the old NotContain list did not name can no longer slip through. Every old fragment appears in the accepted literals. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * ApiExplorer tests: snapshot array bodies, content type pickers and tag pages The array response render, the array request summary, the request content type picker (heading and body) and the tag and product landing pages compare rendered markup to snapshots. These tests became stricter: the old fragments are in the accepted snapshots, and the vague checks on the tag pages (Contain 'api-overview', Contain 'Spaces') now pin the whole page content, including the description wrapper and the grouped endpoint list. The array limits and CommonMark checks stay as targeted Contains. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * ApiExplorer tests: inline snapshots for the synthetic curl samples Sixteen tests that asserted a few fragments of the generated curl command now pin the whole command with an inline snapshot. These tests became stricter: each old Contain fragment is in the accepted literal and each old NotContain fragment is absent (checked with a script against the accepted literals), and a stray or reordered header or flag now fails the test. The [Arguments] rows keep their fragment checks. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Scrub paths the same way on every operating system The repository and temp folder scrubbers replaced only the root, so a Windows run printed {repo}\docs\index.md where the verified text says {repo}/docs/index.md, and about 160 diagnostics snapshots failed there. The path that follows the root is now written with forward slashes. The temp scrubber also swallowed everything up to the next whitespace, including the :line after a file name. A path now ends at a colon too, and the repository root is tried before the temp root because the repository can sit under the temp folder. The doc comment claimed it scrubbed git SHAs; it does not, and no longer says so. Tested with a Windows-style root, which also runs on Linux and macOS. Removing the separator replacement makes that test fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Scrub random folder names under the temp directory A test that works in a Guid-named folder in the temp directory printed that name in its diagnostics, so a snapshot of them could not be stable. The temp scrubber now replaces a run of 32 hex digits in the path with {id}. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Make diagnostics snapshots and snapshot paths the same on every operating system CI on Windows failed five of the stacked PRs for reasons a laptop never shows: - DiagnosticsSnapshot now writes paths with forward slashes and drops a leading drive letter, so `C:\docs\toc.yml`, `C:/docs/toc.yml` and `/docs/toc.yml` read the same, and `testing\req.md` inside a message reads `testing/req.md`. - ToSnapshotText(firstLineOnly: true) keeps only the first line of each message, for diagnostics that carry an exception: its stack trace has source line numbers and the exception type differs per platform. - SnapshotPathLengthTests fails when a verified snapshot's path, with the received file's longer folder and the CI checkout root, would pass Windows' 260 characters. git on the runner cannot check such a file out, so the whole job failed before any test ran. Each has a test. Lowering the length limit makes the guard fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Match the repository root in the form DiagnosticsSnapshot writes it DiagnosticsSnapshot drops a leading drive letter and writes forward slashes, so on Windows D:\a\docs-builder\docs-builder\docs\index.md reaches the {repo} scrubber as /a/docs-builder/docs-builder/docs/index.md, which it no longer recognised. Every diagnostics snapshot that names a file failed on the Windows runner. The scrubber now also matches the root without its drive letter. Removing that variant makes the new test fail. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>

akira28 added the chore label Nov 2, 2026

akira28 requested a review from a team as a code owner November 2, 2026 16:32

akira28 requested a review from theletterf November 2, 2026 16:32

infra-vault-gh-plugin-prod Bot approved these changes Nov 2, 2026

View reviewed changes

Comment thread ...ionHeadingLevelTests.Render_SearchOperation_UsesOneH1ThenH2SectionsAndH3Nested.verified.html Resolved

Comment thread tests/Elastic.ApiExplorer.Tests/VerifyInitializer.cs Resolved

akira28 mentioned this pull request Nov 2, 2026

Bump Vitest to 5.1.0 and TUnit to 1.78.0 #4656

Merged

infra-vault-gh-plugin-prod Bot approved these changes Nov 2, 2026

View reviewed changes

akira28 force-pushed the test/verify-pilot branch from d5ddcab to c5fbda1 Compare November 2, 2026 20:11

akira28 changed the base branch from main to chore/bump-vitest November 2, 2026 20:11

Base automatically changed from chore/bump-vitest to main November 3, 2026 07:53

akira28 and others added 2 commits November 3, 2026 10:00

akira28 force-pushed the test/verify-pilot branch from c5fbda1 to cbd2a69 Compare November 3, 2026 09:01