Skip to content

Warn when links to the API docs point at missing pages - #4653

Open
akira28 wants to merge 13 commits into
mainfrom
feature/request-link-checking-related
Open

akira28 wants to merge 13 commits into
mainfrom
feature/request-link-checking-related

Conversation

@akira28

@akira28 akira28 commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

The assembler now checks links from narrative docs and redirects.yml into /docs/api/doc/... against the API pages it generated, and warns about each link that does not resolve.

Affects: Assembler builds, API reference, Authoring

Prompt summary: Implement link checking for the API docs from elastic/docs-eng-team#997. Links into the auto-generated API reference can disappear without notice, for example when an endpoint is discontinued, and nothing validates them today. Report broken links as warnings plus a report file, and check page existence only.

Why

https://www.elastic.co/docs/api/doc/... links pass through the Markdown build unchecked, and redirect targets with :// are only parsed as URIs. API pages come from external specs, so an endpoint can vanish and the links to it break with no signal.

What

Recording API links

Links are recorded where they are parsed: the inline link parser, the autolink parser and DirectiveLinkValidator, which hero actions, card targets and card intros all pass through. Every page, including API children: pages and snippets, reaches these parsers, so each link is recorded with its file, line and column. Links inside API descriptions, which parse without link validation, are recorded in HtmlWriter.RenderApiDescription through OutboundLinkExtractor and reported against the page's description. Redirect targets in redirects.yml are recorded separately and reported against that file. ApiLinkMatcher normalizes each URL by dropping the query and fragment, collapsing repeated slashes, trimming the trailing slash and lowercasing it.

Generated page set

OpenApiGenerator.GeneratedUrls collects every page the generator renders, across all version trees, plus the redirect stubs for collapsed operations and the .json and .yaml spec downloads. URLs are recorded under /docs whatever path prefix the build uses, so PR previews compare like with like. AssemblerOpenApiBuildStep returns the union of these sets with the catalog entries.

Validation and report

After the API step, ApiLinkValidator compares recorded links with the generated set. It follows the old bump.sh redirects the edge serves (ApiLegacyRedirects, mirrored from docs-infra and shared with ApiSitemapParityTests) before comparing, reports each URL once per file, warns per broken link and writes api-link-report.json next to redirects.json. The path is also a GitHub Actions output, api-link-report-path. The step is skipped when the API Explorer produced no pages, so builds with the flag off are unchanged.

Out of scope: #anchor checking, links from product source code, and probing production URLs.

Verify

dotnet test tests/Elastic.Documentation.Tests
dotnet test tests/Elastic.Markdown.Tests --treenode-filter "/*/*/*ApiLink*/*"
dotnet test tests/Elastic.Documentation.Build.Tests --treenode-filter "/*/*/ApiLinkValidatorTests/*"
dotnet test tests/Elastic.ApiExplorer.Tests --treenode-filter "/*/*/OpenApiGeneratorCatalogSplitTests/GenerateProducts_Records*"

Closes elastic/docs-eng-team#997

🤖 Generated with Claude Code

akira28 and others added 2 commits November 2, 2026 17:26
Links from narrative docs and redirects into /docs/api/doc/ were never
checked, so a removed endpoint broke them silently. The assembler now
records those links, compares them with the pages the API Explorer
generated, warns per broken link, and writes api-link-report.json.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The redirect check recorded API targets against docset.yml, so a warning
pointed at the wrong file. ConfigurationFile now exposes the redirects
file and a test covers the recorded location.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…a BOM

An assembler run found 52 links written as .../elasticsearch//operation/...
that the validator reported as broken. Normalizing repeated slashes treats
them like the single-slash form. The report also began with a UTF-8 byte
order mark, which strict JSON parsers reject.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Docs preview (local build)

Handbook preview: https://docs-v3-preview.elastic.dev/elastic/docs-builder/pull/4653/

Sample changed pages:

@akira28
akira28 marked this pull request as ready for review November 3, 2026 09:54
@akira28
akira28 requested a review from a team as a code owner November 3, 2026 09:54
@akira28
akira28 requested a review from technige November 3, 2026 09:54
akira28 and others added 2 commits November 3, 2026 11:43
The three parse-time hooks missed autolinks and recorded link cards twice.
An ApiLinkExporter now reuses the document each page already parsed and
the OutboundLinkExtractor from the search indexer, which now also returns
link positions. Redirect targets are still recorded separately.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…prefix

The exporter route missed API children pages and the hero, card-group and
what's-new fields. Links are recorded again at the inline, autolink and
directive parse points, which every page and directive passes through.
A PR preview serves pages under another path prefix, so the generated
URLs are now recorded under /docs. Each URL is reported once per file.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
About 70 reported links used the old serverless and elastic-cloud-serverless
roots, which the edge redirects. The validator now follows the same redirect
table and checks the target page, so a link that works today is not flagged
but one whose target is gone still is. The table moves out of
ApiSitemapParityTests, which now uses it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Markdig parses <https://...> links itself, so they never reached the API
link hooks. A recorder in front of its autolink parser now sees them
without consuming input. The bare-URL and angle-bracket paths skip
% comment lines, as the inline link parser already does.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Markdig builds an AutolinkInline for <https://...>, not a LinkInline, so
the generic test base failed its own non-null check.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
akira28 and others added 2 commits November 5, 2026 10:15
API descriptions parse without link validation, so the links between API
pages that come from the OpenAPI specs were never checked. HtmlWriter now
records them, using the outbound link extractor, and spec download files
count as generated pages. A run on the staging checkouts finds 10 broken
description links across 5 targets.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…lated' into feature/request-link-checking-related
Comment on lines +80 to +89
// Descriptions parse without link validation, so the links they hold between API pages are recorded here for the API link check.
private void RecordApiLinks(MarkdownDocument parsed, IFileInfo source)
{
var context = DocumentationSet.Context;
foreach (var url in OutboundLinkExtractor.Extract(parsed) ?? [])
{
if (ApiLinkMatcher.TryCanonicalize(url, context.UrlPathPrefix, out var canonical))
context.Collector.EmitApiLink(new ApiLinkReference(canonical, source.FullName, 1, 1, canonical.Length));
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: P2 (Medium)

RenderApiDescription runs on every page that shows a shared description, such as a schema, parameter or response description. Each call passes a different virtual description.md path (one per page URL), so the (Url, File) dedupe in ApiLinkValidator keeps every copy. One broken link in a shared spec description then produces a warning on every operation page that renders it, which can be hundreds of near-identical warnings, and it inflates checked in the report. Each warning also points at line 1 of a file in the output directory that doesn't exist. Dedupe description links by URL plus spec or product root, or record them once per description text, rather than once per rendering page.

Generated by Libra. Specialist: libra-complexity. React with 👍 or 👎 to give feedback on this comment.

Comment on lines +84 to +87
foreach (var url in OutboundLinkExtractor.Extract(parsed) ?? [])
{
if (ApiLinkMatcher.TryCanonicalize(url, context.UrlPathPrefix, out var canonical))
context.Collector.EmitApiLink(new ApiLinkReference(canonical, source.FullName, 1, 1, canonical.Length));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: P2 (Medium)

Include raw HTML anchor targets when recording description links. ApiDescriptionPipeline supports <a href="..."> links, but OutboundLinkExtractor only collects Markdown link nodes and directive targets, so an HTML link to a missing API page never reaches the validator and produces neither a warning nor a report entry.

Generated by Libra. React with 👍 or 👎 to give feedback on this comment.

This branch was successfully deployed

2 active deployments
docs-preview — b115859c Deployed Nov 5, 2026 by github-actions[bot]
assembler-preview — b115859c Deployed Nov 5, 2026 by github-actions[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants