Skip to content

Index each API operation's words, group and id word count for search - #4698

Merged
reakaleek merged 1 commit into
mainfrom
api-search/index-words
Nov 4, 2026
Merged

reakaleek merged 1 commit into
mainfrom
api-search/index-words

Conversation

@reakaleek

@reakaleek reakaleek commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

Every indexed API operation now carries its own api.words, api.group and api.id_words fields. Nothing changes for readers yet; the API search query that reads them follows in the next pull request.

Affects: Search, API reference

Prompt summary: The API search box ranks search retriever perfectly but returns bogus results for cluster flat_settings and nothing useful for es|ql drop_null_columns, although both are real query parameters. Asked why, and then asked to design the fix properly instead of patching the rule, the work became a per-token query model delivered as a stack of three changes. This is the bottom layer, the index fields that model reads.

Why

The API search reads an operation's name words only inside search_title, where they sit next to the product label, the HTTP method and the group. A query cannot weight a word of the operation id (cluster, esql) apart from a word of its title or its group, and it has no way to prefer esql-query over esql-async-query-stop when both carry the typed field. The new query model needs each of those signals as a field of its own.

What

Words and group as fields of their own

ApiOperationMetadata gains Words, the plain words of the operation id and its literal path segments that ApiSearchWords.Of already computes for search_title, and Group, the title of the operation's group as the explorer shows it. Both are text fields with the synonym analyzers, and api.words also has a completion search-as-you-type multi-field for prefix matching. #4666 kept the group out of api.* because its query never read it; the test that asserted the absence now asserts the field.

A specificity prior

IdWords counts the words of the operation id alone and is mapped as a rank_feature with negative score impact, like navigation.depth. Fewer id words is a more specific operation, so a query can break ties toward it.

The exporter fills them

OpenApiDocumentExporter sets the three fields next to the existing operation metadata. The document hash already covers the whole api record, so an incremental index run re-syncs every operation.

Verify

dotnet test tests/Elastic.ApiExplorer.Tests/
# OfOperationId_OperationId_ReturnsIdWordsOnly, ConvertToDocuments_TaggedOperations_IndexGroupAndWordsInSearchTitle
dotnet test tests/Elastic.Documentation.Api.Tests/
# Mapping_ApiWordsAndGroup_AreSynonymText, Mapping_ApiIdWords_IsRankFeatureWithNegativeImpact

Out of scope: the query that reads the new fields, and the removal of the two-reading parse it replaces. Both follow in this stack.

Risk: the api mapping changes, so the first index run after deploy rolls the indices over to new dated indices, about six minutes on dev, as the last mapping change did.

Stack: 1 of 3. #4699 (the per-word query) and #4700 (the removal of the legacy readings) sit on top of this branch.

🤖 Generated with Claude Code

The API search is moving to a per-token model where a typed word may name
an operation by its endpoint, by a word of its id or path, by its group, or
by a field. Those words and the group existed only concatenated into
`search_title`, so they could not be weighted apart from the title and the
method. `api.words` and `api.group` give each its own field, and
`api.id_words` is a rank feature that breaks ties toward the more specific
operation (`esql-query` over `esql-async-query-stop`). The query that reads
them follows in the next change.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@reakaleek
reakaleek requested a review from a team as a code owner November 4, 2026 21:33
@reakaleek reakaleek added the chore label Nov 4, 2026
@reakaleek
reakaleek requested a review from Mpdreamz November 4, 2026 21:33
@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

Docs preview (local build)

Handbook preview: https://docs-v3-preview.elastic.dev/elastic/docs-builder/pull/4698/

@reakaleek
reakaleek added this pull request to stack #4701 November 4, 2026 21:59
@reakaleek
reakaleek merged commit 6aad302 into main Nov 4, 2026
41 of 42 checks passed
@reakaleek
reakaleek deleted the api-search/index-words branch November 4, 2026 22:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant