Skip to content

Share one web host across the page feedback endpoint tests - #4694

Merged
Mpdreamz merged 1 commit into
mainfrom
perf/api-tests-shared-host
Nov 4, 2026
Merged

Mpdreamz merged 1 commit into
mainfrom
perf/api-tests-shared-host

Conversation

@Mpdreamz

@Mpdreamz Mpdreamz commented Nov 4, 2026

Copy link
Copy Markdown
Member

The page feedback endpoint tests start one web host for the whole class instead of one per test. Elastic.Documentation.Api.Tests uses about a quarter of the CPU it did.

Affects: Internals only

Prompt summary: Find out which individual tests in the slow test projects do far more work than they need, and share setup between them. These ten tests were nearly all of this project's test time.

Why

Each of the 10 PageFeedbackEndpointTests tests (7 methods, one with 4 [Arguments] rows) built its own ApiWebApplicationFactory: a full ASP.NET host, about 0.75 s each. Run one at a time they were 8.5 of the 9.0 seconds the project's 134 tests take. The only difference between them is the IPageFeedbackService fake each one installs.

What

One host per class, a swappable service

New SwappableService<TService> holds the implementation for the current test. The class builds one ApiWebApplicationFactory lazily, registers IPageFeedbackService through Replace<IPageFeedbackService>(_ => FeedbackService.Current), and each test installs its own fake before sending its request. The registration is scoped, so every request resolves whichever fake is current. The host is disposed in an [After(Class)] hook.

Serialised within the class

The tests share the current fake, so the class is [NotInParallel(nameof(PageFeedbackEndpointTests))]. Other test classes still run in parallel with it.

Out of scope: EuidEnrichmentTests also starts a host, but it has a single test and reads that host's own exported telemetry, so sharing would not save anything and would mix telemetry between tests.

Results

Elastic.Documentation.Api.Tests, release build, 4 simulated cores, 3 runs each:

Before After
Wall time in the test process 2.9–4.3 s 1.3–1.4 s
CPU (user + sys) 5.0–5.2 s 1.2–1.4 s

The CI effect is not measured yet. On Windows CI this project took 45 s with 4 projects running at once.

Verify

dotnet test tests/Elastic.Documentation.Api.Tests

All 134 tests pass. Because the tests now share state, I ran the suite 10 times at each of 1, 4 and 18 cores: no failures. Two mutations fail tests: not swapping the fake (24 failures over 6 runs), and removing [NotInParallel] (22 failures over 6 runs), so the serialisation is required.

🤖 Generated with Claude Code

Each of the 10 page feedback endpoint tests started its own `WebApplicationFactory`, about 0.75 s per host, which was 8.5 of the 9 seconds the project's tests take when run one at a time. The only thing that differed between them was the `IPageFeedbackService` fake each one installed.

The class now builds one host whose `IPageFeedbackService` registration resolves a `SwappableService`, and every test installs its fake before it sends its request. The tests share the current fake, so the class is marked `[NotInParallel]`; other classes still run in parallel with it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@Mpdreamz
Mpdreamz requested a review from a team as a code owner November 4, 2026 18:28
@Mpdreamz Mpdreamz added the chore label Nov 4, 2026
@Mpdreamz
Mpdreamz requested a review from theletterf November 4, 2026 18:28
@Mpdreamz Mpdreamz added the chore label Nov 4, 2026
@Mpdreamz
Mpdreamz enabled auto-merge (squash) November 4, 2026 18:29
@Mpdreamz
Mpdreamz disabled auto-merge November 4, 2026 18:29
@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

Docs preview (local build)

Handbook preview: https://docs-v3-preview.elastic.dev/elastic/docs-builder/pull/4694/

@Mpdreamz
Mpdreamz merged commit 70bcfb5 into main Nov 4, 2026
48 checks passed
@Mpdreamz

Mpdreamz commented Nov 4, 2026

Copy link
Copy Markdown
Member Author

Merged by heed (squash) because every merge rule held:

  • Green: every green rule holds
  • Reviewed after push: approved by @reakaleek on edc4e71
  • Quiet: nothing new for 15m
  • Gate checks: Analyze (actions), Analyze (csharp), Analyze (javascript-typescript), CodeQL, Libra passed
  • No hold label: none

@Mpdreamz
Mpdreamz deleted the perf/api-tests-shared-host branch November 4, 2026 18:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants