Repository navigation
perf(api): speed up API tests by lazy IM imports and smaller contract docs - #45381
Merged
Merged
Conversation
create_app() reaches the adapter factory, which imported every provider adapter and its vendor SDK at module load. lark_oapi alone takes ~3.5s, so every app start (each xdist worker and subprocess test) paid ~3.7s.
Contributor
Pyrefly Type Coverage
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #45381 +/- ##
===========================================
+ Coverage 68.81% 89.80% +20.98%
===========================================
Files 6103 6239 +136
Lines 368940 390949 +22009
Branches 77825 79291 +1466
===========================================
+ Hits 253897 351075 +97178
+ Misses 112233 33774 -78459
- Partials 2810 6100 +3290
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
The controller error contract check builds an OpenAPI document for each ExternalApi that serves an error. Test APIs borrowed models by copying all ~2200 surface models into api.models, so a one-path document carried every schema and took ~90ms; with ~1200 test APIs per run that was ~18% of the controller suite. Borrow through a fallback mapping that resolves names on lookup but iterates only the API's own models, so a document carries just the borrowed models its operations reference. Documents for all 1200 test APIs keep identical paths and referenced components.
hyoban
marked this pull request as ready for review
November 3, 2026 04:20
lyzno1
approved these changes
Nov 3, 2026
2 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
API unit and controller test jobs have gotten much slower since early October (merge-queue medians: unit shard ~4m → ~8m, controllers ~3m → ~10m50). This PR fixes the two largest causes that aren't just "more tests".
1. IM provider SDKs imported by
create_app()(from #44864)create_app()reachescore/human_input_v2/im_integration/adapters/factory.py, which imported every IM provider adapter and its vendor SDK at module load.lark_oapialone costs ~3.5s, so each xdist worker and every subprocess-based test paid it, and existing tests that happened to run first in a worker jumped from ~0.05s to 3–6s.Each adapter import now lives in its
matchbranch, matching the pattern already used by_card_decoderinservices/human_input_v2/im_webhook_runtime.py.create_app()no longer loadslark_oapi,slack_sdk,botbuilder,botframework,alibabacloud_dingtalk, orwechatpy.python -c "from app_factory import create_app; create_app()": ~11.6s → ~7.9s.2. Error contract check copies every surface model into each test API (from #44001)
The autouse
error_response_contractfixture builds an OpenAPI document for eachExternalApithat serves an error response. Test-built APIs borrowed models by copying all ~2200 surface models intoapi.models, so a one-path document carried every schema and took ~90ms infinalize_openapi_payload. About 1200 test APIs per run made that ~107s of CPU, about 18% of the controller suite locally.Models are now borrowed through a fallback mapping. Name lookups (the only way Swagger resolves models) fall back to the surfaces, but iteration (the way document finalization merges models) yields only the API's own models. The swap is scoped to the build, so
api.modelsitself is no longer mutated.paths, every$refresolves, and every referenced component is identical to the old document.-n 8, two runs each): 74s / 98s → 62s / 79s.test_documents_only_the_surface_models_a_test_api_references; it fails on the old helper.Checklist
make lint && make type-check(backend) andvp staged(frontend) to appease the lint godsFrom Claude Code