Skip to content

Resolving a large schema is quadratic in the number of items - #1284

Merged
theoephraim merged 1 commit into
dmno-dev:mainfrom
voiys:fix-large-schema-resolve
Nov 13, 2026
Merged

theoephraim merged 1 commit into
dmno-dev:mainfrom
voiys:fix-large-schema-resolve

Conversation

@voiys

@voiys voiys commented Nov 9, 2026

Copy link
Copy Markdown
Contributor
# generated schema: ports, urls, and concat() refs to them
PORT_0=3000
URL_1=https://svc-1.example.com
LABEL_2=concat("svc-", $PORT_0)
...

load + resolve, median of 15 runs:

items    main       here
1000     36 ms      19 ms
3000     245 ms     50 ms
6000     1051 ms    119 ms

cause:

markItemCompleted(key):
  delete pending[key]
  if (Object.keys(pending).length === 0) resolve()     // counts every pending key, once per item

fix: keep a counter.

every item's resolved value and error count is the same before and after on the 3,000-item schema. no new test: behaviour is unchanged and a timing test would be flaky.

context: found while timing a language server prototype (#469) on generated schemas. thousands of items is bigger than most real schemas, so this is a stretch too, but the fix is small.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes: Reviewed the completion-counter optimization and release note, including dependency scheduling, error handling, and partial resolution; all 127 focused tests passed.

  • Completion bookkeeping: Counts pending items once and decrements on completion instead of enumerating pending keys after every item.
  • Release note: Adds a varlock patch entry describing the large-schema performance improvement.

Pullfrog  | View workflow run | Using gpt-6.1-sol | 𝕏

@theoephraim

Copy link
Copy Markdown
Member

Nice find. Thanks!

resolveEnvValues counted the keys of the pending-items object each time an
item finished, which is O(n^2) in the number of items. It now keeps a
counter. 3,000-item schema, load + resolve: about 245 ms to 50 ms.
@theoephraim
theoephraim force-pushed the fix-large-schema-resolve branch from 59e0f5c to 3185771 Compare November 13, 2026 00:47
@pkg-pr-new

pkg-pr-new Bot commented Nov 13, 2026

Copy link
Copy Markdown

Open in StackBlitz

varlock

npm i https://pkg.pr.new/varlock@1284

@varlock/astro-integration

npm i https://pkg.pr.new/@varlock/astro-integration@1284

@varlock/cloudflare-integration

npm i https://pkg.pr.new/@varlock/cloudflare-integration@1284

@varlock/nextjs-integration

npm i https://pkg.pr.new/@varlock/nextjs-integration@1284

@varlock/nuxt-integration

npm i https://pkg.pr.new/@varlock/nuxt-integration@1284

@varlock/vite-integration

npm i https://pkg.pr.new/@varlock/vite-integration@1284

@varlock/1password-plugin

npm i https://pkg.pr.new/@varlock/1password-plugin@1284

@varlock/aws-secrets-plugin

npm i https://pkg.pr.new/@varlock/aws-secrets-plugin@1284

@varlock/azure-key-vault-plugin

npm i https://pkg.pr.new/@varlock/azure-key-vault-plugin@1284

@varlock/bitwarden-plugin

npm i https://pkg.pr.new/@varlock/bitwarden-plugin@1284

@varlock/keepass-plugin

npm i https://pkg.pr.new/@varlock/keepass-plugin@1284

@varlock/pass-plugin

npm i https://pkg.pr.new/@varlock/pass-plugin@1284

@varlock/proton-pass-plugin

npm i https://pkg.pr.new/@varlock/proton-pass-plugin@1284

commit: 3185771

@theoephraim
theoephraim merged commit c577346 into dmno-dev:main Nov 13, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants