diff --git a/scripts/perf/README.md b/scripts/perf/README.md index c6c719bd644..f0b0109cdf8 100644 --- a/scripts/perf/README.md +++ b/scripts/perf/README.md @@ -9,88 +9,40 @@ http://www.apache.org/licenses/LICENSE-2.0 - Unless required by applicable law or agreed to in writing, - software distributed under the License is distributed on an - "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - KIND, either express or implied. See the License for the - specific language governing permissions and limitations - under the License. + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --> -# Renderer performance probes +# Runtime SQLite VACUUM benchmark -For the two manual CI lanes, raw reports, fixtures and their coverage limits, see [CI.md](./CI.md). Those lanes verify DOM events and consumer state; they do not use the legacy native-input helpers below. +Run `node scripts/perf/runtime-vacuum-benchmark.mjs 100 1024 3072`. It uses Node's built-in SQLite with WAL, creates incompressible 32 KiB rows, deletes one third to leave free pages, converts with `PRAGMA auto_vacuum=INCREMENTAL; VACUUM`, and includes connection close/checkpoint time. It then verifies `auto_vacuum = 2`, an empty freelist, and `integrity_check = 'ok'`. -Measuring what a session switch costs in the running Desktop app, over CDP. The -findings in #4109 were produced with these; they live here so the next -measurement is a command rather than a rebuild. +This is a synthetic conversion-cost fixture, not a production database workload. It does not yet measure peak WAL size or platform-specific free-space requirements. Preserve the emitted JSON, including OS, Node and SQLite versions, with results. -They are ad-hoc tools, not a benchmark suite: they attach to a dev app you -started, and they answer "how much" only for the interaction they drive. +## Current Windows run -## Running +Windows x64, Node v24.15.0, SQLite 3.51.3; rerun 2026-10-10: -Start the dev app with the debugger open: +| Fixture size | Source bytes before deletion | Conversion and close | +| ---: | ---: | ---: | +| 100 MiB | 106,504,192 | 1.384 s | +| 1,024 MiB | 1,090,572,288 | 16.397 s | +| 3,072 MiB | 3,271,692,288 | 64.150 s | -```bash -npm run dev -w @maka/desktop -- --remote-debugging-port=9334 -``` +All three post-close checks passed (`auto_vacuum=2`, `freelist_count=0`, `integrity_check=ok`). Earlier pre-close WAL timings were 0.58 s, 10.73 s and 140.36 s; those are superseded. The 36.18 s 3 GiB figure was from the excluded DELETE-mode run and is not a WAL result. -Then, with a populated sidebar (about 30 rows is what the #4109 numbers used): +Only Windows has been measured so far. The requested second-platform run is outstanding; a prior one-off benchmark workflow was removed from this repository. The corrected 1-to-3 GiB increase is still super-linear (16.397 s to 64.150 s); WAL/journal I/O, SQLite page-cache behavior, storage and antivirus scanning are plausible contributors, but this fixture does not isolate their individual contributions. -```bash -node scripts/perf/session-switch-commits.mjs before -node scripts/perf/session-switch-busy-js.mjs -``` +## Placement notes for issue #6000 -- `session-switch-commits.mjs` — React commits per switch, how many are - full-tree renders, and how many fibers actually re-rendered. Reloads the page - first, so `react-commit-probe.js` is in place before React boots. -- `session-switch-busy-js.mjs` — renderer busy JS per switch, with the top - self-time entries. It reports the running build, one configuration. A - before/after needs both configurations in one instance; see below. -- `react-commit-probe.js` — the in-page half: a minimal - `__REACT_DEVTOOLS_GLOBAL_HOOK__` plus a `Function.prototype.bind` wrapper that - catches React creating a `dispatchSetState`, so a commit can be attributed to - the `setState` that caused it. -- `cdp-client.mjs` — the protocol client and the row-clicking helper. -- `xterm-hidden-selection.mjs ` — functional regression for the patched - xterm CJS/ESM bundles in real Chromium: hidden selection updates must not - redraw rows, and showing the terminal must paint the latest selection without - another write. Use a disposable Electron fixture; the probe briefly embeds - its own terminal frame and bypasses CSP, then removes the frame and restores - CSP. This is a behavior check, not a performance score. +- `runtime.sqlite` has one process-local owner connection; repositories borrow leases. A worker's second connection departs from that ownership model, and a Host-only gate cannot fence secondary processes using `require_current`. +- A worker thread keeps the event loop responsive and reads available where SQLite permits. It does not remove `VACUUM`'s write lock: concurrent writers still wait or hit the configured 5 s `SQLITE_BUSY` timeout. +- In WAL mode, reserve close to 3x the database size plus operational headroom until measured peak usage supports a tighter bound. The prior 2x estimate omitted WAL growth. +- Under WAL with `synchronous=FULL`, a crash before the VACUUM transaction commits discards the uncommitted conversion; the original database remains. Reopen and verify integrity. +- `VACUUM INTO` can build a staging copy while the live database stays available. Swap it on the next startup before listen; do not plan a live swap on Windows while handles are open. Maka already uses a private copy for context offload (`context-offload-snapshot.ts`). +- The smallest first implementation is to create new `runtime.sqlite` databases with `auto_vacuum=INCREMENTAL` before schema creation and WAL activation. No full VACUUM is required for new databases. Converting existing databases is a later, separate decision. -## The one rule - -**Never compare numbers from two app launches.** Restarting the app shifts these -metrics by orders of magnitude, while the spread inside a single running -instance is small. An A/B means: one instance, a `globalThis` switch to select -the configuration, at least three repetitions per configuration, alternating, -compared pairwise. Two runs of "before" and "after" against two launches will -produce a confident number that means nothing. - -The switch is a throwaway, not a shipped affordance. Branch a worktree, put a -`globalThis` flag in the one place that reads it — one that restores the old -behaviour when set, so the regressed configuration is reachable without -rebuilding the defect by hand — alternate the flag inside the running instance, -then delete the worktree. Measurement discipline is ours; product code carries -none of it. A flag that reaches `main` is a branch nothing executes and a -defect replica that has to be maintained against the code it replicates. - -Two smaller ones that cost time to rediscover: - -- Clicks must be `Input.dispatchMouseEvent`. A synthesised `element.click()` - does not activate an Astryx `SideNavItem`, so the app does nothing and the - measurement describes it. -- Disarm the commit probe (`__MAKA_PROBE__.arm = false`) before sampling. Its - fiber walk otherwise shows up in the profile as the app's own work; - `session-switch-busy-js.mjs` does this for you. - -## What these do not tell you - -The commit count is unweighted, and busy JS on this workload is spread across -about a thousand cheap fibers rather than concentrated in one component. A -change that lowers the commit count has not necessarily lowered the time, and a -change that moves the top self-time entry has not necessarily lowered anything. -Read both, and read the medians. +No placement recommendation is made until the required second-platform measurement and owner-process fencing design are settled. The full analysis and option-by-option effects belong in the #6000 issue discussion; this README only preserves the fixture and its caveats. diff --git a/scripts/perf/runtime-vacuum-benchmark.mjs b/scripts/perf/runtime-vacuum-benchmark.mjs new file mode 100644 index 00000000000..83f7547841c --- /dev/null +++ b/scripts/perf/runtime-vacuum-benchmark.mjs @@ -0,0 +1,93 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import { DatabaseSync } from 'node:sqlite'; +import { cpSync, mkdtempSync, rmSync, statSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { performance } from 'node:perf_hooks'; + +const MiB = 1024 * 1024; +const sizes = (process.argv.slice(2).length ? process.argv.slice(2) : ['100', '1024', '3072']).map( + Number, +); +const root = mkdtempSync(join(tmpdir(), 'maka-vacuum-bench-')); + +try { + for (const targetMiB of sizes) { + const source = join(root, `runtime-${targetMiB}.sqlite`); + const db = new DatabaseSync(source); + db.exec('PRAGMA page_size=4096; PRAGMA auto_vacuum=NONE; PRAGMA journal_mode=WAL;'); + db.exec('CREATE TABLE runtime_events (event_id INTEGER PRIMARY KEY, event_json BLOB NOT NULL)'); + const insert = db.prepare('INSERT INTO runtime_events(event_json) VALUES (?)'); + const payload = Buffer.alloc(32 * 1024); + for (let i = 0; i < payload.length; i += 1) payload[i] = (i * 37 + 19) & 0xff; + const started = performance.now(); + db.exec('BEGIN'); + let inserted = 0; + while (inserted * payload.length < targetMiB * MiB) { + insert.run(payload); + inserted += 1; + } + db.exec('COMMIT'); + const populatedMs = performance.now() - started; + const before = statSync(source).size; + + // Leave realistic free pages for the one-time conversion to incremental vacuum. + db.exec('DELETE FROM runtime_events WHERE event_id % 3 = 0'); + db.close(); + const copy = join(root, `runtime-${targetMiB}-conversion.sqlite`); + cpSync(source, copy); + const conversion = new DatabaseSync(copy); + conversion.exec('PRAGMA busy_timeout=5000; PRAGMA auto_vacuum=INCREMENTAL;'); + + const vacuumStart = performance.now(); + conversion.exec('VACUUM'); + conversion.close(); + // Include close(), which checkpoints the WAL and is part of completion. + const conversionMs = performance.now() - vacuumStart; + const verified = new DatabaseSync(copy); + const autoVacuum = verified.prepare('PRAGMA auto_vacuum').get().auto_vacuum; + const freelistCount = verified.prepare('PRAGMA freelist_count').get().freelist_count; + const integrity = verified.prepare('PRAGMA integrity_check').get().integrity_check; + verified.close(); + if (autoVacuum !== 2 || freelistCount !== 0 || integrity !== 'ok') { + throw new Error( + `post-conversion verification failed: auto_vacuum=${autoVacuum}, freelist=${freelistCount}, integrity=${integrity}`, + ); + } + console.log( + JSON.stringify({ + platform: `${process.platform}-${process.arch}`, + node: process.version, + sqlite: process.versions.sqlite, + targetMiB, + sourceBytesBeforeDelete: before, + insertedRows: inserted, + populationMs: Math.round(populatedMs), + conversionAndCloseMs: Math.round(conversionMs), + autoVacuum, + freelistCount, + integrity, + }), + ); + } +} finally { + rmSync(root, { recursive: true, force: true }); +}