perf: measure four balanced process pairs per benchmark

[mrousavy]

Each benchmark currently gets one base/head process pair, so one slow launch can dominate the PR comparison. Measure four adjacent pairs per case in AB, BA, BA, AB order, with a fresh process for each launch and each revision first twice.

Build and install each app once, retain the existing five warmups and twenty measured batches per process, and preserve all four numbered base/head runs and per-case JSON. HEAD report generation requires the declared pair count and correct identities, pools all four runs using the existing median calculation, and still accepts retained one-pair measurements from earlier attempts. The table layout and reporting threshold are unchanged.

The one-second app-start sleep was already removed by #1616. Two initial animation frames and five warmups remain; the one-second Android crash-monitor interval runs concurrently. With 46 cases, measurement uses 368 fresh processes per platform. Reducing the sample budget or sharding cases across runners can be evaluated separately.

Validation after rebasing onto the merged #1623 reporting pipeline: 103 performance-tool tests passed, including exact process order and lifecycle on both controllers, same-binary reuse, changed/reordered/unequal suites, all-four-pair reporting, missing/duplicate/extra results, and retaining an earlier iOS measurement while rerunning Android. Package builds, tooling/app typechecks and app lint passed.

The same-binary CI diagnostic passed on both Blacksmith platforms at 376e2ae93f32b0326112b948e2909f91f4b7f729. Downloaded base/head app artifacts were byte-for-byte identical. The raw report artifact passed this revision's reporter against GitHub's actual run/artifact metadata. All 736 process results followed the intended order with matching work counts and twenty samples each.

A/A diagnostic	Android	iOS
Measurement job duration	25m21s	23m36s
Individual-pair median absolute delta (range across four pairs)	1.33–1.94%	1.87–2.72%
Pooled median absolute delta	0.91%	1.22%
Individual-pair cases at least 5% apart	5–10 / 46	6–13 / 46
Pooled cases at least 5% apart	3 / 46	2 / 46
Largest pooled absolute delta	7.33%	5.73%

These are descriptive results from one same-binary job per platform, not a calibrated false-positive rate or a regression gate. Remaining apparent differences are measurement variation.

The diagnostic ran before #1623 merged. The current PR is rebased onto that reporting pipeline and applies the four-pair reader in generate-report.ts; publication-envelope validation stays unchanged. Replaying the diagnostic's complete raw artifact through the rebased generator produced byte-identical Markdown and Android/iOS BMF to the original validated report. The controller's execution order is unchanged by the rebase.

The historical diagnostic's separate publishing run failed because main switched to requiring performance-publication-1 while the older workflow produced only performance-report-1. The diagnostic measurements and local report validation succeeded. The rebased PR now produces the publication artifact using HEAD code, so its report can be reviewed before merge. Same-branch manual diagnostics are not main history and should not upload to Bencher.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
nitro-docs	Ready	Preview	Sep 7, 2026 7:33pm UTC

[nitro-modules-bot]

Performance Report

⚠️ Advisory: Results do not fail this PR.

iOS

Benchmark	Before	After	Difference
Swift synchronousCallback()	242.3 ns	223.6 ns	🟢 -7.72% faster
Swift create()	24.53 µs	23.20 µs	🟢 -5.45% faster

All Benchmarks

Benchmark	Before	After	Difference
JavaScript addNumbers()	24.0 ns	22.8 ns	🟢 -4.76% faster
C++ bounce(1 MiB)	230.9 ns	233.4 ns	🔴 +1.06% slower
C++ bounce(4 KiB)	242.7 ns	240.4 ns	🟢 -0.94% faster
C++ bounce native-owned buffer (1 MiB)	3.26 µs	3.29 µs	🔴 +0.79% slower
C++ bounce native-owned buffer (4 KiB)	342.0 ns	345.1 ns	🔴 +0.92% slower
C++ copy(1 MiB)	66.33 µs	64.92 µs	🟢 -2.13% faster
C++ copy(4 KiB)	942.5 ns	947.7 ns	🔴 +0.56% slower
C++ large array (1,024)	48.00 µs	47.55 µs	🟢 -0.93% faster
C++ small array (16)	915.9 ns	917.8 ns	🔴 +0.22% slower
C++ synchronousCallback()	188.7 ns	188.8 ns	🔴 +0.07% slower
C++ create()	414.4 ns	403.4 ns	🟢 -2.65% faster
C++ returnExisting()	192.4 ns	187.2 ns	🟢 -2.72% faster
C++ typed map (8 entries)	1.78 µs	1.76 µs	🟢 -1% faster
C++ optional trailing string	137.1 ns	136.5 ns	🟢 -0.43% faster
C++ addNumbers()	82.0 ns	81.9 ns	🟢 -0.07% faster
C++ simpleFunc()	61.0 ns	62.3 ns	🔴 +2.28% slower
C++ deferred worker Promise (includes trigger call)	17.30 µs	17.38 µs	🔴 +0.47% slower
C++ immediatePromise()	792.5 ns	792.8 ns	🔴 +0.04% slower
C++ number property get/set	127.4 ns	127.1 ns	🟢 -0.21% faster
C++ short ASCII string	136.7 ns	132.8 ns	🟢 -2.82% faster
C++ Unicode string	279.6 ns	278.0 ns	🟢 -0.56% faster
C++ nested Car struct	2.41 µs	2.44 µs	🔴 +1.11% slower
C++ number | string variant	128.1 ns	127.1 ns	🟢 -0.83% faster
Swift bounce(1 MiB)	218.8 ns	220.6 ns	🔴 +0.81% slower
Swift bounce(4 KiB)	214.6 ns	208.8 ns	🟢 -2.74% faster
Swift bounce native-owned buffer (1 MiB)	3.52 µs	3.56 µs	🔴 +1.03% slower
Swift bounce native-owned buffer (4 KiB)	342.9 ns	338.6 ns	🟢 -1.26% faster
Swift copy(1 MiB)	62.81 µs	63.61 µs	🔴 +1.27% slower
Swift copy(4 KiB)	816.7 ns	805.0 ns	🟢 -1.44% faster
Swift large array (1,024)	45.45 µs	46.10 µs	🔴 +1.44% slower
Swift small array (16)	1.01 µs	981.7 ns	🟢 -2.93% faster
Swift returnExisting()	221.8 ns	221.6 ns	🟢 -0.08% faster
Swift typed map (8 entries)	2.14 µs	2.18 µs	🔴 +1.43% slower
Swift optional trailing string	153.0 ns	154.3 ns	🔴 +0.85% slower
Swift addNumbers()	74.1 ns	73.4 ns	🟢 -1.04% faster
Swift simpleFunc()	60.0 ns	61.7 ns	🔴 +2.76% slower
Swift deferred worker Promise (includes trigger call)	24.09 µs	24.28 µs	🔴 +0.8% slower
Swift immediatePromise()	1.23 µs	1.22 µs	🟢 -1.07% faster
Swift number property get/set	129.5 ns	127.4 ns	🟢 -1.64% faster
Swift short ASCII string	153.3 ns	157.5 ns	🔴 +2.75% slower
Swift Unicode string	340.2 ns	341.6 ns	🔴 +0.42% slower
Swift nested Car struct	2.38 µs	2.31 µs	🟢 -3.12% faster
Swift number | string variant	179.4 ns	179.4 ns	🔴 +0.02% slower
TurboModule addNumbers()	912.7 ns	879.8 ns	🟢 -3.61% faster

Android

No observed change reached the 5% reporting threshold.

All Benchmarks

Benchmark	Before	After	Difference
JavaScript addNumbers()	19.0 ns	19.0 ns	🟢 -0.27% faster
C++ bounce(1 MiB)	450.2 ns	453.3 ns	🔴 +0.7% slower
C++ bounce(4 KiB)	451.0 ns	450.8 ns	🟢 -0.03% faster
C++ bounce native-owned buffer (1 MiB)	3.90 µs	3.80 µs	🟢 -2.61% faster
C++ bounce native-owned buffer (4 KiB)	434.2 ns	433.6 ns	🟢 -0.15% faster
C++ copy(1 MiB)	26.83 µs	26.21 µs	🟢 -2.33% faster
C++ copy(4 KiB)	922.3 ns	903.1 ns	🟢 -2.09% faster
C++ large array (1,024)	34.56 µs	34.40 µs	🟢 -0.46% faster
C++ small array (16)	751.6 ns	748.3 ns	🟢 -0.44% faster
C++ synchronousCallback()	347.6 ns	340.0 ns	🟢 -2.19% faster
C++ create()	588.5 ns	572.7 ns	🟢 -2.69% faster
C++ returnExisting()	197.0 ns	196.8 ns	🟢 -0.08% faster
C++ typed map (8 entries)	1.73 µs	1.69 µs	🟢 -2.16% faster
C++ optional trailing string	142.4 ns	143.1 ns	🔴 +0.48% slower
C++ addNumbers()	80.0 ns	80.5 ns	🔴 +0.65% slower
C++ simpleFunc()	64.1 ns	63.7 ns	🟢 -0.73% faster
C++ deferred worker Promise (includes trigger call)	33.70 µs	35.00 µs	🔴 +3.84% slower
C++ immediatePromise()	797.6 ns	783.4 ns	🟢 -1.78% faster
C++ number property get/set	134.9 ns	133.6 ns	🟢 -0.93% faster
C++ short ASCII string	130.2 ns	130.8 ns	🔴 +0.43% slower
C++ Unicode string	279.1 ns	276.7 ns	🟢 -0.86% faster
C++ nested Car struct	2.10 µs	2.09 µs	🟢 -0.4% faster
C++ number | string variant	127.1 ns	126.8 ns	🟢 -0.2% faster
Kotlin bounce(1 MiB)	1.16 µs	1.16 µs	🟢 -0.11% faster
Kotlin bounce(4 KiB)	1.18 µs	1.18 µs	🔴 +0.03% slower
Kotlin bounce native-owned buffer (1 MiB)	5.02 µs	4.94 µs	🟢 -1.65% faster
Kotlin bounce native-owned buffer (4 KiB)	1.29 µs	1.30 µs	🔴 +0.94% slower
Kotlin copy(1 MiB)	311.70 µs	313.93 µs	🔴 +0.72% slower
Kotlin copy(4 KiB)	6.75 µs	6.78 µs	🔴 +0.56% slower
Kotlin large array (1,024)	42.40 µs	42.41 µs	🔴 +0.01% slower
Kotlin small array (16)	1.10 µs	1.10 µs	🔴 +0.28% slower
Kotlin synchronousCallback()	1.13 µs	1.14 µs	🔴 +1.18% slower
Kotlin create()	1.85 µs	1.86 µs	🔴 +0.37% slower
Kotlin returnExisting()	607.4 ns	604.4 ns	🟢 -0.49% faster
Kotlin typed map (8 entries)	7.53 µs	7.51 µs	🟢 -0.33% faster
Kotlin optional trailing string	315.8 ns	314.2 ns	🟢 -0.53% faster
Kotlin addNumbers()	113.0 ns	113.5 ns	🔴 +0.52% slower
Kotlin simpleFunc()	95.9 ns	95.7 ns	🟢 -0.18% faster
Kotlin deferred worker Promise (includes trigger call)	47.08 µs	46.37 µs	🟢 -1.51% faster
Kotlin immediatePromise()	2.34 µs	2.35 µs	🔴 +0.28% slower
Kotlin number property get/set	225.6 ns	223.2 ns	🟢 -1.05% faster
Kotlin short ASCII string	473.4 ns	473.8 ns	🔴 +0.1% slower
Kotlin Unicode string	734.5 ns	735.2 ns	🔴 +0.1% slower
Kotlin nested Car struct	5.11 µs	5.14 µs	🔴 +0.67% slower
Kotlin number | string variant	406.8 ns	418.9 ns	🔴 +2.97% slower
TurboModule addNumbers()	227.7 ns	227.2 ns	🟢 -0.18% faster

Benchmarking Code Diff 54377134...15c760fa (view raw output)

Raw measurements: performance-report-1 (JSON artifact). Run 34155950400, attempt 1. Download requires GitHub access.

Android: measurements, attempt 1, apps, attempt 1.
iOS: measurements, attempt 1, apps, attempt 1.