diff --git a/.ci/bundle-size-budgets.json b/.ci/bundle-size-budgets.json new file mode 100644 index 000000000..ec3f9b0f1 --- /dev/null +++ b/.ci/bundle-size-budgets.json @@ -0,0 +1,9 @@ +{ + "web": { + "javascript": { + "measurement": "bundle", + "softKiB": 35, + "hardKiB": 50 + } + } +} diff --git a/.ci/bundle-size-budgets.md b/.ci/bundle-size-budgets.md new file mode 100644 index 000000000..f2dd235f9 --- /dev/null +++ b/.ci/bundle-size-budgets.md @@ -0,0 +1,109 @@ +# Bundle size budgets + +`.ci/bundle-size-budgets.json` defines named budgets for each platform. Each budget +explicitly selects a `measurement` and may select a `file` within a package. +The platform adapter defines what its `bundle` contains; for web, that is all +shipped JavaScript under `dist`. +Limits are KiB (1,024 bytes), including fractions. Comparisons use exact +bytes, not the rounded numbers displayed in PR reports. + +Web starts with a 35 KiB soft limit and 50 KiB hard limit on all shipped JavaScript, +using `"measurement": "bundle"`. This currently measures `dist/index.js` +and will include any additional JavaScript chunks shipped in `dist`. Declarations, +source maps, and other package contents are excluded from this measurement. +The report also includes deterministic gzip size and package sizes. These remain +informational until a budget is configured for their metric. + +## Accepting an increase + +The **Size budgets** check combines all configured metrics for affected platforms: + +- At or below the soft limit: pass. +- Already above a limit on the PR base, and unchanged or smaller: pass. +- Growing above the soft limit, but at or below the hard limit: require acceptance. +- Growing above the hard limit: require a reviewed budget-file change. +- Missing a configured head measurement: fail. Missing base measurements do not + exempt an increase from its limits. + +Once the current revision's report is posted, a repository writer (including the +PR author) can add a new PR comment: + +```text +/accept-size web Required for the new checkout capability. +``` + +The reason is mandatory. The command accepts every currently unaccepted soft-limit +breach for that platform, recording an independent byte cap for each metric, the +actor, and a link to the reason. Multiple commands may share one comment: + +```text +/accept-size web Includes the new checkout capability. +/accept-size android Includes its native implementation. +``` + +Every breached metric must be satisfied before the aggregate check passes. +Acceptance survives subsequent commits when the accepted measurement stays the same +size or gets smaller. Further growth or a newly breached metric requires a new +comment. Changing a budget's measurement or file also requires fresh acceptance. +A comment cannot override a hard limit, failed build, or missing data. +It does not modify the configured budget. Edited comments are not processed; post +a new command instead. Editing or deleting an already accepted reason does not +erase the recorded decision in the bot report. + +Commands posted before the current report is ready cannot pre-approve future +measurements. If the PR head or base has changed, rerun **Package Size** and wait +for the updated report. Expired measurement artifacts also require a rerun. + +## Metrics and additional platforms + +| Platform key | `measurement` | Scope | +| --- | --- | --- | +| `web` | `bundle` | Sum of raw shipped `.js`, `.mjs`, and `.cjs` files in `dist` | +| `web` | `bundleGzip` | Sum of those files compressed individually with `gzip -n -9` | +| `web` | `package` | Whole compressed npm tarball | +| `react-native` | `package` | Whole compressed wrapper npm tarball | +| `android` | `package` | Whole compressed release AAR | + +With `measurement: "package"`, an optional `file` selects that exact file's +**uncompressed** size inside the package. Paths are relative to the published +package root (or AAR root); globs are not supported. A missing file fails the check. +For example, this budgets only the entry file, rather than every JavaScript chunk: + +```json +{ + "web": { + "entryPoint": { + "measurement": "package", + "file": "dist/index.js", + "softKiB": 35, + "hardKiB": 50 + } + } +} +``` + +Omit `file` from a `package` budget to measure the complete compressed artifact. +Budget names such as `entryPoint` are labels; the selector fields determine what +is measured, and the PR report displays that scope explicitly. + +Unknown platforms, measurements, units, and invalid limits fail configuration +validation. Adding a new measurement (for example a Swift framework) requires a +reproducible collector in `measure-package-size`, its changed-path detection and +build setup in `package-size.yml`, and an adapter in `bundle-size-budgets.cjs`. +The evaluator, comment commands, and aggregate check need no platform-specific policy. + +## CI integration + +**Package Size** builds the explicit PR head and base SHAs with a read-only token, +including on drafts. It uploads measurements and the informational file breakdown. +**Bundle Size Budgets** runs trusted default-branch code to read that data, check +commenter permissions, and publish the report and **Size budgets** check. It never +executes code from a PR in the job with write permissions. Saved acceptances are +read only from the GitHub Actions bot's report, and concurrent updates are serialized +per PR. Older runs cannot overwrite a newer revision's result. + +The publisher and comment commands become available after this workflow lands on +the default branch. At that point, add **Size budgets** (GitHub Actions) to the +repository's required status checks, alongside **CI Required**. Requiring it before +then would block PRs waiting for a check that cannot yet run. Keep it separate from +the build gate so accepting an increase can update its result without rebuilding. diff --git a/.github/scripts/bundle-size-budgets.cjs b/.github/scripts/bundle-size-budgets.cjs new file mode 100644 index 000000000..d690944b1 --- /dev/null +++ b/.github/scripts/bundle-size-budgets.cjs @@ -0,0 +1,302 @@ +const fs = require("node:fs"); + +// Measurement adapters map the existing artifact report to explicit selectors. +// Policy and acceptance handling below do not depend on a particular platform. +const platforms = { + web: { + label: "Web", + packageLabel: "Whole npm package (gzip)", + measurements: { + bundle: "JavaScript", + bundleGzip: "JavaScript (gzip)", + package: "npm tarball", + }, + }, + "react-native": { + label: "React Native", + packageLabel: "Whole npm package (gzip)", + measurements: { package: "npm tarball" }, + }, + android: { + label: "Android", + packageLabel: "Whole AAR package (ZIP)", + measurements: { package: "release AAR" }, + }, +}; +const marker = ""; +const statePattern = //; + +function object(value) { + return value !== null && typeof value === "object" && !Array.isArray(value); +} + +function validateBudgets(budgets) { + if (!object(budgets)) throw new Error("Budgets must be an object"); + for (const [platform, metrics] of Object.entries(budgets)) { + if (!Object.hasOwn(platforms, platform) || !object(metrics)) + throw new Error(`Unknown platform: ${platform}`); + for (const [metric, budget] of Object.entries(metrics)) { + if (!/^[a-zA-Z][a-zA-Z0-9-]*$/.test(metric)) + throw new Error(`Invalid budget name: ${platform}.${metric}`); + if ( + !object(budget) || + typeof budget.measurement !== "string" || + !Object.hasOwn(platforms[platform].measurements, budget.measurement) + ) + throw new Error(`Unknown measurement for ${platform}.${metric}`); + if ( + Object.hasOwn(budget, "file") && + (budget.measurement !== "package" || + typeof budget.file !== "string" || + !budget.file || + /[\\\t\r\n*?[\]{}]/.test(budget.file) || + budget.file.split("/").some((part) => !part || part === "." || part === "..")) + ) + throw new Error(`File must be an exact package-relative path for ${platform}.${metric}`); + if ( + Object.keys(budget).some( + (key) => !["measurement", "file", "softKiB", "hardKiB"].includes(key), + ) || + !Number.isFinite(budget.softKiB) || + !Number.isFinite(budget.hardKiB) || + budget.softKiB <= 0 || + budget.hardKiB < budget.softKiB || + budget.hardKiB * 1024 > Number.MAX_SAFE_INTEGER + ) { + throw new Error(`Invalid budget for ${platform}.${metric}: require 0 < softKiB <= hardKiB`); + } + } + } + return budgets; +} + +function measurements(tsv) { + const values = {}; + for (const line of tsv.split("\n").filter(Boolean)) { + const columns = line.split("\t"); + const [platform, metric, bytes, file] = columns; + if ( + ![3, 4].includes(columns.length) || + (columns.length === 4 && !file) || + !/^\d+$/.test(bytes) || + !Number.isSafeInteger(Number(bytes)) + ) { + throw new Error("Invalid measurement row"); + } + const key = `${platform}\t${metric}${file ? `\t${file}` : ""}`; + if (Object.hasOwn(values, key)) throw new Error(`Duplicate measurement: ${key}`); + values[key] = Number(bytes); + } + return values; +} + +function evaluate({ budgets, base, head, measuredPlatforms }, acceptances = {}) { + validateBudgets(budgets); + if ( + !Array.isArray(measuredPlatforms) || + measuredPlatforms.some((p) => !Object.hasOwn(platforms, p)) + ) { + throw new Error("Invalid measured platforms"); + } + const rows = []; + for (const [platform, metrics] of Object.entries(budgets)) { + if (!measuredPlatforms.includes(platform)) continue; + for (const [metric, budget] of Object.entries(metrics)) { + const key = `${platform}.${metric}`; + const measurementKey = `${platforms[platform].label}\t${platforms[platform].measurements[budget.measurement]}${budget.file ? `\t${budget.file}` : ""}`; + const before = base[measurementKey]; + const after = head[measurementKey]; + const acceptance = acceptances[key]; + let status; + if (!Number.isSafeInteger(after) || after < 0 || (after === 0 && !budget.file)) + status = "missing"; + else if (after <= budget.softKiB * 1024) status = "within"; + else if (before !== undefined && after <= before) status = "no-growth"; + else if (after > budget.hardKiB * 1024) status = "hard"; + else if ( + acceptance && + acceptance.measurement === budget.measurement && + acceptance.file === budget.file && + after <= acceptance.bytes + ) + status = "accepted"; + else status = "soft"; + rows.push({ key, platform, metric, before, after, ...budget, status, acceptance }); + } + } + return rows; +} + +function parseCommands(body) { + const commands = []; + for (const line of body.split(/\r?\n/)) { + if (!line.startsWith("/accept-size")) continue; + const match = /^\/accept-size ([a-z][a-z-]*) (\S.*)$/.exec(line); + if (!match || !Object.hasOwn(platforms, match[1]) || match[2].trim().length > 1000) { + throw new Error("Use /accept-size (reason: 1–1000 characters)"); + } + commands.push({ platform: match[1], reason: match[2].trim() }); + } + return commands; +} + +function accept(rows, commands, actor, comment, previous = {}) { + const accepted = { ...previous }; + const notes = []; + for (const { platform, reason } of commands) { + const eligible = rows.filter((row) => row.platform === platform && row.status === "soft"); + for (const row of eligible) { + accepted[row.key] = { + bytes: row.after, + measurement: row.measurement, + ...(row.file ? { file: row.file } : {}), + actor, + reason, + commentId: comment.id, + url: comment.html_url, + }; + } + const hard = rows.some((row) => row.platform === platform && row.status === "hard"); + notes.push( + hard + ? `${platform}: hard budget exceeded; edit the budget file for review.` + : eligible.length + ? `${platform}: accepted ${eligible.length} exceeded metric(s).` + : `${platform}: no soft-budget breaches to accept.`, + ); + } + return { accepted, notes }; +} + +function readState(body) { + const match = body?.match(statePattern); + if (!match) return null; + const state = JSON.parse(Buffer.from(match[1], "base64").toString("utf8")); + if ( + state.version !== 1 || + !object(state.acceptances) || + !Array.isArray(state.processedComments) + ) { + throw new Error("Invalid saved budget report"); + } + return state; +} + +function escape(value) { + return String(value) + .replace(/&/g, "&") + .replace(//g, ">") + .replace(/\|/g, "|") + .replace(/\r?\n/g, " ") + .replace(/([\\`*_[\]])/g, "\\$1"); +} + +function kib(bytes) { + return bytes === undefined ? "unavailable" : `${(bytes / 1024).toFixed(2)} KiB`; +} + +function render(rows, state, packageComment, notes = []) { + const labels = { + within: "Within budget", + "no-growth": "Above budget; no increase", + soft: "Acceptance required", + hard: "Budget change required", + missing: "Measurement missing", + }; + const lines = [ + marker, + "## Size budgets", + "", + `Measured head: \`${state.headSha}\`; base: \`${state.baseSha}\`.`, + "", + "| Platform / budget | Measurement | Base | Head | Delta | Soft | Hard | Result |", + "| --- | --- | ---: | ---: | ---: | ---: | ---: | --- |", + ]; + for (const row of rows) { + const status = + row.status === "accepted" + ? `Accepted by @${escape(row.acceptance.actor)}: ${escape(row.acceptance.reason)} ([comment](${row.acceptance.url}))` + : labels[row.status]; + const delta = + row.before === undefined || row.after === undefined + ? "unavailable" + : `${row.after > row.before ? "+" : ""}${kib(row.after - row.before)}`; + const scope = row.file + ? `${escape(row.file)} (uncompressed)` + : { + bundle: "Bundle (uncompressed)", + bundleGzip: "Bundle (gzip)", + package: platforms[row.platform].packageLabel, + }[row.measurement]; + lines.push( + `| ${row.platform} / ${row.metric} | ${scope} | ${kib(row.before)} | ${kib(row.after)} | ${delta} | ${row.softKiB} KiB | ${row.hardKiB} KiB | ${status} |`, + ); + } + if (!rows.length) lines.push("| — | — | — | — | — | — | — | No configured budgets affected |"); + lines.push( + "", + "Repository writers, including the PR author, can accept current soft-budget breaches with a reason:", + "", + "```text", + "/accept-size web Explain why this increase is necessary.", + "```", + "", + "Use one command per platform; several lines can share a comment. Post after this report is ready for the current head. Commands in edited comments are not accepted.", + "", + "Acceptance applies to each currently exceeded metric up to its recorded size. Further growth or a newly exceeded metric needs fresh acceptance. Hard-budget increases require a reviewed change to `.ci/bundle-size-budgets.json`.", + "", + ); + for (const note of notes) lines.push(`- ${escape(note)}`); + lines.push( + "", + packageComment.replace(//g, ""), + "", + ``, + ); + const body = lines.join("\n"); + if (body.length > 65000) throw new Error("Size report exceeds GitHub comment limit"); + return body; +} + +function conclusion(rows) { + return rows.some((row) => ["soft", "hard", "missing"].includes(row.status)) + ? "failure" + : "success"; +} + +module.exports = { + platforms, + marker, + validateBudgets, + measurements, + evaluate, + parseCommands, + accept, + readState, + render, + conclusion, +}; + +if (require.main === module) { + const [basePath, headPath, budgetsPath, outputPath] = process.argv.slice(2); + const event = JSON.parse(fs.readFileSync(process.env.GITHUB_EVENT_PATH, "utf8")); + const report = { + version: 1, + pr: event.pull_request.number, + headSha: event.pull_request.head.sha, + baseSha: process.env.BASE_SHA || event.pull_request.base.sha, + budgets: validateBudgets(JSON.parse(fs.readFileSync(budgetsPath, "utf8"))), + base: measurements(fs.readFileSync(basePath, "utf8")), + head: measurements(fs.readFileSync(headPath, "utf8")), + measuredPlatforms: Object.entries({ + web: process.env.MEASURE_WEB, + "react-native": process.env.MEASURE_REACT_NATIVE, + android: process.env.MEASURE_ANDROID, + }) + .filter(([, enabled]) => enabled === "true") + .map(([platform]) => platform), + }; + evaluate(report); // Reject malformed configuration before publishing an artifact. + fs.writeFileSync(outputPath, JSON.stringify(report)); +} diff --git a/.github/scripts/bundle-size-budgets.test.cjs b/.github/scripts/bundle-size-budgets.test.cjs new file mode 100644 index 000000000..b6c6966e1 --- /dev/null +++ b/.github/scripts/bundle-size-budgets.test.cjs @@ -0,0 +1,185 @@ +const { test } = require("node:test"); +const assert = require("node:assert/strict"); +const policy = require("./bundle-size-budgets.cjs"); + +const KiB = 1024; +const report = (size, base = 34 * KiB) => ({ + budgets: { web: { javascript: { measurement: "bundle", softKiB: 35, hardKiB: 50 } } }, + base: base === null ? {} : { "Web\tJavaScript": base }, + head: { "Web\tJavaScript": size }, + measuredPlatforms: ["web"], +}); +const comment = { id: 42, html_url: "https://github.com/example/repo/pull/1#issuecomment-42" }; + +test("enforces exact limits, exempts no growth, and handles missing measurements", () => { + for (const [size, base, status] of [ + [35 * KiB, 34 * KiB, "within"], + [35 * KiB + 1, 34 * KiB, "soft"], + [50 * KiB, 34 * KiB, "soft"], + [50 * KiB + 1, 34 * KiB, "hard"], + [60 * KiB, 60 * KiB, "no-growth"], + [59 * KiB, 60 * KiB, "no-growth"], + [40 * KiB, null, "soft"], + [undefined, 34 * KiB, "missing"], + [0, 34 * KiB, "missing"], + ]) { + assert.equal(policy.evaluate(report(size, base))[0].status, status, `${base} → ${size}`); + } + const input = report(35.5 * KiB); + input.budgets.web.javascript.softKiB = 35.5; + assert.equal(policy.evaluate(input)[0].status, "within"); + input.head["Web\tJavaScript"]++; + assert.equal(policy.evaluate(input)[0].status, "soft"); + input.measuredPlatforms = []; + assert.equal(policy.conclusion(policy.evaluate(input)), "success"); +}); + +test("platform acceptance covers each current breach, with independent caps", () => { + const input = report(40 * KiB); + input.budgets.web.javascriptGzip = { + measurement: "bundleGzip", + softKiB: 10, + hardKiB: 15, + }; + input.head["Web\tJavaScript (gzip)"] = 11 * KiB; + input.budgets.android = { aar: { measurement: "package", softKiB: 100, hardKiB: 200 } }; + input.head["Android\trelease AAR"] = 150 * KiB; + input.measuredPlatforms.push("android"); + const { accepted } = policy.accept( + policy.evaluate(input), + [{ platform: "web", reason: "New capability" }], + "writer", + comment, + ); + assert.deepEqual( + policy.evaluate(input, accepted).map((row) => row.status), + ["accepted", "accepted", "soft"], + ); + assert.equal(policy.conclusion(policy.evaluate(input, accepted)), "failure"); + const all = policy.accept( + policy.evaluate(input, accepted), + [{ platform: "android", reason: "Native support" }], + "writer", + comment, + accepted, + ).accepted; + assert.equal(policy.conclusion(policy.evaluate(input, all)), "success"); + for (const [size, status] of [ + [39 * KiB, "accepted"], + [40 * KiB, "accepted"], + [40 * KiB + 1, "soft"], + ]) { + assert.equal(policy.evaluate(report(size), all)[0].status, status); + } + input.budgets.web.npmTarball = { measurement: "package", softKiB: 100, hardKiB: 200 }; + input.head["Web\tnpm tarball"] = 120 * KiB; + assert.equal( + policy.evaluate(input, all).find((row) => row.metric === "npmTarball").status, + "soft", + ); +}); + +test("comments and existing acceptances cannot override a hard budget", () => { + const input = report(51 * KiB); + const result = policy.accept( + policy.evaluate(input), + [{ platform: "web", reason: "Please override" }], + "writer", + comment, + ); + assert.deepEqual(result.accepted, {}); + assert.equal( + policy.evaluate(input, { + "web.javascript": { bytes: 60 * KiB, measurement: "bundle" }, + })[0].status, + "hard", + ); +}); + +test("selects whole packages or individual files without reusing acceptance for another scope", () => { + const input = report(40000); + Object.assign( + input.head, + policy.measurements( + "Web\tnpm tarball\t90000\nWeb\tnpm tarball\t40000\tdist/index.js\nWeb\tnpm tarball\t39999\tdist/other.js\n", + ), + ); + const budget = input.budgets.web.javascript; + budget.measurement = "package"; + assert.equal(policy.evaluate(input)[0].after, 90000); + budget.file = "dist/index.js"; + assert.equal(policy.evaluate(input)[0].after, 40000); + const { accepted } = policy.accept( + policy.evaluate(input), + [{ platform: "web", reason: "New capability" }], + "writer", + comment, + ); + assert.equal(policy.evaluate(input, accepted)[0].status, "accepted"); + budget.file = "dist/other.js"; + assert.equal(policy.evaluate(input, accepted)[0].status, "soft"); + budget.file = "dist/missing.js"; + assert.equal(policy.evaluate(input, accepted)[0].status, "missing"); + input.head["Web\tnpm tarball\tdist/missing.js"] = 0; + assert.equal(policy.evaluate(input, accepted)[0].status, "within"); + delete budget.file; + budget.measurement = "bundle"; + assert.equal(policy.evaluate(input, accepted)[0].status, "soft"); +}); + +test("rejects unknown configuration keys and invalid limits", () => { + for (const budgets of [ + { ios: {} }, + { web: { javascript: { softKiB: 35, hardKiB: 50 } } }, + { web: { typo: { measurement: "unknown", softKiB: 1, hardKiB: 2 } } }, + { web: { javascript: { measurement: "bundle", soft: 35, hard: 50 } } }, + { web: { javascript: { measurement: "bundle", softKiB: "35", hardKiB: 50 } } }, + { web: { javascript: { measurement: "bundle", softKiB: 51, hardKiB: 50 } } }, + { web: { javascript: { measurement: "bundle", softKiB: 0, hardKiB: 50 } } }, + ...["", "/index.js", "../index.js", "dist/*.js", "dist\\index.js", 42].map((file) => ({ + web: { entry: { measurement: "package", file, softKiB: 35, hardKiB: 50 } }, + })), + { + web: { + entry: { + measurement: "bundle", + file: "dist/index.js", + softKiB: 35, + hardKiB: 50, + }, + }, + }, + ]) + assert.throws(() => policy.validateBudgets(budgets)); +}); + +test("measurement parsing keeps package and file sizes separate and rejects invalid rows", () => { + assert.deepEqual( + policy.measurements("Web\tJavaScript\t34073\nWeb\tnpm tarball\t42000\tdist/index.js\n"), + { "Web\tJavaScript": 34073, "Web\tnpm tarball\tdist/index.js": 42000 }, + ); + for (const text of [ + "Web\tJavaScript\t-1", + "Web\tJavaScript\t1.5", + "Web\tJavaScript\t12\nWeb\tJavaScript\t13", + ]) { + assert.throws(() => policy.measurements(text)); + } +}); + +test("artifact text cannot inject saved acceptance state into the report", () => { + const state = { + version: 1, + headSha: "abc", + baseSha: "def", + acceptances: {}, + processedComments: [], + }; + const body = policy.render( + policy.evaluate(report(40 * KiB)), + state, + "", + ); + assert.deepEqual(policy.readState(body), state); + assert.match(body, /Acceptance required/); +}); diff --git a/.github/scripts/measure-package-size b/.github/scripts/measure-package-size index 99123ef29..13915765f 100755 --- a/.github/scripts/measure-package-size +++ b/.github/scripts/measure-package-size @@ -99,6 +99,28 @@ measure_web_package() { ) tarball=$(find "$pack_dir" -name "*.tgz" -type f -print -quit) + # Measure every shipped JS chunk, excluding declarations and source maps. + # Sum gzip sizes per file, matching separately compressed HTTP responses. + local js_dir + local js_file + local javascript_bytes=0 + local gzip_bytes=0 + local js_count=0 + js_dir=$(mktemp -d "${TMPDIR:-/tmp}/checkout-kit-web-js.XXXXXX") + tar -xzf "$tarball" -C "$js_dir" + while IFS= read -r js_file; do + javascript_bytes=$((javascript_bytes + $(size_bytes "$js_file"))) + gzip_bytes=$((gzip_bytes + $(gzip -n -9 -c "$js_file" | wc -c))) + js_count=$((js_count + 1)) + done < <(find "$js_dir/package/dist" -type f \( -name '*.js' -o -name '*.mjs' -o -name '*.cjs' \) | sort) + if [[ "$js_count" -eq 0 ]]; then + echo "No shipped web JavaScript found" >&2 + rm -rf "$js_dir" + return 1 + fi + printf "Web\tJavaScript\t%s\n" "$javascript_bytes" >> "$output_file" + printf "Web\tJavaScript (gzip)\t%s\n" "$gzip_bytes" >> "$output_file" + rm -rf "$js_dir" record_artifact "Web" "npm tarball" "$tarball" record_tarball_breakdown "Web" "npm tarball" "$tarball" } @@ -180,6 +202,26 @@ render_comment() { return (bytes == "") ? "—" : human(bytes); } + function measurement(artifact) { + if (artifact == "JavaScript" || artifact == "JavaScript (gzip)") { + return "JavaScript bundle"; + } + if (artifact == "npm tarball") { + return "npm package (`.tgz`)"; + } + if (artifact == "release AAR") { + return "Library package (`.aar`)"; + } + return artifact; + } + + function compression(artifact) { + if (artifact == "JavaScript") return "Uncompressed"; + if (artifact == "JavaScript (gzip)" || artifact == "npm tarball") return "gzip"; + if (artifact == "release AAR") return "ZIP"; + return "Unspecified"; + } + function delta(head, base, diff) { if (head == "" || base == "") { return "unavailable"; @@ -263,19 +305,22 @@ render_comment() { cap = 20; print ""; - print "## Package Size"; + print "## Bundle and package size"; print ""; - print "| Platform | Artifact | Base | Head | Delta |"; - print "| --- | --- | ---: | ---: | ---: |"; + print "Web bundle sizes cover shipped runtime JavaScript. Package sizes cover the full published archive, including any source maps, declarations, and documentation it contains."; + print ""; + print "| Platform | Measurement | Compression | Base | Head | Delta |"; + print "| --- | --- | --- | ---: | ---: | ---: |"; if (order_count == 0) { - print "| - | - | - | - | - |"; + print "| - | - | - | - | - | - |"; } else { for (i = 1; i <= order_count; i++) { key = order[i]; - printf "| %s | %s | %s | %s | %s |\n", + printf "| %s | %s | %s | %s | %s | %s |\n", platforms[key], - artifacts[key], + measurement(artifacts[key]), + compression(artifacts[key]), human(base_bytes[key]), human(head_bytes[key]), delta(head_bytes[key], base_bytes[key]); @@ -310,7 +355,9 @@ render_comment() { } print ""; - print "
" platform " file breakdown"; + print "
" platform " package files (uncompressed)"; + print ""; + print "These are uncompressed file sizes; they do not sum to the compressed package size above."; print ""; print "| File | Base | Head | Delta |"; print "| --- | ---: | ---: | ---: |"; @@ -333,7 +380,7 @@ render_comment() { } print ""; - print "_Measured from the PR base SHA and PR head SHA. The file breakdown shows uncompressed sizes within each package artifact, so individual files do not sum to the compressed artifact total. This comment reports package artifact sizes only; it is not a final app binary-size report._"; + print "_Measured from the PR base SHA and PR head SHA. Web bundle rows sum shipped `.js`, `.mjs`, and `.cjs` files under `dist/`, excluding source maps and declarations. The gzip bundle size sums files compressed individually with `gzip -n -9`. npm package sizes are gzip-compressed `.tgz` archives; Android AAR sizes are ZIP archives. Package sizes are not final app binary sizes._"; } ' "$base_file" "$head_file" > "$comment_file" } diff --git a/.github/scripts/publish-bundle-size.cjs b/.github/scripts/publish-bundle-size.cjs new file mode 100644 index 000000000..d2ccf45b1 --- /dev/null +++ b/.github/scripts/publish-bundle-size.cjs @@ -0,0 +1,284 @@ +const fs = require("node:fs"); +const os = require("node:os"); +const path = require("node:path"); +const { execFileSync } = require("node:child_process"); +const policy = require("./bundle-size-budgets.cjs"); + +const checkName = "Size budgets"; +const workflow = "package-size.yml"; +const isReport = (comment) => + comment.user?.login === "github-actions[bot]" && + comment.user.type === "Bot" && + comment.body?.startsWith(policy.marker); + +async function resolvePR({ github, context }) { + if (context.eventName === "issue_comment") + return context.payload.issue.pull_request ? context.payload.issue.number : null; + const run = context.payload.workflow_run; + if (run.event !== "pull_request" || run.path !== `.github/workflows/${workflow}`) return null; + // workflow_run.pull_requests can be empty for a fork PR. + const prs = run.pull_requests.length + ? run.pull_requests + : await github.paginate(github.rest.repos.listPullRequestsAssociatedWithCommit, { + ...context.repo, + commit_sha: run.head_sha, + per_page: 100, + }); + const pr = prs.find( + (pr) => pr.head.sha === run.head_sha && pr.head.repo?.id === run.head_repository?.id, + ); + return pr?.number ?? null; +} + +async function readArtifact(github, repo, runId) { + const artifacts = await github.paginate(github.rest.actions.listWorkflowRunArtifacts, { + ...repo, + run_id: runId, + per_page: 100, + }); + const artifact = artifacts.find((item) => item.name === "bundle-size-report" && !item.expired); + if (!artifact || artifact.size_in_bytes > 5 * 1024 * 1024) + throw new Error("Missing or oversized measurement artifact; rerun Package Size."); + const { data } = await github.rest.actions.downloadArtifact({ + ...repo, + artifact_id: artifact.id, + archive_format: "zip", + }); + const directory = fs.mkdtempSync(path.join(os.tmpdir(), "bundle-size-report-")); + try { + const archive = path.join(directory, "report.zip"); + fs.writeFileSync(archive, Buffer.from(data)); + // Read only known text entries. Never extract paths or execute PR-provided code. + const read = (name) => + execFileSync("unzip", ["-p", archive, name], { encoding: "utf8", maxBuffer: 1024 * 1024 }); + return { report: JSON.parse(read("report.json")), packageComment: read("comment.md") }; + } finally { + fs.rmSync(directory, { recursive: true, force: true }); + } +} + +function validateReport(report, pr, run) { + if ( + report.version !== 1 || + report.pr !== pr.number || + report.headSha !== pr.head.sha || + report.baseSha !== pr.base.sha || + run.head_sha !== pr.head.sha + ) { + throw new Error("Measurements do not match the current PR head and base; rerun Package Size."); + } + for (const values of [report.base, report.head]) { + if ( + !values || + typeof values !== "object" || + Array.isArray(values) || + Object.values(values).some((value) => !Number.isSafeInteger(value) || value < 0) + ) { + throw new Error("Invalid size measurements"); + } + } + policy.evaluate(report); +} + +async function latestRun(github, repo, pr) { + const runs = await github.paginate(github.rest.actions.listWorkflowRuns, { + ...repo, + workflow_id: workflow, + event: "pull_request", + head_sha: pr.head.sha, + per_page: 100, + }); + return runs + .filter((run) => run.head_repository?.id === pr.head.repo.id && run.head_branch === pr.head.ref) + .sort((a, b) => b.id - a.id)[0]; +} + +async function updateCheck(github, repo, pr, status, summary, detailsUrl) { + const { data: current } = await github.rest.pulls.get({ ...repo, pull_number: pr.number }); + if ( + current.head.sha !== pr.head.sha || + current.base.sha !== pr.base.sha || + current.state !== "open" + ) + return; + const checks = await github.paginate(github.rest.checks.listForRef, { + ...repo, + ref: pr.head.sha, + check_name: checkName, + per_page: 100, + }); + const externalId = `bundle-size-budgets:${pr.number}`; + const existing = checks.find( + (check) => check.external_id === externalId && check.app?.slug === "github-actions", + ); + const params = { + ...repo, + name: checkName, + external_id: externalId, + status: status === "pending" ? "in_progress" : "completed", + output: { + title: + status === "success" + ? "All size budgets satisfied or accepted" + : status === "pending" + ? "Waiting for size measurements" + : "Size budgets need attention", + summary, + }, + details_url: detailsUrl, + }; + if (status !== "pending") { + params.conclusion = status; + params.completed_at = new Date().toISOString(); + } + if (existing) await github.rest.checks.update({ ...params, check_run_id: existing.id }); + else await github.rest.checks.create({ ...params, head_sha: pr.head.sha }); +} + +async function processCommands({ github, repo, comments, state, report, mayAccept }) { + let acceptances = { ...state.acceptances }; + const processed = new Set(state.processedComments); + const notes = []; + const permissions = new Map(); + for (const comment of comments) { + if ( + processed.has(comment.id) || + !comment.body?.split(/\r?\n/).some((line) => line.startsWith("/accept-size")) + ) + continue; + processed.add(comment.id); + if (!mayAccept || comment.created_at < state.publishedAt) { + notes.push( + `Comment ${comment.id}: wait for the current size report, then post a new command.`, + ); + continue; + } + if (comment.created_at !== comment.updated_at) { + notes.push(`Comment ${comment.id}: post a new command; edited comments are not accepted.`); + continue; + } + const actor = comment.user.login; + if (comment.user.type !== "User") continue; + if (!permissions.has(actor)) { + try { + const { data } = await github.rest.repos.getCollaboratorPermissionLevel({ + ...repo, + username: actor, + }); + permissions.set( + actor, + ["admin", "maintain", "write"].includes(data.permission) || + data.user?.permissions?.push === true, + ); + } catch (error) { + if (error.status !== 404) throw error; + permissions.set(actor, false); + } + } + if (!permissions.get(actor)) { + notes.push(`Comment ${comment.id}: repository write access is required.`); + continue; + } + let commands; + try { + commands = policy.parseCommands(comment.body); + } catch (error) { + notes.push(`Comment ${comment.id}: ${error.message}`); + continue; + } + const result = policy.accept( + policy.evaluate(report, acceptances), + commands, + actor, + comment, + acceptances, + ); + acceptances = result.accepted; + notes.push(...result.notes); + } + return { acceptances, processedComments: [...processed], notes: notes.slice(-10) }; +} + +async function publish({ github, context, core, prNumber, loadArtifact = readArtifact }) { + const repo = context.repo; + const { data: pr } = await github.rest.pulls.get({ ...repo, pull_number: prNumber }); + if (pr.state !== "open") return; + const run = await latestRun(github, repo, pr); + if (!run) return; // A comment cannot create an approval before measurements exist. + if (context.eventName === "workflow_run" && context.payload.workflow_run.id !== run.id) return; + try { + if (run.status !== "completed") { + await updateCheck( + github, + repo, + pr, + "pending", + "Package Size is measuring this revision. Wait for its report before accepting an increase.", + run.html_url, + ); + return; + } + if (run.conclusion !== "success") + throw new Error( + "Package Size did not succeed. Fix or rerun the measurement workflow; a comment cannot bypass build failures.", + ); + const { report, packageComment } = await loadArtifact(github, repo, run.id); + validateReport(report, pr, run); + const comments = await github.paginate(github.rest.issues.listComments, { + ...repo, + issue_number: pr.number, + per_page: 100, + }); + const existing = comments.find(isReport); + const previous = existing ? policy.readState(existing.body) : null; + // A command only accepts sizes already shown for this exact measurement run. + // Saved per-metric caps survive subsequent runs and unrelated commits. + const mayAccept = + previous?.headSha === report.headSha && + previous?.baseSha === report.baseSha && + previous?.runId === run.id && + previous?.runAttempt === run.run_attempt; + const state = previous ?? { + version: 1, + acceptances: {}, + processedComments: [], + publishedAt: new Date().toISOString(), + }; + const processed = await processCommands({ github, repo, comments, state, report, mayAccept }); + const next = { + ...state, + ...processed, + notes: undefined, + headSha: report.headSha, + baseSha: report.baseSha, + runId: run.id, + runAttempt: run.run_attempt, + publishedAt: mayAccept ? state.publishedAt : new Date().toISOString(), + }; + const rows = policy.evaluate(report, next.acceptances); + const body = policy.render(rows, next, packageComment, processed.notes); + const { data: current } = await github.rest.pulls.get({ ...repo, pull_number: pr.number }); + if ( + current.head.sha !== pr.head.sha || + current.base.sha !== pr.base.sha || + current.state !== "open" + ) + return; + const { data: comment } = existing + ? await github.rest.issues.updateComment({ ...repo, comment_id: existing.id, body }) + : await github.rest.issues.createComment({ ...repo, issue_number: pr.number, body }); + await updateCheck( + github, + repo, + pr, + policy.conclusion(rows), + body.replace(//gs, "").slice(0, 60000), + comment.html_url, + ); + } catch (error) { + await updateCheck(github, repo, pr, "failure", error.message, run.html_url); + core.setFailed(error.message); + } +} + +module.exports = { resolvePR, publish, processCommands, validateReport, isReport }; diff --git a/.github/scripts/publish-bundle-size.test.cjs b/.github/scripts/publish-bundle-size.test.cjs new file mode 100644 index 000000000..4523d9d9c --- /dev/null +++ b/.github/scripts/publish-bundle-size.test.cjs @@ -0,0 +1,109 @@ +const { test } = require("node:test"); +const assert = require("node:assert/strict"); +const policy = require("./bundle-size-budgets.cjs"); +const publisher = require("./publish-bundle-size.cjs"); + +function fixture(permission = "write") { + const report = { + version: 1, + pr: 1, + headSha: "head", + baseSha: "base", + budgets: { + web: { javascript: { measurement: "bundle", softKiB: 35, hardKiB: 50 } }, + }, + base: { "Web\tJavaScript": 34000 }, + head: { "Web\tJavaScript": 40000 }, + measuredPlatforms: ["web"], + }; + const comment = { + id: 42, + body: "/accept-size web New checkout capability", + user: { login: "writer", type: "User" }, + created_at: "2026-10-02T12:01:00Z", + updated_at: "2026-10-02T12:01:00Z", + html_url: "https://github.com/example/repo/pull/1#issuecomment-42", + }; + const state = { acceptances: {}, processedComments: [], publishedAt: "2026-10-02T12:00:00Z" }; + const github = { + rest: { + repos: { + getCollaboratorPermissionLevel: async () => { + if (permission === "missing") + throw Object.assign(new Error("Not found"), { status: 404 }); + return { data: { permission } }; + }, + }, + }, + }; + const process = (overrides = {}) => + publisher.processCommands({ + github, + repo: {}, + comments: [comment], + state, + report, + mayAccept: true, + ...overrides, + }); + return { report, comment, state, process }; +} + +test("writer acceptance records the actor, reason and size, and is processed only once", async () => { + const f = fixture(); + const result = await f.process(); + assert.deepEqual(result.acceptances["web.javascript"], { + bytes: 40000, + measurement: "bundle", + actor: "writer", + reason: "New checkout capability", + commentId: 42, + url: f.comment.html_url, + }); + assert.equal(policy.conclusion(policy.evaluate(f.report, result.acceptances)), "success"); + Object.assign(f.state, result); + f.report.head["Web\tJavaScript"]++; + const replay = await f.process(); + assert.deepEqual(replay.acceptances, result.acceptances); + assert.equal(policy.conclusion(policy.evaluate(f.report, replay.acceptances)), "failure"); +}); + +test("unauthorized, malformed, edited and premature commands cannot accept sizes", async () => { + for (const scenario of ["read", "missing", "reason", "edited", "stale", "premature"]) { + const f = fixture(["read", "missing"].includes(scenario) ? scenario : "write"); + if (scenario === "reason") f.comment.body = "/accept-size web"; + if (scenario === "edited") f.comment.updated_at = "2026-10-02T12:02:00Z"; + if (scenario === "premature") f.state.publishedAt = "2026-10-02T12:02:00Z"; + const result = await f.process({ mayAccept: scenario !== "stale" }); + assert.deepEqual(result.acceptances, {}, scenario); + assert.deepEqual(result.processedComments, [42], scenario); + assert.ok(result.notes.length, scenario); + } +}); + +test("all pending platform commands are processed when comment events are coalesced", async () => { + const f = fixture(); + f.report.budgets.android = { aar: { measurement: "package", softKiB: 100, hardKiB: 200 } }; + f.report.head["Android\trelease AAR"] = 150 * 1024; + f.report.measuredPlatforms.push("android"); + const android = { ...f.comment, id: 43, body: "/accept-size android Native support" }; + const result = await f.process({ comments: [f.comment, android] }); + assert.equal(policy.conclusion(policy.evaluate(f.report, result.acceptances)), "success"); + assert.deepEqual(result.processedComments, [42, 43]); +}); + +test("rejects stale or mismatched reports and trusts only the Actions bot's saved state", () => { + const { report } = fixture(); + const pr = { number: 1, head: { sha: "head" }, base: { sha: "base" } }; + const run = { head_sha: "head" }; + assert.doesNotThrow(() => publisher.validateReport(report, pr, run)); + for (const field of ["headSha", "baseSha", "pr"]) { + assert.throws(() => publisher.validateReport({ ...report, [field]: "wrong" }, pr, run)); + } + for (const [login, type, trusted] of [ + ["github-actions[bot]", "Bot", true], + ["writer", "User", false], + ]) { + assert.equal(publisher.isReport({ user: { login, type }, body: policy.marker }), trusted); + } +}); diff --git a/.github/workflows/bundle-size-budgets.yml b/.github/workflows/bundle-size-budgets.yml new file mode 100644 index 000000000..331267f78 --- /dev/null +++ b/.github/workflows/bundle-size-budgets.yml @@ -0,0 +1,66 @@ +name: Bundle Size Budgets + +on: + workflow_run: + workflows: [Package Size] + types: [requested, completed] + issue_comment: + types: [created] + +permissions: + contents: read + +jobs: + resolve: + if: >- + github.event_name == 'workflow_run' || + (github.event.issue.pull_request && contains(github.event.comment.body, '/accept-size')) + runs-on: ubuntu-latest + timeout-minutes: 5 + permissions: + contents: read + pull-requests: read + outputs: + pr: ${{ steps.resolve.outputs.pr }} + steps: + # These events run trusted default-branch code. Never check out the PR here. + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ github.event.repository.default_branch }} + persist-credentials: false + - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 + id: resolve + with: + script: | + const {resolvePR} = require('./.github/scripts/publish-bundle-size.cjs'); + const pr = await resolvePR({github, context}); + if (pr) core.setOutput('pr', pr); + + report: + name: Publish size budgets + needs: resolve + if: needs.resolve.outputs.pr != '' + runs-on: ubuntu-latest + timeout-minutes: 5 + permissions: + contents: read + actions: read + checks: write + pull-requests: write + # Serialize report writes and acceptance comments for this PR. Each run + # processes all pending commands, including events coalesced by concurrency. + concurrency: + group: bundle-size-report-${{ needs.resolve.outputs.pr }} + cancel-in-progress: false + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ github.event.repository.default_branch }} + persist-credentials: false + - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 + env: + PR_NUMBER: ${{ needs.resolve.outputs.pr }} + with: + script: | + const {publish} = require('./.github/scripts/publish-bundle-size.cjs'); + await publish({github, context, core, prNumber: Number(process.env.PR_NUMBER)}); diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 945f5fe1c..bf0b4664e 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -113,6 +113,11 @@ jobs: - '.ci/changed-file-filters.yml' - '.github/workflows/ci.yml' scripts: + - '.github/scripts/measure-package-size' + - '.github/scripts/*bundle-size*' + - '.github/workflows/package-size.yml' + - '.github/workflows/bundle-size-budgets.yml' + - '.ci/bundle-size-budgets.json' - 'platforms/swift/Scripts/api' - 'platforms/swift/Scripts/normalize-api.jq' - 'scripts/lib/**' @@ -245,6 +250,7 @@ jobs: - uses: ruby/setup-ruby@e8944e80fb94b20106697132f8c20c665fab29e9 # v1.325.0 with: ruby-version: .ruby-version + - run: node --test .github/scripts/*bundle-size*.test.cjs - run: ./scripts/test_ruby - run: ./e2e/scripts/check_hide_keyboard_usage diff --git a/.github/workflows/package-size.yml b/.github/workflows/package-size.yml index 4e01b020f..2787ef269 100644 --- a/.github/workflows/package-size.yml +++ b/.github/workflows/package-size.yml @@ -2,11 +2,11 @@ name: Package Size on: pull_request: - types: [opened, synchronize, reopened, ready_for_review] + types: [opened, synchronize, reopened, ready_for_review, edited] permissions: contents: read - pull-requests: write + pull-requests: read concurrency: group: package-size-${{ github.ref }} @@ -15,7 +15,6 @@ concurrency: jobs: changes: name: Detect Changed Packages - if: github.event_name == 'pull_request' && github.event.pull_request.draft == false runs-on: ubuntu-latest timeout-minutes: 5 outputs: @@ -37,39 +36,60 @@ jobs: filters: | android: - '.github/scripts/measure-package-size' + - '.github/scripts/*bundle-size*' + - '.ci/bundle-size-budgets.json' + - '.github/workflows/bundle-size-budgets.yml' - '.ci/changed-file-filters.yml' - '.github/workflows/package-size.yml' reactNative: - '.github/actions/setup/**' - '.github/scripts/measure-package-size' + - '.github/scripts/*bundle-size*' + - '.ci/bundle-size-budgets.json' + - '.github/workflows/bundle-size-budgets.yml' - '.ci/changed-file-filters.yml' - '.github/workflows/package-size.yml' web: - '.github/actions/setup/**' - '.github/scripts/measure-package-size' + - '.github/scripts/*bundle-size*' + - '.ci/bundle-size-budgets.json' + - '.github/workflows/bundle-size-budgets.yml' - '.ci/changed-file-filters.yml' - '.github/workflows/package-size.yml' measure: name: Measure Package Size needs: changes - if: | - needs.changes.outputs.android == 'true' || - needs.changes.outputs.reactNative == 'true' || - needs.changes.outputs.web == 'true' runs-on: ubuntu-latest timeout-minutes: 30 steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: + ref: ${{ github.event.pull_request.head.sha }} submodules: true + persist-credentials: false + + - name: Check budget policy + run: node --test .github/scripts/*bundle-size*.test.cjs + + - name: Resolve current PR base + uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 + id: base + with: + script: | + const {data: pr} = await github.rest.pulls.get({ + ...context.repo, pull_number: context.issue.number, + }); + core.setOutput('sha', pr.base.sha); - name: Checkout PR base uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: - ref: ${{ github.event.pull_request.base.sha }} + ref: ${{ steps.base.outputs.sha }} path: .package-size-base submodules: true + persist-credentials: false - name: Setup base Web dependencies if: needs.changes.outputs.web == 'true' @@ -136,41 +156,21 @@ jobs: touch /tmp/package-size-base.tsv .github/scripts/measure-package-size render /tmp/package-size-base.tsv /tmp/package-size-head.tsv /tmp/package-size-comment.md - - name: Comment on PR - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 + - name: Prepare budget report env: - COMMENT_FILE: /tmp/package-size-comment.md - PR_NUMBER: ${{ github.event.pull_request.number }} - with: - script: | - const fs = require('fs'); - - const marker = ''; - const body = fs.readFileSync(process.env.COMMENT_FILE, 'utf8'); - const issue_number = Number(process.env.PR_NUMBER); - const {owner, repo} = context.repo; - - const comments = await github.paginate(github.rest.issues.listComments, { - owner, - repo, - issue_number, - per_page: 100, - }); + BASE_SHA: ${{ steps.base.outputs.sha }} + MEASURE_ANDROID: ${{ needs.changes.outputs.android }} + MEASURE_REACT_NATIVE: ${{ needs.changes.outputs.reactNative }} + MEASURE_WEB: ${{ needs.changes.outputs.web }} + run: | + mkdir -p /tmp/bundle-size-report + node .github/scripts/bundle-size-budgets.cjs /tmp/package-size-base.tsv /tmp/package-size-head.tsv .ci/bundle-size-budgets.json /tmp/bundle-size-report/report.json + cp /tmp/package-size-comment.md /tmp/bundle-size-report/comment.md - const existing = comments.find((comment) => comment.body?.includes(marker)); - - if (existing) { - await github.rest.issues.updateComment({ - owner, - repo, - comment_id: existing.id, - body, - }); - } else { - await github.rest.issues.createComment({ - owner, - repo, - issue_number, - body, - }); - } + - name: Upload measurements + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: bundle-size-report + path: /tmp/bundle-size-report/ + if-no-files-found: error + retention-days: 90 diff --git a/dev.yml b/dev.yml index 4694d9899..b27ef8170 100644 --- a/dev.yml +++ b/dev.yml @@ -69,6 +69,7 @@ open: "PRs": "https://github.com/Shopify/checkout-kit/pulls" check: + bundle-size-budget-tests: node --test .github/scripts/*bundle-size*.test.cjs ejson-plaintext: ./scripts/ejson_lint generate-env-tests: ./scripts/test_generate_env_files storefront-env-tests: ./scripts/test_setup_storefront_env