Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .claude/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -613,6 +613,29 @@ download models or call live APIs); `WORKGLOW_TEST_TARGET=dist` turns the rewrit
exercise the bundles instead, and refuses coverage there. The include/exclude globs are
repo-root-relative, so coverage and `--project` cannot be combined.

The rewrite is attached to every project unconditionally, so with the default in force NO
vitest job resolves a `@workglow/*` specifier through `exports` — and the blocking workflow
runs no `bun test` job at all. The nightly Bun parity run does resolve `exports` natively,
but it is informational, never blocks a merge, runs on a cron and excludes six sections. So
the `test-vitest-dist` job in `.github/workflows/test.yml` is what keeps bundle integrity
blocking: it reuses the `build-output` artifact and runs the unit tier under
`WORKGLOW_TEST_TARGET=dist`.

That job runs the UNIT tier only, which on its own would make the blocking check
**tier-shaped**: a bundle reachable only from an `.integration.test.ts` file would be
loaded by no blocking job at all. What closes that gap is
`packages/test/src/test/util/PublishedEntryImports.test.ts`, a unit-tier file that
enumerates every workspace manifest's `exports`, resolves each subpath under the Node
conditions, and dynamically imports the result, asserting the module is non-empty. Under
`WORKGLOW_TEST_TARGET=dist` that one file loads every published bundle regardless of which
tier a suite happens to exercise it from; under the default target it costs nothing,
because it loads the same sources the rest of the suite already does. Its exemption maps
are deliberately tiny and each entry states its reason, so a new package defaults to being
checked and a stale exemption fails the test rather than silently exempting nothing.
Adding a published `exports` subpath therefore needs no CI change, and importing
`workglow` — the meta-package `packages/test` devDepends on — is what pulls the provider
bundles into the sweep transitively.

## Developing without building

`bun run use-source` makes every package resolve to its source. It does **not** touch
Expand Down
34 changes: 34 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,39 @@ jobs:
# of files, and the shards past the first then hold none of them.
- name: Run unit tests via vitest (shard ${{ matrix.shard }})
run: bun run test:vitest:unit -- --shard ${{ matrix.shard }} ${{ env.TEST_CHANGED }}

# The one blocking job that resolves `@workglow/*` through `exports` rather
# than through the source-rewrite plugin. Every other vitest job here runs
# against `src`, and there is no `bun test` job in this workflow at all, so
# without this a `bun build` entry that silently dropped a re-export would
# reach main and surface only in the nightly parity run, which never blocks.
# Unsharded: it exists to load every bundle once, and the artifact download
# dominates its runtime either way.
test-vitest-dist:
runs-on: ubuntu-latest
needs: build
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
- uses: actions/setup-node@v6
with:
node-version: 24
- uses: oven-sh/setup-bun@v2
with:
bun-version: 1.4.0
- run: bun i
- name: Download build artifacts
uses: actions/download-artifact@v8
with:
name: build-output
path: .
# Invokes the script rather than restating the variable: one definition
# of "the dist target". An inline `env:` block here is how a workflow
# edit silently demotes this job to a duplicate source run — nothing
# in-process can notice, since an unset target resolves to `source`.
- name: Run unit tests against the built bundles
run: bun run test:vitest:dist -- ${{ env.TEST_CHANGED }}
test-vitest-integration:
runs-on: ubuntu-latest
needs: build
Expand Down Expand Up @@ -307,6 +340,7 @@ jobs:
needs:
[
test-vitest-unit,
test-vitest-dist,
test-vitest-integration,
test-vitest-rag,
test-vitest-ai-provider-hft,
Expand Down
1 change: 1 addition & 0 deletions bun.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
"build:types": "turbo run build-types --concurrency=15",
"typecheck:budget": "bun scripts/typecheck-budget.ts",
"typecheck:tests": "for f in packages/*/tsconfig.test.json; do echo \"typecheck $f\" && tsc -p \"$f\" || exit 1; done",
"typecheck:scripts": "tsc -p tsconfig.scripts.json",
"clean": "rm -rf node_modules packages/*/node_modules packages/*/*tsbuildinfo packages/*/dist packages/*/src/**/*\\.d\\.ts packages/*/src/**/*\\.map integrations/*/node_modules integrations/*/dist integrations/*/src/**/*\\.d\\.ts integrations/*/src/**/*\\.map examples/*/node_modules examples/*/dist examples/*/src/**/*\\.d\\.ts examples/*/src/**/*\\.map .turbo */*/.turbo",
"dev": "turbo run dev --concurrency=15",
"docs": "typedoc",
Expand All @@ -43,6 +44,7 @@
"test:bun:ai-provider-nodellama": "bun scripts/test.ts bun integration provider-nodellama",
"test:bun:ai-provider-api": "bun scripts/test.ts bun integration ai provider-api",
"test:vitest:unit": "bun scripts/test.ts vitest unit",
"test:vitest:dist": "WORKGLOW_TEST_TARGET=dist bun scripts/test.ts vitest unit",
"test:vitest:integration": "bun scripts/test.ts vitest integration --except rag,browser,provider-hft,provider-nodellama,provider-api,provider-cactus",
"test:vitest:rag": "bun scripts/test.ts vitest integration rag",
"test:vitest:ai-provider": "bun scripts/test.ts vitest integration ai provider",
Expand Down
3 changes: 2 additions & 1 deletion packages/test/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@
"node-llama-cpp": "catalog:",
"pg": "catalog:",
"playwright": "catalog:",
"vitest": "catalog:"
"vitest": "catalog:",
"workglow": "workspace:*"
}
}
219 changes: 219 additions & 0 deletions packages/test/src/test/util/PublishedEntryExportParity.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,219 @@
/**
* @license
* Copyright 2026 Steven Roussey <sroussey@gmail.com>
* SPDX-License-Identifier: Apache-2.0
*/

// @vitest-environment node
//
// The tag is what keeps this file off the Bun runner, which is the point of it
// here rather than any DOM: the target this sweep branches on is handed down by
// `vitest.config.ts`'s `test.env`, and `bun test` never reads that config. Under
// Bun the validated value would simply be absent, and the anti-vacuity case
// below would fail on the plumbing instead of on anything under test. `node` is
// vitest's default environment, so the tag costs nothing there.

import { existsSync, readdirSync, readFileSync } from "node:fs";
import { dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import { describe, expect, it } from "vitest";

const repoRoot = resolve(dirname(fileURLToPath(import.meta.url)), "../../../../..");

/**
* The workspace groups, derived from the root manifest's `workspaces` field.
*
* Duplicated derivation code rather than a duplicated list, for the reason
* `PublishedEntryImports.test.ts` spells out: `packages/test` is a `composite`
* project rooted at `./src`, so importing the tooling's own copy under
* `scripts/` would pull those files into its program and break `build-types`.
*/
const WORKSPACE_GROUPS: readonly string[] = (
JSON.parse(readFileSync(join(repoRoot, "package.json"), "utf8")) as { workspaces: string[] }
).workspaces.map((pattern) => pattern.replace(/^\.\//, "").replace(/\/?\*.*$/, ""));

/**
* The conditions a plain `import` from Node activates, and only those. Node's
* own algorithm: walk the keys IN ORDER and take the first active one, so a
* `types`-first block resolves to whatever follows it rather than to nothing.
*/
const NODE_CONDITIONS: ReadonlySet<string> = new Set(["node", "import", "default"]);

/** Source extensions a built entry can have come from, in resolution order. */
const SOURCE_EXTENSIONS = [".ts", ".tsx"] as const;

/**
* Entries no CI job can import, each with the reason — the same map, for the
* same entry and the same reason, as `PublishedEntryImports.test.ts`. Kept
* deliberately small so a NEW package defaults to being checked. Every key is
* proved below to still name a published entry.
*/
const UNCHECKABLE: Readonly<Record<string, string>> = {
"@workglow/cli":
"an example app, and `packages/test` does not depend on it — under bunfig's " +
"isolated linker the specifier does not resolve from here at all, so importing " +
"it would test the linker rather than the bundle",
};

interface EntryPair {
/** The specifier a consumer writes, e.g. `@workglow/util/schema`. */
readonly specifier: string;
/** Absolute path of the source module the built entry was built from. */
readonly sourcePath: string;
}

/** The target a conditional `exports` value resolves to under Node. */
function resolveUnderNode(value: unknown): string | undefined {
if (typeof value === "string") return value;
if (typeof value !== "object" || value === null || Array.isArray(value)) return undefined;
for (const [condition, child] of Object.entries(value as Record<string, unknown>)) {
if (!NODE_CONDITIONS.has(condition)) continue;
const resolved = resolveUnderNode(child);
if (resolved !== undefined) return resolved;
}
return undefined;
}

/**
* `<pkg>/dist/<entry>.js` → `<pkg>/src/<entry>.{ts,tsx}` — the inverse of what
* the build emits, and the same mapping `use-source` writes its stubs from.
*
* Re-derived here rather than imported from `scripts/lib/workspaceSource.ts`
* for the composite-project reason above. Deliberately returns `undefined`
* rather than guessing for a target with no counterpart (generated or copied
* build output), so such an entry is reported as skipped instead of failing.
*/
function sourceCounterpart(packageDir: string, target: string): string | undefined {
const match = /^\.\/dist\/(?<entry>.+)\.(?:js|mjs|cjs)$/.exec(target);
if (!match?.groups) return undefined;
for (const extension of SOURCE_EXTENSIONS) {
const candidate = join(packageDir, "src", `${match.groups.entry}${extension}`);
if (existsSync(candidate)) return candidate;
}
return undefined;
}

/** Every published Node entry that has a source counterpart to compare against. */
function collectEntryPairs(): { pairs: EntryPair[]; unmapped: string[] } {
const pairs: EntryPair[] = [];
const unmapped: string[] = [];
for (const group of WORKSPACE_GROUPS) {
let directories: string[];
try {
directories = readdirSync(join(repoRoot, group));
} catch {
continue;
}
for (const directory of directories) {
const packageDir = join(repoRoot, group, directory);
let manifest: { name?: unknown; exports?: unknown };
try {
manifest = JSON.parse(readFileSync(join(packageDir, "package.json"), "utf8"));
} catch {
continue; // not a package directory
}
const { name, exports } = manifest;
if (typeof name !== "string") continue;
if (typeof exports !== "object" || exports === null || Array.isArray(exports)) continue;
for (const [subpath, value] of Object.entries(exports as Record<string, unknown>)) {
if (!subpath.startsWith(".")) continue; // a bare condition map, not a subpath
const target = resolveUnderNode(value);
if (target === undefined) continue; // browser-only entry; nothing for Node to load
const specifier = name + subpath.slice(1);
const sourcePath = sourceCounterpart(packageDir, target);
if (sourcePath === undefined) {
unmapped.push(`${specifier} -> ${target}`);
continue;
}
pairs.push({ specifier, sourcePath });
}
}
}
pairs.sort((a, b) => a.specifier.localeCompare(b.specifier));
return { pairs, unmapped };
}

const { pairs, unmapped } = collectEntryPairs();
const checkable = pairs.filter((pair) => !(pair.specifier in UNCHECKABLE));

/**
* Whether this run exercises the built bundles.
*
* The VALIDATED target handed down by `vitest.config.ts`'s `test.env`, not
* re-derived: `packages/test` is a composite program rooted at `./src` and
* cannot import `scripts/lib/*`, so a second `=== "dist"` comparison here would
* be exactly the silent-typo bug `resolveTestTarget` exists to remove.
*/
const RUNS_AGAINST_BUNDLES = process.env.WORKGLOW_TEST_TARGET === "dist";

/**
* Every published entry, imported twice — once by the specifier a consumer
* writes, once by the source file it was built from — with the export NAME sets
* compared.
*
* Why names, and why both sides: a bundle that lost a re-export still resolves
* and still evaluates cleanly, so `PublishedEntryImports.test.ts`'s "loads and
* exports something" check passes over it unchanged. That test's `> 0` bound is
* satisfied by a bundle carrying one symbol out of ninety. The export list is
* the only observable that says the entry point is intact, and the source file
* is the only available statement of what it should be.
*
* Under the default `source` target the source-resolving plugin rewrites
* `import(specifier)` to exactly the path `sourceCounterpart()` computes, so
* both sides ARE the same module and every case asserts `X === X`. Those cases
* are therefore SKIPPED under source, so the report distinguishes "checked"
* from "not applicable" rather than showing ~90 green rows that compared
* nothing. The run that means something is `test-vitest-dist`
* (`WORKGLOW_TEST_TARGET=dist`), where the left side is the real bundle; hence
* a unit-tier file, since that is the tier the dist job runs.
*
* Skipping loses no loading: `PublishedEntryImports.test.ts` imports every
* published specifier unconditionally, and that file DOES carry signal under
* source — it is what proves each entry resolves and evaluates at all.
*/
describe("published entry export parity", () => {
it("enumerates every workspace manifest, so an empty sweep cannot pass", () => {
// Anti-vacuity: a typo in the walk yields a short list rather than an
// error, and a short list passes every assertion below.
expect(pairs.length).toBeGreaterThan(60);
});

it("maps every published entry back to a source file", () => {
// A published entry whose target is not `./dist/<entry>.js`, or whose
// source twin is missing, is silently dropped from the sweep above — the
// same hole in a different shape. Listed here so it fails loudly instead.
expect(unmapped).toEqual([]);
});

it("keeps every exemption pinned to an entry that still exists", () => {
// An exemption that outlives its package silently exempts nothing, and
// reads as if a real hole were still open.
const published = new Set(pairs.map((pair) => pair.specifier));
expect(Object.keys(UNCHECKABLE).filter((specifier) => !published.has(specifier))).toEqual([]);
for (const reason of Object.values(UNCHECKABLE)) {
expect(reason.length).toBeGreaterThan(20);
}
});

it("is handed a validated target, so the skip cannot swallow the dist sweep", () => {
// If the `test.env` plumbing broke, `RUNS_AGAINST_BUNDLES` would be false
// in every job: every case below would report skipped and
// `test-vitest-dist` would go green having compared nothing at all.
expect(["source", "dist"]).toContain(process.env.WORKGLOW_TEST_TARGET);
});

it
.skipIf(!RUNS_AGAINST_BUNDLES)
.each(checkable.map((pair) => [pair.specifier, pair.sourcePath] as const))(
"%s exports the same names as its source",
async (specifier, sourcePath) => {
const [published, source] = await Promise.all([

Check failure on line 210 in packages/test/src/test/util/PublishedEntryExportParity.test.ts

View workflow job for this annotation

GitHub Actions / test-vitest-dist

[test] src/test/util/PublishedEntryExportParity.test.ts > published entry export parity > @workglow/a2ui/catalog exports the same names as its source

Error: [workglow:workspace-source] cannot resolve "@workglow/a2ui/catalog" (imported from /home/runner/work/libs/libs/packages/test/src/test/util/PublishedEntryExportParity.test.ts). It is owned by the workspace package @workglow/a2ui. Resolution goes through that package's "exports", which point at ./dist/*, so the built entry has to exist even though this plugin then rewrites it to src. @workglow/test does not list @workglow/a2ui in any of its dependency blocks. bunfig sets linker = "isolated", so a workspace package resolves only what it declares: nothing is linked into /home/runner/work/libs/libs/packages/test/node_modules for this specifier no matter what @workglow/a2ui has built. Add @workglow/a2ui to @workglow/test's package.json and re-run `bun i`. ❯ src/test/util/PublishedEntryExportParity.test.ts:210:35

Check failure on line 210 in packages/test/src/test/util/PublishedEntryExportParity.test.ts

View workflow job for this annotation

GitHub Actions / test-vitest-dist

[test] src/test/util/PublishedEntryExportParity.test.ts > published entry export parity > @workglow/a2ui/catalog exports the same names as its source

Error: [workglow:workspace-source] cannot resolve "@workglow/a2ui/catalog" (imported from /home/runner/work/libs/libs/packages/test/src/test/util/PublishedEntryExportParity.test.ts). It is owned by the workspace package @workglow/a2ui. Resolution goes through that package's "exports", which point at ./dist/*, so the built entry has to exist even though this plugin then rewrites it to src. @workglow/test does not list @workglow/a2ui in any of its dependency blocks. bunfig sets linker = "isolated", so a workspace package resolves only what it declares: nothing is linked into /home/runner/work/libs/libs/packages/test/node_modules for this specifier no matter what @workglow/a2ui has built. Add @workglow/a2ui to @workglow/test's package.json and re-run `bun i`. ❯ src/test/util/PublishedEntryExportParity.test.ts:210:35

Check failure on line 210 in packages/test/src/test/util/PublishedEntryExportParity.test.ts

View workflow job for this annotation

GitHub Actions / test-vitest-dist

[test] src/test/util/PublishedEntryExportParity.test.ts > published entry export parity > @workglow/a2a/util exports the same names as its source

Error: [workglow:workspace-source] cannot resolve "@workglow/a2a/util" (imported from /home/runner/work/libs/libs/packages/test/src/test/util/PublishedEntryExportParity.test.ts). It is owned by the workspace package @workglow/a2a. Resolution goes through that package's "exports", which point at ./dist/*, so the built entry has to exist even though this plugin then rewrites it to src. @workglow/test does not list @workglow/a2a in any of its dependency blocks. bunfig sets linker = "isolated", so a workspace package resolves only what it declares: nothing is linked into /home/runner/work/libs/libs/packages/test/node_modules for this specifier no matter what @workglow/a2a has built. Add @workglow/a2a to @workglow/test's package.json and re-run `bun i`. ❯ src/test/util/PublishedEntryExportParity.test.ts:210:35

Check failure on line 210 in packages/test/src/test/util/PublishedEntryExportParity.test.ts

View workflow job for this annotation

GitHub Actions / test-vitest-dist

[test] src/test/util/PublishedEntryExportParity.test.ts > published entry export parity > @workglow/a2a/util exports the same names as its source

Error: [workglow:workspace-source] cannot resolve "@workglow/a2a/util" (imported from /home/runner/work/libs/libs/packages/test/src/test/util/PublishedEntryExportParity.test.ts). It is owned by the workspace package @workglow/a2a. Resolution goes through that package's "exports", which point at ./dist/*, so the built entry has to exist even though this plugin then rewrites it to src. @workglow/test does not list @workglow/a2a in any of its dependency blocks. bunfig sets linker = "isolated", so a workspace package resolves only what it declares: nothing is linked into /home/runner/work/libs/libs/packages/test/node_modules for this specifier no matter what @workglow/a2a has built. Add @workglow/a2a to @workglow/test's package.json and re-run `bun i`. ❯ src/test/util/PublishedEntryExportParity.test.ts:210:35

Check failure on line 210 in packages/test/src/test/util/PublishedEntryExportParity.test.ts

View workflow job for this annotation

GitHub Actions / test-vitest-dist

[test] src/test/util/PublishedEntryExportParity.test.ts > published entry export parity > @workglow/a2a/tasks exports the same names as its source

Error: [workglow:workspace-source] cannot resolve "@workglow/a2a/tasks" (imported from /home/runner/work/libs/libs/packages/test/src/test/util/PublishedEntryExportParity.test.ts). It is owned by the workspace package @workglow/a2a. Resolution goes through that package's "exports", which point at ./dist/*, so the built entry has to exist even though this plugin then rewrites it to src. @workglow/test does not list @workglow/a2a in any of its dependency blocks. bunfig sets linker = "isolated", so a workspace package resolves only what it declares: nothing is linked into /home/runner/work/libs/libs/packages/test/node_modules for this specifier no matter what @workglow/a2a has built. Add @workglow/a2a to @workglow/test's package.json and re-run `bun i`. ❯ src/test/util/PublishedEntryExportParity.test.ts:210:35

Check failure on line 210 in packages/test/src/test/util/PublishedEntryExportParity.test.ts

View workflow job for this annotation

GitHub Actions / test-vitest-dist

[test] src/test/util/PublishedEntryExportParity.test.ts > published entry export parity > @workglow/a2a/tasks exports the same names as its source

Error: [workglow:workspace-source] cannot resolve "@workglow/a2a/tasks" (imported from /home/runner/work/libs/libs/packages/test/src/test/util/PublishedEntryExportParity.test.ts). It is owned by the workspace package @workglow/a2a. Resolution goes through that package's "exports", which point at ./dist/*, so the built entry has to exist even though this plugin then rewrites it to src. @workglow/test does not list @workglow/a2a in any of its dependency blocks. bunfig sets linker = "isolated", so a workspace package resolves only what it declares: nothing is linked into /home/runner/work/libs/libs/packages/test/node_modules for this specifier no matter what @workglow/a2a has built. Add @workglow/a2a to @workglow/test's package.json and re-run `bun i`. ❯ src/test/util/PublishedEntryExportParity.test.ts:210:35

Check failure on line 210 in packages/test/src/test/util/PublishedEntryExportParity.test.ts

View workflow job for this annotation

GitHub Actions / test-vitest-dist

[test] src/test/util/PublishedEntryExportParity.test.ts > published entry export parity > @workglow/a2a/server exports the same names as its source

Error: [workglow:workspace-source] cannot resolve "@workglow/a2a/server" (imported from /home/runner/work/libs/libs/packages/test/src/test/util/PublishedEntryExportParity.test.ts). It is owned by the workspace package @workglow/a2a. Resolution goes through that package's "exports", which point at ./dist/*, so the built entry has to exist even though this plugin then rewrites it to src. @workglow/test does not list @workglow/a2a in any of its dependency blocks. bunfig sets linker = "isolated", so a workspace package resolves only what it declares: nothing is linked into /home/runner/work/libs/libs/packages/test/node_modules for this specifier no matter what @workglow/a2a has built. Add @workglow/a2a to @workglow/test's package.json and re-run `bun i`. ❯ src/test/util/PublishedEntryExportParity.test.ts:210:35

Check failure on line 210 in packages/test/src/test/util/PublishedEntryExportParity.test.ts

View workflow job for this annotation

GitHub Actions / test-vitest-dist

[test] src/test/util/PublishedEntryExportParity.test.ts > published entry export parity > @workglow/a2a/server exports the same names as its source

Error: [workglow:workspace-source] cannot resolve "@workglow/a2a/server" (imported from /home/runner/work/libs/libs/packages/test/src/test/util/PublishedEntryExportParity.test.ts). It is owned by the workspace package @workglow/a2a. Resolution goes through that package's "exports", which point at ./dist/*, so the built entry has to exist even though this plugin then rewrites it to src. @workglow/test does not list @workglow/a2a in any of its dependency blocks. bunfig sets linker = "isolated", so a workspace package resolves only what it declares: nothing is linked into /home/runner/work/libs/libs/packages/test/node_modules for this specifier no matter what @workglow/a2a has built. Add @workglow/a2a to @workglow/test's package.json and re-run `bun i`. ❯ src/test/util/PublishedEntryExportParity.test.ts:210:35
import(/* @vite-ignore */ specifier) as Promise<Record<string, unknown>>,
import(/* @vite-ignore */ sourcePath) as Promise<Record<string, unknown>>,
]);
// Sorted, so the diff on failure names the missing symbols rather than
// reporting two shuffled lists as unequal.
expect(Object.keys(published).sort()).toEqual(Object.keys(source).sort());
}
);
});
Loading
Loading