Skip to content

feat: Concurrent sessions, a vitest integration, and an npm package - #28

Merged
dcramer merged 2 commits into
mainfrom
feat/vitest-concurrent-sessions
Nov 4, 2026
Merged

dcramer merged 2 commits into
mainfrom
feat/vitest-concurrent-sessions

Conversation

@sentry-junior

@sentry-junior sentry-junior Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Setting up Warden showed that each Roach consumer has to write its own glue. Junior copies the client (packages/junior-evals/src/roach/, "delete it when @sentry/roach is on npm") and has about 300 lines of vitest setup. Roach also allows one open session per run, so Junior runs its evals in one worker. Warden's 63 evals take about 45 minutes in parallel workers, so run one at a time they would not fit in its 90-minute job.

This change moves that glue into Roach. A consumer now needs only config.

Concurrent sessions. POST /session returns { url }, the proxy URL of the session (user <run id>-<session id>). A request through that URL belongs to that session, even when other sessions are open. A request through the run URL still belongs to the open session when only one is open. The action, and Junior's copied client, therefore work as before. A run can have 64 open sessions, and it keeps at most 1000 ended ones.

vitest integration. It has two entries:

globalSetup: ["@sentry/roach/vitest"],
setupFiles: ["@sentry/roach/vitest/setup"],

The global setup starts one run from roach.json. It reads ROACH_TOKEN, so workers do not get the token, and also reads ROACH_URL and ROACH_MODE. It masks the run token in CI, and it gives the workers the proxy variables and the CA. At the end it reports the run to the job summary. The setup file opens one session for each test. It points fetch, http, and child processes at the session URL with http.setGlobalProxyFromEnv (Node 24.14+). It also fails a test when replay mode missed one of its requests. A project that has its own global setup calls startRoach(project). With ROACH_MODE=off the global setup starts no run, so local runs without a token go direct. Before, the service refused off without the write token.

roach.json. The action and vitest now share one loader (src/config.ts). It also takes tenant, secretEnv, noProxy, and { "builtin": "uuid" } for VALUE_PATTERNS. Without { "builtin": ... }, a JSON config would have to repeat regexes such as gitCommit inline.

npm. The client no longer imports server.ts and service.ts, because CONTROL_PATH, RunConfig, and RemoteRun moved to types.ts. The package is now public, and Craft publishes the tarball from a new package CI job. I removed preReleaseCommand: "" so that Craft writes the version to package.json, and I removed bin. The package has TypeScript sources, as the action does. Vitest compiles them. Plain Node cannot strip types in node_modules, so a JS build is a follow-up for non-vitest consumers.

tests/vitest.test.ts runs vitest in a child process on a two-file project, against an in-process service. It covers per-test sessions in parallel workers, child processes, dropped recordings from failed tests, a replay with no live calls, a replay miss that fails a test, and off mode with no service. I also checked a pnpm pack tarball installed into a scratch vitest 4 project, with globalSetup: ["@sentry/roach/vitest"].

Notes:

via @dcramer.

--

View Session [Sentry]

Roach needed custom glue in each consumer: Junior copies the client and
has about 300 lines of vitest setup, and one session per run forced its
evals to run in one worker.

- A run can have many open sessions. Each session has its own proxy
  URL, so test workers record at the same time. A request through the
  run URL still belongs to the only open session.
- @sentry/roach/vitest and @sentry/roach/vitest/setup start a run from
  roach.json and open a session for each test.
- roach.json is shared by the action and vitest, and adds tenant,
  secretEnv, noProxy, and built-in value patterns.
- The client no longer imports server code. Craft publishes the
  package to npm.

Co-Authored-By: David Cramer <23610+dcramer@users.noreply.github.com>
ROACH_MODE=off now starts no run, so requests go direct. Local eval runs
without a write token were refused by the service before, since off mode
needs the token.

Co-Authored-By: David Cramer <23610+dcramer@users.noreply.github.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 956b0c5. Configure here.

Comment thread src/config.ts Resolved
@dcramer
dcramer merged commit 6f0fcc9 into main Nov 4, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant