Repository navigation
feat: Concurrent sessions, a vitest integration, and an npm package - #28
Merged
Merged
Conversation
Roach needed custom glue in each consumer: Junior copies the client and has about 300 lines of vitest setup, and one session per run forced its evals to run in one worker. - A run can have many open sessions. Each session has its own proxy URL, so test workers record at the same time. A request through the run URL still belongs to the only open session. - @sentry/roach/vitest and @sentry/roach/vitest/setup start a run from roach.json and open a session for each test. - roach.json is shared by the action and vitest, and adds tenant, secretEnv, noProxy, and built-in value patterns. - The client no longer imports server code. Craft publishes the package to npm. Co-Authored-By: David Cramer <23610+dcramer@users.noreply.github.com>
ROACH_MODE=off now starts no run, so requests go direct. Local eval runs without a write token were refused by the service before, since off mode needs the token. Co-Authored-By: David Cramer <23610+dcramer@users.noreply.github.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 956b0c5. Configure here.
3 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Setting up Warden showed that each Roach consumer has to write its own glue. Junior copies the client (
packages/junior-evals/src/roach/, "delete it when@sentry/roachis on npm") and has about 300 lines of vitest setup. Roach also allows one open session per run, so Junior runs its evals in one worker. Warden's 63 evals take about 45 minutes in parallel workers, so run one at a time they would not fit in its 90-minute job.This change moves that glue into Roach. A consumer now needs only config.
Concurrent sessions.
POST /sessionreturns{ url }, the proxy URL of the session (user<run id>-<session id>). A request through that URL belongs to that session, even when other sessions are open. A request through the run URL still belongs to the open session when only one is open. The action, and Junior's copied client, therefore work as before. A run can have 64 open sessions, and it keeps at most 1000 ended ones.vitest integration. It has two entries:
The global setup starts one run from
roach.json. It readsROACH_TOKEN, so workers do not get the token, and also readsROACH_URLandROACH_MODE. It masks the run token in CI, and it gives the workers the proxy variables and the CA. At the end it reports the run to the job summary. The setup file opens one session for each test. It pointsfetch,http, and child processes at the session URL withhttp.setGlobalProxyFromEnv(Node 24.14+). It also fails a test when replay mode missed one of its requests. A project that has its own global setup callsstartRoach(project). WithROACH_MODE=offthe global setup starts no run, so local runs without a token go direct. Before, the service refusedoffwithout the write token.roach.json. The action and vitest now share one loader (src/config.ts). It also takestenant,secretEnv,noProxy, and{ "builtin": "uuid" }forVALUE_PATTERNS. Without{ "builtin": ... }, a JSON config would have to repeat regexes such asgitCommitinline.npm. The client no longer imports
server.tsandservice.ts, becauseCONTROL_PATH,RunConfig, andRemoteRunmoved totypes.ts. The package is now public, and Craft publishes the tarball from a newpackageCI job. I removedpreReleaseCommand: ""so that Craft writes the version topackage.json, and I removedbin. The package has TypeScript sources, as the action does. Vitest compiles them. Plain Node cannot strip types innode_modules, so a JS build is a follow-up for non-vitest consumers.tests/vitest.test.tsruns vitest in a child process on a two-file project, against an in-process service. It covers per-test sessions in parallel workers, child processes, dropped recordings from failed tests, a replay with no live calls, a replay miss that fails a test, andoffmode with no service. I also checked apnpm packtarball installed into a scratch vitest 4 project, withglobalSetup: ["@sentry/roach/vitest"].Notes:
.craft.yml.@sentry/roachpackage to be allowed in getsentry/publish.via @dcramer.
--
View Session [Sentry]