[sentry-junior]

Follow-up to #2338. pnpm dev no longer needs Vercel Queues access or a Vercel OIDC token. It delivers conversation work, plugin tasks and snapshot jobs through BullMQ on the Redis in compose.yaml. Production and tests still use Vercel Queues.

What changes

• chat/queue/bullmq.ts: a QueueProvider backed by BullMQ. The Nitro process only sends jobs. BullMQ workers run in one separate worker process (see below).
• Worker process: scripts/dev-server.mjs starts scripts/dev-queue-worker.mjs next to Nitro. It imports apps/example/server.ts (the same createApp() options as Nitro) and calls the new startQueueWorker(app) from @sentry/junior. The worker owns its BullMQ workers, queues, Redis client, state adapter and database. On SIGINT/SIGTERM it stops taking jobs, waits up to 10 seconds for running ones, then closes each worker and queue, the Redis client, state and the database. A second signal exits at once.
• Restarts: a new Junior build (packages/junior/dist, dashboard dist) restarts Nitro and the worker. A change to apps/example/server.ts, plugins.ts, dashboard.ts or apps/example/app/ restarts only the worker. Nitro-only HMR does not touch it. The supervisor starts the new worker only after the old one has exited. If the worker crashes, pnpm dev stays up and starts it again on the next watched change.
• Contract: QueueProvider gains an optional startWorker(consumers) that returns a QueueWorker with close(). registerDevConsumer becomes optional; only the Vercel provider implements it. The Vercel provider and production behavior do not change.
• State: the worker is a separate process, so the default memory state can't be shared. When pnpm dev picks BullMQ and JUNIOR_STATE_ADAPTER is unset, it uses Redis state on the queue Redis (and points REDIS_URL there, so a shared preview REDIS_URL from dev:env is never used by default). JUNIOR_STATE_ADAPTER=memory with BullMQ stops pnpm dev with an error.
• chat/queue/config.ts: JUNIOR_QUEUE_PROVIDER (vercel default, or bullmq), JUNIOR_QUEUE_REDIS_URL and JUNIOR_QUEUE_WORKER_CONCURRENCY. getQueueProvider() reads it.
• scripts/dev-server.mjs: defaults pnpm dev to bullmq on redis://localhost:6382. Explicit env wins. Other local commands (pnpm cli, ACP) and test/eval config keep Vercel, because only the dev server starts workers. apps/example/.env.example is unchanged, since tests and evals load it.
• bullmq and redis are optional peer dependencies of @sentry/junior (dev deps of packages/junior only; the example app does not list them, so the junior init scaffold stays the same). Pinned to versions older than minimumReleaseAge. msgpackr-extract: false in allowBuilds.

Mapping to the contract

• idempotencyKey → hashed jobId (BullMQ forbids : in ids, and Junior keys contain it).
• delaySeconds → delay. retentionSeconds → the worker drops jobs older than it.
• retry() returning { acknowledge: true } ends the job. Otherwise BullMQ retries with backoff (cap 50 attempts, since conversation work uses maxDeliveries: null).
• Worker concurrency defaults to 4 per queue topic (3 topics, one process). JUNIOR_QUEUE_WORKER_CONCURRENCY overrides it (positive integer). The worker logs the value at startup in queue.bullmq.worker.started.
• createCallback returns 404: nothing pushes to those routes with BullMQ.

Known differences from Vercel Queues (dev only, documented in chat/queue/README.md)

• Dedupe holds only while BullMQ still stores the first job (one hour after completion).
• BullMQ renews a running job's lock, so a slow job is not sent again when visibilityTimeoutSeconds runs out.

Production bundle
A literal import("bullmq") made Nitro bundle a ~1.7 MB BullMQ chunk plus native msgpackr files into the production function. bullmq.ts now imports by a variable name. check-vercel-output.mjs fails if bullmq or msgpackr reach a deployed function.

Verification (head 920d5c483, rebased on main at 7b5558468)

• pnpm dev with no .env.local, no VERCEL_OIDC_TOKEN, and no state setting: one dev-queue-worker.mjs process starts and logs app.queue.worker_concurrency=4. POST /api/conversations → 200 accepted → conversation.work.lease.acquired in the worker → the turn runs until the model call fails on auth (no gateway key in the sandbox) → conversation.work.completed.
• touch apps/example/plugins.ts: the worker restarted with a new pid, the Nitro pid stayed the same, and only one worker process was running afterwards.
• SIGTERM to the worker: exit code 0 in about 0.5s. With process.exit removed, the process still exited by itself, so no Redis, database or timer handles stay open. SIGINT to the dev server group stopped Nitro, the worker and the watchers.
• tests/integration/queue-bullmq.test.ts against real Redis: dedupe with : keys, retries then acknowledge, concurrency, and a new case that close() waits for a running job.
• @sentry/junior unit + component suites (312 files, 2339 tests), chat-config tests (concurrency default, override and invalid value), load-env-files.test.mjs, init-cli.test.ts, typecheck, @sentry/junior lint, ast-grep:lint, package:lint, Prettier, file-length:check.
• nitro build (Vercel preset) of the example app, then check-vercel-output.mjs passes. The full apps/example pnpm build needs Vercel credentials for junior snapshot create, so I ran nitro build directly.

Not in this PR

• Queue isolation between concurrent worktrees that share one Redis.
• Making createCallback optional.
• The worker's traces don't go to the Spotlight server that sentry local run starts for Nitro. Its logs print in the pnpm dev terminal.

Open questions

• bullmq.ts still caches the Redis client and queues in module state, but the worker's close() now releases and resets them. app.ts keeps a WeakMap from each app to its queue consumers so startQueueWorker(app) can find them without changing server.ts (which the junior init scaffold test pins). Is that acceptable, or would you rather createApp() return the consumers some other way?
• In the Nitro process, the BullMQ sender's Redis client lives until the process exits. A Junior build restarts that process, and Nitro HMR doesn't reload @sentry/junior, so it isn't duplicated. I didn't add a Nitro close hook for it.
• Should pnpm dev default to BullMQ (as in this PR), or should it stay opt-in until more people have tried it?

--

View Junior Session [Sentry]

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
junior-docs	Ready	Preview	Oct 30, 2026 4:14am UTC

[sentry-evals]

Sentry Evals

junior-behavioral

Run	Status	Passed	Failed	Errored	Cost	Tokens	Duration
Head	Error	79	2	1	$0.71	0	631.69s

Base commit 8c45aeb has no eval run to compare against.

junior-integration

Run	Status	Passed	Failed	Errored	Cost	Tokens	Duration
Head	Passed	102	0	0	$0.60	0	351.11s

Base commit 8c45aeb has no eval run to compare against.

[thomasjiangcy]

I tested the current head locally. The BullMQ delivery path works: the Redis integration tests pass, pnpm dev selects BullMQ, a real Conversation request was accepted, the worker acquired its lease, and the Turn completed.

I do not think the PR should proceed with the current worker lifecycle.

The development consumers start inside createApp(). Each registration returns a stop function, but createApp() discards those functions. The BullMQ provider also keeps its loaded Redis connection and queue map in module state. No app or Nitro close hook stops the workers, closes the queues, or disconnects Redis.

This creates a concrete Nitro HMR problem. An in-process reload can evaluate the app again and register another set of workers while the old workers remain active. Old code can then receive new jobs. A full child-process restart cleans up through process exit, but that should not be the normal resource lifecycle.

I suggest that local BullMQ consumers run in one sibling worker process managed by scripts/dev-server.mjs, rather than inside the Nitro request process:

• Nitro accepts requests and sends queue messages.
• The worker process owns the BullMQ workers, queues, Redis connection, SQL executor, and state adapter that it uses.
• The development supervisor starts and restarts Nitro and the worker.
• On SIGINT or SIGTERM, the worker stops accepting jobs, awaits active work with a bounded shutdown period, closes each worker and queue, disconnects Redis, and closes its other resources.
• Junior runtime or plugin configuration changes restart the worker. Nitro-only HMR does not.

The PR also sets concurrency to 20 for every BullMQ worker. There are three queue topics, so one process can admit up to 60 active jobs. For development, I suggest one worker process and a default BullMQ concurrency of 4 per queue. Please allow a positive integer override through an environment variable such as JUNIOR_QUEUE_WORKER_CONCURRENCY. The effective value should be logged at startup and documented as per-queue concurrency.

This can remain a development-only change. The shared queue contract and production Vercel Queue behavior do not need to change.

I would keep concurrent-worktree queue isolation outside this PR. The blocking issue here is clear ownership and cleanup of the local workers, plus a safe development concurrency default.

[thomasjiangcy]

Thanks for the update. I checked the new head locally. It addresses the main points from the first review:

• BullMQ consumers now run in one sibling worker process.
• Nitro only sends jobs.
• The default concurrency is 4 per queue, with a validated JUNIOR_QUEUE_WORKER_CONCURRENCY override.
• A real Conversation completed through BullMQ.
• A watched app change replaced the worker without leaving a duplicate process.
• The focused queue and config tests pass.

I still think this needs changes before it can proceed.

1. The worker shutdown timeout is not effective. closeWorker() first calls worker.close(). BullMQ 6.5.0 then stores that close promise. If the ten-second timer expires, worker.close(true) returns the same promise, so it does not change the close to a forced close. A long-running job can therefore block a worker restart without limit. Please enforce the grace period at the child-process boundary. Send SIGTERM, then terminate the worker process if it has not exited after the grace period.

2. The change still adds mutable runtime globals. loaded and queues in bullmq.ts own the Redis connection and queues. appQueueConsumers in app.ts is another mutable module-level registry. This conflicts with the repository rule against mutable runtime globals. Please move these resources into an explicit provider or worker instance that owns its dependencies and exposes its lifecycle.

3. Cleanup should attempt every step. In startQueueWorker(app).close(), a failure from worker.close() prevents state and database cleanup. Please use finally or an all-settled cleanup path so every owned resource gets a cleanup attempt.

The PR also currently conflicts with main (packages/junior/package.json and pnpm-lock.yaml). Please rebase or merge main, resolve the conflicts, and rerun the checks. The external eval status is also red, with 74 passed, 2 failed, and 2 errors, so that result needs review after the branch is current.

[github-actions]

106 screenshot changes — 0 changed · 106 added · 0 removed

Review screenshots in Frameshift

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit f5a7b3f. Configure here.}