feat: cap the jobs one token may hold

.claude/rules/worker.md

@@ -16,10 +16,10 @@ paths:
   Test-only tasks exist only with `BRIDGE_TEST_TASKS=1` (compose's `fault-worker`).
 - `MODEL_NAMES` is what every result reports as `model` and what the MCP server's `BACKENDS` table must
   equal (`tests/mcp/test_models.py`); change both together.
-- Limits live in three places: concurrency, capacity and retention are `Settings` in `jobs.py` (defaults
-  from the environment in `server.main`), the body size and the token limits are `MAX_BODY`, `MAX_TOKENS`,
-  `MAX_TOKENS_FILE` and `TOKEN_NAME` in `server.py`, and the field, count and character limits are the
-  `MAX_*` constants in `tasks.py`; the deadline range is checked in `JobStore.submit`. `docs/protocol.md`
+- Limits live in three places: concurrency, capacity, the per-owner cap and retention are `Settings` in
+  `jobs.py` (defaults from the environment in `server.main`), the body size and the token limits are
+  `MAX_BODY`, `MAX_TOKENS`, `MAX_TOKENS_FILE` and `TOKEN_NAME` in `server.py`, and the field, count and
+  character limits are the `MAX_*` constants in `tasks.py`; the deadline range is checked in `JobStore.submit`. `docs/protocol.md`
   documents them; change the doc with the limit.
 - Tests run offline with `python -m unittest discover -s tests/python` inside `python-tests`
   (`network_mode: none`). A change to the model path belongs on `model-smoke.yml`'s `paths:` list, so

CHANGELOG.md

@@ -11,6 +11,7 @@ All notable changes to this project are documented in this file. The format foll
 - A graceful drain: on SIGTERM or SIGINT the worker answers `POST /v1/jobs` with 503 `draining` and `Retry-After: 1`, fails queued jobs as `service_restarted` at once, keeps answering `GET`, and lets running jobs finish for up to `BRIDGE_DRAIN_SECONDS` (integer 0 to 300, default 30) before it stops; a second signal stops it at once.
 - `GET /readyz`, behind any configured token like `/healthz`: 200 when the worker is not draining and a child is idle with its backend's models loaded, otherwise 503 with a reason (`draining`, `no_idle_child`, `models_loading`). A child now loads its backend's models as it starts instead of on its first job. Compose's MCP workers report healthy only once ready.
 - Optional structured logging: `BRIDGE_LOG=json` writes one JSON object per line on stderr, best effort (a full queue or the exit can drop lines), a `request` line after each reply (method, route template, status, duration, the token's name), a `job` line when a job ends (job ID, task, owner, terminal status, queue and run times), and `start`, `drain` and `stop` lines. A fixed schema checked value by value keeps inputs, results, headers, query strings, tokens and exception text out; a stderr that blocks or fails never holds a request. `off`, the default, writes nothing, and any other value stops the worker at start naming `BRIDGE_LOG`.
+- `BRIDGE_OWNER_CAPACITY`, a cap on one token's queued, running and retained jobs (1 to the capacity; unset, the whole capacity, so nothing changes without it). A submission over it is the same 429 `capacity_exceeded` as a full worker, with a `Retry-After` from that token's own first expiry; a repeated idempotency key is still answered, and the total capacity still applies. The `start` log line carries `owner_capacity`.
 - A full connection budget (32) is answered 503 `service_busy` with `Retry-After: 1` and `Connection: close` instead of a closed socket, and a request line over 8 KiB or a header block over 16 KiB is refused with 431 `headers_too_large`.
 
 ### Changed

docker/compose.yaml

@@ -280,6 +280,9 @@ services:
       PYTHONDONTWRITEBYTECODE: "1"
       PYTHONPATH: /work/worker
       BRIDGE_TOKENS: app=test-only-app-client-token-never-use-in-production,other=test-only-other-client-token-never-use-in-production
+      # Small enough for tests/integration.py, which repeats both numbers, to fill one token's cap and then the worker.
+      BRIDGE_CAPACITY: "12"
+      BRIDGE_OWNER_CAPACITY: "8"
     command: [python, -m, ai_bridge.server, --host, 0.0.0.0]
 
   hostile:

docs/protocol.md

@@ -21,7 +21,7 @@ The worker takes its tokens from exactly one of three variables, read once at st
 
 A name is 1–64 characters of `A-Z`, `a-z`, `0-9`, `_`, `.` and `-`; a token is at least 32 non-whitespace ASCII characters, everything after the first `=` of its entry. At most 64 tokens; no name or token may appear twice. An empty variable counts as unset, so compose's `${X:-}` forms work. Setting `BRIDGE_TOKEN` together with either list, or both lists, is refused. The worker refuses to start on any of these, and on a file it cannot read or that group or others can, with a message naming the variable and never a token. `BRIDGE_TOKEN` and `BRIDGE_TOKENS` leave the worker's environment before it starts any child process; the file's path stays, its content does not enter the environment.
 
-A request is authenticated by comparing its bearer with every configured token in constant time; the matching token's name is the request's owner. A job belongs to the owner that submitted it: `GET /v1/jobs/{id}` and `POST /v1/jobs/{id}/cancel` with another token answer 404 `not_found`, exactly as for a job that does not exist, never 403. `GET /healthz`, `GET /readyz` and `GET /metrics` answer any configured token. The owner is not part of a job's snapshot. Capacity, concurrency and retention are shared by every owner.
+A request is authenticated by comparing its bearer with every configured token in constant time; the matching token's name is the request's owner. A job belongs to the owner that submitted it: `GET /v1/jobs/{id}` and `POST /v1/jobs/{id}/cancel` with another token answer 404 `not_found`, exactly as for a job that does not exist, never 403. `GET /healthz`, `GET /readyz` and `GET /metrics` answer any configured token. The owner is not part of a job's snapshot. Capacity, concurrency and retention are shared by every owner; `BRIDGE_OWNER_CAPACITY`, with the other limits below, caps how much of the capacity one owner may hold.
 
 Submit:
 
@@ -45,9 +45,9 @@ Submit `{"task":"redact","input":{"text":"...","entities":["PER"],"min_score":0.
 
 Failures use generic error objects with `code` and `message`. Task errors include `task_failed`, `worker_crashed`, `worker_start_failed`, `deadline_exceeded` and `service_restarted`. Internal exception text is not sent to clients. HTTP contract failures return 400, missing auth 401, missing routes/jobs 404, a connection idle for five seconds while its body is read 408 (a stall while reading headers closes the connection without a response), oversized bodies 413, an oversized request line or header block 431 and capacity exhaustion 429. Two answers are 503 with `Retry-After: 1` and create no job: `draining` for a submission to a worker that is stopping ([below](#drain-and-readiness)), and `service_busy` when all 32 of the worker's connections are taken; that one is written as the connection is accepted, before anything of the request is read, so it carries no auth check and answers any route.
 
-Every 429 carries `Retry-After` in whole delay seconds (an integer of at least 1, never an HTTP date): when every slot is taken and some jobs have finished, the time until the first of them leaves retention, rounded up; when every slot holds an unfinished job, the worker is shutting down or the job table cannot be written, 1. A 429 created no job, so resubmitting after that delay cannot duplicate work. The PHP client exposes the value as `BridgeException::$retryAfterSeconds`, through the cURL and the PSR-18 transport alike, and the MCP server waits for it within a call's budget; both read one header of 1 to 6 digits and ignore anything else, a repeated header included.
+Every 429 carries `Retry-After` in whole delay seconds (an integer of at least 1, never an HTTP date): when every slot is taken and some jobs have finished, the time until the first of them leaves retention, rounded up; when every slot holds an unfinished job, the worker is shutting down or the job table cannot be written, 1. A submission over its owner's cap (`BRIDGE_OWNER_CAPACITY`) is the same 429 `capacity_exceeded` with the same body, and its delay counts only that owner's jobs: the time until the first of its own finished jobs leaves retention, or 1 when it holds only unfinished ones. The owner's cap is checked before the total, and a repeated idempotency key is answered at either. A 429 created no job, so resubmitting after that delay cannot duplicate work. The PHP client exposes the value as `BridgeException::$retryAfterSeconds`, through the cURL and the PSR-18 transport alike, and the MCP server waits for it within a call's budget; both read one header of 1 to 6 digits and ignore anything else, a repeated header included.
 
-The default capacity is 128 total queued, running and retained jobs, concurrency is 2, and terminal retention is 300 seconds. A full result cache rejects submissions until entries expire; it does not silently evict a result early. Configure these with `BRIDGE_CAPACITY`, `BRIDGE_CONCURRENCY`, and `BRIDGE_RETENTION_SECONDS`. The worker keeps as many child processes as its concurrency and replaces a child after `BRIDGE_WORKER_MAX_JOBS` jobs (default 1000, between 1 and 1000000), as well as after a cancellation, a deadline or a crash. A numeric setting that does not parse stops the worker at start with a message naming its variable; one outside its range is refused before any child starts.
+The default capacity is 128 total queued, running and retained jobs, concurrency is 2, and terminal retention is 300 seconds. A full result cache rejects submissions until entries expire; it does not silently evict a result early. Configure these with `BRIDGE_CAPACITY`, `BRIDGE_CONCURRENCY`, and `BRIDGE_RETENTION_SECONDS`. `BRIDGE_OWNER_CAPACITY` caps the queued, running and retained jobs of one owner (one token), an integer from 1 to the capacity; unset, it is the whole capacity, so one token may fill the worker as before. The total still applies on top, so caps that add up to more than the capacity share what is left. The worker keeps as many child processes as its concurrency and replaces a child after `BRIDGE_WORKER_MAX_JOBS` jobs (default 1000, between 1 and 1000000), as well as after a cancellation, a deadline or a crash. A numeric setting that does not parse stops the worker at start with a message naming its variable; one outside its range is refused before any child starts.
 
 With `BRIDGE_STATE_DIR` set, the job table (never a job's input) is written through to SQLite at `<BRIDGE_STATE_DIR>/jobs.sqlite` on every status change; unset, it stays in memory and a restart loses every job. When the worker starts on an existing file, a job still `queued` or `running` there becomes `failed` with error `service_restarted` (its input was never stored, so it is not replayed; a shutdown reports such jobs as `cancelled` to whoever polls the stopping process but leaves them unfinished in the file), a finished job whose retention has not run out by the wall clock is served by GET as before until it does, and an expired one is deleted. The file records each job's owner, the token's name and never its value. A file written before owners existed has no `owner` column: the worker adds it when it opens the file, and its rows belong to `default`, the single-token name, so a worker moved to a token list keeps serving them only to a token named `default`. A 404 for a well-formed ID therefore means the job expired, was never submitted to this instance, belongs to another token, or was lost to a restart without a state directory; the worker cannot tell these apart.
 
@@ -90,7 +90,7 @@ The worker writes no log by default: `BRIDGE_LOG` unset, empty or `off` writes n
 | --- | --- | --- |
 | `request` | After the reply to a request is sent, a 401, 408, 413, 431 and 503 included; a connection that ends before any reply writes none | `method` (`GET`, `POST`, `other`, or `null` when the request line could not be read or parsed), `route` (`/healthz`, `/readyz`, `/metrics`, `/v1/jobs`, `/v1/jobs/{id}`, `/v1/jobs/{id}/cancel`, or `other` for any other path, a query string included), `status`, `duration_ms` (from the start of reading the request to the reply sent), `owner` (the token's name, `null` when the request was not authenticated) |
 | `job` | A job reaches `succeeded`, `failed`, `timed_out` or `cancelled` | `job_id`, `task`, `owner`, `job_status`, `queue_ms` (submission to `running`, `null` if no child started it), `run_ms` (`running` to the terminal status, `null` likewise) |
-| `start` | The port is bound and the children are starting | `backend`, `concurrency`, `capacity`, `retention_seconds`, `max_jobs`, `drain_seconds`, `tokens` (how many tokens are configured; never a name or a token) |
+| `start` | The port is bound and the children are starting | `backend`, `concurrency`, `capacity`, `owner_capacity` (the capacity when `BRIDGE_OWNER_CAPACITY` is unset), `retention_seconds`, `max_jobs`, `drain_seconds`, `tokens` (how many tokens are configured; never a name or a token) |
 | `drain` | A drain ends, before the worker stops serving | `duration_ms` (from the signal), `running` (jobs still running when it ended, which the stop then cancels), `interrupted` (a second signal ended it) |
 | `stop` | The worker has stopped its children and closed the job table | `cancelled` (unfinished jobs the stop ended as `cancelled`) |
 

docs/security.md

@@ -2,7 +2,7 @@
 
 This experimental service is designed for trusted application-to-service traffic on a private network. Do not expose the example routes or Python server directly to the internet. Across hosts, put a TLS proxy in front of the worker as [Cross-host use behind TLS](integration.md#cross-host-use-behind-tls) shows.
 
-- Generate a service token of at least 32 non-whitespace ASCII characters. Never commit it. A token reaches only the jobs it submitted: another token's job answers 404 as if it did not exist, and idempotency keys are per token. Give each application, and the MCP server, its own token in `BRIDGE_TOKENS` or `BRIDGE_TOKENS_FILE` (mode 0600 or stricter), so one can be revoked by removing its entry and restarting the worker without touching the others, and a leaked token exposes only its own client's jobs. Capacity is still shared, so any token can fill the worker; `/healthz`, `/readyz` and `/metrics` answer every token, and the metrics carry the token names, never the tokens. The job table stores a job's owner by name only.
+- Generate a service token of at least 32 non-whitespace ASCII characters. Never commit it. A token reaches only the jobs it submitted: another token's job answers 404 as if it did not exist, and idempotency keys are per token. Give each application, and the MCP server, its own token in `BRIDGE_TOKENS` or `BRIDGE_TOKENS_FILE` (mode 0600 or stricter), so one can be revoked by removing its entry and restarting the worker without touching the others, and a leaked token exposes only its own client's jobs. Capacity is shared, so by default any token can fill the worker and starve the others; set `BRIDGE_OWNER_CAPACITY` below `BRIDGE_CAPACITY` to cap one token's queued, running and retained jobs, which is off by default. A token at its cap gets the same 429 as a full worker, so it learns nothing of the other tokens' load; the cap is the same for every token, so where it times the number of tokens exceeds the capacity the tokens together can still fill the worker: set it at most the capacity divided by the number of tokens where every token must always find room; `/healthz`, `/readyz` and `/metrics` answer every token, and the metrics carry the token names, never the tokens. The job table stores a job's owner by name only.
 - Enforce end-user authentication and per-job ownership in the PHP application. The example deliberately omits those application-specific policies and publishes no ports.
 - Only trusted task implementations are allowed. The service removes `BRIDGE_TOKEN` and `BRIDGE_TOKENS` from its environment before it starts its child processes, and reads `BRIDGE_TOKENS_FILE` without putting its content there, so no token is in either their environment nor their initial environment block (`/proc/<pid>/environ`), which a removal inside the child would leave as it was; the children never speak HTTP. Everything else in the service environment stays visible to them. A child serves many jobs in turn and keeps its loaded models; it keeps no reference to a finished job's input or result, but the memory they used is freed, not wiped, so a core dump of a long-lived child can still hold earlier jobs' data where a process that died after each job could not. They are not suitable for arbitrary user-provided scripts.
 - Container configuration uses a non-root user, read-only filesystem, dropped capabilities, no-new-privileges, CPU/memory/PID bounds, and an internal-only network for execution. It mounts only this checkout and the optional dependency/model cache, never a Docker socket or home directory.