Repository navigation
feat: add optional structured JSON logging - #91
Merged
Merged
Conversation
BRIDGE_LOG=json writes one JSON line on stderr per request, per finished job and at start, drain and stop, from a fixed schema whose values are checked against fixed sets: route templates, never the raw path, token names, never tokens, and no input, result, header or exception text. Lines go through a bounded queue to a writer thread, so a stuck or broken stderr never holds a request; delivery is best effort. The default, off, writes nothing, and the server no longer prints a traceback when a peer resets a connection.
# Conflicts: # docs/testing.md
# Conflicts: # docs/testing.md
# Conflicts: # docs/testing.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #75.
What
Optional structured logging for operators, without giving up the reason access logs were off: nothing a client sends can reach a line.
BRIDGE_LOG=off|json.off, the default, writes nothing, as before; an empty value meansoff, and any other value stops the worker at start with a message namingBRIDGE_LOG.json, one JSON object per line on stderr:ts(UTC, milliseconds),level(infoorwarning),event, and only that event's keys:requestafter every reply (401, 408, 413, 431 and the busy 503 included):method(GET,POSTorother),route(the template, such as/v1/jobs/{id}, orother; never the raw path or a query string),status,duration_ms,owner(the token's name, null when unauthenticated);jobat each terminal status:job_id,task,owner,job_status,queue_ms,run_ms(null for a job that never started);start(backend, concurrency, capacity, retention, max jobs per child, drain deadline, the number of tokens),drain(how long, jobs still running, whether a second signal cut it short) andstop(jobs cancelled at the stop).worker/ai_bridge/log.pyholds the schema. Its formatter reads only the fieldslog.event()passes, drops any key outside the event's schema and writes null for a value that fails its check (fixed sets, the task allowlist, the token-name and job-id patterns, numbers). A plainloggingcall never reaches the output.BridgeServer.handle_errorwrites nothing, so a peer resetting a connection no longer prints a traceback and its address to stderr in either mode.Files
worker/ai_bridge/log.py(new);worker/ai_bridge/server.py(request, busy, drain, start and stop lines,BRIDGE_LOG, the drain thread joined inserver_close,handle_error);worker/ai_bridge/jobs.py(the job line,_started_at,close()returns the cancelled count).tests/python/test_log.py(new),tests/python/test_server.py,tests/python/test_jobs.py.docs/protocol.md(a Logging section with the schema),docs/security.md,README.md,CHANGELOG.md,docs/testing.md(the Python row and count),.github/workflows/model-smoke.yml(log.pyonpaths:, sincejobs.pyimports it).Tests
Every line parses and holds exactly the keys the test writes out for its event (a literal key set, not the implementation's table). A run whose input, idempotency key,
Authorizationheader, query string, wrong token and failing task's exception text all carry a sentinel never writes the sentinel, a token,Beareror?.offwrites nothing; a badBRIDGE_LOGstops the start. The route template for every route andother; the 401 line has no owner; one line each for 408, 413, 431 and the busy 503; a connection that sends nothing or half a header block writes none. Job timings agree with the metrics stamps, each job has exactly one line, andqueue_msandrun_msare null for a job that never started. A stream that raises or blocks never fails or holds a request, and a writer left with a full queue still ends once its stream is released. Throughmain(): start and stop lines,cancelled >= 1at warning when a job is running at shutdown, and an interrupted drain's singledrainline withinterrupted: truebeforestop. A peer reset leaves stderr with JSON lines only injsonand empty inoff. The new tests were seen failing before the code, and the review-round tests were seen failing before their fixes (the stuck writer, the traceback on reset, the lost drain line); the request-line race fix holds over three full runs in a row.Checks
The full command block of
AGENTS.md, run as one command on this head (a8295ba: the slice commit 05ede27 merged withmainat 2bd4b4b, which brought in the drain, images, load, quality and supply-chain slices) in a worktree, PHP 8.5:--profile test config: okpython-tests: Ran 395 tests, OKphp-tests: PASS: 148 PHP contract checks;php -l: no syntax errorsphp-integration: PASS: 192 total PHP checks;integration: PASS (12 concurrent jobs, metrics, jobs scoped to their token)mcp-tests: Ran 125 tests, OK;composer-install: okphp-analysis: [OK] No errors;php-psr18-tests: PASS: 178laravel-tests: 94 checks;symfony-tests: 96 checks;tls-check: 7 + 3 checksBefore the merge,
python-testsalso passed three times in a row on the slice alone (316 tests each). PHP 8.2, 8.3 and 8.4 are left to CI.Review
Two independent read-only reviews: codex (another model family) and the delivery reviewer. They agreed on these findings, all fixed in this branch before the push:
stop;handle_errorput a traceback and the client's address on stderr, contrary to the new docs;log.close();__main__guard;The missing tests both reviews named are added (above). Security: nothing open; no client-controlled text reaches a line, and the one low-severity point (the traceback on a peer reset, older than this branch) is fixed here.
Not verified here: a container log driver reading the worker's stderr, since no compose service sets
BRIDGE_LOG;model-smoke.ymlruns on this pull request becauselog.pyis on itspaths:.