Repository navigation
feat(middleware): Log request context with large RSS growth - #129458
Draft
mwarkentin wants to merge 1 commit into
Draft
mwarkentin wants to merge 1 commit into
mwarkentin wants to merge 1 commit into
Conversation
The request.rss_growth log line now also records what made the request large: duration, status, request and response sizes, user agent, webhook event type, the organization from the URL, and the values of a few paging and dataset query parameters (only the length of a search query). It also records how many requests the worker had served and its uptime, so a new worker's warm-up can be told apart from real offenders. Reporting can no longer fail the request. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lpv3UTF9m93wLvaw2ruhQ8
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ticket: PRODENG-2971
Description:
Environments affected:
Only processes with
SENTRY_REQUEST_RSS_METRICSset (currently the getsentry web canaries in SaaS US, DE, S4S2). Off everywhere else, including self-hosted.Change plan:
AI Commentary
Why. After ~55 hours on the web canaries,
request.rss_growthnames the endpoints behind most large RSS jumps (repository.get_repositories, the issue events lists, the GitHub webhook,chunk-upload, ...). Reading the code points to specific causes, but confirming them needs to know how each request was made: e.g. whether the events lists are hit with 100-row pages by API clients, and whether the GitHub webhook spikes are bigpushevents.What it does. The log line gains these fields (the metric and its tags are unchanged, so no new cardinality):
duration_ms,status_code,request_bytes(fromContent-Length),response_bytes(none for streaming responses)user_agent(truncated to 200 characters)webhook_event, fromX-GitHub-Event,X-Gitlab-EventorX-Event-Keyorganization, theorganization_id_or_slugURL argument, to tell whether one organization accounts for the outliersparam_<name>for an allowlist of query parameters (per_page,limit,full,dataset,sort,expand,collapse,statsPeriod,referrer), plusfield_countandquery_length. Search text and other parameters are never logged, only the query's length.worker_request_numberandworker_uptime_s. A new worker's first requests grow RSS while it imports modules and fills caches; these separate warm-up from real offenders.Keys are flat so each is a queryable attribute in Sentry logs.
Safety. Reporting runs in the middleware's
finally, so it is now wrapped: any failure while building the log line is logged withlogger.exceptionand never fails the request or masks the view's exception (e.g. Django raising on a query string with more than 1,000 parameters). The only new per-request cost is twotime.monotonic()calls; everything else runs only for requests over the threshold.Tests.
tests/sentry/middleware/test_request_memory.py, 22 tests, adding:prekand mypy pass on both files.Generated by Claude Code
Generated by Claude Code