[6/6] Attribute requests to a caller and a client tool - #393
Merged
Merged
Conversation
This was referenced Oct 1, 2026
- Add GetEcosystemStats, aggregating packages, versions, artifacts, cache size, downloads and downloaded bytes per ecosystem - Publish six proxy_ecosystem_* gauges from it on the existing one-minute cache-stats tick - Set and selectively delete rather than Reset, so no scrape lands on a half-populated vector - Report artifacts with no package row under "unattributed" rather than dropping them, so the figures still add up Part 2 of 6 splitting git-pkgs#381 up. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Ring of download size by ecosystem, cache figures, per-ecosystem table, vulnerability overview and a Runtime card - Add metrics.Gather so the page can render counters that were never in the database - Add proxy_response_bytes_total and response-writer byte counting - Serve a retained snapshot behind a staleness banner when the aggregation fails, rather than rendering old figures as current - Extract the security overview into a shared component Part 3 of 6 splitting git-pkgs#381 up. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Add downloaded_bytes, downloads and an ecosystems array, served from the snapshot the gauges and the page already share - Add stats_unavailable so a failed aggregation is distinguishable from an idle proxy - Regenerate the OpenAPI spec Part 4 of 6 splitting git-pkgs#381 up. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- 33 panels, no hardcoded data source UID - Three ecosystem variables, because the label means three different things across /metrics: the package record, the request path, and the handler's own name Part 5 of 6 splitting git-pkgs#381 up. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Track per-caller request and byte counts in a bounded table - Export proxy_client_requests_total and proxy_client_response_bytes_total over a closed label set; addresses are never labels - Gate the sources card behind ui_request_sources, off by default, since /ui carries no authentication of its own - Move access_log.trust_forwarded_for to a top-level key - Keep remote as the TCP peer and add remote_ip alongside it Part 6 of 6 splitting git-pkgs#381 up. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
wickedOne
force-pushed
the
analytics-6-source-attribution
branch
from
October 1, 2026 14:19
6916614 to
3655000
Compare
The flag exists to withhold caller addresses. The by-client table carries none, and the same figures are already public at /metrics, so gating it left two metrics with no tile on the page in the default config.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Client and request-source attribution
Package managers do not say who invoked them. A request from
piporgocarries aHost, anAcceptand aUser-Agent-- noReferer, no originating URL, nothing naming a repository, pipeline or job. So the proxy attributes requests by the two things always present: the caller's address, and the tool name taken from the leading User-Agent token.Client names are exported as
proxy_client_requests_total{client}andproxy_client_response_bytes_total{client}because they come from a closed set -- a User-Agent is attacker-controlled, so anything unrecognised collapses tootherrather than minting a time series per request. Addresses are not exported as labels at all: the caller set is unbounded and outside the proxy's control. Per-address detail lives in the access log and, when enabled, on the page.ui_request_sources, off by defaultThe review's point: the sources card publishes caller IP addresses, tool names and per-caller volumes on
/ui/analytics, and the proxy has no authentication of its own, with gating deferred to a reverse proxy (#123). Until now/uiexposed what was cached, not who called.So the card is behind a new top-level
ui_request_sources, defaulting to false, and the README says plainly what it exposes and that/uiis unauthenticated. The tracker still runs when the flag is off, so the Prometheus client metrics are unaffected -- only the addresses are withheld. Tests cover both states end to end.trust_forwarded_formoves to the top levelAlso from the review:
access_log.trust_forwarded_forreads narrower than it acts, since it drives the structured log and the analytics table as well as the access log. It is now top-leveltrust_forwarded_for/PROXY_TRUST_FORWARDED_FOR. Nothing has been released with the old key.The header is honoured only when the flag is set, and the value must parse as an IP: a load balancer that appends leaves the leftmost entry caller-controlled, and that string becomes a map key held for the process lifetime and a line in the access log.
Log fields
remotekeeps its meaning. #381 changed it fromhost:portto host-only; per the review it stays as the TCP peer and a newremote_ipcarries the attributed address, matching whataccesslog.Entryalready does withRemoteAddr/RemoteIP. The access log gainsremote_ip,user_agent,client,ecosystemandbytes.The table
It tracks 200 callers, evicting the least recently seen once full. Evicting rather than refusing matters in the deployment this is for: containerised CI gives every job a fresh address, so a refusing table would freeze on whoever arrived first.
The overflow row reports two counts separately -- callers currently tracked below the display cut, and fold-ins from eviction -- because only the first is exact. A caller evicted, returning and evicted again counts twice, and counting distinct callers instead would mean keeping every key ever seen, which is the unbounded set the limit exists to avoid.