You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Document what the observability stack costs in RPS #187
There is currently nothing in the docs about what turning an instrument on costs. It is a lot, and the ordering is not what people expect.
Measured on a trivial async def endpoint returning {"ok": True}, uvicorn single worker, ab -k -c 16 -n 20000. Apple M2, CPython 3.14.7, fastapi 0.141.1, sentry-sdk 2.67.1, opentelemetry-sdk 1.44.0, prometheus-fastapi-instrumentator 8.1.0.
config
RPS
vs bare
bare FastAPI
7978
-
full lite-bootstrap stack (otel + prometheus + structlog + sentry)
2389
−70%
same stack, tuned
4187
−48%
Per instrument, measured alone, driving the ASGI app in-process so only library cost shows (baseline 15.8 µs/req):
instrument
+µs/req
share
LoggingInstrument (configured, no logs emitted)
+0.1
~0%
PrometheusInstrument
+17.5
8%
SentryInstrument (tracing off)
+58.5
27%
OpenTelemetryInstrument
+120.1
55%
all four
+217.1
Things a page like this should say, all of which surprised me:
OpenTelemetry is twice Sentry, and is the dominant cost in the stack. Most people assume Sentry is the expensive one.
LoggingInstrument is free until you actually log. The cost is per record (~8 µs with Sentry on), not per request.
The Sentry knobs people reach for do nothing.attach_stacktrace=False, max_breadcrumbs=0 and dropping the default integrations all measured within noise. The entire cost is the ASGI integration, and over half of that is a Transaction built and discarded because sentry_traces_sample_rate is unset.
sentry_traces_sample_rate=1.0 costs +274 µs/request on top of that (2486 RPS, −68% vs bare). Worth stating next to the setting.
Each saving has a price, and the page should be explicit about it: no breadcrumbs on error events, no release health, no Sentry-side trace correlation, sampled-away OTel traces.
Suggested home: a "Performance" or "What observability costs" page under Introduction, linked from each instrument's docs, with a short "if you are RPS-constrained, start here" section.
Caveats to include: these ratios are for a handler that does nothing. A service doing real work per request (a DB round trip, a downstream call) will see a much smaller relative hit - the absolute per-request cost is what transfers, not the percentage.
The benchmark harness that produced these numbers is reproducible and I can attach it if useful. Sibling issues cover the two OpenTelemetry knobs and the Sentry ones.
benchmarks/README.md there is the full write-up this page would be sourced from, including the trade-off table (what each saving costs you in Sentry data) that a docs page needs more than it needs the RPS figures.
Activity
lesnik512 commentedon Sep 7, 2026
Harness landed in #189 (
benchmarks/), so these numbers are re-runnable rather than a claim in an issue body:benchmarks/README.mdthere is the full write-up this page would be sourced from, including the trade-off table (what each saving costs you in Sentry data) that a docs page needs more than it needs the RPS figures.chore(benchmarks): add the observability-overhead harness and its fin…
chore(benchmarks): add the observability-overhead harness and its fin…