Feat: Add the lineage demo — per-request lineage from the sidecar

[JoshSag]

What this adds

A demo under authbridge/demos/lineage/ showing how to get per-request data
lineage out of the AuthBridge sidecar: attach the envoy-sidecar to an agent or
tool, switch on the lineage-telemetry plugin, and every HTTP exchange becomes
two facts-only spans that go to any OTLP consumer — on a stock install, the
platform's own collector, whose default pipeline exports to debug, so a reader
can read them straight from its log with nothing extra deployed. (Phoenix is not
installed by default; the README says so and shows both routes.)

Six files, five of them the mechanism and one the argument:

file	lines	what
attach-lineage.sh	384	the one generator — emits every YAML byte: full manifest, the plugin ConfigMap alone, or a sidecar patch
README.md	360	why the shim exists, the ownership fork, the envelope, a worked example, limits
build-otel-shim.sh	88	builds the shim onto an app image; refuses images it cannot safely wrap
sidecar-patch.sh	83	the patch path, for a Deployment you do not own
Dockerfile.otel-shim	63	the propagate-only shim layer
container-runtime.sh	39	docker-vs-podman detection and kind loading

The interesting half: why a sidecar alone is not enough

The sidecar sees every hop with parsed bodies, but it lives outside the
app's execution context and cannot know which internal coroutine issued which
outbound call. Attributing an outbound call to the inbound request that caused
it needs a token carried with the execution scope through the app — which is
what W3C traceparent is for, and only code inside the request's context can
propagate it.

Without it, the plugin falls back to "this agent's current inbound span". That
is correct only while the agent handles one request at a time. Under
concurrency it collapses: with 6 concurrent requests, all 6 outbound calls
attach to whichever inbound updated the process-wide pointer last — 1/6,
measured.

So the demo ships a propagate-only OpenTelemetry shim: stock
auto-instrumentation for the mainstream Python HTTP stack, layered onto an app
image without touching its source, running under
opentelemetry-instrument --traces_exporter none. It extracts the inbound
traceparent, injects it on outbound calls, carries the context across thread
boundaries — and exports nothing. An app that configures its own exporter
keeps exporting exactly as before; the shim silences only its own
instrumentation.

The threading instrumentor is load-bearing rather than incidental: frameworks
that run the LLM call in a worker thread lose the context at the thread
boundary, and propagation silently reverts to 1/N.

Two paths, and the demo says which one to use

This is the part we would most like reviewed, because choosing wrong destroys
resources. The question is not what your app is — it is who owns the
Deployment:

• You own it → attach-lineage.sh with EMIT=manifest emits a complete
  ConfigMap + Service + Deployment. It is a replacement by design.
• An operator, controller or UI owns it → sidecar-patch.sh applies a
  strategic-merge patch that only adds the sidecar containers, leaving the
  owner's spec alone.

Point the first at a name some other controller manages and you have silently
rewritten that controller's object. The README states this as a decision table
before any command, with the consequence spelled out.

It also states two limits we would rather a reader hit in prose than in
production: a patch is not durable (any platform-side rewrite silently drops
the sidecar — there is no error, lineage just stops), and the uninstrumented +
operator-owned quadrant is unsolved, because the shim needs an image change
you cannot make for a workload you do not control.

Runs against a stock platform, with no service of ours

Everything defaults to your side of the fence: rossoctl.io/* workload labels,
otel-collector.rossoctl-system as the OTLP endpoint, kind cluster rossoctl,
and ghcr.io/rossoctl/cortex/{authbridge-envoy,proxy-init} for the sidecar
images — all overridable by environment variable. The demo names no service,
repository or endpoint that is not either in this repo or supplied by the
reader.

One caveat we should state rather than have you discover: a published
sidecar image only carries the lineage-telemetry plugin once lane 2 has merged
and a release is cut. Until then, build the two images from this repo and
point SIDECAR_IMAGE / PROXY_INIT_IMAGE at your local tags. The README says
this where a reader will hit it.

It has been run, exactly as written

The worked example uses this repo's own weather_service image, and its
facts are measured from that image rather than assumed — its Cmd is
["uv","run","--no-sync","server"], its venv is at /app/.venv, it runs as UID
1001, and it already bundles OpenTelemetry instrumentation. That last fact is
what makes it a good example: build-otel-shim.sh refuses to wrap it,
because wrapping an already-instrumented app would double-instrument it. The
honest recipe for such an app is sidecar-only, and the README shows that.

Executed against a live cluster: the manifest applies, the pod comes up 2/2, and
one A2A request produces

weather-lineage a2a message/send            role=request   direction=inbound  protocol=a2a
weather-lineage a2a message/send response   role=response  direction=inbound  protocol=a2a  outcome=ok

both with the same lineage.exchange.id.

What is deliberately not here

A considerably larger operational kit exists behind this demo — a fleet catalog
and deployer, a global on/off switch, a probe application, concurrency
harnesses, per-app expectation cards, from-zero cluster runbooks. We are not
offering it, and that is a decision rather than an oversight.

The cut line is "runs with no other service and no lab of ours". The rest of
the kit fails it: it scales named objects in a service you do not have, encodes
our own 14-app inventory, and computes paths into a sibling clone in our
workspace layout. Asking a maintainer to host that would be asking you to carry
our lab. What is here is ~1,000 lines of generic mechanism that works against
Phoenix, Jaeger, or any OTLP endpoint.

Gates

gate	result
shellcheck --severity=error, as your Security Scans job runs it	exit 0 repo-wide
hadolint --failure-threshold error with your ignore list	exit 0 on Dockerfile.otel-shim (one DL3066 info: non-numeric UID)
pre-commit hygiene	clean — no trailing whitespace, no missing final newline, no CRLF
Python gates (bandit / ruff)	not applicable; no Python ships in this demo

---

Assisted-By: Claude (Anthropic AI) noreply@anthropic.com

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 96c7d7c7-1704-4a46-8aa5-06fd76b59d94

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}