Correct frontend telemetry metadata and SLO inputs

[abhi1693]

What changes and why?

Browser telemetry reported version development while production ran 0.0.50, and frontend request histograms supplied neither response status nor stream classification to the existing SLO rules. Collector 403s also had no diagnostic reason, so their volume could not establish browser data loss.

• Use the version embedded in the Next artifact for web/admin Faro metadata, both in browser settings and when the receiver forwards a batch. The environment remains a separate setting.
• Add bounded outcome labels to devfeed_faro_deliveries_total, distinguishing missing/mismatched/unconfigured origins, encoded requests, body/type/size/rate/time rejection, configuration problems, and upstream rejection or transport failure. Labels contain no request values or identities. Existing origin validation and receiver limits remain enforced.
• Emit the Node duration histogram as http_server_request_duration_seconds, with response status and devfeed_request_kind. The existing GitOps normalization already consumes this schema and excludes stream. Classify known Chimely inbox streams before bounding their paths, and recognize other SSE responses by content type. Ordinary chunked HTML remains a request; disconnected responses record 499 once.
• Add real Faro browser/receiver checks to the reader and admin browser CI jobs, plus regression coverage and an operational explanation of the metrics and rollout.

Production evidence

Read-only checks on 2026-10-04, 00:22–00:24 UTC:

• The preceding 24-hour increase window showed approximately 215.2 accepted web batches and 668.5 web 403s: 75.7% rejected, versus roughly 79% in the earlier audit window. These are Prometheus estimates, not an exact request ledger. Current production counters do not expose outcome reasons.
• Loki returned 177 web measurement records with app_version="development" and app_environment="production". All four web/admin replicas exposed build 0.0.50 / production. The incorrect value is the version, not the environment.
• The normalized frontend duration metric had 77 web and 74 admin series without a response-status label. Notification gateway duration series also entered normalization without labels distinguishing streams from ordinary inbox calls.
• Web/admin pods were ready with zero restarts. No production configuration or workload was changed.

Genuine browser data loss remains unproven. A controlled, sampled browser request succeeds against both built applications. The installed Faro SDK samples using Web Crypto; overriding Math.random alone did not make the earlier probe deterministic. After rollout, correlate the new rejection reasons with a real sampled browser's network response and collector ingestion in the same window before attributing 403s to browser loss.

Validation

Updated against master after #69, #71, #72 and #73. Current commit: 574a700f2f45874a12546384889bb3c0c9b04d88. The telemetry-test conflict preserves both release-version/outcome checks and authentication-redaction regressions.

• All 32 focused telemetry regressions, formatting, whitespace checks and affected-project commit checks passed after updating the branch.
• Current-head CI: 2,620 backend unit tests, 1,132 integration tests, 790 web tests and 647 admin tests passed, with no skips. Populated migration upgrades, web/Chrome/Edge reader parity, authenticated API journeys, admin browser journeys, ARM64 image builds/scans/smoke checks and all three CodeQL analyses passed. CI required and both Sonar checks passed.
• Sonar MCP confirms the quality gate is OK, with 98.5% new-code coverage, 0% new duplication, and zero new bugs, vulnerabilities or code smells.
• Hosted reader/admin browser jobs run the real Faro SDK against production builds. They verify browser Origin acceptance, forwarding, release/environment metadata before and after the receiver, spoofed metadata replacement, server-only collector keys, and acceptance/rejection metrics. Local collector/API fixtures isolate these checks from production; sampling is forced only inside the tests.
• The six-sample API performance gate passed with zero request failures: three route comparisons were inconclusive and five were within threshold. This unchanged-backend benchmark does not establish browser performance or a production latency improvement.
• promtool check rules: 25 current GitOps rules passed. All six existing scenarios plus a new frontend scenario passed, including seven assertions proving web/admin error ratios and exclusion of stream counts, sums and buckets. Validation used a disposable copy of the current rules and fixtures, without repository changes in home-lab.
• The unmodified home-lab fixture suite fails on three stale runbook URL expectations (v0.0.46 versus rules pointing to v0.0.50). Only those expected URLs were corrected in the disposable validation copy. This existing fixture maintenance issue remains a separate follow-up.
• GitHub's separate managed AI findings scan remains unavailable because the monthly quota is exhausted (HTTP 402). Standard CodeQL passed; no paid service was enabled.
• Full updated-head browser and backend integration verification is supplied by hosted CI rather than a complete local rerun. No visible UI changed, so screenshots are not applicable.

Rollout notes

The Node histogram changes from devfeed_http_request_duration_seconds to http_server_request_duration_seconds. Current GitOps SLO/dashboard recording rules already accept the new metric; direct consumers of the old histogram must migrate. devfeed_http_requests_total remains available. During a rolling update, older replicas still contribute the earlier unclassified duration series; verify replacement replicas and allow the selected rate windows to age past the rollout.

Adding outcome labels starts new receiver counter series. Old 403 records cannot be classified retroactively. Status, method, route and outcome dimensions remain bounded, but status/kind labels add metric series; inspect scrape size and cardinality after rollout.

No migration or new dependency is required. Local validation used Linux AMD64; current-head hosted ARM64 image/security checks, CI required, CodeQL and Sonar passed. Production rollout and post-rollout measurement remain pending release.

Ready for review

• The description explains the change and the validation results, including gaps.
• Relevant documentation and regression tests are updated.

[github-actions]

PR reader API performance

Result: INCONCLUSIVE

3 inconclusive · 5 within threshold. Zero request failures across all six runs.

Base df25ae0498a13feb0dd765c8a6e18de59b7d8cd6 → PR 574a700f2f45874a12546384889bb3c0c9b04d88

Three independent runs per revision, each on a fresh GitHub-hosted ARM64 runner. 16 users, 60 seconds/run, 1,000 synthetic articles, cache off, five PgBouncer slots.

Latency comparison

All latency values are in milliseconds; lower is better. Values are medians of three runs.

Route	Base p95	PR p95	Change	Result
All requests	87	89	+2 (+2.3%)	Within threshold
Latest feed	120	110	-10 (-8.3%)	Within threshold
Feed pagination	87	86	-1 (-1.1%)	Within threshold
Article detail	68	61	-7 (-10.3%)	Inconclusive
Source feed	85	81	-4 (-4.7%)	Within threshold
Topic feed	74	71	-3 (-4.1%)	Inconclusive
Feed filters	76	74	-2 (-2.6%)	Within threshold
Sources	40	46	+6 (+15.0%)	Inconclusive
Topics	49	42	-7 (-14.3%)	Within threshold

Additional latency and throughput metrics

Each cell shows base → PR. Latencies are in milliseconds.

Endpoint	p50	p99	Requests/s
Aggregated	26 → 26	130 → 130	30.09 → 29.76
/v1/feed [latest]	41 → 41	150 → 130	9.02 → 9.13
/v1/feed [page]	25 → 24	130 → 110	3.64 → 3.54
/v1/articles/[id]	16 → 16	96 → 110	6.39 → 6.45
/v1/feed [source]	26 → 27	140 → 110	2.04 → 1.96
/v1/feed [topic]	24 → 24	130 → 110	1.75 → 1.89
/v1/feed/options	20 → 19	110 → 100	2.43 → 2.40
/v1/sources	10 → 10	70 → 68	2.43 → 2.40
/v1/topics	10 → 11	68 → 78	2.43 → 2.40

Individual runs

Run	Requests	Failures	Exit code
Base 1	1695	0	0
Base 2	1685	0	0
Base 3	1674	0	0
PR 1	1666	0	0
PR 2	1672	0	0
PR 3	1671	0	0

How to interpret this report

Regressed / Improved: a p95 regression needs >20% and >20 ms increase in the medians and at least two head samples versus the base median. Improvements use the inverse threshold. Within threshold means the change did not meet both thresholds.

Inconclusive: a within-revision p95 range >35% of its median and >20 ms is too noisy for a confident latency verdict.

Inconclusive latency comparisons are non-blocking: the check passes without claiming an improvement. Confirmed regressions and test/report errors fail the PR gate. Throughput is descriptive: paced users do not measure maximum capacity. This uncached public API test does not establish production health, browser performance or worker capacity. Separate runners can differ in hardware or host load; repetitions and noise checks reduce but cannot eliminate that uncertainty.

Full HTML/CSV reports and logs · attempt 1

[gitar-bot]

Code Review ✅ Approved

🔴 High risk · Replaces the exported request histogram schema, requiring direct metric consumers to migrate

Corrects frontend telemetry metadata to report the actual release version instead of development, adds bounded outcome labels to receiver rejection counters for diagnostic visibility, and emits request duration with response status and stream classification for SLO enforcement. All 32 focused telemetry tests, 775 web tests, 647 admin tests, real browser validation against both production builds, and 25 GitOps SLO rules passed. No issues found.

Review coverage

📋 Rules No rules evaluated

🧪 Functional validation Not enabled · Set up

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Counting what did not apply, without listing it.

Comment with these commands to change the behavior for this request:

Auto-apply	Compact
gitar auto-apply:on	gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Gitar}

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues
0 New dependency risks

Measures
0 Security Hotspots
98.5% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud