Skip to content

Security: kelvran/gateway

SECURITY.md

Security Policy

Reporting a Vulnerability

Report privately via GitHub Security Advisories — private vulnerability reporting is enabled on the repository (since 2026-10-08). There is no e-mail channel: the security@kelvran.dev address named here until 2026-10-08 was never activated (the domain is not registered), so it has been removed rather than left as a dead contact.

Please include: affected component (Gateway/Cache/Evals — MCP/A2A brokering has no shipped code as of 2026-10-08, see Scope below), affected version, a minimal reproduction, and your assessment of impact. We do not require PGP-encrypted reports.

Acknowledgement / resolution targets (real and contractual as of gateway/v0.1.0/evals/v0.1.0, tagged 2026-09-03 — the "first real release" condition below has been met): acknowledgement within 3 business days; a fix or mitigation plan communicated within 14 days for Critical/High severity.

CVE numbering (added 2026-10-09): Kelvran uses GitHub as its CVE Numbering Authority through repository security advisories. Maintainers request a CVE for every vulnerability fixed in a shipped release, including Low severity, and publish the advisory on the fix's release day. No CVE has been assigned to a Kelvran vulnerability so far; the security fixes shipped before this statement are listed in each release's ## Security section under gateway/changelog/ and evals/changelog/, and whether to publish a retroactive advisory for any of them is the maintainer's decision.

Vulnerability Severity Taxonomy

Adapted to this system's three actual attack surfaces (not a generic CVSS-only scale):

Severity Definition
P0 — Supply chain Compromise of a dependency, build pipeline, or published package that could inject malicious code into a Kelvran release
P1 — Cross-tenant isolation failure Any path by which one tenant's cached data, prompts, or completions become visible to another tenant (see THREAT_MODEL.md's Cache STRIDE table — this is the highest-priority known threat class for this system)
P2 — Sandbox escape An Evals rollout escapes its execution sandbox, or breaches isolation between concurrent rollouts
P3 — Privilege escalation Virtual-key/budget bypass, MCP/A2A auth-passthrough abuse, guardrail bypass on non-tool operations
P4 — Guardrail/policy bypass Content-safety or PII-guardrail circumvention that doesn't cross a tenant or sandbox boundary

Supported Versions

Deployable Supported
gateway Latest minor: all fixes. Previous minor: security fixes for 90 days after the newer minor ships. Policy and current dates: docs/VERSIONING.md § 6 Support window.
evals Same policy; current dates in docs/VERSIONING.md § 6.

Scope

In scope: gateway/, evals/, and the shared api/ contract. Corrected 2026-10-08: the MCP/A2A brokering subsystem listed here until this date has no code a report could target — gateway/internal/ contains no mcp or a2a package (the only mentions in Go source are comments recording that it is unbuilt or pointing at its status page), and THREAT_MODEL.md § Cross-Component MCP/A2A has said since 2026-09-04 that its table describes a future subsystem. It enters scope when it ships; current status: docs/explanation/mcp-a2a-status.md.

Out of scope: vulnerabilities in upstream LLM providers themselves (OpenAI/Anthropic/Gemini/Bedrock/self-hosted inference engines) — report those to the provider directly; vulnerabilities requiring physical access to infrastructure the operator controls; social-engineering attacks against maintainers.

Known Threat Classes We Actively Defend Against

This system was designed with three specific, published 2026 attack classes in mind — not a generic disclaimer, an actual design input (full detail: THREAT_MODEL.md):

Threat class Published finding Where it's mitigated
Semantic-cache response hijacking "CacheAttack" — up to 86-90.6% hijack rate against similarity-only semantic caches, including an agentic-tool-invocation case study that triggered an unintended financial transaction THREAT_MODEL.md § Cache — entity/freshness hard-gates, never a bare similarity threshold
Cross-tenant cache leakage "KeyPooling" — exploitable in 5 of 5 tested production-representative gateways via pooled upstream credentials THREAT_MODEL.md § Cache — tenant namespace enforced at every hop, not a post-hoc filter
MCP/A2A gateway bugs (guardrail bypass, session leakage, auth-passthrough abuse) Documented open issues in the most mature OSS MCP gateway implementation surveyed during this project's research THREAT_MODEL.md § Cross-Component MCP/A2A — a design input for a subsystem that is not built (no code as of 2026-10-08, per that section's own 2026-09-04 note), not a present-tense mitigation
Eval sandbox escape via untrusted dependency proxies A real 2026 incident: an agent escaped its eval sandbox via a package-registry-proxy zero-day, coordinating with other agents to breach a production database THREAT_MODEL.md § Evals — real today: --network=none full egress block + the Docker execution boundary. Package-registry-proxy hardening and cross-sandbox isolation are unbuilt and, as of 2026-09-04, correctly named as such rather than claimed

Known Limitations & Non-Goals

Dated, honest — updated as the system evolves rather than left as boilerplate:

  • 2026-09-02: Pre-scaffolding. No code exists yet, so no vulnerability disclosure is possible against a running system — this document describes the intended security posture, to be verified against the actual implementation once Phase 0 (per each deployable's ARCHITECTURE.md) ships.
  • 2026-09-05 (corrected — stale since the 2026-09-03 release): the line above no longer describes reality. gateway/v0.1.0/evals/v0.1.0 shipped 2026-09-03 with real attack surface a report could target today — virtual keys/budgets (internal/identity/internal/budget), the live-mutable Admin API (internal/admin, per docs/rfcs/2026-09-05-gateway-admin-api.md), and the Evals sandbox executor (evals/evals/rollout/sandbox.py). This document's own severity taxonomy and threat-class table above are real, verified security posture, not aspirational — see THREAT_MODEL.md for the current, actively-reviewed detail. Corrected 2026-10-08: the Admin API was not in gateway/v0.1.0 — internal/admin landed on main on 2026-09-05 (the day this entry was written) and first shipped in gateway/v0.2.0 (2026-09-09); corrected against the tagged trees (gateway/v0.1.0 has no gateway/internal/admin/) and gateway/changelog/0.2.0.md, the first changelog to name internal/admin.
  • Kelvran does not currently support (and has no near-term plan to support) a hosted/managed offering — self-hosting is the only deployment model, per PRD.md's non-goals.
  • 2026-09-04: The Evals sandbox's threat mitigations are narrower than this document previously implied — see the corrected THREAT_MODEL.md § Evals table row above. Package-registry-proxy hardening, cross-sandbox isolation, scoped per-tool credentials, and audit-trail-tied-to-a-trace are all unbuilt; splitting these into separate future RFCs, not fixed in one pass, per that document's own change log.
  • 2026-09-15: Server-side prompt/template management (docs/rfcs/2026-09-13-gateway-prompt-management.md) shipped with prompts deliberately GLOBAL, not tenant-scoped, per that RFC's own resolved design fork — any virtual key may resolve any prompt_id, with no ownership/authorization concept anywhere in internal/prompt. This is a direct instance of the P1 threat class above (that class's own definition already names "prompts" explicitly, predating this feature) — see THREAT_MODEL.md's Gateway Information Disclosure row for the full detail. Not a bug: it's the accepted, disclosed cost of the global-config design. Operator guidance below.
  • Fixed 2026-09-26: until this date, a client-facing error response for an upstream provider failure (any non-2xx upstream HTTP status, after the fallback chain — if configured — exhausted) included the RAW upstream response body verbatim, not just its status code. For a provider: "bedrock" deployment this could disclose the operator's real AWS account ID and IAM role ARN (a real AccessDeniedException body names both); more generally it let any authenticated tenant fingerprint upstream infrastructure (self-hosted backend stack traces, internal hostnames) by causing upstream failures. This never affected server-side observability — the full raw body was, and still is, captured in Kelvran's own structured error logs (gateway/internal/gateway/dataplane/dataplane.go's logRequest/logEmbeddingsRequest) regardless. Fixed: cmd/gateway/main.go's writeErrorResponse now sends only a generic, status-naming message (UpstreamHTTPError.ClientSafeMessage(), gateway/internal/gateway/dataplane/fallback.go) for this error class — the status code itself is unchanged and still disclosed, matching every other error class this function already maps. See THREAT_MODEL.md's 2026-09-26 Change Log entry for the full detail.

Security Best Practices for Operators

  • Never store upstream provider API keys in Kelvran's own config files in plaintext — use environment variables or a secrets manager, per this project's global coding-security conventions (see AGENTS.md).
  • Added 2026-10-11 (item 13c): the optional Prometheus pull endpoint (telemetry.prometheus.enabled: true) is unauthenticated by design and its exposition names virtual-key ids as label values, spend aggregates, model and deployment names and Go runtime detail. It binds to 127.0.0.1:9464 unless telemetry.prometheus.listen_addr says otherwise; a non-loopback bind belongs behind a NetworkPolicy or firewall that admits the scraper only (the recipe is in docs/how-to/deploy/kubernetes-kustomize.md), never on a host port the tenants can reach.
  • Added 2026-09-14, real and recurring, not hypothetical: never run docker compose config (or any command that prints a fully-resolved/interpolated environment) against a real .env/config.yaml — it prints every real secret in cleartext, including AWS/provider credentials. This exact mechanism has exposed the same live AWS access key twice during this project's own development (2026-09-13, 2026-09-14), with no rotation between occurrences and no technical control in place to prevent a third. If you must inspect resolved Compose config for debugging, scope it to a single non-secret key, or redact known secret env-var names before viewing the output (make config-safe, per AGENTS.md's own Gotchas entry). If a credential is ever exposed this way — to a terminal, a screen-share, a CI log, or an AI coding agent's own transcript — rotate it immediately; do not assume a private/local context makes exposure safe. See THREAT_MODEL.md's 2026-09-14 Change Log entry and AGENTS.md's Gotchas section for the full incident history.
  • Corrected 2026-09-18: eliminate the long-lived Bedrock credential, don't just rotate it — config.yaml's deployments.*.session_token_env (already a real, shipped, optional field for every provider: "bedrock" deployment) lets you populate .env with a short-lived AWS STS session triple (aws sts get-session-token/assume-role) instead of a permanent IAM user's static keypair, per AWS's own current guidance that access keys should be replaced, not rotated. This requires zero Kelvran code changes — only a credential-generation step you run against your own AWS account. See docs/upgrade-research/secrets-management-lifecycle-2026-09-15.md Finding 1 for the full rationale, and terraform/iam-access-analyzer/ for a complementary, different hygiene control (catching forgotten, not leaked, credentials) for whatever long-lived keys you can't eliminate this way. Corrected 2026-10-08: "zero Kelvran code changes" holds for generating the session, but a session_token_env (or any other *_env) credential is read once at startup, so every lapsed STS session needs a gateway restart to pick up its replacement. The non-restart path is the optional *_file fields — access_key_id_file/secret_access_key_file/session_token_file/api_key_file on DeploymentConfig (gateway/internal/gateway/controlplane/config.go), which dataplane.Pipeline.RunCredentialReloadLoop (gateway/internal/gateway/dataplane/credential_reload.go, started from cmd/gateway/main.go) re-reads periodically; *_file wins when both are set. Shipped in gateway/v0.16.0, including matching *_file fields for the Bedrock Guardrails and embed-sim detectors' own AWS credentials. Moving a deployment from *_env to *_file is itself one config change plus one restart. Procedure: docs/how-to/rotate-credentials.md.
  • Added 2026-09-25, found missing from this list by a 26-agent production-readiness audit despite being real, shipped operator controls since 2026-09-23 (the 8-phase upgrade round's own Phase 2): a virtual key can be restricted to a CIDR allowlist of client source IPs (AllowedSourceCIDRs, unset = unrestricted, never trusts a spoofable X-Forwarded-For by default); a self-hosted deployment's base_url must be https:// unless you explicitly set AllowInsecureHTTP for a legitimate localhost/dev case (fails closed at config-load time); and a specific self-hosted backend can be pinned to its own CA/client-cert pair (TLSConfig) instead of sharing the global transport. None of these are on by default — use them for any deployment where source-IP restriction or backend-specific TLS trust is part of your own threat model.
  • Issue least-privilege virtual keys per team/agent, not one shared key across an entire organization.
  • Restrict network exposure of the Gateway's admin API to a private network or VPN; it is not designed to be internet-facing.
  • Terminate TLS at or before the Gateway — plaintext prompt/completion traffic should never traverse an untrusted network segment.
  • Never embed tenant-specific secrets, PII, or confidential business logic in a shared prompt template (internal/prompt) — prompts are global config, not tenant-scoped, so any virtual key can resolve and indirectly probe any prompt's content by design (see the 2026-09-15 entry above).
  • Added 2026-09-23, per docs/upgrade-research/api-key-abuse-anomaly-detection-2026-09-22.md Finding 1: GitHub's secret-scanning partner program (which auto-detects and reports a leaked OpenAI/Anthropic/Google API key in a public repo to the issuing provider within seconds, triggering that provider's own automatic revocation) covers Kelvran's upstream provider credentials, but has no equivalent for a Kelvran-issued virtual key — VirtualKey.KeyHash is a Kelvran-native secret format no external partner program's regex patterns recognize, so a leaked virtual key would only ever be caught by Kelvran itself, or not at all. If you commit configs, scripts, or logs to a public (or even shared-private) repository, run a generic secret-scanner (Gitleaks, TruffleHog) against it yourself for the virtual-key shape Kelvran issues — this is an operator action, not something Kelvran's own admin API can currently do on your behalf.
  • Added 2026-09-23, per docs/upgrade-research/multi-agent-fanout-concurrency-composition-2026-09-23.md: if one integration's agent runs fan out into many parallel sub-calls, a single runaway or misbehaving run can consume that entire virtual key's own MaxInFlight/RPM/TPM/budget allowance, starving any OTHER concurrent use of the same key (a different agent run, or ordinary traffic) — extending the "least-privilege virtual keys per team/agent" guidance above with the specific case fresh, dedicated research confirms has no safer automated fix industry-wide: no production LLM gateway safely nests a per-agent-run sub-limit inside a parent key's own allowance, because doing so would require trusting a client-supplied run identifier, which every gateway surveyed (and Kelvran's own agent_run_id) treats as spoofable and unsafe for enforcement, not just as a Kelvran gap. If different agent runs sharing one integration need isolation from each other, issue each run its own virtual key rather than sharing one key across unrelated runs — every existing cap already enforces independently per key, with zero new configuration needed. GET /admin/virtual_keys/{name}/inflight (Admin/Viewer credential) surfaces a live, per-agent-run breakdown of one key's current in-flight count — self-reported and never used for enforcement, but useful for spotting "most of this key's concurrency is one agent run" before deciding whether to split it out.

Provider & Data-Flow Inventory

See docs/operations/PROVIDERS.md for exactly which upstream providers receive what data, under what auth mechanism.

Compliance Evidence for Operators

Added 2026-09-14, per docs/upgrade-research/ai-compliance-regulatory-readiness-2026-09-14.md Finding 2. SOC 2 is an attestation an operating organization produces via an independent audit of its own controls over time — it is not a property that attaches to open-source software itself, and Kelvran the project has no such organization to certify. If your own organization needs to represent your Kelvran deployment's controls to an auditor or customer (for your own SOC 2, ISO 27001, or an equivalent framework), the artifacts already maintained in this repository are the evidence to point at, not a substitute for your own audit:

  • THREAT_MODEL.md — STRIDE/OWASP-LLM-Top-10/NIST-AI-600-1 crosswalk, actively reviewed
  • This document's own severity taxonomy and disclosure process (above)
  • SECURITY-INSIGHTS.yml — machine-readable OpenSSF Security Insights metadata
  • The published container image's cosign signature, CycloneDX SBOM, and SLSA Build Level 2 provenance (RELEASE.md's verification section)
  • docs/operations/PROVIDERS.md — the provider/data-flow inventory, including the cross-border-transfer-safeguard disclosure added alongside this section

None of these artifacts constitute a SOC 2 report, an ISO 42001 certification, or a formal AI-Policy governance document on their own — those require an operating organization with named leadership and a management-review cadence to certify against, which this OSS project structurally is not. Pursue them only once a real operator or customer requires that specific attestation, not speculatively ahead of one.

Data Retention & Right to Erasure

Added 2026-09-18, per docs/upgrade-research/data-retention-right-to-erasure-2026-09-15.md. Kelvran itself is almost never the GDPR/CCPA-obligated party — that's typically the organization operating a deployment — but the operator's ability to comply is gated entirely on what this software actually does. Retention windows below are disclosed provisional defaults (mirroring the same honest framing docs/rfcs/2026-09-07-evals-trace-ingestion-object-storage.md §4 already uses for its own 90-day S3 lifecycle rule), not numbers derived from a specific compliance requirement.

Store Default retention Configurable? Erasure mechanism
Cache L1 (exact-match) 300s (cache.ttl_seconds) Yes POST /admin/cache/erase (see below) or wait out the TTL
Cache L2 (normalized-match) 75s (cache.l2.ttl_seconds) Yes Same as L1
Cache L3-lite (lexical near-duplicate) 300s (cache.l3.ttl_seconds) Yes None — POST /admin/cache/erase deliberately does not cover this layer (no Delete method exists on LexicalCache today); only the TTL removes an L3 entry
Budget-spend history (internal/budget, persisted via budget.persist_path when configured) Indefinite — no automatic expiry/rolling-deletion exists No automatic window; per-key deletion is real (see below) DELETE /admin/virtual_keys/{name} — budget.Tracker.Delete removes both the live in-memory record and, via boltstore.Store.Delete, the persisted bbolt record too
Spend ledger (internal/spendledger; in memory, ledger.persist_path bbolt or the budget's Redis; added 2026-10-11, item 13d) — hourly spend and request counts by virtual key, served model, client tool and session (the Claude Code session id or a grammar-passing agent_run_id) ledger.retention_hours, default 168 h: an hourly sweep drops whole hour buckets older than that (the Redis backend also carries a per-hash TTL of retention + 1 h as a backstop) Yes, ledger.retention_hours (never 0 — a ledger is never unbounded); attribution.capture_ids: false / attribution_capture_ids: false keep the Claude Code session id out of it (the Baggage agent_run_id still reaches it) By key: DELETE /admin/virtual_keys/{name} cascades to every row of the key; by session: POST /admin/spend/erase (Admin only; once per replica in bbolt/in-memory modes, once in Redis mode; the audit line carries a 16-hex SHA-256 prefix, never the id). Rows folded into other cannot be erased on their own
Admin audit log (gateway/internal/admin, slog.Info lines) Indefinite by design (see below), bounded only by whatever process captures stdout Yes — admin.enable_audit_log: false disables it entirely None at the record level (only whole-log disablement)
OTel span attributes exported by telemetry.exporter (row added 2026-10-09; kelvran.agent_run_id has been exported since 2026-09-02, and kelvran.client.tool plus the Claude Code kelvran.claude_code.* session / agent / parent-agent / prompt ids and agent type were added 2026-10-09) The tracing backend's own retention — Kelvran retains nothing once the exporter has shipped the span (the stdout exporter prints it to the process's stdout, retained by whatever captures that) Not in Kelvran. attribution.capture_ids: false (gateway-wide) or attribution_capture_ids: false (per key) stops the Claude Code identifiers from being recorded at all (a request that fails authentication never carries them, whatever the switches say); agent_run_id has no switch None in Kelvran — erase in the tracing backend. A disclosed provisional posture, not a number derived from a compliance requirement
Per-request decision log (row added 2026-10-15, round-5 item 29): the chat_completion slog line and its gatewayevents_v1 field, one GatewayDecisionEvent per chat completion — trace and span ids, virtual key id, requested model, outcome, fallback fields, budget spent, cost, savings, agent_run_id, billing_subject_id, finish reason (no event on embeddings or count_tokens) None applied by Kelvran — a JSON line on stdout, retained by whatever captures it: the ECS recipe's CloudWatch group keeps 30 days (log_retention_days), the Vector → S3/GCS pipeline keeps objects 90 days under the lifecycle rule in docs/rfcs/2026-09-07-evals-trace-ingestion-object-storage.md § 4, with OUTCOME_OK sampled at 10 % in the Kubernetes configs. Distinct from the spend ledger's ledger.retention_hours above, which bounds the gateway's own store and never this line Only in the sink — see "Decision-log retention" below None in Kelvran — erase in the log sink. The line never carries prompt or completion text, a bearer credential or key hash, or the Claude Code identifiers (THREAT_MODEL.md's Information Disclosure row; docs/reference/metrics-and-logs.md); on a failed request its error field, and whenever a fallback happened the event's fallbackReason — even on an OUTCOME_OK line — do carry the abandoned hop's raw provider error body (the 2026-09-26 entry above)
Admin audit trail, durable copy (internal/admin/auditstore, added 2026-09-25) Indefinite — an append-only JSONL file with no built-in rotation or size cap; unlike the slog.Info line above (bounded only by whatever external process captures stdout), this is a specific on-disk file Kelvran itself writes and never prunes Governed by the same admin.enable_audit_log switch as the slog.Info line above — disabling it disables both paths together None at the record level, same limitation as the slog.Info line above; unlike that line, an operator wanting erasure here must manage the file directly — no Delete/Rotate/Prune method exists on auditstore today, and since 2026-10-16 (round-5 Tier S item 26) each line is chained to the previous one (seq, prev_hash) and signed when admin.audit_signing_secret_env is set, so a removed or edited line is reported by kelvran audit verify to anyone without the key: erasure here is detectable by design, not merely unoffered — whole-file deletion, a tail cut at a line boundary or a restored older copy are visible only against the audit_seq/audit_hash the stdout sink keeps, once the gateway has restarted

Cache erasure: POST /admin/cache/erase (Admin-tier) services a per-request erasure against L1/L2 for one specific request shape — the caller must already know the original request's defining fields, since Kelvran's cache is keyed by a content hash, not a per-tenant index. See dataplane.Pipeline.EraseCacheEntry's own doc comment for the full mechanism and its L3 limitation.

Audit log retention basis (disclosed default posture, not a legal determination): the admin audit log's lack of automatic expiry is deliberate, not an oversight — an administrative action log (who changed a virtual key, a deployment weight, a prompt, when) serves this software's own accountability/incident-investigation purpose in a way that plausibly falls under UK GDPR Article 17(3)(b)/(e) (compliance with a legal obligation; establishment, exercise, or defence of legal claims) as a real, available basis for declining an erasure request against it. This is disclosed as an available basis an operator may invoke, not a settled legal conclusion — per the research doc's own Finding 2, that exemption "must be affirmatively claimed and documented" with a genuine statutory basis specific to the invoking organization's own jurisdiction and circumstances. Confirm this applies to your own deployment (or adopt a different basis/retention policy) before relying on it; this is not legal advice.

Decision-log retention (added 2026-10-15, per docs/upgrade-research/kelvran-deep-research-round5-2026-10-14.md item 29): two retentions are easy to conflate. ledger.retention_hours (168 h by default) bounds the spend ledger — the gateway's own hourly spend-and-request-count store behind GET /admin/spend — and nothing else. The per-request decision log (the chat_completion line and its gatewayevents_v1 event) has no retention inside Kelvran: the gateway keeps no copy, and the line lives as long as the log sink keeps it — 30 days on the ECS recipe's CloudWatch group, 90 days under the Vector → S3/GCS lifecycle rule (the S3 module terraform/vector-s3-shipper ships it, default 90; on GCS the operator applies the equivalent Object Lifecycle rule — the Vector file cannot), and in the Kubernetes Vector configs only a 10 % sample of OUTCOME_OK events reaches object storage. An operator whose own downstream system is a high-risk AI system under Regulation (EU) 2024/1689 must keep the logs it generates automatically, to the extent they are under the operator's control, for at least six months (Article 26(6)) — applicable from 2 December 2027 for Annex III systems and 2 August 2028 for Annex I systems under Regulation (EU) 2026/1744, recital 40 (see THREAT_MODEL.md's EU AI Act Scoping Rationale); the minimum itself stands — Articles 12 and 26 unamended as far as the round-5 lane could see — but is not yet applicable. Such a deployer should set the sink's retention to six months or more and ship every outcome (rate: 1 in the Vector gatewayevents_sample transform, as the Compose variant already does; log_retention_days at or above 180 on the ECS recipe), since a sampled copy is not a complete record. This is a recommendation for that deployer, not a Kelvran default or requirement: Kelvran as router and cache is not the AI system under its own scoping, and whether Article 26 reaches a given deployment, and whether this line counts as such a log, is that deployer's own determination — not legal advice.

See docs/operations/DATA-SUBJECT-REQUESTS.md for the manual, interim data-subject-request procedure across all three stores, including what today's tooling can and cannot do.

Bug Bounty

Not yet adopted. Tracked as a future decision, not a current commitment.

Contact

General questions about this policy: open a GitHub issue. Vulnerability reports: use the reporting channel above, not a public issue — this separation exists specifically so a live vulnerability is never disclosed publicly before a fix ships.

There aren't any published security advisories