Skip to content

feat(dwctl): per-process internal credential for calls into /ai (DW-1009) - #2154

Merged
tobydigz merged 5 commits into
mainfrom
feat/dw-1009/internal-credential
Nov 3, 2026
Merged

tobydigz merged 5 commits into
mainfrom
feat/dw-1009/internal-credential

Conversation

@tobydigz

@tobydigz tobydigz commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Part of DW-1005 (DW-1009). The playground, health probes and continuation authenticated as hidden keys by reading their plaintext secrets. They now authenticate by key id with a per-process internal credential.

What changes

Internal credential (dwctl/src/auth/internal_credential.rs): dwi-<key id>.<expiry>.<HMAC-SHA256>, signed with a 32-byte key generated at startup and held only in memory, so only the process that minted a token accepts it. 60s TTL for playground and probes.

Verifying layer on /ai: verifies a dwi- bearer, looks the key id up in onwards' live key set and passes its hash on as an onwards PresentedKey. An invalid, expired, foreign or deleted-key token is a 401 invalid_api_key, never passed through as an ordinary key. Layers that hashed the bearer (inference middleware, error enrichment, prompt cache, continuation, image normaliser, prices, rejections, /models) use the presented hash, so spend caps and attribution still apply.

onwards: PresentedKey request extension, preferred over hashing the bearer; the key must still be in the key set. Targets::key_ids maps the api_key_id label to the key hash. Adds a public field to Targets, so it rides the pending onwards 2.0.0 (#1802).

Callers: playground and continuation use the hidden key's id; probes and deployment checks use the system key's id, and ak.secret is gone from those queries. Hidden-key repository functions return the key id.

Not yet: batch, file, flex and sample-file creation still hand fusillade a stored key, via ApiKeys::legacy_dispatch_secret, until DW-1010 dispatches by key id.

Testing

  • Forged, malformed, expired, other-process, unknown-key and deleted-key tokens are rejected with no upstream call; a valid token reaches the upstream with no dwi- header; playground and test_probe work end to end over HTTP.
  • cargo nextest run -p dwctl -p onwards (3690 passed) and just lint rust pass locally.

View guided diff Turn on auto-fix

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Deploying control-layer with  Cloudflare Pages  Cloudflare Pages

Latest commit: 39dc3e2
Status: ✅  Deploy successful!
Preview URL: https://4021c8e3.control-layer.pages.dev
Branch Preview URL: https://feat-dw-1009-internal-creden.control-layer.pages.dev

View logs

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 2, 2026

Copy link
Copy Markdown

Running ultrareview automatically — this PR appears to change authentication, authorization, or input validation, where a missed bug can expose data or grant the wrong access, so a deeper multi-pass review is worth running. I'll post findings when complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ultrareview ran automatically — this PR appears to change authentication, authorization, or input validation, where a missed bug can expose data or grant the wrong access, so a deeper multi-pass review is worth running.

All reported issues were addressed across 66 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread onwards/src/strict/handlers.rs Resolved
Comment thread dwctl/src/auth/internal_credential.rs Resolved
Comment thread dwctl/src/prompt_cache/principal.rs Resolved
Comment thread dwctl/src/lib.rs Resolved
Comment thread dwctl/src/auth/internal_credential.rs Resolved
An embedding gateway can now authenticate a request itself and hand onwards
the key's hash as a PresentedKey request extension, instead of a bearer
token onwards hashes. The key must still be in the target's key set.
Targets::key_ids maps the api_key_id label to the key hash, kept in step
with key_labels, so the gateway can find a key it authenticated by id.
The playground, health probes and the continuation layer authenticated
as hidden keys by reading their plaintext secrets. They now authenticate
by key id: dwctl mints a short-lived `dwi-<id>.<expiry>.<hmac>` token
signed with a key generated at startup and held only in memory, so only
this process accepts it. A layer at the front of /ai verifies the token,
looks the key up in onwards' live key set and hands its hash on as an
onwards PresentedKey; every layer that identified the caller by hashing
the bearer now uses that hash, so deleted keys and spend caps still apply.

Hidden-key repository functions return the key id. Batch, file, flex and
sample-file creation still store a plaintext key in fusillade until it
dispatches by key id (DW-1010), through ApiKeys::legacy_dispatch_secret.

Refs DW-1009
Request logging attributed an internal credential through onwards'
AuthenticatedApiKeyId, which is missing when an inner layer (translation,
image normalisation) refuses the request first. A background playground
request refused that way left its row processing. Logging now resolves the
captured token itself: signature checked, expiry not, since every token it
sees was admitted at the edge.

Refs DW-1009
@tobydigz
tobydigz force-pushed the feat/dw-1009/internal-credential branch from e1a074a to 1f8834c Compare November 2, 2026 13:19

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 4 reviewed files. 1 file intentionally excluded from review.

Requires human review: Auto-approval blocked by 5 unresolved issues from previous reviews.

View guided diff | Turn on auto-fix | Re-trigger cubic

Strict-mode refusals hashed the Authorization bearer, so a request an
embedder pre-authenticated was refused without its caller's identity.
Request logging re-resolved an internal credential through onwards' live
key set, so a key deleted mid-request lost its attribution and the
response record was dropped. The verified key now rides on the response
from just inside request logging; re-resolving is only the fallback for
abandoned requests. Also expire tokens at their expiry second and fix
stale docs.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 8 files (changes from recent commits).

Requires human review: Per-process internal credentials and presented-key auth on /ai change production request-path authentication, with security and spend/attribution impact; this requires human approval.

View guided diff | Turn on auto-fix | Re-trigger cubic

@tobydigz
tobydigz added this pull request to the merge queue Nov 3, 2026
Merged via the queue into main with commit 33457cb Nov 3, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants