Skip to content

feat(vanity-gateway): pass the LLM Gateway endpoint through the stack - #1026

Open
Max-NV wants to merge 1 commit into
mxing/vanity-gateway-llm-endpoint-chartfrom
mxing/vanity-gateway-llm-stack-wiring
Open

feat(vanity-gateway): pass the LLM Gateway endpoint through the stack#1026
Max-NV wants to merge 1 commit into
mxing/vanity-gateway-llm-endpoint-chartfrom
mxing/vanity-gateway-llm-stack-wiring

Conversation

@Max-NV

@Max-NV Max-NV commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Why

The self-managed stack does not pass the Vanity Gateway's config block through wholesale. It enumerates the keys one at a time when rendering chart values, so llmGatewayEndpoint was dropped even after #1025 added it to the chart.

The mappingConfig half of the values does pass through untouched. The result is that an operator can mark an openai model with functionType: LLM and have the mapping land correctly, while the endpoint the gateway needs to act on it never arrives, so the container fails at startup with LLM_GATEWAY_ENDPOINT is required when a model sets functionType LLM.

What changed

Adds llmGatewayEndpoint to the rendered vanityGateway.config block in global.yaml.gotmpl, defaulted to the in-cluster LLM Gateway service, and documents it in base.yaml next to the other Vanity Gateway config keys.

Customer Release Notes

Self-managed installs can now configure the Vanity Gateway's LLM Gateway endpoint.

Plan Summary

One additional key in the values rendered for the vanity-gateway release. No resource changes for installs that do not use LLM-routed models.

Usage

addons:
  vanityGateway:
    config:
      llmGatewayEndpoint: http://llm-api-gateway.nvcf.svc.cluster.local:8080

Models that set functionType: LLM also need addons.llm.enabled, which deploys the LLM Gateway.

Testing

Rendered the self-managed stack and confirmed the key reaches the chart values.

Not covered by the end-to-end run recorded on #1022. That cluster ran the chart installed
directly from the working tree rather than through the stack, so this rendering path was not
exercised live. The value it needs to produce is the same one that run used.

No QA needed.

Notes

Chart and image pins are not bumped here. Both need a published artifact carrying the #1022 service change, which does not exist yet.

Stacked on #1025, which adds the chart key. Merging this first would break installs: the chart config block is additionalProperties: false, so it would reject the value the stack now sends. Base is set to that branch and will retarget to main when it merges.

References

None

Related Pull Requests

Dependencies

None

Issues

Relates to #1021

@Max-NV
Max-NV requested a review from a team as a code owner August 20, 2026 00:55
@Max-NV
Max-NV requested a review from arpitad-nv August 20, 2026 00:55
@coderabbitai

coderabbitai Bot commented Aug 20, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (1)
  • main

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e7c582c7-729a-4097-9664-93a96075db26

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@Max-NV
Max-NV marked this pull request as draft August 20, 2026 00:58
@Max-NV
Max-NV force-pushed the mxing/vanity-gateway-llm-endpoint-chart branch from a404d9a to 98cbd52 Compare August 26, 2026 20:10
@Max-NV
Max-NV force-pushed the mxing/vanity-gateway-llm-stack-wiring branch 2 times, most recently from 55ca6f7 to 6a65d40 Compare August 26, 2026 22:14
The self-managed stack enumerates the Vanity Gateway config keys one by one
when rendering chart values, so llmGatewayEndpoint was dropped and the gateway
failed to start with "LLM_GATEWAY_ENDPOINT is required when a model sets
functionType LLM". The mappingConfig half already passed through untouched, so
an LLM-routed model could be configured but never worked.

Adds the key to the rendered config block and documents it in base.yaml with
the in-cluster LLM Gateway as the default.

Chart and image pins are not bumped here. Both need a published artifact
carrying the service-side change, which does not exist yet.

Signed-off-by: Max Xing <mxing@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants