Skip to content

Add support for nvidia driver selection at boot time - #1041

Draft
vigh-m wants to merge 3 commits into
bottlerocket-os:developfrom
vigh-m:nvidia-multi-driver/core-kit
Draft

Add support for nvidia driver selection at boot time#1041
vigh-m wants to merge 3 commits into
bottlerocket-os:developfrom
vigh-m:nvidia-multi-driver/core-kit

Conversation

@vigh-m

@vigh-m vigh-m commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Description of changes:
Adds boot-time NVIDIA driver-branch selection for dual-branch images.

The kernel-kit change (bottlerocket-kernel-kit#536) makes the lts and pb driver branches co-installable and namespaces each branch's units as nvidia-<branch>-*

ghostdog match-nvidia-branch

  • Implement new match-nvidia-branch subcommand. It resolves the target branch from settings.kernel.drivers.nvidia.branch (added in bottlerocket-settings-sdk#154), falling back to instance-type detection when the setting is unset
  • writes a /run/nvidia/branch-<branch> marker file

os package wiring

  • New nvidia-select-branch subpackage requiring nvidia-lts, nvidia-pb, only installed when nvidia-dual-branch image feature is enabled (twoliter#747), and building and nvidia variant
  • Adds nvidia-select-branch.service to execute ghostdog match-nvidia-branch in a dual driver image

Service reordering

  • ecs-gpu-init.service and generate-cdi-specs.service now order After=drivers.target instead of the branch-specific load-*-kernel-modules.service units, since those unit names are now per-branch.

Testing done:

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

Add `match-nvidia-branch`, which resolves the active NVIDIA driver
branch (settings.kernel.drivers.nvidia.branch, else PCI device
detection) and writes a marker file.

Make `match-nvidia-driver` flavour matching branch-aware so co-installed
lts and pb drivers select the right units.

Signed-off-by: Vighnesh Maheshwari <vighmah@amazon.com>
Order these GPU servces after drivers.target to ensure that they
wait for the GPU devices and kernel modules to be loaded regardless
of a multi driver or single driver variant definition

Signed-off-by: Vighnesh Maheshwari <vighmah@amazon.com>
On dual-branch images both NVIDIA driver branches (lts and pb) are
co-installed, and one is selected at boot via the services in this
subpackage

- nvidia-select-branch.service: a oneshot, ordered before
  drivers.target, that runs `ghostdog match-nvidia-branch` to pick the
  branch and write the marker file.
- nvidia-<branch>-.service.d/10-branch-gate.conf: a systemd prefix
  drop-in. The trailing dash means the drop-in applies to every unit
  whose name begins with nvidia-<branch>- (the overlay service and all
  per-branch module-load units), gating them on
  ConditionPathExists=/run/nvidia/branch-<branch> and ordering them
  after nvidia-select-branch.service.
- drivers.target drop-in ordering drivers.target after the selection
  oneshot.

The subpackage is pulled in only on nvidia/nvidia-fips variants built
with the nvidia-dual-branch image feature; single-driver images omit it
and run their units unconditionally.

Signed-off-by: Vighnesh Maheshwari <vighmah@amazon.com>
@vigh-m vigh-m changed the title Nvidia multi driver/core kit Add support for nvidia driver selection at boot time Sep 11, 2026
@vigh-m
vigh-m force-pushed the nvidia-multi-driver/core-kit branch from ea035aa to 8fa54ff Compare September 11, 2026 22:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant